Abstract
Background
Medial humeral epicondyle fractures of the elbow are one of the most common injuries in childhood and often require surgery. There are currently no standardised outcome measures to assess progress after an elbow injury in a child. Wide variation in currently reported outcomes makes comparison of treatment difficult. This study aims to identify outcome measures that have previously been reported in studies evaluating the management of medial epicondyle fractures in children and to facilitate the development of a consensus core outcome set (COS) suitable for use in all future studies of medial humeral epicondyle fractures in children.
Methods/design
This study will include a systematic review of the academic literature to identify a list of outcome measures that have previously been reported. The list of outcome measures will be used in a consensus setting exercise with focus groups of key stakeholders to identify key outcomes. A Delphi process to include two rounds will then be used to define the most important outcomes to all stakeholders forming the COS.
Discussion
Core outcomes represent the minimum expected data reported for a specific condition and will improve the quality of future studies reducing bias, allowing easier comparison and enhancing opportunities for larger meta-analysis. It is anticipated that this COS will form part of the feasibility to a National Institute for Health Research (NIHR) Health Technology Assessment (HTA)-funded trial concerning the management of elbow fractures in children.
Trial registration
Core Outcome Measures in Effectiveness Trials Initiative (COMET), registration number:949. Registered on 17 January 2017.
Prospero International prospective register of systematic reviews, registration number: CRD 42017057912. Registered on 16 April 2017.
Keywords: Core outcome set, Delphi, Consensus methods, Elbow fracture in children, Medial epicondyle
Background
Medial humeral epicondyle fractures are one of the most common injuries in childhood and often require surgery [1, 2]. Long-term sequelae may include non-union, mal-union, instability, nerve injury, growth arrest, deformity and loss of function [1, 3]. There are currently no standardised outcome measures to assess progress after an elbow injury in a child. Wide variation in currently reported outcomes (radiographic-, surgeon- and patient-reported) makes comparison of the treatment of elbow fractures in children difficult.
In particular, the management of medial epicondyle fractures in children is controversial [1]. National Institute for Health Research (NIHR) Health Technology Assessment (HTA) have made children’s orthopaedic surgery a priority area and are considering a clinical trial in the management of medial epicondyle fractures in children [4]. A major barrier to these trials is the lack of a clear outcome, i.e. pain, function, fracture union, cosmesis, quality of life.
An emerging strategy to overcome these issues has been the development of core outcome sets (COSs). COSs represent the minimum required dataset for all randomised control trials (RCT) of a certain condition [5–14].
The use of COSs aim to reduce heterogeneity in research allowing easy comparison of studies, improve accuracy of data interpretation and reduce outcome reporting bias. Standardisation of outcomes will also lead to a reduction in omissions and increased statistical power for meta-analysis [15].
In recent years a small number of COSs have been developed for use in adult orthopaedics; however, these are limited to specific conditions (hip fracture [16], back pain [17] and total joint replacement [18]). There are currently no COSs available for use in clinical trials across the whole of paediatric orthopaedics.
Aims and objectives
Aim
The aim of this study is to develop a COS suitable for use in observational research, clinical trials and routine treatment of acute medial humeral epicondyle fractures in children.
Objectives
The specific study objectives are:
To identify outcomes that have previously been reported in RCT, cohort studies, case-control studies and case series from a systematic review of the academic literature
To identify outcomes important to children and parents
To prioritise outcomes from the perspective of key stakeholder groups using a two-round Delphi
To conduct a consensus meeting, compare outcomes considered important to all stakeholders and to integrate important outcomes into a combined COS
Methods/design
Systematic review
Search methods for identification of studies
The search strategy will be applied to the Cochrane Central Register of Controlled Trials (CENTRAL), SCOPUS and MEDLINE. Multiple databases will be used to maximise the sensitivity of the search (January 2000 to December 2015)
The advantages conferred by using CENTRAL in addition to the other databases is that trials from other sources of research (e.g. journals not indexed in MEDLINE and conference proceedings) are hand-searched, and controlled trials from these are included. This improves the chances of identifying all relevant studies.
Eligibility of studies
Two reviewers (SD and JS) will independently screen all titles and abstracts of papers identified in the initial search. Titles of articles will be reviewed and included or excluded by using Rayyan [19]. Full-text manuscripts of any titles/abstracts that may be eligible for inclusion, or for which there is insufficient data in the title and abstract to make a clear decision, will be obtained.
The full-text papers will be assessed independently by two review authors (SD and JS) and any disagreement on the eligibility of included studies resolved through discussion. Where resolution is not possible, a third review author (DP) will be consulted.
The purpose of this study is to identify all outcomes reported irrespective of study quality. In addition as there is no synthesis of outcome data from the included studies, a critique of the methodological quality of the studies is not necessary.
This process will be documented as per the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidance [20].
Data extraction
The following data will be extracted from each study: paper and author details; year and journal of publication; study type; inclusion criteria (Table 1) and exclusion criteria, duration of follow-up; sample size; injury and intervention(s) under investigation, primary and secondary outcomes, method of measurement and time points at which outcomes were measured.
Table 1.
Inclusion criteria for study selection
| Study design | All study designs except systematic reviews, case studies (< 10 cases) and expert opinion. |
| Patient population | Study exclusively involving children (< 18 years) at injury, with a fracture of the medial humeral epicondyle |
| Interventions | Any non-operative or operative intervention for management of acute medial humeral epicondyle fractures |
| Outcomes | All outcomes |
| Other considerations | All studies must involve at least 10 cases of medial humeral epicondyle All studies must involve humans All studies must be in the English language |
Outcomes
The primary aim of the systematic review is to generate a list of all outcomes and measurement instruments reported historically in eligible studies.
Data analysis and presentation
A comprehensive framework of health can be beneficial in developing a COS, favouring the content validity of the end product. A new framework has been developed that aims at including all key aspects of a health condition to ensure comprehensiveness of COSs.
Outcome terms will be assigned to one of the five core domains from the Dodd-Williamson classification (Table 2). The five core areas that should be covered by outcome measures to ensure a full breadth of reporting are: (1) adverse events, (2) death, (3) physiological/clinical, (4) life impact and (5) resource use [21].
Table 2.
Overview of modified Dodd-Williamson [21] classification of outcomes
| Core area | Core domains | Example |
|---|---|---|
| Adverse events | Adverse events | Unintended consequences |
| Death | N/A | N/A |
| Physiological/clinical | Musculoskeletal | Mal-union, non-union, range of motion |
| Life impact | Physical/social/role/emotional/cognitive functioning Health-related quality of life (HRQL) Delivery of care* |
PROMS, activities of daily living, satisfaction |
| Resource use | Economic/hospital/need for intervention, societal burden | Length of stay, further surgery, physiotherapy |
| Technical considerations | Technical/surgical considerations | Radiographic measurements |
*Delivery of care does not refer to the resource delivery, but instead includes patient satisfaction, patient preference, adherence, withdrawal, tolerability; PROMS, patient-reported quality of life, N/A not applicable
A sixth domain of technical consideration will be added for technical or surgical outcomes relevant to surgeons not covered by the existing framework.
Within each domain we will evaluate the number of different outcomes used and the frequency of selection for each individual outcome measure. We will also record the method of measurement and the time points at which they were measured.
Identification of potential outcomes
A list of all potential outcomes will be identified from the systematic review as described above. Outcomes will be listed both individually and by domain to aid interpretation. All outcome domains and included outcomes will be reviewed by the Study Steering Group (SSG) to assess suitability of domain name and grouping. The SSG will consist of authors SD, JJK and DP.
To identify the outcomes of importance to all stakeholder groups a Delphi approach will be used. An overview of the COS developmental process is shown in Fig. 1. This will enable participants to provide anonymous opinions with equal influence given to all participants. This method also avoids individual participants’ responses being influenced by the opinion of their peers. Key stakeholder groups will consist of children, parents (Delphi stream 1) and clinicians (Delphi stream 2).
Fig. 1.
Overview of the core outcome set (COS) development process
Identification of outcomes of importance to patients and parents
It is essential that consideration is given to the opinions of parents and children regarding the treatment of medial humeral epicondyle fractures. They should be given the opportunity to identify the most important outcomes and domains based own their own experiences and beliefs.
The opinions of children are key as they represent the group upon which the short- and long-term benefits and adverse effects of treatment will have the largest impact.
Medial epicondyle injuries typically affect older children and, as such, it is unlikely that our cohort will include any young children due to the nature of the injury. A recent national audit of medial epicondyle injuries indicated that 7-year-olds were the youngest children presenting with this injury pattern. Children over the age of 7 years with a previous medial humeral epicondyle fracture will be identified from a UK tertiary paediatric centre and invited to interview. A minimum of 10 children will be recruited with no upper limit.
Eligibility will be based on history of an acute medial humeral epicondyle fracture treated non-operatively or operatively and the ability to complete the interview in English without the need for a translator. Eligible children will be invited to attend a one-off interview structured by the use of a questionnaire. The authors feel this will improve compliance and accurate completion of the questionnaire. This will not be audio-recorded. It will consist of a researcher reading out the questions and recording the answers. The survey questionnaire (Delphi stream 1) has been designed to meet the developmental needs of a broad array of children using a traffic-light system for grading of outcome importance (Green ‘not important’, Amber ‘important but not critical’ and Red ‘critical importance’. The traffic-light scoring system has been approved by a Young Person Advisory Group (YPAG).
Children will also have the opportunity to add any additional outcome they feel may have been missed. There will be no formal qualitative interview or qualitative analysis.
A separate sample of 20 parents will be identified from an existing trauma database at a UK tertiary paediatric centre. Parents will be invited to complete an online Delphi questionnaire (Delphi stream 1). Parents will complete all rounds of the Delphi process.
A plain English explanation for all outcome measures listed will be included in the stream-1 questionnaire for parents and children. Language will be approved by the YPAG.
Consultation with the HRA deemed this study a service evaluation project with no requirement for ethical approval (reference 60/89/81). Informed consent will be assumed if participants agree to fill in the surveys.
Identification of outcomes of importance to clinicians
Overview
The Delphi (stream 2) questionnaire will consist of the same outcomes used for children and parents but additional technical and surgical considerations relevant only to clinicians will be added.
Participants
The Delphi study will be conducted with clinicians who have a specialist interest in paediatric trauma and orthopaedic surgery. Three clinician stakeholder groups will be surveyed, comprising UK surgeons, international trialists and UK physiotherapists. Clinicians will only be invited to participate if they are currently involved in the clinical care of children with medial humeral epicondyle fractures.
Clinical leads for surgeons’ groups will be identified via the British Orthopaedic Surgery Surveillance (BOSS) collaborative and the Infrastructure for Musculoskeletal Paediatric Acute Care Clinical Trials (IMPACCT) international collaborative. With the exception of IMPACCT members, participants are not required to have previous experience in clinical research. Eligible participants will contacted via email and asked to complete an online Delphi questionnaire (stream 2). A minimum of 20 participants will be sought from the UK surgeons’ group, and 10 from the group of international trialists.
The number of clinicians at each stage of the process will be recorded including: total number invited to participate; participants recruited to round 1 and numbers completing subsequent rounds. Attrition rates will be documented and analysed. Each participant will be given a unique registration number to enable tracking of attrition at each stage of the Delphi process. Reminder emails will be sent for those failing to complete each round.
Bespoke Delphi manager software will be used to ensure that all information is recorded against the participant’s unique registration number only. Participants will not be able to access information about other participants or other individual’s responses.
Delphi survey
Delphi round 1
In the first round the online questionnaire will also be used to request demographic information for registration. Information collected will include; participant’s name, stakeholder group, clinical role, place of work and email address. Personal information will be stored in a separate database with a unique registration number.
At each stage of the Delphi process participants will be given 3 weeks to complete the questionnaire. A reminder email will be sent at the end of week 2 to encourage completion and reduce attrition rates.
Participants who do not complete round 1 will be excluded from participation in further rounds.
Round-1 survey format
All data will be collected using an online format. Content for round 1 will include: the participant demographics as outlined above, a list of outcomes to be scored, listed alphabetically and by domain. Participants will be asked to score listed outcomes and will have the option of adding any additional outcomes of importance not currently listed.
Participants will be asked the key question, ‘What outcomes may influence how you treat fractures of the medial humeral epicondyle in children’?
The Grading of Recommendations, Assessment, Development and Evaluations scale will be used to score each outcome. Participants will be asked to grade each listed outcome in the format 1–9, with 1–3 deemed ‘not important’; 4–6 ‘important but not critical’ and 7–9 ‘critical importance’.
Analysis of round 1
All additional outcomes proposed by participants will be reviewed by two assessors (SD and JS) to ensure that they represent new outcomes not already listed. In case of uncertainty or disagreement a third assessor will be consulted (DP).
The number of participants who scored each individual outcome will be recorded. The distribution of scores will also be summarised by the stakeholder group. All outcomes will be carried forward to round 2.
Response rate in round 1
The response rate will be assessed and presented as: total number of participants registered; number by stakeholder group; number completing round 1; and the percentage of registered participants vs. invited based on information from clinical leads.
Continuation to round 2 will be determined based on response rate of round 1. In case of low numbers (< 10) the protocol for future Delphi rounds will be reviewed. Where responses do not differ greatly, an SSG review may suggest combining appropriate stakeholder groups.
Delphi round 2
Round 2 data will be presented and recorded using an online format. Participants will be presented with data from all stakeholder groups. Data presented will include the number of respondents and distribution of scores for all listed outcomes and any additional outcomes added after round 1. Participants will also be able to view their individual score from round 1.
Participants will then be asked to rescore the outcome in light of the additional information provided. New outcomes added in round 1 will also be scored. Any changes to scoring from round 1 will be recorded.
Analysis of round 2
The total number of participants invited to participate in round 2 will be documented. The number of participants who scored each individual outcome will be recorded. The distribution of scores will also be summarised by the stakeholder group.
For each stakeholder group, each outcome will be classified as ‘consensus in’, ‘consensus out’ or ‘no consensus’ according to the consensus criteria (defined below).
Additional rounds of Delphi may be introduced if it is felt by the SSG that consensus had not yet been achieved.
Consensus meeting
The final phase of the study will involve a consensus setting exercise by the Consensus Focus Group (CFG). The CFG will consist of representatives from all stakeholder groups (UK surgeon, physiotherapist, patient and parent), SSG and an independent COMET representative.
Results from round 2 of the Delphi survey will be presented and discussed followed by voting to reach a final consensus COS.
Definition of consensus
The classification of consensus (Table 3) will be used to determine whether a consensus has been reached or not for each individual outcome.
Table 3.
Classification of consensus [22]
| Consensus classification | Description | Definition |
|---|---|---|
| Consensus in | Consensus that outcome should be included in the core outcome set | 70% or more participants scoring as 7 to 9 and < 15% participants scoring as 1 to 3 |
| Consensus out | Consensus that outcome should not be included in the core outcomes set | 70% or more participants scoring as 1 to 3 and < 15% of participants scoring as 7 to 9 |
| No consensus | Uncertainty about importance of outcome | Anything else |
In order to reach a consensus that the outcome should be included in the COS requires agreement by the vast majority (> 70%) that the outcome in question is of ‘critical importance’ with only a minority (< 15%) deeming it to be of ‘no clinical importance’ [22].
For an outcome to be excluded from the COS the vast majority (> 70%) must score the outcome as of ‘no clinical importance’ with the minority (< 15%) of participants scoring it as ‘critically important’.
The threshold for definition of consensus for this study has been predefined to prevent any bias of the end results towards beliefs of the research team.
In the event of a ‘no consensus’ outcome, the final decision will be made by the CFG. The structured expert CFG will enable representatives from key stakeholder groups to discuss differences of opinion, justify their perspectives and make an informed decision using a Nominal Group Technique (NGT). It will be undertaken through face-to-face meeting. Final consensus will be reached by means of a vote of stakeholders.
Discussion
There is currently no published COS for medial humeral epicondyle fracture in children. The development of COSs in this clinical area will improve the quality of future studies reducing bias, allowing easier comparison and enhance opportunities for a larger meta-analysis. It is anticipated that this COS will form part of the feasibility to an NIHR HTA-funded trial concerning the management of medial humeral epicondyle fractures in children.
Acknowledgments
Acknowledgments
None.
Funding
This work was supported by an AOUK foundation grant. Views and opinions expressed therein are those of the authors and do not necessarily reflect those of the AO Foundation. This article presents independent research supported by a National Institute for Health Research (NIHR) clinician scientist fellowship (to Daniel C Perry; grant number NIHR/CS/2014/14/012). All authors carried out this research independently of the funding bodies. The findings and conclusions in this report are those of the authors and do not necessarily represent the views of the UK Department of Health.
Availability of data and materials
Data sharing is not applicable to this article as no datasets were generated or analysed during the current study.
Authors’ contributions
DP, SD, JS and JJK conceived of the study and participated in its design. DP and SD designed the protocol for the systematic review. SD and JS will complete the systematic review search and extraction of eligible papers. SD and JS will extract outcomes for the outcomes list and draft the list of outcomes for the online Delphi. SD, RAS, JJK and DP developed the Delphi protocol. SD will complete structured interviews with children. SD wrote the first draft the manuscript. All authors edited the manuscript and read and approved the final version.
Ethics approval and consent to participate
Consultation with the HRA deemed this study a service evaluation project with no requirement for ethical approval (reference 60/89/81). Informed consent will be assumed if participants agree to fill in the surveys.
Consent for publication
Not applicable
Competing interests
The authors declare that they have no competing interests.
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Contributor Information
Sara L. Dorman, Email: sara.dorman@nhs.net
James A. Shelton, Email: james.shelton@nhs.net
Robert Allen Stevenson, Email: rastevenson@doctors.org.uk.
Kenneth Linkman, Email: k.linkman@liverpool.ac.uk.
Jamie Kirkham, Email: J.J.Kirkham@liverpool.ac.uk.
Daniel C. Perry, Email: D.C.Perry@liverpool.ac.uk
References
- 1.Beck JJ, Bowen RE, Silvia M. What’s new in pediatric medial epicondyle fractures? J Pediatr Orthop. 2016. doi:10.1097/BPO.0000000000000902. [DOI] [PubMed]
- 2.Tarollo L, Mugnai R, Fiacchi F, Adani R, Zambianchi F, Catani F. Pediatric medial epicondyle fractures with intra-articular elbow incarceration. J Orthop Traumatol. 2015;16:117–123. doi: 10.1007/s10195-014-0310-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Vallila N, Sommarhem A, Paavola M, Nietosvaara Y. Pediatric distal humeral fractures and complications of treatment in Finland: a review of compensation claims from 1990 through 2010. J Bone Joint Surg Am. 2015;97(6):494–499. doi: 10.2106/JBJS.N.00758. [DOI] [PubMed] [Google Scholar]
- 4.National Institute for Health Research Health Technology Assessment Programme. NIHR HTA Commissioning Board Minutes of the 67th Meeting. 2017. https://www.nihr.ac.uk/about-us/documents/board-and-panel-documents/hta/HTA-Minutes-September-2017.pdf. Accessed 30 Jan 2018.
- 5.Williamson PR, Altman DG, Blazeby JM, Clarke M, Devane D, Gargon E, et al. Developing core outcome sets for clinical trials: issues to consider. Trials. 2012;13:132. doi: 10.1186/1745-6215-13-132. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Sinha IP, Smyth RL, Williamson PR. Using the Delphi technique to determine which outcomes to measure in clinical trials: recommendations for the future based on a systematic review of existing studies. PLoS Med. 2011;8(1):e1000393. doi: 10.1371/journal.pmed.1000393. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Sinha I, Jones L, Smyth RL, Williamson PR. A systematic review of studies that aim to determine which outcomes to measure in clinical trials in children. PLoS Med. 2008;5(4):e96. doi: 10.1371/journal.pmed.0050096. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Clarke M. Standardising outcomes for clinical trials and systematic reviews. Trials. 2007;8:39. doi: 10.1186/1745-6215-8-39. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Chan AW, Altman DG. Identifying outcome reporting bias in randomised trials on PubMed: review of publications and survey of authors. BMJ. 2005;330:753. doi: 10.1136/bmj.38356.424606.8F. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Tugwell P, Boers M, Brooks P, Simon L, Strand V, Idzerda L. OMERACT: an international initiative to improve outcome measurement in rheumatology. Trials. 2007;8:38. doi: 10.1186/1745-6215-8-38. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Lux AL, Osborne JP. A proposal for case definitions and outcome measures in studies of infantile spasms and West syndrome: consensus statement of the West Delphi group. Epilepsia. 2004;45:1416–1428. doi: 10.1111/j.0013-9580.2004.02404.x. [DOI] [PubMed] [Google Scholar]
- 12.Ruperto N, Ravelli A, Murray KJ, Lovell DJ, Andersson-Gare B, Feldman BM, et al. Preliminary core sets of measures for disease activity and damage assessment in juvenile systemic lupus erythematosus and juvenile dermatomyositis. Rheumatology. 2003;42:1452–1459. doi: 10.1093/rheumatology/keg403. [DOI] [PubMed] [Google Scholar]
- 13.Harman NL, Bruce IA, Callery P, Tierney S, Sharif MO, O’Brien K, et al. MOMENT— Management of Otitis Media with Effusion in Cleft Palate: protocol for a systematic review of the literature and identification of a core outcome set using a Delphi survey. Trials. 2013;14:70. doi: 10.1186/1745-6215-14-70. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.AMI W, Smith CT, Young B, Jones TM. The CONSENSUS study: protocol for a mixed methods study to establish which outcomes should be included in a core outcome set for oropharyngeal cancer. Trials. 2014;15:168. doi: 10.1186/1745-6215-15-168. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Core Outcome Measures in Effectiveness Trials (COMET Initiative). 2017. http://www.comet-initiative.org/about/background. Accessed 21 Apr 2017.
- 16.Haywood KL, Griffin XL, Achten J, Costa ML. Developing a core outcome set for hip fracture trials. Bone Joint J Aug. 2014;96-B(8):1016–1023. doi: 10.1302/0301-620X.96B8.33766. [DOI] [PubMed] [Google Scholar]
- 17.Chiarotto A, Deyo RA, Terwee CB, Boers MA, Buchbinder R, Corbin TP, et al. Core outcome set for clinical trials on non-specific low back pain. Eur Spine J. 2015;24(6):1127–1142. doi: 10.1007/s00586-015-3892-3. [DOI] [PubMed] [Google Scholar]
- 18.Singh JA, Dohm M, Choong PF. Consensus on draft OMERACT core domains for clinical trials of Total Joint Replacement outcome by orthopaedic surgeons: a report from the International Consensus on Outcome Measures in TJR Trials (I-COMiTT) group. BMC Musculoskelet Disord. 2017;18:1. doi: 10.1186/s12891-016-1361-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Ouzzani M, Hammady H, Fedorowicz Z, Elmagarmid A. Rayyan—a web and mobile app for systematic reviews. Syst Rev. 2016;5:210. doi: 10.1186/s13643-016-0384-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Preferred Reporting Items for Systematic Reviews and Meta-Analyses. PRISMA checklist. 2009. http://www.prisma-statement.org/PRISMAStatement/Checklist.aspx. Accessed 19 Apr 2017.
- 21.Dodd S. Classification of COS outcomes. 2017. [Google Scholar]
- 22.Harman NL, Bruce IA, Kirkham JJ, Tierney S, Callery P, O’Brien K, et al. The importance of integration of stakeholder views in core outcome set development: otitis media with effusion in children with cleft palate. PLoS One. 2015;10(6):e0129514. doi: 10.1371/journal.pone.0129514. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Data Availability Statement
Data sharing is not applicable to this article as no datasets were generated or analysed during the current study.