Developing an index for the orthodontic treatment need in paediatric patients with obstructive sleep apnoea: a protocol for a novel communication tool between physicians and orthodontists

Abstract

Introduction

Sleep disordered breathing in the paediatric population can manifest as an array of different systemic symptoms; among them is a distinct malocclusion and craniofacial phenotype. Emerging research suggests that the treatment of this malocclusion and/or craniofacial phenotype through orthodontic intervention may help with the symptoms of these patients. Selecting the patients who would benefit from orthodontic treatment can be a difficult task for the physician with minimal dental training. Therefore the aim of this study is to develop a simple index to be used by medical professionals to identify those paediatric patients with orthodontic treatment needs who may benefit their obstructive sleep apnoea (OSA) symptoms.

Methods and analysis

The methodology in this project has been devised through the WHO's recommendations on developing an index, with modifications based on the specific needs of this study. Based on the available literature, a draft index will be produced and subjected to multiple iterative revisions based on the feedback from: the Index Development Group, a group of multidisciplinary and internationally acclaimed experts in the field; the External Review Group, a group of potential end users and interested parties and the Steering Committee. Once the index has been formalised, it will be subjected to a pair of reliability tests using physicians and orthodontists scored 2 weeks apart. Subsequently, the index will be validated using dichotomous responses from orthodontists on whether they would treat a patient for OSA symptoms, and comparing the responses to the score of the index on the same patient.

Ethics and dissemination

The index will be translated into French and will be presented in orthodontic and medical conferences, workshops, seminars, round table discussions, and free copies for download will be made available on the website of the University of Alberta Interdisciplinary Airway Research Clinic (iarc.ualberta.ca). Furthermore, the index will be published in a peer-reviewed medical journal to further increase the exposure of the index.

Strengths and limitations of this study.

• This index will help physicians and other medical professionals understand and identify which patients with obstructive sleep apnoea with malocclusions and craniofacial phenotypes are likely to benefit from targeted orthodontic treatment and will allow them to refer these patients accordingly.

• This index development acknowledges the multifactorial nature of sleep disorder breathing (SDB) and the need for multidisciplinary care. The ideal end result of this index is to facilitate and enhance effective collaboration between invested dental and medical specialties.

• Development of a validated index will facilitate future epidemiology studies, allow for quality assurance and guide funding allocation. It will also allow long-term follow-up and audit in order to enter into comparisons with other centres.

Introduction

Snoring, although ubiquitous in the adult population, is considered abnormal in children and adolescents.¹ More importantly, it may serve as an indicator of a more severe respiratory problem that presents as a continuum, from primary snoring to obstructive sleep apnoea (OSA). Reports vary on the prevalence of OSA, ranging from 0.7% to 5% of the population under 18 years old.^2–8 Moreover, breathing-induced sleep disorders have been proven to have a profound effect on the child's behaviour, growth and development; the myriad of symptoms include: morning tension-type headaches, excessive morning thirst, excessive fatigue and sleepiness, abnormal shyness, withdrawn and depressive presentation, pattern of attention-deficit/hyperactivity disorder, memory impairments, aggressiveness and irritability, among many others.^9–13 Other physiological processes that can be affected include stunted growth,¹⁴ high blood pressure,¹⁵ damage to the heart: ventricular hypertrophy,¹⁶ and cor pulmonale.¹⁷ In addition to behavioural and systemic health consequences, craniofacial development is also affected. These patients generally have a craniofacial component contributing to their OSA, which would manifest as a retrognathic maxilla¹⁸ or mandible, a long lower face height and restriction in the space of the upper airway.^{19 20} Furthermore, when evaluating the polysomnogrophy of these patients, the evidence suggests that palatal expansion and mandibular advancement appliances^21–26 can be of benefit at reducing the severity of OSA. Reverse pull headgear,^27–32 and maxillary and mandibular advancement surgery³³ have also been shown to have great promise at helping this group of patients. Since orthodontic treatment of the OSA craniofacial phenotype is an integral component to multidisciplinary care, it is essential for medical professionals (physicians, nurses, etc) to recognise the phenotype that would benefit from orthodontic treatment. Unfortunately, there are no guidelines for non-dental trained practitioners to help identify which children with sleep disorder breathing (SDB) would benefit from orthodontic treatment. Therefore, this study aims to develop an index that can summarise the need for orthodontic treatment, in select cases of children with OSA, to physicians and adjunct medical professionals. Once the index is developed, it will be assessed for reliability and will be validated. On completion, this index will equip medical professionals with a simple way to assess which patients have malocclusions that contribute to their OSA and may benefit from orthodontic treatment.

Methods and analysis

This research is classified as an applied interdisciplinary medical research. The overall study design involves components of qualitative analysis and quantitative analysis. The qualitative components comprise focus groups and committee meetings. The quantitative components comprise reliability tests and a cross-sectional validity test.

Initial development

In accordance with the WHO's recommendations on developing an index,³⁴ development of the index will be achieved through the following objectives:

1. Establishing a Steering Committee;

2. Scoping the index;

3. Reviewing the literature;

4. Drafting the index;

5. Organising an Index Development Group (IDG);

6. Organising an External Review Group (ERG).

Steering Committee

The Steering Committee will be established a priori, and will be identified by the principal investigator based on the required expertise in the involved fields and based on availability and access. The Steering Committee will be responsible for overseeing every aspect of the study. It will be composed of a representative group of three experts in orthodontics, paediatric sleep medicine and a methodologist specialising in psychometric property analysis. Their responsibilities will include: scoping the index, overseeing evidence retrieval, drafting the index, selecting members of the IDG and ERG, and finalising the index. In these meetings, unanimous consensus through discussion must be reached on all decisions before proceeding to the next step.

Scoping the index

Scoping is the process of defining what factors will be investigated in the literature for inclusion in the index. Scoping will be achieved through the combined experience and expertise of the Steering Committee, and each factor that is suggested will be further investigated in the literature to establish an evidence-based approach to the development of the index.

Reviewing the literature

For each of the scoped factors, the literature will be reviewed to establish relevance. Specifically, the evidence must demonstrate that appropriate treatment of the craniofacial factor in question will lead to an improvement in the OSA symptoms. The literature search will not be a systematic review, and will include results from PubMed, MEDLINE, EMBASE, Web of Science, Scopus, the Cochrane Library and grey literature. The search strategy, including inclusion criteria, exclusion criteria and key words, will be devised through the advice of the Steering Committee. The searches themselves will be done in duplicate, using two different assessors. Of the relevant articles, hand searches will be performed on the bibliography lists. The results will be discussed and consensus will be reached between both assessors for the results of the search. Priority will be given to the results of well-conducted and well-reported systematic reviews and randomised control trials. Each of the factors will be assessed on its effect on paediatric OSA. Furthermore, the literature will be searched for the craniofacial morphology of paediatric patients with OSA. All of this information will be brought back to the Steering Committee for discussion of inclusion/elimination of factors in the index.

Drafting the index

Since this index will have visual-rating scales for each of the craniofacial and occlusal factors, the Steering Committee will devise an outline suitable for displaying each of the factors representing the index. The factors in the drafted index will also have a number of levels that will divide the factor into categories of severity. The number of levels for a given factor will also be determined by the Steering Committee. Details describing the nature of the illustrations, the amount of levels that each of the factors will have and the general layout of the index will be agreed on by the Steering Committee, and the graphic artist will then design a preliminary version of the index based on the relevant factors and the feedback from the Steering Committee.

The index development meeting

The IDG is formed by a group of external multidisciplinary experts who provide evidence-based recommendations on the content, layout and development process of the index. This eclectic group will be chosen by the Steering Committee based on experience in the field, expertise and ability to contribute, availability and access. The group should be small enough to be able to have effective discussions, yet large enough to ensure appropriate representation from all stakeholders. In this study, we plan on including a broad spectrum of medical specialists, family physicians, orthodontists and methodologists. The goals of this meeting are to gain additional information to strengthen the index, as well as to gain the professional credibility of a broad range of experts on the topic of paediatric OSA.

Once members have been chosen by the Steering Committee and have accepted the invitation to participate, the IDG members will be emailed a document 4 days prior to the meeting, explaining the purpose of the index and meeting, how the meeting will proceed, what to expect in the meeting and a brief literature review of the orthodontic techniques currently available to help with the symptoms of OSA. Moreover, every member will be randomly assigned a number in order to maintain anonymity of the responses.

Procedure of the IDG meeting

The meeting will start with a brief introduction summarising the literature review, the purpose of the index and its relevance. The meeting will then proceed to collect feedback (see online supplementary appendix 1) on the aforementioned chosen factors using a modified Delphi technique,^{35 36} which is a communication technique that structures the meeting and minimises bias in responses. Participants’ responses will be collected through a web-based response portal. Each factor will be explored through yes or no questions, and yes responses followed up with a scale of 1–9 based on its importance for decision-making. Once all the feedback is received on a particular factor, the summary of the results will be displayed for everyone to see. Discussion will ensue for a maximum of 5 min, with each person talking no more than 1 min. Then everyone will be asked to re-enter their feedback on the website for the same factor in light of the discussion. This cycle will continue until a consensus of greater than 80% of the members is reached. Once consensus is reached we will move on to the next factor and the participants will be asked to give feedback on this new factor in a similar manner.

In addition, two negative control factors will be used to calibrate the responses. The first negative control will be of orthodontic relevance but will have no effect on the amelioration of the symptoms of OSA (ie, crowding or impacted canine). The second negative control will be of relevance to OSA symptoms, but cannot be changed by orthodontic treatment (ie, body mass index or neck circumference).

Once all the factors have been discussed and consensus reached, we will get feedback through the website on the: alignment—importance of the index and whether the development process is appropriate; relevance—content analysis and whether all the factors identified are important; representation—if there is anything that needs to be added to the index (see online supplementary appendix 2). Finally, additional written feedback will be accepted at the very end of the session.

All the feedback from the IDM will be summarised and presented to the Steering Committee, and decisions will be made to remove factors, modify factors and/or modify the outline and layout of the index. These modifications will subsequently be presented to the graphic artist and a second draft will be procured.

The ERG meeting

The ERG is composed of end users and interested parties. The group will also be identified based on the advice of the Steering Committee based on the ability of the group to assess the index and contribute to its development, availability and access. This group is not responsible for any content analysis, instead it will be responsible for reviewing the layout, simplicity and ease of use. It will also be responsible for assessing the usefulness of the index in the healthcare setting and to give feedback on the feasibility of implementing the index in practice. The goal of this meeting is to gain end user approval for the ease and feasibility of administering the index. This group should be large enough to be a representative sample of the population, yet small enough to allow for ease of explanation and healthy discussion. It is not as structured as the IDM, and allows for the participants to freely express their opinions in an open forum.

In this meeting, we will explain the theory behind the index by briefly reviewing the literature and then explain the purpose of the index. Subsequently we will show the group a pilot version of the index and a paper will be distributed to receive their feedback based on the following questions:

1. Do you understand the purpose of the index?

2. Do you understand what each factor is assessing?

   • A. If not, which one(s) do you not understand and why?

3. On a scale of 1–10, how simple would you rate this index to understand and use?

4. Would you use this index in your practice?

5. Other recommendations.

By compiling the comments and scores onto a table, the results will display the feedback from the ERG meeting. All these results will be presented to the Steering Committee, and decisions will be made to modify the outline and layout of the index. These modifications will subsequently be presented to the graphic artist and a third draft will be procured.

Reliability

Reliability of the index will be tested within a group that represents the typical end user population. This includes physicians of family medicine, paediatricians, paediatric ENT or paediatric sleep physicians. Reliability will also be assessed within a group of orthodontists who, by their training, are experts at assessing malocclusion and craniofacial morphology. Therefore 10–20 physicians and orthodontists will be recruited as examiners in this study.

A pool of 15–40 randomly selected patient charts from the University of Alberta Inter-disciplinary Airway Research Clinic (I-ARC) will be recruited as reliability test participants, and their intraoral and extraoral photographs will be used in the reliability assessment. After a brief explanation about the use and application of the index, the physicians as well as the orthodontists will apply the developed index to the sample patient pool's pictures once. In order to diminish recall bias, application of the index will be repeated 2 weeks later.

Intra-rater reliability and inter-rater reliability between the physicians, between the orthodontists, and between the physicians and orthodontists will be compared. The reliability will be assessed using interclass correlation coefficients and Bland-Altman plots.

Validation

The index will be validated using dichotomous responses from orthodontists on whether specific patients would require orthodontic treatment to help their OSA symptoms and comparing them to the score that the index gave those same patients. This will be achieved by setting up a website where 30 orthodontists with experience in dealing with paediatric OSA will be recruited to take the assessment. The website will contain extraoral and intraoral pictures of patients randomly selected from the Interdisciplinary Airway Research Clinic diagnosed with OSA, and the orthodontists will be asked to rate these patients, using a ‘yes’ or ‘no’ response, on whether they would benefit from orthodontic treatment for their OSA symptoms. The index will then be applied on the same patients by the principal investigator and the score of each patient will be recorded. Using a stepwise multiple logistic regression, each of the identified factors will be given a weight; this will represent the relative importance of the factor. Once analysed, if the correlation is high between the expert scores and the cluster groups then the clusters are meaningful and valid. Furthermore, a cut-off for most efficient score above which to refer will be chosen using a graph and observing the value that optimises the sensitivity, specificity and overall accuracy of the index. Finally, four grades of treatment need will be determined using the 25th centile ranges. The grades will be:

1. Minimal need;

2. Mild need;

3. Moderate need;

4. Severe need.

Discussion

The development plan of this index has been conceived through a modification of the WHO handbook for guideline development,³⁴ as well as reviewing the orthodontic literature for ways indices have been previously developed. The WHO provided an excellent starting point; from there modifications were made to cater to the specifics of this study, given that there are differences between developing a guideline and an index for orthodontic treatment need. The literature was useful, and among the index development protocols reviewed, some assessed the orthodontic treatment need within the entire population,^37–40 while others assessed it for a given subpopulation^41–46; each had strengths and weaknesses, and thus we further modified our methods, synthesising a protocol for our particular needs from the available literature and using the experience and expertise of the authors. Through this protocol we aim to develop an index that fulfils all of the following criteria⁴⁷:

1. Gradient of numeric values: the severity of the orthodontic treatment need within the paediatric patients with OSA should be defined within a numerical scheme that demonstrates a finite and progressive gradient from low need to high need.

2. Equal sensitivity: should demonstrate equal sensitivity throughout the scale.

3. Clinical importance: the numerical scale should correspond with the clinically appraised orthodontic treatment need of paediatric patients with OSA.

4. Statistical ease: should be amendable to statistical analysis.

5. Reliability: should have a high intra-rater and inter-rater reliability.

6. Practical: the instruments required to score the index should be practical to the setting in which it will be administered.

7. Minimal judgement: applying the index should require minimal judgement.

8. Simple: the index should not have a high financial or time cost, and thereby should be simple enough to administer to many patients.

9. Detect change: The index should be able to detect changes in orthodontic treatment need in paediatric patients with OSA.

10. Validity: should be valid over time.

Validity can be characterised into different types: face validity, content validity, construct validity and criterion validity.⁴⁸ In this study, we will examine these kinds of validity at different stages of development. The first draft of the index will focus on establishing face validity. Feedback from the Steering Committee and IDG will assist in establishing content and construct validity. Assuming that the ‘gold standard’ in assessing the orthodontic treatment need in paediatric patients with OSA is an orthodontist with experience in dealing with paediatric patients with OSA, then the subsequent modification of the index based on the reliability tests and the dichotomous responses from the orthodontists provides the index with the necessary criterion-related validity evidence through statistical means.⁴⁹

Significance

This index will help physicians and other medical professionals identify which craniofacial phenotypes may benefit from orthodontic treatment as part of their multidisciplinary OSA management. Furthermore, due to the diverse medical effects of sleep deprivation, there will be a trend to make sleep apnoea into a centralised service, where the main focus is for a highly trained multidisciplinary team to treat a high volume of patients to a standardised protocol, where meticulous documentation is exercised. This index is part of that documentation process. It will allow for quality assurance, funding allocation and epidemiological studies to be performed. It will also allow long-term follow-up and audit in order to enter into comparisons with other centres.

Dissemination plan

The dissemination of this index will be performed through a variety of ways in order to maximise its reach. Primarily, it will be published in a peer-reviewed journal, which will allow its introduction to the scientific literature. The journal should be a respected medical journal with broad reach, in order to allow the greatest number of physicians to be exposed to the index. Subsequently, it will be translated into French, in order for it to be accessible to the entire Canadian and American population of medical professionals. Moreover, the index will be presented at national and international conferences to increase the awareness of the index among the scientific community. Finally, the index will be used at the University of Alberta's Interdisciplinary Airway Research Center, and more research will be conducted, so that future research in this centre will incorporate it. It will also be placed on the University of Alberta's Interdisciplinary Airway Research Center's website under the physician section, to further educate doctors who visit the site on the index.