Apriori feasibility testing of randomized clinical trial design in patients with cleft deformities and Class III malocclusion

Abstract

Objectives

To assess the feasibility of randomizing treatment (surgical vs. non-surgical) for correction of a Class III malocclusion (underbite) resulting from an earlier repair of cleft lip and palate.

Materials and methods

Surveys about willingness to accept randomized treatment during adolescence were mailed to the parents of cleft lip and palate patients under the care of Children's Hospital Los Angeles between 2005 and 2010. The inclusion criteria were patients with cleft lip and palate, Class III malocclusion due to maxillary deficiency, and absence of medical and cognitive contraindications to treatment.

Results

Out of 287 surveys, 82 (28%) were completed and returned; 47% of the subjects held a strong treatment preference (95% CI, 35–58%), while 30% were willing to accept randomization (95% CI, 20– 41%). Seventy-eight percent would drop out of a randomized trial if dissatisfied with the assigned treatment (95% CI, 67–86%). The three most commonly cited reasons for being unwilling to accept random treatment assignment were 1) the desire for doctors to choose the best treatment, 2) the desire for parents to have input on treatment, and 3) the desire to correct the underbite as early as possible.

Conclusion

Based on this study, parents and patients would be unwilling to accept a randomly assigned treatment and would not remain in an assigned group if treatment did not meet expectations. This highlight the limitations associated with randomization trials involving surgical modalities and provide justification for other research models (e.g., cohort studies) to compare two treatment options when randomization is not feasible.

1. Introduction

Cleft lip and palate is the most common facial birth defect occurring in the United States in 1/700 live births [1]. The cleft deformity is disfiguring and restoring facial functions that control feeding, breathing, speech and hearing as well as facial esthetics is challenging and expensive [2]. Each surgery, while correcting one part of the divided anatomy, also introduces a surgical scar that can resist the further growth of the maxilla across the scar [3]. The combined effects of early repair of the cleft lip, cleft palate, alveolar cleft, palatal fistulas, and narrowing of the velopharyngeal port for improved speech, result in a 3 dimensional growth deficiency of maxilla. The overall effect can be a smaller maxilla functioning against a normal-size mandible and an underbite or Class III malocclusion [3–7]. The standard protocol is to correct the transverse width before the alveolar bone graft and then to wait for the pubertal growth to complete, before correcting the short maxilla with surgery. Orthognathic surgery is often used for lengthening the maxilla and restoring the occlusion from a Class III to a normal Class I occlusion. There are advantages to orthognathic surgery such as the ability to control midlines and to move the maxilla in three dimensions to optimize the bite and facial esthetics. The maxilla can also be surgically sectioned to reposition the segments into better occlusion. The disadvantages are the morbidity and the instability of cleft surgery when advanced against scar tissue [4]. Of the many surgeries a patient with a cleft lip and palate undergoes, orthognathic surgery may be the most difficult for the patient. For patients with cleft lip and palate, there is a long recovery period of 6–10 weeks, a liquid diet, a risk of pain, infection, swelling and transient and sometimes permanent sensory nerve paraesthesia [4–6].

Prior to starting a randomized clinical trial, i.e. surgical or non-surgical treatment for cleft patients, it may be prudent to test the feasibility of the research design before committing resources to a full-fledge study. Certain clinical questions can benefit from a priori testing of the research design, especially when the treatments are dissimilar in morbidity and are given at different ages. In the present study, the alternative treatment to orthognathic surgery for treating the Class III malocclusion is a non-surgical approach involving the loosening of the facial sutures with expanders and orthopedic protraction of the maxilla with reverse-pull facemasks after the maxilla is mobilized [3]. The loosening of the maxillary sutures requires alternating one week of expansion with a screw-driven expander with one week of constriction for eight weeks [8,9]. The back and forth motion of the expansion and constriction if done before the age of 14, will loosen the maxilla and allow larger corrections of the Class III malocclusion [3]. Typically, protraction works well in cases needing 5 mm of maxillary advancement, which is within the surgical range of correction for the LeFort I maxillary advancement surgery in patients with cleft lip and palate [3]. To protract the maxilla, the patient must use the facemask at night to pull the maxilla forward and down, and use continuous Class III elastics during the daytime to maintain the gains when the facemask is not worn. The protocol requires excellent cooperation from the patient to pull at night and hold during the daytime. All patients are told that protraction involves hard work and self-discipline for the technique to work.

When one treatment involves surgery, the need for informed consent from adolescent patients and their parents, could be a barrier to patients accepting random assignment of treatment. Further, when anon-surgical treatment is given at an age 3–4 years earlier than surgical treatment, the difference in age poses a challenge for maintaining treatment groups during the interim periods. The dissimilarity between treatments is a strong indicator to test whether patients would participate in a randomized clinical trial.

The randomized clinical trial relies on the ability of those conducting the trial to randomly assign treatments to subjects in the study population. If a large portion of this population is unwilling to accept a randomly assigned treatment then study enrolment will suffer as will the trial's ultimate power to detect the presence of a true difference between the effects of the treatments. Since acceptance of random treatment assignment and drop-out of participants have such serious and costly implications for a prospective randomized trial, the investigators of this study sought to research these issues in the process of planning the prospective trial. This study tests the hypothesis that patients will indicate a strong preference for one treatment over the other and would be unwilling to accept and stay in a randomly assigned treatment if they were unhappy with an assigned treatment that affects facial appearance. The strategy for the present study was to survey a portion of the study population with a brief questionnaire (Table 1). The questions asked whether or not the children in the study population and their guardians had a strong treatment preference, whether they would accept a random treatment assignment, and whether they would not change groups throughout the entirety of such a trial if so assigned. The a priori testing of the feasibility of a randomized trial was done to avoid low enrolment and high dropout rates, causing the trial to run longer than expected or end with insufficient numbers to determine treatment effects.

Table 1.

Survey questions on participation in randomized trial.

Based on the description above, please mark which treatment you would prefer	Definitely surgery	Maybe surgery	No preference	Maybe headgear	Definitely headgear
Despite your preference marked above, would you be willing to have your child randomly assigned to one of the two treatments described above?	Yes	Probably	Probably not	No
If you chose to participate in the study but ended up not being happy with the treatment your child was assigned to, how likely would you be to leave the study to change to the other treatment?	I would leave	I might leave	I might stay	I would stay

2. Materials and Methods

2.1. Data acquisition

The Children's Hospital Los Angeles (CHLA) IRB approved this survey. A questionnaire was developed asking parents to imagine that their child was in pre-teen years with an underbite resulting from cleft-lip and palate surgery (Table 1). Both the standard surgical procedure and the non-surgical protraction procedure [3] were explained to the parents and patient.

The questionnaire explained how treatment is randomly assigned. Parents were then asked three questions:

1. Would you accept a randomly assigned treatment?

2. What is your treatment preference?

3. If you were involved in a clinical trial and were unhappy with the treatment assigned to your child, would you drop out of the study and change treatment?

If participants were unwilling to accept randomization, then they were asked to choose from a list of predefined reasons for having this opinion as follows:

1. I want to be able to choose myself

2. I want the doctor to consider my child's case then recommend

3. I think that the headgear treatment is better

4. I think that the surgery is better

5. I want my child to avoid surgery if possible

6. I do not think my child would be willing to stick to the headgear treatment regimen

7. I want the earliest correction for my child

8. Other reasons,

The survey also had an option to write in the reason for choosing or discontinuing treatment. Potential participants were selected based on billing records from the Craniofacial and Dental Department at CHLA. To be eligible for the study participants must have had a child under the care of the craniofacial orthodontics department with a cleft palate and Class III malocclusions (underbite). Questionnaires were mailed to 287 patients and their parents on September 1, 2010, and were collected until December 31, 2010. Twelve surveys (4%) were returned to sender and 82 surveys were collected, including 8 that were completed in Spanish. Forty surveys (49%) were returned in the clinic and 42 (51%) were returned by mail. Patients who had appointments during the survey collection window were asked in person whether or not they had received the mailing and, if they had, whether they had understood and completed it. Mailed surveys included a return envelope coded with the subject's age group. Surveys returned in the clinic recorded the child's age, but no further identifying characteristics of the subjects or their children were recorded.

2.2. Statistical analysis

All statistical analysis was performed with SAS statistical software (version 9.2, SAS Institute). The simple percentages of respondents choosing each possible answer to each question were calculated. For determining strong treatment preference, willingness to randomize, and willingness to drop out, simple percentages and 95% confidence intervals (CIs) were calculated. Simple percentages of respondents who marked and ranked reasons for their choices were calculated. Chi-squared tests were used to test differences between answers to the questions according to age groups. Chi-squared tests were also used to test for differences between surveys returned in the clinic vs. surveys returned by mail on how each of the first three questions was answered.

3. Results

Fifteen respondents definitely preferred the surgical treatment to the protraction treatment and 14 said they might prefer surgery. Twenty-two subjects said they would definitely prefer the protraction treatment. While 17 said they might prefer it, from 39 subjects (48%) who preferred the protraction treatment. Eleven subjects (13%) had no treatment preference (Fig. 1a,b). For the question about treatment preference,2 surveys were left blank and 1 respondent marked two answers. Subjects were considered to have a strong treatment preference if they marked that they would definitely prefer the surgery or the protraction. Thirty-seven subjects had a strong treatment preference (46%, N = 79, 95% CI: 35–58%).

Fig. 1.

(a) Treatment preference and number of patients in each group (n = 82). (b) Distribution of treatment preferences according to age group (n = 82): Group 1, age <13 years pre-treatment; Group 2, age >13 years, but <18 years mid-treatment; Group 3, age >18 years after treatment.

Seven respondents (9%) replied that they would accept one of the two aforementioned treatments being randomly assigned to their child and 17 (21%) said they probably would do so (Fig. 2a,b). Forty-eight (59%) said that they would not accept randomization and 9 (11%) said they would probably not accept randomization. For this survey, one respondent did not answer this question. A subject was considered possibly willing to accept randomization if he or she marked ‘yes’ or ‘probably’ to the randomization question. There were at 24 such subjects (30%, N = 81, 95% CI: 20–41%).

Fig. 2.

(a) Acceptance of randomization and number of patients in each group. (b) Distribution according to age group: Group 1, age <13 years pre-treatment; Group 2, age >13 years, but <18 years mid-treatment; Group 3, age >18 years after treatment.

When asked whether they would drop out of the randomized trial if they were unhappy with the assigned treatment, 42 (51%) respondents stated that they would drop out and 20 (24%) said they would probably drop out (Fig. 3a,b). In the situation described, 13 respondents (16%) said they might stay and 4 (5%) said that they would not change. Two respondents left this question blank. A respondent was considered to be willing to drop out of a randomized trial if they responded that they would leave or that they might leave the third question of the survey. Overall, 62 subjects were willing to drop out of a randomized trial (78%, N = 80, 95% CI: 67–86%).

Fig. 3.

(a) Distribution of willingness to stay with the original treatment group for the duration of the trial. (b) Distribution of willingness to stay with the original treatment group for the duration of the trial according to age group: Group 1, age <13 years pre-treatment; Group 2, age >13 years, but <18 years mid-treatment; Group 3, age >18 years after treatment.

Nearly all subjects marked reasons for wanting to reject randomization for their child regardless of how they answered the question about accepting a random treatment assignment. Most respondents did not rank their reasons; rather, they marked all the choices they felt applied to their situation. The most common reason for rejecting randomization was that parents wanted the doctor to consider their child's case and recommend the treatment best for their child with 73 subjects (89%), indicating this as one of their reasons. The second most common reason was that parents wanted to have a say in which treatment their child received (55 respondents, 67%). The third most common reason was that parents wanted the earliest possible correction for their child's underbite (31 respondents, 38%). The other reasons for rejecting the randomization were as follows, I want my child to avoid surgery if at all possible (33%), I think that the headgear treatment is better (28%), I think that the surgery is better (27%), and I do not think my child would be willing to stick to the headgear treatment regimen (24%). Only 7 (9%) subjects felt the need to write in their own reason and these either mirrored one of the provided reasons or used the space as an opportunity to comment on the treatment their child had received at CHLA.

Subjects with children in their pre-teens were more neutral in preference, while those with children in their early teens preferred protraction and those with older teens preferred surgery (p = 0.05). There was also a significant difference in willingness to drop out by age group with the subjects with older children being slightly more willing to stay in a trial (p = 0.04). There was no difference by age group regarding willingness to accept randomization (p = 0.36).

Subjects who returned their surveys by mail were more likely to accept randomization than those who returned their surveys in the clinic (p < 0.0001). Those who mailed in their responses were also more likely to indicate willingness to stay in a randomized trial despite being unhappy with the assigned treatment (p < 0.0001). There was no significant difference in treatment preference between those surveys returned by mail and those returned in the clinic (p = 0.58).

4. Discussion

Not all clinical questions or comparisons lend itself to a randomized trial. In this paper, a randomized research design is tested apriori using a survey to determine whether this design would be possible in a comparison between a surgical and non-surgical approach for correcting a Class III malocclusion in teenagers with cleft lip and palate going through adolescence. Testing the research design allows the final trial to be constructed with advance knowledge about the test population. The survey tested whether patients had strong treatment preferences, would accept random assignment of treatment and stay with the treatment group over four years. It also surveyed for reasons why randomization would not be accepted in this sample population. When randomized trials are not feasible or cannot be completed, alternative research designs such as observational cohort studies may be more pragmatic for comparing clinical efficacy.

Patients' decision to have surgery is often weighed against a failure rate that ranges from 25 to 80% and would then require a second surgery [6,7]. Given the challenges associated with correcting a Class III malocclusion in late adolescence when patients were almost adults, there was much debate about whether adolescent patients would sign up in a randomized trial that compared treatments that would alter their facial appearance.

In designing a clinical trial, the need for randomization of treatment must be weighed against the differences between the two types of treatments. Adolescent patients and their parents may not accept random assignment of treatments if they know in advance that the cooperation needed for protraction is too much to expect for their child, or their child is afraid of facial surgery. Prior history and experiences with earlier surgeries may be a factor in deciding on additional surgeries. A problem with comparing surgical treatment at age 17 with protraction treatment at age 13 is the age difference between treatment groups. If the patients were randomized at age 12, the groups would have to maintain their group assignment during the intervening years.

In cleft palate research, randomized trials have been used to compare different primary repairs of the cleft defect in children under 5 years of age [10], and surgical treatments to improve velopharyngeal competence in children <3 years old [11]. During infancy, the patient is too young to make a choice so parents or guardians make the choice. However, the teenage patient awaiting orthognathic surgery or maxillary protraction is likely to be part of the decision making process.

Randomized trials have also been used to test the outcomes achieved with palatal block for pain management [12] and perioperative steroid therapy for reducing airway distress [13] in cleft patients. In infants, randomized trials have been used to compare feeding methods [14] and to compare orthopedic appliances for improving cephalometric measures [15]. In these examples, the treatment options were between two surgical treatments; there was no non-surgical treatment option. If a clinical trial tests a treatment that has high morbidity and is beyond the alternative for pain and discomfort, such as surgery, then randomly assigning a child to receive or not to receive the treatment is fraught with ethical ambiguity. These complications and risks must be discussed prior to treatment to obtain patient consent. If a patient (or guardian) has a very strong treatment preference, then it is unlikely that the randomly assigned treatment would be acceptable. Specifically, if a patient (or guardian) has an aversion to surgery, then the patient may not be willing to accept the random chance of undergoing surgery when a non-surgical option is available. This decision is further complicated as the early treatment, maxillary protraction, does not prevent the patient from having maxillary surgery later at 17, which gives the patient the option of testing the non-surgical option before the surgical option. A decision to accept the possibility of surgical treatment may be hard for young patients to make without the input of their parents or doctors; thus making the randomization difficult to implement if informed consent is obtained.

In the present survey, most patients had strong treatment preferences, were resistant to accepting a randomly assigned treatment, and were willing to drop out of a trial due to dissatisfaction with treatment. Unlike the patients in most randomized cleft trials, the patients in this population were older and had more opportunity to develop their own preferences. There may be many social obstacles to randomized care. As both treatments have the potential to significantly alter the patient's facial aesthetic [16], the choice of treatment usually involves prior discussion. Teenagers often ask questions about orthognathic surgery and maxillary protraction, the potential benefits and co-morbidities. Parents ask about the costs of treatment, frequency of visits and level of cooperation needed. The discussion between the clinician and patient, to obtain the informed consent, may lead to strong treatment preferences and cause patients to refuse a randomly assigned treatment. Additionally, these patients have a history of being guided by the professionals. If the surgeon or orthodontist has followed the patient from birth, the parents may believe that the doctor knows the risks and benefits and should make the decisions based on the golden rule principle. This dependence on their doctor was the main reason why the patients did not choose to participate in a randomized clinical trial. The willingness to drop out of a trial, if patients were unhappy with their assigned treatment, may be another consequence of a trial and error approach when attempting different therapies to solve challenging clinical problems. The initial choice of treatment may not yield the desired results, especially as it affects facial esthetics.

Attempting a randomized trial that draws from a study population that is unwilling to accept a randomly assigned treatment could lead to a much lower enrollment rate than would otherwise be expected, as well as a bias in the socio-demographics of the participants vs. the non-participants. The trial, consequently, could take much longer than planned due to the extended accrual time. This is a concern when comparing protraction to surgery due to the time between treatments. With the long window of time from randomization (around age 12) to completion of the study (around age 18 for those receiving surgery), patient dropout could accumulate, resulting in fewer patients in the trial.

The randomized clinical trial has been re-examined especially when costly trials did not have sufficient power or could not be completed. If a prospective randomized trial is not being feasible for the intended study, then alternative, non-randomized trial designs should be explored [17]. Removing the element of random treatment assignment would eliminate one reason for patients to refuse to enroll and could reduce the risk of patients dropping out due to dissatisfaction with treatment. One alternative is the prospective cohort study with defined covariates that are adjusted using a repeat measure ANOVA model. A priori covariates for this study include site, age, single parent status, cooperation and compliance of patients, severity at baseline (mild, moderate, and severe), and home distance to the clinic. Propensity scores can also be used in the non-randomized cohort trial to adjust for baseline differences in severity in a manner modelled after Connors et al. [18]. In the hierarchy of comparative trial evidence, cohort studies are often placed lower than the double blind trial and randomized trials [19]. GRADE (Grades of Recommendation, Assessment, Development and Evaluation) considers how well a study is done and suggests that randomized trials that are flawed in their execution should not be at the top of the pyramid in any hierarchy of evidence [20,21]. Well-constructed and executed cohort studies sometimes can provide better evidence than a randomized trial, if the randomized trial is poorly done. In two randomized trials on early vs. late orthodontic treatment of Class II malocclusions [22,23], the two randomized trials confirmed observations recorded in a cohort study [24].

According to a National Academy of Sciences Roundtable on redesigning the clinical effectiveness research paradigm, “Observational studies offer an alternative when trials are impractical or infeasible and also help to accelerate translation of evidence into practice and risk management and minimization efforts” [25]. However, common problems include elementary design errors, failure to identify a clinically meaningful time point or start of follow-up, exposure and disease misclassification, the use of overly broad endpoints for safety studies, confounding by patient behaviour effects, and marginal sample size [26]. Alternatively, observational trials can start patient registries that can be followed over periods and be part of a comprehensive pool of shared research data. Cohort trials also have pragmatic value by providing information on how effective the treatments are in real world practice settings [27].

This study indicates that feedback from patients can provide information to investigators regarding logistical barriers to conducting a study in the manner initially planned. A questionnaire is a low-cost and effective measure that provides information on the feasibility of conducting a randomized trial. This data may save significant time, inefficiency, and costs if it detects flaws in the research design at an early stage; thus, avoiding the possibility of conducting a trial that ultimately would have proved unable to detect any differences between treatments.