A randomised controlled trial on the effectiveness of bracing patients with idiopathic scoliosis: failure to include patients and lessons to be learnt

Abstract

Trials often do not succeed in including as many patients as anticipated beforehand. The aim of this paper was to describe why we were not able to include more than a few patients in our randomized controlled treatment trial on the effectiveness of bracing patients with idiopathic scoliosis, and to describe which lessons can be learnt. A pilot study on the willingness to participate in such a trial was conducted amongst 21 patients and their parents. A description of how we prepared and designed this trial, the problems we faced and how we tried to improve the inclusion are given. A total of four patients were included, and 14 refused to participate in an 18-month period. There were a lot less eligible patients than anticipated (40 instead of 100 per year), and the patients’ participation rate was much lower than we had found in our pilot study (21% instead of 70%). The trial failed to include more than a few patients because of an overestimation of the number of eligible patients and because a lot less eligible patients were willing to participate compared to our pilot study. One reason for a low participation rate could be that this trial evaluated a frequently used existing treatment instead of a new treatment, and patients and parents might be afraid of not being treated (despite an intensive secure system for the control arm).

Introduction

Idiopathic scoliosis (IS) is a lateral curvature with concomitant rotation of the spine of unknown origin with a minimal Cobb angle of 10°. Progression of scoliosis usually occurs just before or during puberty. Early treatment by bracing is thought to prevent further progression of the curvature and thereby to prevent the need for surgery [6, 7]. The effectiveness of bracing, however, has not been sufficiently established due to a lack of randomized controlled trials (RCT) [15, 24]. The studies on bracing that have been done were mostly retrospective studies, or studies without a control group. Only one prospective study was performed [18]. The authors concluded that bracing was effective. However, this study was non-randomized, non-blinded and with baseline differences between the groups [15, 24]. Several authors have stated that a RCT, in which the control group (initially) is not offered brace treatment, should be conducted [8, 9, 14]. Therefore, we designed a multicentre RCT on the effectiveness of bracing patients with IS in reducing further progression of the scoliosis. Exact data on incidence numbers are (inter)nationally lacking, but based on a questionnaire sent out to Dutch orthopaedic surgeons, and reported estimated international incidence rates if a screening programme would exist, an estimated few hundred patients need brace treatment each year in the Netherlands [5, 22]. These estimations range from at least 200 (based on data from 25 Dutch hospitals, excluding the data of at least three large scoliosis clinics) to 600 (based on estimations in the literature if a screening programme were performed).

To successfully implement a clinical guideline, or in this case a trial protocol, a number of factors should be considered during the developmental process [25]. To start with, the topic must be relevant for the clinicians who have to work with it. A balanced working group should be formed that describes the protocol, which should involve clinical experts. To promote support, the draft has to be presented to the users, so they can comment on it and give suggestions. Furthermore, there should be attention to the impact on resources, materials and facilities, and the protocol should be presented in an attractive design [25].

Of course, taking such factors into consideration does not guarantee that the implementation will succeed. Other factors like barriers at the level of the patient, the individual professional, or the wider environment can complicate the success of a trial [10]. Unfortunately, the above-mentioned brace treatment trial had to be halted, because we were hardly able to include patients. The aim of this report was to describe how we prepared this trial, why we were not able to include the anticipated patients we had counted on beforehand in the trial, and to describe which lessons can be learnt.

Methods

In 2000, we held a national meeting on the need for research on screening and bracing for idiopathic scoliosis. This meeting was attended by orthopaedic surgeons, school doctors, school nurses, a representative of the Dutch Scoliosis Foundation and researchers. It was agreed that a RCT on bracing for scoliosis and a case–control study on the effectiveness of screening for scoliosis were needed. A first step on designing the protocol for the RCT on bracing was then made, in collaboration with members from the Dutch Spine Society. The case–control study was designed and performed between 2002 and 2006 [3, 4].

Since we were not sure whether patients with scoliosis would be willing to participate in such a trial, in 2002, we performed a pilot study to evaluate whether patients and their parents would be willing to participate in such a trial. These were IS patients who did not need brace treatment at that moment, but could need a brace in the near future. This was the most realistically possible method to estimate the percentage of patients and parents that would be willing to participate. In this pilot study (n = 21), we found that 87% of the patients and 70% of the parents were willing to cooperate [1].

In 2004 and 2005, the study protocol for the treatment trial was further designed by the authors and several orthopaedic surgeons, and government funding was obtained in 2005. The semi-final protocol and logistics of the trial were discussed in the first telephone meeting with the participating orthopaedic surgeons in September 2005. Orthopaedic surgeons of 11 Dutch hospitals (3 university and 8 non-university) agreed to cooperate in this trial and after some adjustments were made, they approved the protocol. Furthermore, orthopaedic surgeons made beforehand an estimation on the number of eligible patients (i.e., patients that meet the inclusion criteria) during 1 year in their practice, based on experiences the last couple of years. Together, this would result in about 100 eligible patients per year.

The design of the trial is extensively described in a protocol paper [2]. In short, the main aim of the trial was to establish whether bracing patients with IS in an early stage will result in at least 5° less mean progression of the curvature compared to a control group in 2 years of follow-up. We aimed at including 100 patients with IS, 50 of which would be randomized to the intervention arm and 50 to the control arm. With about 100 eligible patients per year and a participation rate of 70%, this would take about one and a half years. The intervention arm would be treated with a Boston brace for 18–23 h/day. The control arm would initially not be braced. Eligible patients were girls and boys in the age group of 8–15 years whose diagnosis of IS has been established by an orthopaedic surgeon, who had not yet been treated by bracing or surgery, and for whom further growth of physical height was still expected based on maturation characteristics (Risser ≤ 2). The Cobb angle of the eligible patient should have either been minimally 22° and maximally 29°, with established progression of more than 5°, or should have been minimally 30° and maximally 35°; established progression for the latter was not necessary. For every 4 months, all patients would have had a physical examination and an X-ray of the spine. In case the curvature of a patient in the control arm would have progressed with 10° or more compared to inclusion, it could be decided to start brace treatment.

The primary outcome was the Cobb angle 2 years after inclusion. The secondary outcomes were health-related quality of life and costs.

The Medical Ethics Committee of the coordinating centre approved the trial in December 2005, after which all local Medical Ethics Committees approved the trial (between February 2006 and June 2007). All Medical Ethics Committees approved the randomization process. Furthermore, the Medical Ethics Committees approved that the control group would not be offered brace treatment, and that in case the curvature of a patient in the control arm would have progressed with 10° or more compared to inclusion, it could be decided to start brace treatment.

Results

We were able to include four patients in about one and a half years. We faced (mostly administrative) delays in obtaining approval from local Medical Ethics Committees, and more importantly in including patients. Figure 1 gives an overview of these results during the 2 years of the study. An outline of the number of eligible participants, trial inclusion and refusal rates for the year 2006 is given in Fig. 2.

Fig. 1.

Progress of approval from local Medical Ethics Committees, the inclusion of patients and the refusal of patients to participate. The patient that withdrew from the trial is included in the line “refusal of patients to participate”

Fig. 2.

Number of eligible participants, trial inclusion and refusal rates in the scoliosis treatment trial for the year 2006

Delay in getting the trial started

By the end of December 2005, so before the trial started in January 2006, we had obtained approval from the Medical Ethics Committee of the coordinating centre (Erasmus MC, University Medical Center Rotterdam). After that, we had to obtain approval from all other centres. After 6 months, we had obtained approval to start the trial in seven of the ten participating hospitals, and after one and a half years, we obtained approval from all participating hospitals. The delay in obtaining approval was mostly attributable to logistics in the paperwork, and not because the local Committee disapproved the trial. We needed a multicentre trial to be able to recruit enough patients, and beforehand it was anticipated that the process of approval by all Medical Ethics Committees would take less than 3 months.

Two major problems

Eligible patients

Originally, there were 11 hospitals cooperating in this trial. Unfortunately, one hospital had to withdraw from the trial a few months after receiving approval from their local Medical Ethics Committee, because that hospital stopped treating scoliosis patients.

After the trial had started in the first few hospitals, we noticed that the number of eligible patients (i.e., fulfilling the inclusion criteria) was strikingly lower than the number we had anticipated on beforehand. Therefore, a new estimation of eligible patients was made by reviewing the medical files of all scoliosis patients who visited the participating hospitals in 2006. Instead of 100 eligible patients, we found about 40 eligible patients during that year. A note was made in the medical files of patients who were not eligible for the trial yet, but who could be in the near future (e.g., the Cobb angle was too small to be included yet, but could progress to the inclusion criteria in the nearby future). An interesting finding was that in one hospital there were eight eligible patients in 2006, while there was not even one eligible patient in 2007.

By reviewing the medical files of all scoliosis patients who visited the participating hospitals in 2006, we noticed that in one hospital seven eligible patients (i.e., they met the inclusion criteria) had not been approached by their orthopaedic surgeon for participating in the trial in that year, and were therefore considered “missed”; these patients all started brace treatment. This happened partly due to a combination of busy out-patient consultation hours and the fact that only one in so many scoliosis visits is made by a patient that is eligible for the trial. A large part of the visits is made by patients who are already being treated for scoliosis, and, in some hospitals relatively more than in others, not all patients have idiopathic scoliosis, but some other type of scoliosis. Besides this, a part of the new patients who visit the orthopaedic surgeon for the first time is too young or too old to participate or their curvature does not fulfil the inclusion criteria. We therefore made a plastic card with the inclusion criteria and contact information of the coordinating researcher, which was supposed to help remind the orthopaedic surgeon whom to include. This card fitted in a breast pocket of a “doctor’s coat”.

Higher refusal rate than anticipated

In the trial, we found that only four in 19 patients (or in 2006, 3 in 16 patients), i.e., about 21%, of the patients were willing to participate (see Fig. 1). We briefly asked for reasons not to participate in the trial. All patients and parents who refused to participate stated that they wanted to start brace treatment immediately. Some of them indicated that they considered the risk to be too high, i.e., they felt that postponing treatment would have a negative impact.

It is good to note here that in the trial for each patient every 4 months, X-rays were taken to examine whether the curvature had progressed. The patients in the control group were therefore also closely watched, and had a “safety net”; in case their curvature would show more than 10° progression compared to inclusion, the patient, parents and her/his orthopaedic surgeon could decide to start brace treatment.

Actions to improve inclusion

Every 3–4 months, we had a telephone conference with the orthopaedic surgeons to discuss the progress of the trial, and to ask what help we could offer to increase the inclusion of patients. We tried to include extra hospitals in the study between April 2007 and September 2007. The already participating orthopaedic surgeons recommended four hospitals. These were approached, but only one could be reached and was willing to cooperate.

To decrease the refusal to participate, we wrote an article about the trial in the patients’ magazine of the Dutch Scoliosis Foundation (summer 2007) in which we explained the trial. Possibly, this would have been more effective if we had published that in an earlier stage of the trial, although new patients are probably not yet a member at the moment they are eligible for the trial, because, in most cases, that will be relatively shortly after diagnosis.

Discussion

The phenomenon that, during a trial, there appear to be less eligible patients than anticipated on beforehand is known as “Lasagna’s Law” [12]. Grol et al. [10], describes barriers to change in practice (i.e., implication of clinical guidelines). These barriers can arise at different levels, amongst others at the level of the patient, the individual professional, or the wider environment, and we feel these can also arise in trials. We will describe these factors below.

Lasagna’s law

Literature shows that a lot of trials do not succeed in including as many patients as expected beforehand [11, 17, 23]. A review of 114 trials in the UK showed that approximately one-third succeeded in including their targeted numbers. Half of the trials were awarded an extension. In 10% of the trials, enrolment was halted before the end of the recruitment period, because of poor recruitment. Reasons for slow enrolment were: less eligible patients than anticipated and a higher refusal rate. A total of 45% of the trials failed to recruit to within 80% of target [17]; our recruitment was far worse.

One of the difficulties in our trial could be that we focused mostly on incidence cases. Literature shows that trials were less successful in including patients if the study focused on incidence cases, rather than on prevalence cases [11, 23]. We tried to keep attention for our trial by mailing the orthopaedic surgeons, having telephone meetings and giving them small plastic cards with the inclusion criteria and contact information of the coordinating researcher.

Level of the patient

The willingness to participate in the trial was much lower than found in the pilot study (25 vs. 70%). Apparently, even though the situation in our pilot study was a near-future situation, it did not reflect the choices that were made in the actual trial. The information for the patients and parents was almost the same for the pilot and the trial. One could argue that the 10° worsening in the control group before starting brace treatment might have been the reason why the refusal rate was high. However, in our pilot study, we also explained that patients in the control group would be offered brace treatment in case the curvature progresses with 10° or more. Furthermore, all patients would have been examined every 4 months to closely watch possible progression. This protocol was acceptable for patients and parents in the pilot study, and they did not give us any indication that 5° would have been more acceptable. In the USA, a RCT on the effectiveness of bracing patients is currently running. In this trial brace wear versus watchful waiting will be compared. In their protocol, as far as we know, brace treatment in case of progression is not standardly offered to patients in the control group [13].

The main difference was the type of person that approached the potential participants. In the pilot study, the patients and parents were approached by a medical student. The patients and parents received written information on what such a trial would concern. Then the medical student visited them and explained the trial verbally and asked them whether they would participate. In the trial, the study was explained by the orthopaedic surgeon or, in a few hospitals, by a research nurse. Although numbers are too small to draw conclusions, we do not have indications that the participation rate is higher in the hospitals that had a research nurse explaining the trial to the patient and her parent(s). Group seminars with potential participants may be a useful strategy for maximising recruitment (at least from general practices) [19]. For our trial, however, this would have been too complicated to carry out in practice, because there were only a couple of eligible patients per month, and they lived scattered over the whole country.

Level of the individual professional

It is possible that, even though all orthopaedic surgeons agreed to participate and agreed with the protocol, they perceived some conflict between their roles as scientific investigator and personal physician [16]. They braced many patients before the trial, and now they had to randomize their patients to treatment or watchful waiting. This brace is a regular treatment, and is preferred amongst most of the orthopaedic surgeons, and apparently also by patients and parents, even though evidence is not convincingly established. In this case, the only way for patients to be certain that they would be treated was not to participate in the trial. Usually, in RCTs that test new medicines or devices, the only way to have access to that treatment for patients is to participate in that trial. This can make a big difference in inclusion rates. In the before-mentioned review of 114 UK trials, it seemed that cancer or drug trials were associated with successful recruitment, even as trials in which one or more interventions were tested that were only available inside the trial, although these finding should be interpreted carefully [17]. We know of at least one published trial on a popular treatment, but based on weak evidence, which also failed to include patients [20]. Taking this and our trial recruitment into consideration, we also feel that it is harder to abolish or postpone a treatment in a RCT than to add a new treatment.

Project leadership

Furthermore, it could be argued that whether it would have been better if an orthopaedic surgeon had project leadership. Perhaps, an orthopaedic surgeon could have been better able to assist the participating orthopaedic surgeons in combining their role as scientific investigator and personal physician.

Level of the wider environment

Screening for scoliosis is necessary for detecting cases in an early stage of the clinical course [3]. In our recently published case–control study on the effectiveness of screening for scoliosis, we did not find evidence that screening leads to a reduction in the need for surgery, which is the ultimate goal of screening [4]. One of the reasons why we did not find a beneficial effect of screening could be that brace treatment is not effective (enough) in (some of) these earlier detected patients. These results justified a RCT on bracing even more. In the Netherlands, the screening is performed by nearly 50% of the municipal health services (MHSs) [4, 14]. Although we feel that abolishing screening for scoliosis is justified [4], we probably “need” screening to identify patients in an early stage to be eligible for this trial. We do not have indications that additional MHSs stopped screening between 2006 and the end of 2007. Otherwise, this could have (partly) explained the lower actual number of eligible patients for the trial than the expected number.

Another issue could be that some orthopaedic surgeons feel that the incidence of idiopathic scoliosis might be declining, but we do not have data neither to prove nor to disprove this. It is, however, not very likely that a decline in incidence can completely explain the lower number of eligible patients.

Internet is an important information source for people who want to learn more about their disease or condition. Supposing eligible patients would consult the Internet, before they visit an orthopaedic surgeon, they would now mostly find that bracing is a (effective) strategy to prevent them from worsening. Perhaps, an Internet site with balanced information on the trial could have resulted in a higher participation rate, although the value of audio-visual interventions for people considering participating in clinical trials is unclear [21].

Conclusion

Our RCT on the effectiveness of bracing patients with IS failed to include patients, despite a good preparation and a pilot study that showed good participation rates. This latter teaches us that making a choice in a near-future situation can be something else than making a choice in the actual situation. This has probably much to do with the fact that bracing is the regular treatment and once patients have progressive curvatures, they want to act and try their best to stop progression, even if evidence of effectiveness is not convincing. Another important lesson learnt is that beforehand as precise as possible estimations on the number of eligible patients need to be made. This seems obvious, but in about 10% of trials enrolment was halted, because of poor recruitment [17]. In retrospect, we should have done that apparently better before the trial started.

Finally, we feel that it is harder to perform a RCT that abolishes or postpones a treatment than a RCT that adds a new treatment, and this should probably lead to a standard adjustment in sample size calculations beforehand in these situations.