The reporting of methods for reducing and detecting bias: an example from the WHO Misoprostol Third Stage of Labour equivalence randomised controlled trial

Abstract

Background

The aim of this article is to explore ways in which selection bias and ascertainment bias can be reduced and investigated in trials, by using the example of a drug trial carried out in both developed and developing countries in hospital delivery wards.

Methods

We describe an innovative and practical design for the boxes for packing the drugs as a way of increasing the security of allocation concealment and blinding. We also assess ascertainment bias using sensitivity analyses, as some unblinding could have occurred due to a potential side effect of one of the drugs.

Results

The sensitivity analyses indicated that the conclusions about the relative effects of the treatments could be maintained even in the unlikely worst-case scenarios.

Conclusions

Detailed description of the procedures protecting against common biases and of the assessment of ascertainment bias in this trial should allow readers to confidently appraise and interpret the results obtained. In addition, our experiences will assist others in planning trials in the future.

Background

In the attempt to reduce the risk of bias in clinical trials, the method used to assign treatments to trial participants is crucial for reducing selection bias. Randomisation is the method of choice, and also permits the use of probability theory to express the likelihood that any difference in outcome between treatment groups merely reflects chance. Successful randomisation in practice depends both on adequately generating an unpredictable allocation sequence, and also on concealing that sequence until assignment occurs [1,2]. Blinding the identity of treatments to the investigators, participants, and evaluators (sometimes using placebo) also reduces the risk of bias after assignment of treatments. Knowledge of treatment allocation could lead to selective withdrawals before the treatment starts, selective loss to follow-up (both selection biases) and/or to a subjective assessment of the outcome (ascertainment bias).

Inadequate methodological reporting of randomised clinical trials (RCTs) is associated with bias in the estimation of treatment effects [3]. Incomplete or inaccurate reporting impairs the critical appraisal of the quality of clinical trials, which is important to assess the validity of the results of the individual trial and in conducting systematic reviews. This realisation has been a factor in the recent calls for full and clear reporting of trials. To address these issues, the CONSORT statement [4] was published in 1996 and comprises a checklist, for two-group parallel designs, of essential items that should be included in reports of RCTs and a diagram to document the flow of participants through the trial. This has been accepted by many editors of international journals, and recently been revised [5] with further explanation and elaboration [6]. It is also being extended to other trial designs http://www.consort-statement.org. Nevertheless, concentration in Journals on the clinical issues often restricts the ability of those planning new trials to learn from the experiences of innovative methods gained in completed trials.

Hence the aim of this paper is to use the World Health Organization (WHO) Misoprostol Third Stage of Labour Trial in the Management of the Third Stage of Labour [7] as a case study to fully describe the details of procedures for sequence generation, allocation concealment and blinding in the context of a large equivalence RCT conducted in both developing and developed countries. We also describe how we evaluated the success of blinding by assessing the existence of ascertainment bias in the main outcomes due to differential side-effects in the two treatments.

Presentation of the Trial and its Procedures

The WHO Misoprostol Third Stage of Labour Trial

The incidence of postpartum haemorrhage can be reduced by the use of injectable uterotonic agents in the management of the third stage of labour (the period in which the placenta is delivered). However, some of those injectable agents (mostly ergot alkaloids) are associated with side-effects. An alternative is misoprostol, a prostaglandin E1 analogue that has strong uterotonic effects and was regarded as a promising agent for use in the third stage of labour in some uncontrolled unblinded studies [8,9]. It is administered orally or vaginally rather than intravenously (IV) or intramuscularly (IM) and therefore it could be used more easily than injectable preparations.

The aim of the WHO Misoprostol Third Stage of Labour Trial was to evaluate the effectiveness of routine oral misoprostol in the management of the third stage of labour, as compared with one of the injectable preparations, oxytocin. The trial was a multicentre, double-blind, randomised controlled trial conducted to determine whether the efficacy of 600 mcg misoprostol orally for use during the third stage of labour (after the baby is born) is equivalent to that of 10 IU of oxytocin in terms of measured postpartum vaginal blood loss of 1000 mls or more and the use of additional uterotonics, without an unacceptable level of side-effects. It was conducted in 14 hospitals in Argentina, China, Egypt, Ireland, Nigeria, South Africa, Switzerland, Thailand and Viet Nam. Women about to deliver vaginally were randomly allocated to receive 600 mcg misoprostol orally or 10 IU oxytocin (IV or IM) with corresponding identical placebos. The medications were administered immediately after delivery of the baby as part of the active management of the third stage of labour. A total of 18530 women were randomised into the trial (9264 women allocated to the misoprostol group and 9266 allocated to the oxytocin group), between April 1998 and November 1999. The two primary outcomes were the postpartum vaginal blood loss of 1000 mls or more measured from the time of delivery of the baby until the mother was transferred to postnatal care, and the use of additional uterotonics. Information about the primary outcomes was available for 99% of randomised women.

Oral misoprostol was associated with a higher occurrence of measured blood loss ≥ 1000 ml (risk difference 1.1%, 95% CI 0.6 to 1.6, relative risk RR = 1.39, 95% CI 1.19 to 1.63) and with more frequent use of additional uterotonics (risk difference 4.3%, 95% CI 3.3 to 5.3, RR = 1.40, 95% CI 1.29 to 1.51).

Sequence Generation, Allocation Concealment and Blinding

Sequence generation

The randomisation sequence was produced centrally in Geneva by the Clinical Trials and Informatics (CTI) support unit of RHR/WHO. Randomisation was stratified by country, allocating subjects to the two groups in blocks of randomly varying sizes of 4 and 6. For each country, random permutations in the blocks were produced using a SAS^® random number generator, with the starting number taken as a multiple of the WHO centre number (an identifying code number for the participating centre in the country). The SAS^® program that produces this allocation for a particular country is included in the Appendix (see Additional file: 1.

Allocation concealment

Allocation concealment was carefully planned to reduce selection bias through the design of special treatment-packs and dispensers [7].

Treatment packs contained three tablets, one ampoule, one syringe with a needle and swabs for injection and were identical, except for the centre identification and the subject number on the label (Figure 1).

Figure 1.

Treatment pack

The participating centres were in many areas of the world – some with poor telecommunications systems. This meant that central telephone randomisation was not always a feasible option. Hence, to preserve the allocation sequence, to conceal it, and to organize the distribution of the treatment packs, we arranged these in sequential order of subject numbers in bigger containers, or 'dispensers', in sets of 25. Each dispenser was sealed at the top and had a sealed slot at the bottom to be ripped open in order to remove the individual treatment packs sequentially, with lower numbers removed first. Each dispenser was labelled with the WHO project number and name, the country identification, the packing date and the range of sequential subject numbers on the treatment packs included (Figure 2).

Figure 2.

Dispenser (where the top seal has been violated)

In each hospital, there was at least one researcher (usually a nurse or a midwife) who had been trained to act as a focal point for the trial in that hospital. When a woman had been admitted after screening and vaginal delivery was imminent, this researcher removed a pack from the dispenser through the bottom open slot, starting with dispensers in ascending order of subject numbers. She wrote the woman's name on the pack. She also wrote the woman's name and her hospital record number on the appropriate line of a subject number list provided by the CTI Unit. At this moment the woman was considered as randomised and entered into the trial. The WHO site visitor (or in his absence, the study country co-ordinator) verified that all names on the subject number list had numbers assigned in sequence. He also verified that the dispensers were used in order of subject numbers and that the used dispensers had been opened by the bottom slot and the top seal had not been removed. Used packs and dispensers were discarded after observation. Unused packs were returned to Geneva. The randomisation sequence was thus preserved in every hospital. This allocation system of treatment packs contained by dispensers proved reliable and convenient, even in developing country settings.

Blinding

Double-blinding implied making placebo vials identical to oxytocin vials for women assigned misoprostol and placebo tablets identical to misoprostol tablets for women assigned oxytocin. A drug company preparing vials with 10 IU of oxytocin also prepared identical placebo vials. Another drug company prepared misoprostol 200 mcg and identical placebo tablets. In the oxytocin group, the vial contained 10 IU of oxytocin and the tablets were placebo. In the misoprostol group, the vial contained placebo liquid and the tablets contained 200 mcg of misoprostol each. The treatment packs were thus identical in shape, colour, weight and feel. The participants, those administering the interventions and those assessing the outcomes were all blinded to the group assignments. The schedule of group assignments was kept in Geneva blinded from the trial co-ordinators. Unblinding for medical reasons was possible by requesting the trial co-ordinator in Geneva to reveal the code, but there were no such cases.

Evaluation of Ascertainment Bias Due to Unblinding by a Side-Effect

Oxytocin alone when used in the management of the third stage of labour is relatively free of side-effects, although it carries the discomfort of injection. Ergot preparations, alone or in combination with oxytocin, cause significant rise in blood pressure, but oxytocin alone is not hypertensive. On the other hand, an association between misoprostol and shivering and other prostaglandin-related side effects (i.e. nausea, vomiting, diarrhoea) after administration has been reported. [8-12]. Some degree of unblinding could, therefore, have occurred due to the knowledge of differential side-effects. If shivering, the most prevalent of the side-effects, starts before blood loss measurement is completed, the chance of the assessor correctly guessing the treatment assignment could have been more than the 50% expected by chance in the absence of knowledge about the association. The chance of a correct guess is however unlikely to be close to 100% because shivering also occurs with delivery in a minority of women regardless of misoprostol.

Adequate correction for ascertainment bias requires knowledge of the degree of mis-estimation of blood loss in women with shivering. This quantity cannot be estimated from the data because high levels of blood loss in women with shivering could be a consequence of ascertainment bias, but they could also be due to a real biological effect of misoprostol on blood loss that takes place through shivering.

We therefore performed sensitivity analyses considering the effects of a range of plausible values of the ascertainment bias in the treatment effect. We assumed that the blood loss of all women with shivering, when measured, was inflated by 100, 200, 300, 400 or 500 mls or decreased by the same amounts. Then we re-analyzed the data subtracting these quantities from all measured blood losses in women with shivering and then repeated the analyses adding the same quantities to the measurements. If a negative value was obtained when subtracting, the blood loss was set to zero. These re-analyses have the advantage, compared to those we reported previously [7], that they use all women included in the main analysis and do not introduce selection bias. They also deal with possible bias in either direction.

The results of these sensitivity analyses indicate that even in the unlikely worst-case scenarios in which blood loss measurements in women with shivering had been inflated or decreased by 500 mls, the conclusions about the relative advantages of oxytocin would not change (Table 1).

Table 1.

Sensitivity analyses for measured blood loss ≥ 1000 ml (severe post-partum haemorrhage, or SPPH)

Assumed bias introduced in SPPH by unblinding due to shivering (mls)	Group	No. of women	No. of cases	%	RR	95% CI
500	Misoprostol	9214	295	3.2	1.23	(1.04 to 1.45)
	Oxytocin	9228	241	2.6
400	Misoprostol	9214	307	3.3	1.27	(1.07 to 1.49)
	Oxytocin	9228	243	2.6
300	Misoprostol	9214	313	3.4	1.29	(1.09 to 1.52)
	Oxytocin	9228	243	2.6
200	Misoprostol	9214	322	3.5	1.28	(1.09 to 1.51)
	Oxytocin	9228	251	2.7
100	Misoprostol	9214	334	3.6	1.31	(1.11 to 1.53)
	Oxytocin	9228	256	2.8
0	Misoprostol	9214	366	4.0	1.39	(1.19 to 1.63)
	Oxytocin	9228	263	2.9
-100	Misoprostol	9214	401	4.4	1.50	(1.29 to 1.75)
	Oxytocin	9228	267	2.9
-200	Misoprostol	9214	453	4.9	1.64	(1.41 to 1.90)
	Oxytocin	9228	277	3.0
-300	Misoprostol	9214	499	5.42	1.74	(1.51 to 2.00)
	Oxytocin	9228	288	3.12
-400	Misoprostol	9214	591	6.41	1.94	(1.70 to 2.22)
	Oxytocin	9228	305	3.31
-500	Misoprostol	9214	688	7.47	2.10	(1.85 to 2.39)
	Oxytocin	9228	328	3.55

Comments

Reporting of the main clinical findings of a trial in a Journal does not always provide sufficient details for others to learn innovative methods from the experience. Indeed, extra clinical questions may be raised after publication of the original report, and further analyses may appear in correspondence columns to aid clinical readers, as for this trial [13-17]. Similarly, publication of the methodological procedures we used for bias reduction and for assessing this bias will be useful for other trialists. Our method of randomisation ensured the generation of an unpredictable assignment sequence for treatment allocation. This was guaranteed by the sequence being centrally produced and by the use of blocks of randomly varying sizes. Blocks of 4 or 6 are small, but this size is reasonable, considering the effective double-blinding [18,19]. Randomization was successful in achieving a balance in baseline variables, for which descriptive statistics were similar between groups [7].

The range of clinical settings for the Misoprostol Third Stage of Labour Trial posed special challenges in achieving the concealment of the allocation and double-blinding. Allocation concealment required the design of identically appearing boxes and dispensers. The dispensers ensured the preservation of the random sequence and avoided the loss of treatment packs in busy labour wards. A subject number list was kept, verifying that names were not wrongly entered or omitted. We believe that the concealment of the allocation sequence was carefully planned and thus prevented selection bias.

Double-blinding was complicated to implement because it implied producing placebo vials identical to the oxytocin vials, as well as placebo tablets identical to the misoprostol tablets. Actually, of the eight randomised controlled trials published to date comparing misoprostol to other uterotonics, only four trials were double-blind [10]. We considered it important to have this feature of double-blinding in the trial to prevent selection bias after assigning the treatment, ascertainment bias and bias due to differential use of a co-intervention. The use of misoprostol implied changing the routine practice of the hospital, and unblinded medical personnel could feel inclined to change the treatment of a woman assigned misoprostol to the routine uterotonic if the woman had a risk condition. They could also be inclined to do the reverse, given that misoprostol is such a popular drug in obstetrics and gynaecology [8,9] and it was therefore expected to perform equally or better than injectable uterotonics. This could introduce selection bias. Ascertainment bias was likely to occur when measuring blood loss if researchers were unblinded, for similar reasons. Providing additional uterotonics as a co-intervention more frequently to women receiving the new treatment could introduce bias in the effect of the treatment on the use of additional uterotonics and also on blood loss, since the additional uterotonics could prevent further blood loss. Ascertainment bias would be also possible in the assessment of side-effects, due to the known association between misoprostol and shivering and other prostaglandin-related clinical events.

Blinding of all involved in implementing the trial was achieved by the identical appearance of the boxes and their contents, including double placebos. There were no cases of unblinding for medical reasons because it was not necessary to know the treatment assignment in order to provide additional uterotonics or any other additional treatment. We believe double-blinding successfully prevented possible biases likely to occur after treatment assignment.

We could have assessed the success of the blinding by interviews to the medical personnel, asking questions about which treatment they thought was being administered, and then evaluating if they were guessing better than chance. We did not apply this procedure for three reasons. In the first place, the interviews, even if short, would have disturbed even more the medical practice in the delivery ward. In the second place, the possibility of inaccurate responses or interpretation difficulties following tests of blindness has led to their usefulness being characterised as being "tests of hunches for adverse effects or efficacy" [20,21]. Third, blinding in the particular case of equivalence trials does not protect against bias towards demonstrating equivalence, since it does not prevent a clinician taking similar measures of blood or applying a co-intervention in the two arms [22].

In spite of the blinding efforts, we recognized there could be some ascertainment bias in the main outcomes due to unblinding by guessing the treatment allocation due to shivering. We conducted sensitivity analyses to evaluate the magnitude of this possible bias, and concluded that even if present, this bias would not substantially alter the main conclusions from the trial.

Trial boxes are often used for ease of administration of the trial medications, ease of enrolment and to achieve satisfactory concealment of the allocation sequence. The dispensers we designed also served the practical purpose of administering drug distribution in the hospital wards and to facilitate enrolment of eligible women, by making them available all the time. This strategy could be used by other researchers in future trials. Especially in some settings in developing countries or busy wards or rural institutions in developing or developed countries, this type of administration of trial medications is a good alternative to central/telephone information of the treatment allocation.

The detailed description of the procedures for protecting against common biases in RCTs in this case study should allow readers the confidence to critically appraise and interpret the results of this trial. In addition, our experiences will assist others in planning trials in the future.

Competing interests

None declared.

Authors' contributions

All the authors contributed to the design, analysis, interpretation and reporting of the original Misoprostol Trial. JV, APYP and AMG designed the boxes used for allocation concealment and APYP prepared the randomisation program. This report was conceived and written by GP, DE and KFS. All authors commented on all versions of the paper.

Pre-publication history

The pre-publication history for this paper can be accessed here:

http://www.biomedcentral.com/1471-2288/3/19/prepub