Skip to main content
PMC home page
BMC Medical Research Methodology logoLink to BMC Medical Research Methodology
. 2001 Jan 10;1:1. doi: 10.1186/1471-2288-1-1

Using evidence from different sources: an example using paracetamol 1000 mg plus codeine 60 mg

Lesley A Smith 1, R Andrew Moore 1,✉, Henry J McQuay 1, David Gavaghan 2
PMCID: PMC32200  PMID: 11231885

Abstract

Background

Meta-analysis usually restricts the information pooled, for instance using only randomised, double-blind, placebo-controlled trials. This neglects other types of high quality information. This review explores using different information for the combination of paracetamol 1000 mg and codeine 60 mg in acute postoperative pain.

Results

Randomised, double-blind, placebo-controlled trials of paracetamol 1000 mg and codeine 60 mg had an NNT of 2.2 (95% confidence interval 1.7 to 2.9) for at least 50% pain relief over four to six hours in three trials with 197 patients. Computer simulation of randomised trials demonstrated 92% confidence that the simulated NNT was within ± 0.5 of the underlying value of 2.2 with this number of patients. The result was supported a rational dose-response relationship for different doses of paracetamol and codeine in 17 additional trials with 1,195 patients. Three controlled trials lacking a placebo and with 117 patients treated with of paracetamol 1000 mg and codeine 60 mg had 73% (95%CI 56% to 81%) of patients with at least 50% pain relief, compared with 57% (48% to 66%) in placebo controlled trials. Six trials in acute pain were omitted because of design issues, like the use of different pain measures or multiple dosing regimens. In each paracetamol 1000 mg and codeine 60 mg was shown to be better than placebo or comparators for at least one measure.

Conclusions

Different designs of high quality trials can be used to support limited information used in meta-analysis without recourse to low quality trials that might be biased.

Background

The use of evidence-based approaches to therapeutic decision making can frequently raise the problem of how to make decisions when evidence is in limited supply. Often systematic reviews limit trial inclusion in an attempt to generate clinical homogeneity and allow sensible meta-analysis. The problem, though, is that other, useful, information is omitted. An example of this is the popular combination of paracetamol with codeine for treatment of acute and, more frequently, chronic pain. For the combination of 1000 mg paracetamol plus 60 mg codeine, for instance, there was information on only 127 patients in two placebo-controlled acute pain studies [1, 2].

Systematic reviews should seek unbiased evidence, which may limit the number of studies available for analysis. One approach to resolving the problem of apparently insufficient evidence may be to assess evidence of lower methodological quality. The amount of bias that could result from this approach would be a concern.

Schultz et al [3] demonstrated that lack of randomisation is the major source of bias in trials; studies which are not randomised can lead to overestimation of treatment effects by up to 40%. Restricting systematic reviews to include only randomised studies therefore makes sense for reviews of effectiveness. A classic example is a review of transcutaneous nerve stimulation for post-operative pain relief. Randomised studies overwhelmingly showed no benefit over placebo, while non-randomised studies did show benefit [4].

Non-blinded studies over-estimate treatment effects by about 17% [3]. In a review of acupuncture for back pain [5], the inclusion of both blinded and non-blinded studies changed the overall conclusion. The blinded studies showed 57% of patients improved with acupuncture and 50% with control, a non-significant relative benefit of 1.2 (95% confidence interval 0.9 to 1.5). Five non-blinded studies showed a difference from control, with 67% improved with acupuncture and 38% with control. Here the relative benefit was significant at 1.8 (1.3 to 2.4).

Trials of poor reporting quality consistently overestimate the effect of treatment. Using a validated scoring system for methodological quality [6], studies of lower quality are likely to overestimate treatment effects [7, 8]. Other sources of bias may include small trials [9,10,11], covert duplication [12], and geography [13]. Vickers and colleagues [13] showed that trials of acupuncture conducted in east Asia were universally positive, while those conducted in Australasia, north America or western Europe were positive only about half the time. Randomised trials of therapies other than acupuncture conducted in China, Taiwan, Japan or Russia/USSR were also overwhelmingly positive.

There is also the issue of the overall validity of a randomised trial. In some areas, like acute pain, valid methods for the conduct of clinical studies have been set out for many years, and are well understood [14]. There are therefore many trials that are randomised and double blind, and conducted on patients with the same initial severity of pain under similar conditions and assessing identical or similar outcomes over the same time periods. Trials with low validity are more likely to have a positive result than those with higher validity [15], seen in acupuncture for head and neck pain.

It is obviously sensible to avoid bias where it is likely to occur. That means avoiding studies with designs or features where bias is possible. What other strategies remain when faced with apparently inadequate information from the literature? There are four that could be applied to the particular combination of paracetamol 1000 mg plus codeine 60 mg. These are:

1. The number of patients required for the number needed to treat (NNT) to be within ± 0.5 of the true value varies with the efficacy of an analgesic [11]. We can therefore calculate how confident we can be in an NNT given the number of patients in the analysis, or calculate the number of patients needed to reach a required level of certainty.

2. We can assess evidence from other dose combinations of paracetamol and codeine and see whether there is a dose response relationship in placebo-controlled trials.

3. We can assess results for paracetamol 1000 mg plus codeine 60 mg against results for active comparators in high quality active controlled and placebo controlled studies. This can then be compared with systematic reviews done in similar clinical settings.

4. We can assess paracetamol 1000 mg plus codeine 60 mg in studies which, though randomised and double blind, had designs that did not allow their inclusion in a meta-analysis.

We applied these approaches to paracetamol 1000 mg plus codeine 60 mg through systematic review of the published literature.

Methods

Full reports of randomised controlled trials of oral paracetamol combined with codeine for acute postoperative pain were sought. Different search strategies were used to identify eligible reports in MEDLINE (1966 - March 2000), EMBASE (1980 - March 2000), the Cochrane Library (Issue 3, 2000) and the Oxford Pain Relief Database (1950 - 1994) [16]. Reference lists of retrieved reports and reviews [1,2] were searched for additional trials. Abstracts, review articles and unpublished reports were not considered.

Criteria for inclusion for postoperative pain were: full journal publication, randomised controlled trials which included single dose treatment groups of oral paracetamol combined with codeine, double blind design, baseline postoperative pain of moderate to severe intensity, patients over 15 years of age, at least 10 patients per group, and the pain outcome measures of total pain relief (TOTPAR) or summed pain intensity difference (SPID) over 4-6 hours or sufficient data provided to allow their calculation. Pain measures allowed for the calculation of TOTPAR or SPID were a standard five point pain relief scale (none, slight, moderate, good, complete), a standard four point pain intensity scale (none, mild, moderate, severe) or a standard visual analogue scale (VAS) for pain relief or pain intensity. Each report was scored for quality using a three item, 1-5 score, quality scale [6].

For each trial, mean TOTPAR, SPID, VASTOTPAR or VASSPID values for each drug group were converted to %maxTOTPAR by division into the calculated maximum value [17]. The proportion of patients in each treatment group who achieved at least 50%maxTOTPAR was calculated using valid equations [18,19,20]. The number of patients with >50%maxTOTPAR was then used to calculate relative benefit and NNT for paracetamol plus codeine versus placebo. Relative benefit and relative risk estimates were calculated with 95% confidence intervals (CI) using a fixed effects model [21]. NNT with 95% confidence intervals was calculated by the method of Cook and Sackett [22]. A statistically significant difference from control was assumed when the 95% confidence interval of the relative benefit did not include 1. Confidence intervals of proportions were calculated according to Morris and Gardner [23]. Calculations were performed using Excel v 5.0 on a Power Macintosh G3. Simulations and calculations of probability were conducted as described in Moore et al, 1998 [11].

Results

There were three placebo controlled [24,25,26] and three active controlled studies [27,28,29] involving paracetamol 1000 mg plus codeine 60 mg (Additional material: Table 3). Quality scores were 3 or better for all six. Information on other dose combinations is in Additional material: Table 4

In placebo controlled studies of paracetamol 1000 mg plus codeine 60 mg 65/114 patients given paracetamol plus codeine had at least 50% pain relief compared with 9/83 for placebo (Table 1). For a single dose of paracetamol/codeine the proportion of patients with at least 50% pain relief was 57% (48% to 66%). The NNT for at least 50% pain relief over four to six hours was 2.2 (1.7 to 2.9).

Table 1.

Summary results for efficacy of paracetamol/codeine combinations from randomised, double-blind, placebo-controlled trials in acute pain

At least 50% pain relief At least 50% pain
with paracetamol and relief with placebo
codeine
Paracetamol + Number of Number Percent Number Percent Relative benefit NNT (95% CI)
codeine dose trials /total (95%CI) /total (95%CI) (95% CI)
1000 mg + 60 mg 3 65/114 57 (48-66) 9/83 11(4-18) 4.8 (2.6 to 8.8) 2.2 (1.7 to 2.9)
600/650 mg + 60 mg 13 191/398 48 (43-53) 78/418 19 (15-22) 2.5 (2.0 to 3.1) 3.4 (2.8 to 4.3)
300 mg + 30 mg 4 56/215 26 (20-32) 14/164 9 (4-13) 3.2 (1.8 to 5.6) 5.6 (4.0 to 9.8)
Open in a new tab

For inclusion there had to be at least two studies.

Additional analyses undertaken to assess the reliability of this assessment of paracetamol 1000 mg plus codeine 60 mg included:

1 Assessment of Accuracy

How accurate is the NNT with currently observed group sizes and event rates? Using the values of the placebo event rate of 0.11 (9/83) and the paracetamol/codeine event rate of 0.57 (65/114), we simulated 10,000 trials with experimental group size of 114 and a control group size of 83 and counted the proportion in which the simulated NNT was within ± 0.5 of the true NNT of 1.9 [11]. The results showed that 91.9% of simulated trials had an NNT of 1.8 to 2.6. The probability that the simulated NNT was within ± 0.5 of the underlying value of 2.2 was 0.92.

How much data would we require to be 95% sure that the observed NNT is within ± 0.5 of its true value? We answered this question in two ways, both of which consider the case of equal group sizes. Again the placebo rate of 0.11 and the paracetamol/codeine rate of 0.57 were used. The first method used the exact probability distribution [11]. Using a group size of 130 we obtained a probability of 0.94, and with 140 we obtained 0.95. So the answer based on the exact distribution is between 130 and 140 per group. The second method used simulation. Using a group size of 130 the probability was 0.94, and with 140 it was 0.95. So the answer based on the simulation was again between 130 and 140 per group, a total of 260 to 280 patients.

2 Other dose Combinations of Paracetamol and Codeine

Information was available on 17 comparable studies [30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46] in which different dose combinations of paracetamol and codeine were available for at least two trials (Additional material: Table 4). Quality scores were 3 or better for all.

There were 13 trials comparing paracetamol 600/650 mg plus codeine 60 mg with placebo in acute pain. For paracetamol plus codeine 191/398 patients had at least 50% pain relief and 78/418 for placebo. For a single dose of paracetamol/codeine the proportion of patients with at least 50% pain relief was 48% (43% to 53%). The NNT for at least 50% pain relief over four to six hours was 3.4 (2.8 to 4.2).

There were four trials comparing paracetamol 300 mg plus codeine 30 mg with placebo (Table 1). For paracetamol plus codeine 56/215 patients had at least 50% pain relief and 14/164 for placebo. For a single dose of paracetamol/codeine the proportion of patients with at least 50% pain relief was 26% (20% to 32%). The NNT for at least 50% pain relief over four to six hours was 5.6 (4.0 to 9.8).

Reducing the dose of paracetamol, and reducing the dose of paracetamol and codeine produced systematic reductions in the proportion of patients with adequate pain relief, and a concomitant increase in the NNT (Table 1; Figure 1). Based on information from 20 trials and 727 patients given paracetamol plus codeine and 665 patients given placebo, there appears to be a rational dose-response relationship for paracetamol/codeine combinations in acute pain.

Figure 1.

Figure 1

Open in a new tab

Relative efficacy: Numbers needed to treat for paracetamol/codeine combinations for at least 50 % pain relief over four to six hours for patients with moderate or severe post-operative pain. The number of patients in the comparison was the number given paracetamol 1000 mg plus codeine 60 mg together with the number given placebo in the data pooled from trials for each combination of paracetamol and codeine where there were at least two trials.

3 Active Controlled Trials

We identified three reports of randomised, double blind trials comparing paracetamol 1000 mg plus codeine 60 mg with active control groups but without a placebo control ([27,28,29] (Additional material: Table 3). In active controlled trials the proportion of patients with at least 50% pain relief over four to six hours was 85/117 patients, or 73% (65% to 81%), compared with 65/114 patients, or 57% (48% to 66%) in placebo controlled trials. Combining data from active and placebo controlled trials, 150 of 229 patients, or 66% (59% to 72%), had at least 50% pain relief with paracetamol 1000 mg plus codeine 60 mg (Table 2; Figure 2).

Table 2.

Summary results for efficacy of paracetamol/codeine combinations and active comparators in randomised, double-blind, active- and placebo-controlled trials in acute pain

Drug/dose Number of Patients with at Percentage
trials least 50% pain (95%CI)
relief/total
Paracetamol 300 mg + codeine 30 mg 3 67/272 25 (20 to 30)
Codeine 60 mg 3 24/72 33 (22 to 44)
Ibuprofen 400 mg 2 28/72 39 (28 to 50)
Paracetanol 600/650 mg 10 135/311 43 (38 to 49)
Paracetamol 1000 mg 4 64/151 42 (35 to 50)
Paracetamol 600 mg + codeine 60 mg 13 191/398 48 (43 to 53)
Diflusinal 500 mg 3 54/90 60 (50 to 70)
Ketorolac 10 mg 3 75/117 64 (55 to 73)
Diflusinal 1000 mg 3 61/92 66 (57 to 76)
Paracetamol 1000 mg + codeine 60 mg 6 150/229 66 (59 to 72)
Flurbiprofen 50 mg 3 69/99 70 (61 to 79)
Flurbiprofen 100 mg 3 70/92 76 (67 to 85)
Open in a new tab

Figure 2.

Figure 2

Open in a new tab

Efficacy of paracetamol/codeine combinations and active comparators in randomised, double-blind, active- and placebo-controlled trials in acute pain.

Information on other paracetamol/codeine combinations and other active comparators from active and placebo controlled studies involving paracetamol and codeine combinations is shown in Table 2 and Figure 2. Again there was a rational dose response for paracetamol/codeine combinations, and paracetamol 1000 mg plus codeine 60 mg compared favourably with effective doses of non-steroidal anti-inflammatory drugs (NSAIDs) [14].

4 Trials Omitted from the Meta-Analysis

There were six trials in acute pain, which, though randomised and double blind, had design features that did not allow their inclusion in the meta-analysis ([47,48,49,50,51,52] Additional material: Table 5). The trials could not be included because they used different pain measures, over relatively short periods, or had multiple dosing regimens that obscured the efficacy of the first dose. Three had a placebo control. Quality scores were 3 or better for all six (Additional material: Table 5). The studies enrolled 907 patients, and there was information on 260 patients who took paracetamol 1000 mg plus codeine 60 mg with initial pain intensity of at least moderate intensity.

The outcomes of these six trials were not the same as those used for the meta-analysis. Despite this, the combination of paracetamol 1000 mg plus codeine 60 mg could be ascertained. In each of the studies paracetamol 1000 mg plus codeine 60 mg was shown to be better than placebo or comparators for at least one measure (Additional material: Table 5). Where effects of paracetamol 1000 mg plus codeine 60 mg on pain intensity was graphically displayed [49, 51, 52], large reductions in pain intensity were obtained within about one hour.

Discussion

It can be the case that attempts to practice evidence based medicine come up against the problem that there is no evidence, or relatively little evidence. Reviews of paracetamol and codeine [1, 2] revealed that the combination of paracetamol 1000 mg plus codeine 60 mg was apparently highly effective, with an NNT of 1.9, but this was derived from only 127 patients in two trials. Using evidence from sources other than direct comparisons of paracetamol/codeine with placebo buttresses the conclusion of these earlier reviews.

Firstly we uncovered information about another trials that increased the number of patients to 197. The inclusion of the third trial barely changed the NNT from 1.9 to 2.2. Unpublished information from a fourth study is presently being sought.

We know that the amount of information required to be sure that an NNT is within ± 0.5 of the true value varies depending on the efficacy of treatment [11, 53]. Where efficacy is high, the requirement is for small numbers of patients. Where efficacy is low, the requirement is for large numbers of patients. For paracetamol 1000 mg plus codeine 60 mg, calculations show that with 197 patients we can be about 92% confident that the NNT obtained is within ± 0.5 of the true value, close to the conventionally accepted value of 95%. To be within the 95% limits we would need information from a further 60 to 80 patients.

Biological plausibility derives also from a substantial dose response relationship with the placebo controlled studies as the doses of paracetamol and of paracetamol and codeine increase. Seventeen additional studies of high quality (randomised, double blind, with clear entry criteria and outcomes measures) with 1195 additional patients demonstrated this (Table 1; Figure 1). Dose response was also shown when information from placebo and active controlled trials was pooled to examine the overall percentage of patients with at least 50% pain relief (Table 1).

This type of analysis additionally demonstrated that in trials examining paracetamol/codeine combinations, the paracetamol 1000 mg plus codeine 60 mg combination was much better than either paracetamol 1000 mg or codeine 60 mg alone (Table 2; Figure 2). There was clear separation between the lower confidence interval for paracetamol 1000 mg plus codeine 60 mg (59%) and the higher confidence interval for paracetamol 1000 mg (50%) and codeine 60 mg (44%). This confirms the added benefit seen with this combining codeine with paracetamol in a previous review [1].

Moreover, paracetamol 1000 mg plus codeine 60 mg demonstrated efficacy similar to effective doses of NSAIDs drugs tested in the selfsame trials (Table 2; Figure 2). There is now considerable and growing evidence on the effectiveness of NSAIDs as analgesics in acute pain [54, 55]. While ibuprofen 400 mg performed relatively poorly in two trials here, pooled information from large numbers of studies show it to be an effective analgesic with an NNT of 2.7 (2.5 to 3.0) [54]. Ketorolac has also been shown to be effective orally; the 10 mg dose had an NNT of 2.6 (2.3 to 3.1) [55]. In trials involving NSAIDs these effective analgesics are no better than paracetamol/codeine combinations.

Only six randomised comparisons of paracetamol 1000 mg plus codeine 60 mg with placebo or other analgesics without placebo were available for analysis using methods that have become traditional for analysis in acute pain [14]. This number was doubled by examining studies that were randomised and double blind, but which used different pain intensity or relief outcomes, or other study designs. These additional six studies confirmed that paracetamol 1000 mg plus codeine 60 mg was an effective analgesic in acute pain.

Taken together, this approach shows two things. There is considerable supportive evidence for the conclusion that paracetamol 1000 mg plus codeine 60 mg is an effective analgesic with a low NNT. This support could be produced without resource to low quality studies, or those with known propensity for bias.

It will not always be possible to extend knowledge though high quality and essentially bias free trials in all circumstances: acute pain is particularly rich in trials, and especially those of high quality. Strategies like this will always be preferable. When placebo controlled trials are not available conclusions may have to be drawn from other trials. The techniques described here may be useful in circumstances where large numbers of high quality placebo controlled randomised trials are not available to us. Observational studies of good quality need not give results that are different from those of randomised trials, as testified by three recent studies [56,57,58]. But the potential for bias through lack of quality needs always to be addressed.

Pre-publication history

The pre-publication history for this article can be accessed here:

http://www.biomedcentral.com/content/backmatter/1471-2288-1-1-b1.pdf

Supplementary Material

Table 3
click here for additional Table 3 (15.5KB, xls)
Table 4
click here for additional Table 4 (27.5KB, xls)
Table 5
click here for additional Table 5 (13KB, xls)

Acknowledgments

Acknowledgements

The study was supported with Pain Research funds and an unconditional educational grant from Schwarz Pharma. DG is the recipient of a Medical Research Council for Career Development Fellowship.

Glossary

SPID Categorical scales use words to describe the magnitude of the pain. For analysis numbers are given to the verbal categories (for pain intensity, none=0, mild=1, moderate=2 and severe=3). Pain is assessed before the intervention is made and then on multiple occasions. The area under the time-analgesic effect curve for pain intensity, or the sum of pain intensity differences, is SPID. It is usually, but not always, measured over four to six hours.

TOTPAR Categorical scales also use words to describe the magnitude of pain relief. The commonest scale to measure pain relief is the five category scale (none=0, slight=1, moderate=2, good or lots=3 and complete=4). The initial pain relief is taken to be zero, and the pain relief is measured at intervals thereafter. Ideally the area under the time-analgesic effect curve for pain relief, or total pain relief, is TOTPAR.

VASSPID Visual analogue scales (VAS), 100 mm lines with left end labelled "least possible pain" and right end labelled "worst possible pain" can be used to measure pain intensity. Patients mark the line at the point which corresponds to their pain. The scores are obtained by measuring the distance between the least pain end and the patient's mark, usually in millimetres. The area under the time-analgesic effect curve for pain intensity, or the sum of pain intensity differences, is VASSPID.

VASTOTPAR Visual analogue scales (VAS), 100 mm lines with left end labelled "no relief of pain" and right end labelled "complete relief of pain", can be used to measure pain relief. The initial pain relief is taken to be zero, and thereafter patients mark the line at the point which corresponds to their pain relief. The scores are obtained by measuring the distance between the no relief end and the patient's mark, usually in millimetres. The area under the time-analgesic effect curve for pain relief, or the sum of total pain relief, is VASTOTPAR.

maxTOTPAR The maximum pain relief a patient can obtain is complete relief at the first measurement point, maintained for the whole period of observation. Thus using the categorical scale, the maximum relief would be 4, maintained for six hours, giving a maximum value of TOTPAR of 24. If a patient scored their pain relief in such a way as to give a TOTPAR of 15, this would be 15/24, or 63% of maxTOTPAR.

Contributor Information

Lesley A Smith, Email: lesley.smith@pru.ox.ac.uk.

R Andrew Moore, Email: andrew.moore@pru.ox.ac.uk.

Henry J McQuay, Email: henry.mcquay@pru.ox.ac.uk.

David Gavaghan, Email: David.Gavaghan@comlab.oxford.ac.uk.

References

  1. Moore A, Collins S, Carroll D, McQuay H. Paracetamol with and without codeine in acute pain: a quantitative systematic review. Pain. 1997;70:193–201. doi: 10.1016/S0304-3959(96)03319-2. [DOI] [PubMed] [Google Scholar]
  2. Moore A, Collins S, Carroll D, McQuay H, Edwards J. Single dose paracetamol (acetaminophen), with and without codeine, for postoperative pain (Cochrane Review). The Cochrane Library, Issue 4, 2000. Oxford: Update Software. [DOI] [PubMed]
  3. Schulz KF, Chalmers I, Hayes RJ, Altman DG. Empirical evidence of bias: dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA. 1995;273:408–412. doi: 10.1001/jama.273.5.408. [DOI] [PubMed] [Google Scholar]
  4. Carroll D, Tramer M, McQuay H, Nye B, Moore A. Randomization is important in studies with pain outcomes: Systematic review of transcutaneous electrical nerve stimulation in acute postoperative pain. Br J Anaesth. 1996;77:798–803. doi: 10.1093/bja/77.6.798. [DOI] [PubMed] [Google Scholar]
  5. Ernst E, White AR. Acupuncture for back pain. Arch Intern Med. 1998;158:2235–2241. doi: 10.1001/archinte.158.20.2235. [DOI] [PubMed] [Google Scholar]
  6. Jadad AR, Moore RA, Carroll D, et al. Assessing the quality of reports of randomized clinical trials: is blinding necessary? Control Clin Trials. 1996;17:1–12. doi: 10.1016/0197-2456(95)00134-4. [DOI] [PubMed] [Google Scholar]
  7. Khan KS, Daya S, Jadad AR. The importance of quality of primary studies in producing unbiased systematic reviews. Arch Intern Med. 1996;156:661–666. doi: 10.1001/archinte.156.6.661. [DOI] [PubMed] [Google Scholar]
  8. Moher D, Pham B, Jones A, et al. Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses? Lancet. 1998;352:609–613. doi: 10.1016/S0140-6736(98)01085-X. [DOI] [PubMed] [Google Scholar]
  9. Moore RA, Collins SL, McQuay HJ. Variation in the placebo response rate: implications for individual trials and consequences for meta-analysis. In Proceedings of the 8th World Congress on Pain, Progress in Pain Research & Management, Vol 8 Editors Jensen TS, Turner JA & Wiesenfeld-Hallin Z, IASP Press, Seattle, 1997.
  10. Moore RA, Carroll D, Wiffen PJ, Tramèr M, McQuay HJ. Quantitative systematic review of topically-applied non-steroidal anti-inflammatory drugs. BMJ. 1998;316:333–338. doi: 10.1136/bmj.316.7128.333. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Moore RA, Gavaghan D, Tramèr MR, Collins SL, McQuay HJ. Size is everything - large amounts of information are needed to overcome random effects in estimating direction and magnitude of treatment effects. Pain. 1998;78:217–220. doi: 10.1016/S0304-3959(98)00133-X. [DOI] [PubMed] [Google Scholar]
  12. Tramèr MR, Reynolds DJM, Moore RA, McQuay HJ. Impact of covert duplicate publication on meta-analysis: a case study. BMJ. 1997;315:635–639. doi: 10.1136/bmj.315.7109.635. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Vickers A, Goyal N, Harland R, Rees R. Do certain countries produce only positive results? A systematic review of controlled trials. Control Clin Trials. 1998;19:159–166. doi: 10.1016/S0197-2456(97)00150-5. [DOI] [PubMed] [Google Scholar]
  14. McQuay HJ, Moore RA. An evidence-based resource for pain relief. Oxford: Oxford University Press, 1998.
  15. Smith LA, Oldman AD, McQuay HJ, Moore RA. Teasing apart quality and validity in systematic reviews: an example from acupuncture trials in chronic neck and back pain. Pain. 2000;86:119–132. doi: 10.1016/S0304-3959(00)00234-7. [DOI] [PubMed] [Google Scholar]
  16. Jadad AR, Carroll D, Moore A, McQuay H. Developing a database of published reports of randomised clinical trials in pain research. Pain. 1996;66:239–246. doi: 10.1016/0304-3959(96)03033-3. [DOI] [PubMed] [Google Scholar]
  17. Cooper SA. Single-dose analgesic studies: the upside and downside of assay sensitivity. In The design of analgesic clinical trials (Advances in Pain Research and Therapy Vol 18) Edited by Max MB, Portenoy RK, and Laska EM New York: Raven Press, 1991. pp. 117–124.
  18. Moore A, McQuay H, Gavaghan D. Deriving dichotomous outcome measures from continuous data in randomised controlled trials of analgesics. Pain. 1996;66:229–237. doi: 10.1016/0304-3959(96)03032-1. [DOI] [PubMed] [Google Scholar]
  19. Moore A, McQuay H, Gavaghan D. Deriving dichotomous outcome measures from continuous data in randomised controlled trials of analgesics: Verification from independent data. Pain. 1997;69:127–130. doi: 10.1016/S0304-3959(96)03251-4. [DOI] [PubMed] [Google Scholar]
  20. Moore A, Moore O, McQuay H, Gavaghan D. Deriving dichotomous outcome measures from continuous data in randomised controlled trials of analgesics: Use of pain intensity and visual analogue scales. Pain. 1997;69:311–315. doi: 10.1016/S0304-3959(96)03306-4. [DOI] [PubMed] [Google Scholar]
  21. Gardner MJ, Altman DG. Confidence intervals rather than P values: estimation rather than hypothesis testing. BMJ. 1986;292:746–750. doi: 10.1136/bmj.292.6522.746. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Cook RJ, Sackett DL. The number needed to treat: a clinically useful measure of treatment effect. BMJ. 1995;310:452–454. doi: 10.1136/bmj.310.6977.452. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Morris JA, Gardner MJ. Calculating confidence intervals for relative risk, odds ratios and standardised ratios and rates. In Statistics with confidence -confidence intervals and statistical guidelines Edited by Gardner, MJ and Altman DG London: British Medical Journal, 1995. pp. 50–63. [DOI] [PMC free article] [PubMed]
  24. Bentley KC, Head TW. The additive analgesic efficacy of acetaminophen, 1000 mg, and codeine, 60 mg, in dental pain. Clin Pharmacol Ther. 1987;42:634–640. doi: 10.1038/clpt.1987.211. [DOI] [PubMed] [Google Scholar]
  25. Skoglund LA, Skjelbred P, Fyllingen G. Analgesic efficacy of acetaminophen 1000 mg, acetaminophen 2000 mg, and the combination of acetaminophen 1000 mg and codeine phosphate 60 mg versus placebo in acute postoperative pain. Pharmacotherapy. 1991;11:364–369. [PubMed] [Google Scholar]
  26. Stubhaug A, Grimstad J, Breivik H. Lack of analgesic effect of 50 and 100 mg oral tramadol after orthopaedic surgery: a randomized, double-blind, placebo and standard active drug comparison. Pain. 1995;62:111–118. doi: 10.1016/0304-3959(95)00056-X. [DOI] [PubMed] [Google Scholar]
  27. Breivik EK, Barkvoll P, Skovlund E. Combining diclofenac with acetaminophen or acetaminophen-codeine after oral surgery: a randomized, double-blind single-dose study. Clin Pharmacol Ther. 1999;66:625–635. doi: 10.1053/cp.1999.v66.103629001. [DOI] [PubMed] [Google Scholar]
  28. Gertzbein SD, Tile M, McMurty RY, et al. Analysis of the analgesic efficacy of acetaminophen 1000 mg, codeine phosphate 60 mg, and the combination of acetaminophen 1000 mg and codeine phosphate 60 mg in the relief of postoperative pain. Pharmacotherapy. 1986;6:104–107. doi: 10.1002/j.1875-9114.1986.tb03462.x. [DOI] [PubMed] [Google Scholar]
  29. Vangen O, Doessland S, Lindbaek E. Comparative study of ketorolac and paracetamol/codeine in alleviating pain following gynaecological surgery. J Int Med Res. 1988;16:443–451. doi: 10.1177/030006058801600606. [DOI] [PubMed] [Google Scholar]
  30. Cooper SA, Breen JF, Giuliani RL. The relative efficacy of indoprofen compared with opioid-analgesic combinations. J Oral Surg. 1981;39:21–25. [PubMed] [Google Scholar]
  31. Cooper SA, Firestein A, Cohn P. Double blind comparison of meclofenamate sodium with acetaminophen, acetaminophen with codeine and placebo for relief of postsurgical dental pain. J Clin Dent. 1988;1:31–34. [PubMed] [Google Scholar]
  32. Cooper SA, Kupperman A. The analgesic efficacy of flurbiprofen compared to acetaminophen with codeine. J Clin Dent. 1991;2:70–74. [PubMed] [Google Scholar]
  33. Desjardins PJ, Cooper SA, Finizio T. Efficacy of low dose combination analgesics: acetaminophen/codeine, aspirin/butalbital/caffeine/codeine, and placebo in oral surgery pain. Anesth Prog. 1986;33:143–146. [PMC free article] [PubMed] [Google Scholar]
  34. Dionne RA, Snyder J, Hargreaves KM. Analgesic efficacy of flurbiprofen in comparison with acetaminophen, acetaminophen plus codeine, and placebo after impacted third molar removal. J Oral Maxillofac Surg. 1994;52:919–924. doi: 10.1016/s0278-2391(10)80068-0. [DOI] [PubMed] [Google Scholar]
  35. Forbes JA, Beaver WT, White EH, White RW, Neilson GB, Shackleford RW. Diflunisal. A new oral analgesic with an unusually long duration of action. JAMA. 1982;248:2139–2142. doi: 10.1001/jama.248.17.2139. [DOI] [PubMed] [Google Scholar]
  36. Forbes JA, Kolodny AL, Beaver WT, Shackleford RW, Scarlett VR. A 12 hour evaluation of the analgesic efficacy of diflunisal, acetaminophen, and acetaminophen codeine combination, and placebo in postoperative pain. Pharmacotherapy. 1983;3:47S–54S. [PubMed] [Google Scholar]
  37. Forbes JA, Bates JA, Edquist IA, Burchfield WH, Smith FG, Schwartz MK, Kit V, Hyatt J, Bell WE, Beaver WT. Evaluation of two opioid-acetaminophen combinations and placebo in postoperative oral surgery pain. Pharmacotherapy. 1994;14:139–146. [PubMed] [Google Scholar]
  38. Forbes JA, Jones KF, Smith WK, Gongloff CM. Analgesic effect of an aspirin codeine butalbital caffeine combination and an acetaminophen codeine combination in postoperative oral surgery pain. Pharmacotherapy. 1986;6:240–247. doi: 10.1002/j.1875-9114.1986.tb03483.x. [DOI] [PubMed] [Google Scholar]
  39. Forbes JA, Butterworth GA, Burchfield WH, Yorio CC, Selinger LR, Rosenmertz SK, Beaver WT. Evaluation of flurbiprofen, acetaminophen, an acetaminophen-codeine combination, and placebo in postoperative oral surgery pain. Pharmacotherapy. 1989;9:322–330. doi: 10.1002/j.1875-9114.1989.tb04144.x. [DOI] [PubMed] [Google Scholar]
  40. Forbes JA, Butterworth GA, Burchfield WH, Beaver WT. Evaluation of ketorolac, aspirin, and an acetaminophen-codeine combination in postoperative oral surgery pain. Pharmacotherapy. 1990;10:77S–93S. [PubMed] [Google Scholar]
  41. Forbes JA, Kehm CJ, Grodin CD, Beaver WT. Evaluation of ketorolac, ibuprofen, acetaminophen, and an acetaminophen codeine combination in postoperative oral surgery pain. Pharmacotherapy. 1990;10:94S–105S. [PubMed] [Google Scholar]
  42. Heidrich G, Slavic Svircev V, Kaiko RF. Efficacy and quality of ibuprofen and acetaminophen plus codeine analgesia. Pain. 1985;22:385–397. doi: 10.1016/0304-3959(85)90044-2. [DOI] [PubMed] [Google Scholar]
  43. Honig S, Murray KA. An appraisal of codeine as an analgesic: single dose analysis. J Clin Pharmacol. 1984;24:96–102. doi: 10.1002/j.1552-4604.1984.tb02771.x. [DOI] [PubMed] [Google Scholar]
  44. Pande AC, Pyke RE, Greiner M, Cooper SA, Benjamin R, Pierce MW. Analgesic efficacy of the K-receptor agonist, Enadoline, in dental surgery pain. Clin Neuropharmacol. 1996;19:92–97. doi: 10.1097/00002826-199619010-00009. [DOI] [PubMed] [Google Scholar]
  45. Sunshine A, Marrero I, Olson N, McCormick N, Laska EM. Comparative study of flurbiprofen, zomepirac sodium, acetaminophen plus codeine, and acetaminophen for the relief of postsurgical dental pain. Am J Med. 1986;80:50–54. doi: 10.1016/0002-9343(86)90111-7. [DOI] [PubMed] [Google Scholar]
  46. Turek MD, Baird WM. Double blind parallel comparison of ketoprofen (Orudis), acetaminophen plus codeine, and placebo in postoperative pain. J Clin Pharmacol. 1988;28:S23–8. doi: 10.1002/j.1552-4604.1988.tb05973.x. [DOI] [PubMed] [Google Scholar]
  47. Quiding H, Haggquist SO. Visual analogue scale and the analysis of analgesic action. Eur J Clin Pharmacol. 1983;24:475–478. doi: 10.1007/BF00609889. [DOI] [PubMed] [Google Scholar]
  48. Ahlstrom U, Fahraeus J, Quiding H, Strom C. Multiple doses of paracetamol plus codeine taken immediately after oral surgery. Eur J Clin Pharmacol. 1985;2:693–696. doi: 10.1007/BF00547051. [DOI] [PubMed] [Google Scholar]
  49. Jacobson J, Bertilson SO. Analgesic efficacy of paracetamol/codeine and paracetamol/dextropropoxyphene in pain after episiotomy and ruptures in connection with childbirth. J Int Med Res. 1987;15:89–95. doi: 10.1177/030006058701500205. [DOI] [PubMed] [Google Scholar]
  50. Sagne S, Henrikson P, Kahnberg K, Thilander H, Bertilson S. Analgesic efficacy and side effect profile of paracetamol/codeine and paracetamol/dextropropoxyphene after surgical removal of a lower wisdom tooth. J Int Med Res. 1987;15:83–88. doi: 10.1177/030006058701500204. [DOI] [PubMed] [Google Scholar]
  51. Breivik EK, Bjornsson GA. Variation in surgical trauma and baseline pain intensity: effects on assay sensitivity of an analgesic trial. Eur J Oral Sci. 1998;106:844–852. doi: 10.1046/j.0909-8836.1998.eos106403.x. [DOI] [PubMed] [Google Scholar]
  52. Breivik EK, Haanaes HR, Barkvoll P. Upside assay sensitivity in a dental pain model. Eur J Pain. 1998;2:179–186. doi: 10.1016/s1090-3801(98)90010-6. [DOI] [PubMed] [Google Scholar]
  53. Moore RA. Understanding clinical trials: what have we learned from systematic reviews? In Progress in Pain Research & Management Edited by Devor M, Rowbotham MC, Weisenfeld-Hallin Z IASP Press, Seattle, 2000. pp. 757–770.
  54. Collins SL, Moore RA, McQuay HJ, Wiffen PJ. Oral ibuprofen and diclofenac in postoperative pain: a quantitative systematic review. Eur J Pain. 1998;2:285–291. doi: 10.1016/s1090-3801(98)90027-1. [DOI] [PubMed] [Google Scholar]
  55. Smith LA, Carroll D, Edwards JE, Moore RA, McQuay HJ. Single dose ketorolac and pethidine in acute postoperative pain: a systematic review with meta-analysis. Br J Anaesth. 2000;84:48–58. doi: 10.1093/oxfordjournals.bja.a013381. [DOI] [PubMed] [Google Scholar]
  56. Benson K, Hartz AJ. A comparison of observational studies and randomized, controlled trials. New England Journal of Medicine. 2000;342:1878–1886. doi: 10.1056/NEJM200006223422506. [DOI] [PubMed] [Google Scholar]
  57. Concato J, Shah N, Horwitz RI. Randomized, controlled trials, observational studies, and the hierarchy of research designs. New England Journal of Medicine. 2000;342:1887–1892. doi: 10.1056/NEJM200006223422507. [DOI] [PMC free article] [PubMed] [Google Scholar]
  58. MacLehose RR, Reeves BC, Harvey IM, Sheldon TA, Russell IT, Black AMS. A systematic review of comparisons of effect sizes derived from randomised and non-randomised studies. Health Technology Assessment. 2000;4:34. [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Table 3
click here for additional Table 3 (15.5KB, xls)
Table 4
click here for additional Table 4 (27.5KB, xls)
Table 5
click here for additional Table 5 (13KB, xls)

Articles from BMC Medical Research Methodology are provided here courtesy of BMC

RESOURCES