Skip to main content
PMC home page
BMC Medical Research Methodology logoLink to BMC Medical Research Methodology
. 2009 Jan 5;9:1. doi: 10.1186/1471-2288-9-1

Spontaneous improvement in randomised clinical trials: meta-analysis of three-armed trials comparing no treatment, placebo and active intervention

Lasse Theis Krogsbøll 1, Asbjørn Hróbjartsson 1, Peter C Gøtzsche 1,✉
PMCID: PMC2628943  PMID: 19123933

Abstract

Background

It can be challenging for patients and clinicians to properly interpret a change in the clinical condition after a treatment has been given. It is not known to which extent spontaneous improvement, effect of placebo and effect of active interventions contribute to the observed change from baseline, and we aimed at quantifying these contributions.

Methods

Systematic review and meta-analysis, based on a Cochrane review of the effect of placebo interventions for all clinical conditions. We selected all trials that had randomised the patients to three arms: no treatment, placebo and active intervention, and that had used an outcome that was measured on a continuous scale or on a ranking scale. Clinical conditions that had been studied in less than three trials were excluded.

Results

We analysed 37 trials (2900 patients) that covered 8 clinical conditions. The active interventions were psychological in 17 trials, physical in 15 trials, and pharmacological in 5 trials. Overall, across all conditions and interventions, there was a statistically significant change from baseline in all three arms. The standardized mean difference (SMD) for change from baseline was -0.24 (95% confidence interval -0.36 to -0.12) for no treatment, -0.44 (-0.61 to -0.28) for placebo, and -1.01 (-1.16 to -0.86) for active treatment. Thus, on average, the relative contributions of spontaneous improvement and of placebo to that of the active interventions were 24% and 20%, respectively, but with some uncertainty, as indicated by the confidence intervals for the three SMDs. The conditions that had the most pronounced spontaneous improvement were nausea (45%), smoking (40%), depression (35%), phobia (34%) and acute pain (25%).

Conclusion

Spontaneous improvement and effect of placebo contributed importantly to the observed treatment effect in actively treated patients, but the relative importance of these factors differed according to clinical condition and intervention.

Background

It can be challenging for patients and clinicians to properly interpret a change in the clinical condition after a treatment has been given. An improvement will often be ascribed to the treatment, although at least two other factors often play a role.

One factor is spontaneous improvement [1]. Many clinical conditions are self-limiting, e.g. headache, acute low back pain and the common cold, and most chronic disease symptoms fluctuate in intensity, e.g. rheumatoid arthritis, chronic low back pain and psoriasis. Patients will often seek medical attention when their symptoms are worst, and they are most likely to be included in randomised trials at this time. For the purpose of this paper, we regarded regression to the mean effects as being part of the spontaneous improvement. Regression to the mean occurs, for example, when a patient can only be included in a trial if the symptoms are worse than some threshold value; for statistical reasons, the value will then likely be lower at a later time [1,2].

The second factor is the effect of placebo. Patients may feel reassured, change their expectation, or re-interpret their symptoms once a treatment has been commenced. A Cochrane systematic review did not find large effects of placebo, but some effect in trials with patient-reported continuous outcomes, especially pain [3-5].

We have not found any previous reviews of the three main factors affecting the clinical course of patients included in randomised clinical trials: spontaneous improvement, effect of placebos and effect of active interventions (Fig. 1). We aimed at quantifying their relative contribution to change from baseline in randomised trials.

Figure 1.

Figure 1

Open in a new tab

Illustration of approximate contributions of spontaneous improvement and effect of placebo to the estimated effect of active interventions.

Methods

The Cochrane review of the effect of placebo interventions involved a thorough search for trials including a no-treatment arm and a placebo arm. We selected all trials from the updated Cochrane review of placebo interventions [5] that had randomised the patients to three arms: no treatment, placebo and active intervention, and that had used an outcome that was measured on a continuous scale or on a ranking scale. In order to permit analyses of separate clinical conditions, we excluded conditions studied in less than three trials.

Potentially eligible trial reports were read in full by one author (LK), who made preliminary decisions on inclusion and choice of outcome, and extracted the data. The authors of the Cochrane review (AH and PCG) checked the selections and the extracted data. Disagreements were resolved by discussion.

In the Cochrane review, patient-reported outcomes were preferred to observer-reported ones. For this study, we selected the outcome that we found most relevant, disregarding whether it was patient- or observer-reported. We made this decision by consensus; there was very little disagreement. In seven cases, the chosen outcome was different from that in the original review. An example is the selection of the well-known observer-reported Bech-Rafaelsen Melancholia Scale instead of the patient-reported Befindlichkeits-Skala.

Data extraction was done using a pilot-tested chart. For each trial, pre- and post-treatment means, standard deviations and group sizes were extracted for the three arms. Additional information extracted was: clinical condition, acute or chronic problem, name and range of scale used, and type of intervention (physical, pharmacological or psychological).

Meta-analysis was done using Comprehensive Meta Analysis [computer program] version 2.2.030, July 2006.

Standardized mean differences (SMD) with 95% confidence intervals were calculated for each trial. SMD is the difference in means divided by the pooled standard deviation. SMD was calculated as Hedges' g, with adjustment for small sample bias. A negative SMD usually implies a positive effect of the intervention, e.g. a lower pain score means less pain. However, in four trials, a large clinical score meant a beneficial effect, and we therefore changed the sign of the SMD before the analysis in these cases. Thus, a negative SMD in our analyses always means a beneficial effect. When standard deviations were missing, we used those from similar trials.

Due to the clinical diversity of the included patients, we did not investigate one treatment effect, but rather the mean of many different treatment effects. There was also substantial methodological heterogeneity, e.g. some trials did not have adequately concealed treatment allocation. We therefore used a random effects model for the analyses. The degree of heterogeneity was investigated with I2, which describes the percentage of the variability in effect estimates that is due to heterogeneity rather than sampling error [6].

It was not straightforward how to do the analyses, as we needed to compare the effects in the three groups with the condition at baseline. We analyzed the three treatment arms separately by comparing the post-treatment values with the values at baseline. These data were paired, but we analyzed them as if they were independent, as the presentation of data in the articles did not allow paired analyses. Thus, we accepted a moderate loss of statistical power by handling the paired data as unpaired and assumed that the effect of the ignored correlations between pre- and post-intervention measurements was the same in all situations. It should be noted that this approach leads to overestimation of the sampling error, and therefore to underestimation of the heterogeneity.

It was not possible to determine group sizes both pre- and post-treatment for all trials. We therefore used post-treatment sizes in the analyses, which has the advantage that treatment arms with relatively more dropouts receive less weight.

Ten trials had more than one active treatment. In the meta-analysis, these were entered as separate treatment arms and therefore contributed relatively more than trials with only one active treatment arm. However, the same would occur in trials with skewed randomisation ratios, and overall, the numbers of patients contributing to the results of the three treatment arms were not much different.

Results

In- and exclusion of trials

There were 118 trials in the Cochrane review with continuous outcome data. We excluded 61 trials: seven were two-armed; in 27 trials, the clinical condition had been studied in less than 3 trials; and 27 trials did not have a baseline assessment. Almost all of the trials without a baseline addressed acute conditions, for example acute pain during a procedure. Though such trials often had pre-treatment assessments they did not involve an assessment of pain experienced during the procedure, or the treatment was given before the painful procedure was initiated. Thus, we identified 57 eligible trials. Data necessary for meta-analyses could not be obtained from 14 trials, so we initially included 43 trials [7-49] (Fig. 2).

Figure 2.

Figure 2

Open in a new tab

Selection of trials for the review.

We found that the estimates for four hypertension trials [46-49] were unreliable for our purpose. Three of the four trials had run-in periods of 4 to 8 weeks before randomisation, which eliminates the regression to the mean effect, and the changes from baseline were therefore very small and unstable.

We expected that the change from baseline in the no-treatment arm and the placebo arm would covary from trial to trial, so that when it was large in one arm, it also tended to be large in the other. We verified this, but with two clear outliers (Fig. 3, lower right corner). In one nausea trial [10], the placebo therapy consisted of talks about the child's daily life, which might have had a large reassuring effect. In the other trial [33], the smoking rate was monitored for one week before treatment in the placebo group, but not in the no-treatment group.

Figure 3.

Figure 3

Open in a new tab

Change from baseline in the no-treatment and the placebo arms of the 37 analysed trials. The results are shown as standardized mean differences.

Our overall results were very similar, whether or not we excluded the four hypertension trials and the two outliers, but we feel the results for nausea and smoking are more reliable without the outliers. We report below the results for 37 trials (2900 patients), after these six trials were excluded.

Characteristics of included trials

The 37 trials covered eight different clinical conditions. Most active interventions were of a psychological (17 trials) or physical nature (15 trials); 5 trials were of drugs. Typical psychological treatments were cognitive behaviour therapy and hypnosis, and physical treatment was often acupuncture. Only 10 trials investigated conditions defined by us as acute: depression [7-9], nausea [11,12], and acute pain [13-17], while 27 trials investigated chronic conditions: chronic pain [18-28], phobia [29-31], smoking [32,34], obesity [35-39] and insomnia [40-45]. Duration of treatment was highly variable, ranging from a few days to several months. The outcome was patient-reported in 26 trials and observer-reported in 11 trials.

Statistical analyses

Overall, across all conditions and interventions, there was a statistically significant change from baseline in all three arms (Table 1). The SMD was -0.24 (95% confidence interval -0.36 to -0.12, I2 = 25%) for no treatment, -0.44 (-0.61 to -0.28, I2 = 57%) for placebo, and -1.01 (-1.16 to -0.86, I2 = 57%) for active treatment. Thus, on average, the relative contributions of spontaneous improvement and of placebo to the change from baseline in the active intervention groups were 24% (0.24/1.01) and 20% ((0.44-0.24)/1.01), respectively (shown approximately in Fig. 1), but with wide variation related to the studied clinical conditions and interventions (Fig. 4). The most pronounced spontaneous improvements, relative to the change from baseline in the actively treated groups, were seen in nausea 45%, smoking 40%, depression 35%, phobia 34% and acute pain 25% (Fig. 4). When combining the influence of spontaneous remission and placebo, the similar proportions were for nausea 73%, smoking 59%, depression 43%, phobia 74%, and acute pain 23% (Fig. 4).

Table 1.

Standardized mean differences (SMD) for changes from baseline in the three treatment arms separately.

No. of patients No. of treatment arms SMD (95% confidence interval) I2 (%)
No treatment Overall 978 37 -0.24 (-0.36 to -0.12) 25
 Depression 42 3 -0.44 (-1.27 to 0.39) 70
 Nausea 60 2 -0.63 (-0.97 to -0.30) 0
 Pain – acute 63 5 -0.53 (-1.03 to -0.02) 49
 Pain – chronic 265 11 -0.10 (-0.27 to 0.06) 0
 Phobia 28 3 -0.39 (-0.91 to 0.12) 3
 Smoking 391 2 -0.40 (-0.54 to 0.26) 0
 Obesity 46 5 -0.02 (-0.41 to 0.36) 0
 Insomnia 83 6 0.03 (-0.27 to 0.33) 0
Placebo Overall 849 37 -0.44 (-0.61 to -0.28) 57
 Depression 44 3 -0.54 (-1.53 to 0.44) 80
 Nausea 59 2 -1.02 (-1.37 to -0.66) 0
 Pain – acute 62 5 -0.48 (-1.03 to 0.07) 57
 Pain – chronic 241 11 -0.32 (-0.59 to -0.04) 54
 Phobia 32 3 -0.85 (-1.35 to -0.36) 0
 Smoking 272 2 -0.58 (-1.37 to 0.22) 78
 Obesity 58 5 -0.17 (-0.52 to 0.18) 0
 Insomnia 81 6 -0.31 (-0.61 to 0.00) 0
Active treatment Overall 1073 52 -1.01 (-1.16 to -0.86) 57
 Depression 49 3 -1.27 (-1.91 to -0.62) 53
 Nausea 60 2 -1.39 (-2.58 to -0.23) 87
 Pain – acute 61 5 -2.12 (-3.01 to -1.24) 73
 Pain – chronic 373 17 -0.81 (-1.03 to -0.59) 52
 Phobia 48 5 -1.15 (-1.67 to -0.63) 33
 Smoking 275 3 -0.99 (-1.16 to -0.81) 0
 Obesity 61 5 -0.53 (-1.23 to 0.17) 73
 Insomnia 146 12 -1.06 (-1.30 to -0.82) 0
Open in a new tab

Figure 4.

Figure 4

Open in a new tab

Relative contributions of the spontaneous improvement, effect of placebo, and effect of active treatment to the change from baseline seen in the actively treated group.

The point estimates were very similar in trials with patient-reported and observer-reported outcomes (Table 2) whereas trials involving acute conditions tended to have larger improvements in all three arms compared with trials involving chronic conditions (Table 3), as expected.

Table 2.

Standardized mean differences (SMD) for changes from baseline grouped by patient- and observer-reported outcome.

No. of treatment arms SMD (95% confidence interval) I2 (%)
No treatment
Observer-reported 11 -0.28 (-0.56 to -0.01) 20
Patient-reported 26 -0.22 (-0.36 to -0.09) 30
Placebo
Observer-reported 11 -0.50 (-0.81 to -0.20) 40
Patient-reported 26 -0.42 (-0.62 to -0.22) 63
Active treatment
Observer-reported 16 -0.92 (-1.22 to -0.62) 56
Patient-reported 36 -1.04 (-1.22 to -0.87) 59
Open in a new tab

Table 3.

Standardized mean differences (SMD) for changes from baseline grouped by acute or chronic condition.

No. of treatment arms SMD (95% confidence Interval) I2 (%)
No treatment
Acute 10 -0.56 (-0.85 to -0.26) 41
Chronic 27 -0.24 (-0.33 to -0.14) 0
Placebo
Acute 10 -0.65 (-1.03 to -0.27) 65
Chronic 27 -0.37(-0.55 to -0.19) 54
Active treatment
Acute 10 -1.63 (-2.11 to -1.15) 73
Chronic 42 -0.88 (-1.03 to -0.74) 45
Open in a new tab

Discussion

We found that both the spontaneous improvement and the effect of placebo contributed importantly to the observed treatment effect in actively treated patients. As noted above, we have not found other reviews that describe the relative contributions of spontaneous remission and placebo to the improvement clinicians note when they treat patients.

Our findings have two implications. First, they underline that it is a fallacy when patients and clinicians interpret an improvement that occurs after a treatment has been instituted as being caused by that treatment. In fact, we found that, on average, only about half of that improvement could be ascribed to the treatment in the trials we analysed.

Second, our findings show that it is wrong to describe the effect that is observed in a placebo arm of a randomised trial as the effect of placebo, as it includes the spontaneous improvement that would also have occurred without administration of a placebo [50]. This error is very common. We did a full-text search on "placebo effect" on the BMJ's website on 30 April 2008 and found the error in 90% of the articles, even in an obituary.

It is a limitation of our study that a quarter of the eligible trials did not report the data necessary for our meta-analyses. Furthermore, we had to use an unconventional meta-analytic method but find it reassuring that the overall effect of placebo was -0.28, as this agrees closely with our previous estimate of -0.24 in the Cochrane review [5] where we used standard meta-analytic methods. We would not expect more elaborate methods to yield results that differ importantly from those we have reported here.

We considered other approaches and also did more traditional meta-analyses, comparing treatment arms within each trial after treatment and calculating ratios between the three arms before these ratios were pooled, but as the denominators of the per trial ratios had a distribution that crossed zero, these ratios were very unstable because of "division almost by zero" effects. Furthermore, we could not use this standard approach for the spontaneous improvement, as this required comparison with baseline. We did not try to convert our unpaired analyses into paired ones, as this would have required estimations of correlations that were likely to vary between diseases and interventions.

The relative contributions of spontaneous improvement, effects of placebo, and effects of active treatment to the observed change from baseline varied considerably. The eight clinical conditions we analysed were either psychiatric diseases (depression and phobia), involved a high degree of patient cooperation (smoking and obesity) or involved subjective outcomes (acute and chronic pain, nausea, and insomnia); and the interventions were mostly non-pharmacological. It seems likely that spontaneous improvement is more important in trials that include patients with high symptom scores and that do not implement a placebo run-in period, particularly as the regression to the mean is likely to be more pronounced in such settings.

Our Cochrane review suggested that the effect of placebos is smaller when imitating pharmacological interventions and when outcomes are observer-reported [3-5]. It is therefore likely that the effect of placebo is comparatively less important in drug trials and in trials with observer-reported outcomes. The Cochrane review found a small effect of placebo on pain, which we reproduced in this review for chronic pain, but not for acute pain, possibly because we were unable to include many acute pain trials that provided no baseline data.

The active interventions we included seemed to be quite effective, which is surprising, as most of them were unconventional, and as many trials involved acupuncture. We recently did a systematic review of three-armed acupuncture trials and found a small analgesic effect of acupuncture, compared to placebo acupuncture, that seems to lack clinical relevance and could not be clearly distinguished from bias [51]. The apparent effects we noted of active treatments may therefore to some degree reflect bias, e.g. related to unconcealed allocation of patients and unsuccessful blinding.

A major problem related to the interpretation of the outcomes in no-treatment and placebo groups is the lack of blinding. Blinding is important to reduce reporting bias in experiments with subjective outcomes [52], but it is not possible to blind patients who receive no treatment. The lack of blinding favours placebo [52], as patients were often blinded with respect to placebo and active treatment. Patients in the placebo group may think they receive active treatment, or they may tend to please their doctors by exaggerating the improvement, and conversely, patients in the no-treatment group may tend to view their experiences more negatively, as they may feel deprived of treatment.

Conclusion

We conclude that both the spontaneous improvement and the effect of placebo contribute importantly to the observed treatment effect in actively treated patients, and that the relative importance of these factors differ according to clinical condition and intervention.

Competing interests

The authors declare that they have no competing interests.

Authors' contributions

PCG and AH coined the idea, initiated the project, and wrote the protocol with LK; LK extracted data that were checked by PCG and AH; LK and PCG did the analyses; LK wrote the first draft of the paper, PCG and AH the final version. Guarantors: PCG and AH.

Funding

No funding.

Pre-publication history

The pre-publication history for this paper can be accessed here:

http://www.biomedcentral.com/1471-2288/9/1/prepub

Contributor Information

Lasse Theis Krogsbøll, Email: lassetk@hotmail.com.

Asbjørn Hróbjartsson, Email: ah@cochrane.dk.

Peter C Gøtzsche, Email: pcg@cochrane.dk.

Acknowledgements

We thank the statistical peer reviewer, Jesse Berlin, and a statistician we consulted, Peter Dalgaard, for very valuable advice.

References

  1. Morton V, Torgerson DJ. Effect of regression to the mean on decision making in health care. BMJ. 2003;326:1083–4. doi: 10.1136/bmj.326.7398.1083. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Gøtzsche PC. Rational Diagnosis and Treatment. Evidence-Based Clinical Decision-Making. 4. Chichester, Wiley; 2007. [Google Scholar]
  3. Hróbjartsson A, Gøtzsche PC. Is the placebo powerless? An analysis of clinical trials comparing placebo with no treatment. N Engl J Med. 2001;344:1594–1602. doi: 10.1056/NEJM200105243442106. [DOI] [PubMed] [Google Scholar]
  4. Hrobjartsson A, Gøtzsche PC. Placebo treatment versus no treatment. Cochrane Database Syst Rev. 2003. p. CD003974. [DOI] [PubMed]
  5. Hróbjartsson A, Gøtzsche PC. Placebo interventions for all clinical conditions. Cochrane Database Syst Rev. 2004. p. CD003974. [DOI] [PubMed]
  6. Higgins JPT, Green S, editors. Cochrane Handbook for Systematic Reviews of Interventions Version 5.0.0 [updated February 2008]. The Cochrane Collaboration. 2008. http://www.cochrane-handbook.org
  7. Nandi DN, Ajmany S, Ganguli H, Banerjee G, Boral GC, Ghosh A, Sarkar S. A clinical evaluation of depressives found in a rural survey in India. Br J Psychiatry. 1976;128:523–7. doi: 10.1192/bjp.128.6.523. [DOI] [PubMed] [Google Scholar]
  8. Röschke J, Wolf C, Müller MJ, Wagner P, Mann K, Grözinger M, Bech S. The benefit from whole body acupuncture in major depression. J Affect Disord. 2000;57:73–81. doi: 10.1016/S0165-0327(99)00061-0. [DOI] [PubMed] [Google Scholar]
  9. Sumaya IC, Rienzi BM, Deegan JF, Moss DE. Bright light treatment decreases depression in institutionalized older adults: a placebo-controlled crossover study. J Gerontol A Biol Sci Med Sci. 2001;56(6):M356–M360. doi: 10.1093/gerona/56.6.m356. [DOI] [PubMed] [Google Scholar]
  10. Hawkins PJ, Liossi C, Ewart BW, Hatria P, Kosmidis VH, Varvutsi M. Hypnotherapy for control of anticipatory nausea and vomiting in children with cancer: preliminary findings. Psychooncology. 1995;4:101–6. doi: 10.1002/pon.2960040203. [DOI] [Google Scholar]
  11. O'Brien B, Relyea MJ, Taerum T. Efficacy of P6 acupressure in the treatment of nausea and vomiting during pregnancy. Am J Obstet Gynecol. 1996;174:708–15. doi: 10.1016/S0002-9378(96)70454-4. [DOI] [PubMed] [Google Scholar]
  12. Werntoft E, Dykes A. Effect of acupressure on nausea and vomiting during pregnancy. A randomized, placebo-controlled, pilot study. J Reprod Med. 2001;46:835–9. [PubMed] [Google Scholar]
  13. Cupal DD, Brewer BW. Effects of relaxation and guided imagery on knee strength, reinjury anxiety, and pain following anterior cruciate ligament reconstruction. Rehabil Psychol. 2001;46:28–43. doi: 10.1037/0090-5550.46.1.28. [DOI] [Google Scholar]
  14. Forster EL, Kramer JF, Lucy SD, Scudds RA, Novick RJ. Effects of TENS on pain, medications, and pulmonary function following coronary artery bypass graft surgery. Chest. 1994;106:1343–8. doi: 10.1378/chest.106.5.1343. [DOI] [PubMed] [Google Scholar]
  15. Helms JM. Acupuncture for the management of primary dysmenorrhea. Obstet Gynecol. 1987;69:51–6. [PubMed] [Google Scholar]
  16. Kober A, Scheck T, Greher M, Lieba F, Fleischhackl R, Fleischhackl S, Randunsky F, Hoerauf K. Prehospital analgesia with acupressure in victims of minor trauma: a prospective, randomized, double-blinded trial. Anesth Analg. 2002;95:723–7. doi: 10.1097/00000539-200209000-00035. [DOI] [PubMed] [Google Scholar]
  17. Sanders G, Tepe R, Maloney P, Reinert O. The effect of spinal manipulation on subjects with acute low back pain: a comparison of visual analog pain scores and serum beta endorphin levels. J Manipulative Physiol Ther. 1990;13:58. [PubMed] [Google Scholar]
  18. Alfano AP, Taylor AG, Foresman PA, Dunkl PR, McConnell GG, Conaway MR, Gillies GT. Static magnetic fields for treatment of fibromyalgia: a randomised controlled trial. J Altern Complement Med. 2001;7:53–64. doi: 10.1089/107555301300004538. [DOI] [PubMed] [Google Scholar]
  19. Blanchard EB, Appelbaum KA, Radnitz CL, Michultka D, Morrill B, Kirsch C, Hillhouse J, Evans DD, Guarnieri P, Attanasio V, Andrasik F, Jaccard J, Dentinger MP. Placebo-controlled evaluation of abbreviated progressive muscle relaxation and of relaxation combined with cognitive therapy in the treatment of tension headache. J Consult Clin Psychol. 1990;58:210–5. doi: 10.1037/0022-006X.58.2.210. [DOI] [PubMed] [Google Scholar]
  20. Blanchard EB, Appelbaum KA, Radnitz CL, Morrill B, Michultka D, Kirsch C, Guarnieri P, Hillhouse J, Evans DD, Jaccard J, Barron KD. A controlled evaluation of the thermal biofeedback and thermal biofeedback combined with cognitive therapy in the treatment of vascular headache. J Consult Clin Psychol. 1990;58:216–24. doi: 10.1037/0022-006X.58.2.216. [DOI] [PubMed] [Google Scholar]
  21. Chenard JR, Marchand S, Charest J, Jinxue L, Lavignolle B. Evaluation of a behavioral intervention for chronic low-back pain: 'The interactional back school' [Évaluation d'un traitement comportmental de la lombalgie chronique: 'l école interactionelle du dos'] Science et Comportement. 1991;21:225–39. [Google Scholar]
  22. Hong C, Chen Y, Pon CH, Yu J. Immediate effects of various physical medicine modalities on pain threshold of an active myofascial trigger point. J Musculoskeletal Pain. 1993;1:37–53. doi: 10.1300/J094v01n02_04. [DOI] [Google Scholar]
  23. Kotani N, Kushikata T, Suzuki A, Hashimoto H, Muraoka M, Matsuki A. Insertion of intradermal needles into painful points provides analgesia for intractable abdominal scar pain. Reg Anesth Pain Med. 2001;26:532–8. doi: 10.1053/rapm.2001.25897. [DOI] [PubMed] [Google Scholar]
  24. Leibing E, Leonhardt U, Köster G, Goerlitz A, Rosenfeldt JA, Hilgers R, Ramadori G. Acupuncture treatment of chronic low-back pain: a randomized, blinded, placebo-controlled trial with 9-months followup. Pain. 2002;96:189–96. doi: 10.1016/S0304-3959(01)00444-4. [DOI] [PubMed] [Google Scholar]
  25. Moffett JAK, Richardson PH, Frost H, Osborn A. A placebo controlled double blind trial to evaluate the effectiveness of pulsed short wave therapy for osteoarthritic hip and knee pain. Pain. 1996;67:121–7. doi: 10.1016/0304-3959(96)03100-4. [DOI] [PubMed] [Google Scholar]
  26. Parker JC, Smarr KL, Buckelew SP, Stucky-Ropp RC, Hewett JE, Johnson JC, Wright GE, Irvin WS, Walker SE. Effects of stress management on clinical outcomes in rheumatoid arthritis. Arthritis Rheum. 1995;38:1807–18. doi: 10.1002/art.1780381214. [DOI] [PubMed] [Google Scholar]
  27. Thomas VJ, Dixon AL, Milligan P. Cognitive-behaviour therapy for the management of sickle cell disease pain: an evaluation of a community based intervention. Br J Health Psychol. 1999;4:209–29. doi: 10.1348/135910799168588. [DOI] [Google Scholar]
  28. Wojciechowski FL. Behavioral treatment of tension headache: a contribution to controlled outcome research methodology. Gedrag – Tijdschrift voor Psychologie. 1984;12:16–30. [Google Scholar]
  29. Etringer BD, Cash TF, Rimm DC. Behavioral, affective and cognitive effects of participant modeling and an equally credible placebo. Behav Ther. 1982;13:476–85. doi: 10.1016/S0005-7894(82)80010-5. [DOI] [Google Scholar]
  30. Lick J. Expectancy, false galvanic skin response feedback and systematic desensitization in the modification of phobic behavior. J Consult Clin Psychol. 1975;43:557–67. doi: 10.1037/h0076894. [DOI] [PubMed] [Google Scholar]
  31. Rosen GM, Glasgow RE, Barrera M. A controlled study to assess the clinical effcacy of totally self-administrated systematic desensitization. J Consult Clin Psychol. 1976;44:208–17. doi: 10.1037/0022-006X.44.2.208. [DOI] [PubMed] [Google Scholar]
  32. Etter J, Lazlo E, Zellweger J, Perrot C, Perneger TV. Nicotine replacement to reduce cigarette consumption in smokers who are unwilling to quit: a randomized trial. J Clin Psychopharmacol. 2002;22:487–95. doi: 10.1097/00004714-200210000-00008. [DOI] [PubMed] [Google Scholar]
  33. Sipich JF, Russell RK, Tobias LL. A comparison of covert sensitization and 'nonspecific' treatment in the modification of smoking behavior. J Behav Ther Exp Psychiatry. 1974;5:201–3. doi: 10.1016/0005-7916(74)90115-3. [DOI] [Google Scholar]
  34. Spanos NP, Mondoux TJ, Burgess CA. Comparison of multi-component hypnotic and non-hypnotic treatments for smoking. Contemp Hypnosis. 1995;12:12–19. [Google Scholar]
  35. Antonio J, Colker CM, Torina G, Shi Q, Brink W, Kalman D. Effects of standardised guggulsterone phosphate supplement on body composition in overweight adults: a pilot study. Curr Ther Res. 1999;60:220–7. doi: 10.1016/S0011-393X(00)88517-3. [DOI] [Google Scholar]
  36. Block J. Effects of rational emotive therapy on overweight adults. Psychotherapy: Theory, Research and Practice. 1980;17:277–80. doi: 10.1037/h0085923. [DOI] [Google Scholar]
  37. Colker CM, Kalman DS, Torina GC, Perlis T, Street C. Effects of Citrus aurantium extract, caffeine, and St. John's wort on body fat loss, lipid levels, and mood states in overweight healthy adults. Curr Ther Res. 1999;60:145–53. doi: 10.1016/S0011-393X(00)88523-9. [DOI] [Google Scholar]
  38. Roongpisuthipong C, Panpakdee O, Boontawee A, Kulapongse S, Tanphaichitr V. Possible thermogenesis with dexfenfluramine. J Med Assoc Thai. 1999;82(2):150–159. [PubMed] [Google Scholar]
  39. Senediak C, Spence SH. Rapid versus gradual scheduling of therapeutic contact in a family based behavioural weight control programme for children. Behav Psychother. 1985;13:265–87. [Google Scholar]
  40. Ascher LM, Turner RM. Paradoxical intention and insomnia: an experimental investigation. Behav Res Ther. 1979;17:408–11. doi: 10.1016/0005-7967(79)90015-9. [DOI] [PubMed] [Google Scholar]
  41. Espie CA, Lindsay WR, Brooks DN, Hood EM, Turvey T. A controlled comparative investigation of psychological treatments for chronic sleep-onset insomnia. Behav Res Ther. 1989;27:79–88. doi: 10.1016/0005-7967(89)90123-X. [DOI] [PubMed] [Google Scholar]
  42. Lick JR, Heffler D. Relaxation training and attention placebo in the treatment of severe insomnia. J Consult Clin Psychol. 1977;45:153–61. doi: 10.1037/0022-006X.45.2.153. [DOI] [PubMed] [Google Scholar]
  43. Nicassio P, Bootzin R. A comparison of progressive relaxation and autogenic training as treatments for insomnia. J Abnorm Psychol. 1974;83(3):253–260. doi: 10.1037/h0036729. [DOI] [PubMed] [Google Scholar]
  44. Tsay SL, Chen ML. Acupressure and quality of sleep in patients with end-stage renal disease: a randomised controlled trial. Int J Nurs Stud. 2003;40:1–7. doi: 10.1016/S0020-7489(02)00019-6. [DOI] [PubMed] [Google Scholar]
  45. Turner RM, Ascher LM. Controlled comparison of progressive relaxation, stimulus control, and paradoxical intention therapies for insomnia. J Consult Clin Psychol. 1979;47:500–8. doi: 10.1037/0022-006X.47.3.500. [DOI] [PubMed] [Google Scholar]
  46. Canino E, Cardona R, Monsalve P, Acuna FP, López B, Fragachan F. A behavioral treatment program as a therapy in the control of primary hypertension. Acta Cient Venez. 1994;45:23–30. [PubMed] [Google Scholar]
  47. Frankel BL, Patel DJ, Horwitz D, Friedewald WT, Gaarder KR. Treatment of hypertension with biofeedback and relaxation techniques. Psychosom Med. 1978;40:276–93. doi: 10.1097/00006842-197806000-00002. [DOI] [PubMed] [Google Scholar]
  48. Seer P, Raeburn JM. Meditation training and essential hypertension: a methodological study. J Behav Med. 1980;3:59–71. doi: 10.1007/BF00844914. [DOI] [PubMed] [Google Scholar]
  49. Yates RG, Lamping DL, Abram NL, Wright C. Effects of chiropractic treatment on blood pressure and anxiety: a randomized, controlled trial. J Manipulative Physiol Ther. 1988;11:484–8. [PubMed] [Google Scholar]
  50. Hróbjartsson A. What are the main methodological problems in the estimation of placebo effects? J Clin Epidemiol. 2002;55:430–5. doi: 10.1016/S0895-4356(01)00496-6. [DOI] [PubMed] [Google Scholar]
  51. Madsen MV, Gøtzsche PC, Hróbjartsson A. Acupuncture treatment for pain. Systematic review of randomized clinical trials with acupuncture, placebo acupuncture and no-acupuncture groups. BMJ. in press . [DOI] [PMC free article] [PubMed]
  52. Wood L, Egger M, Gluud LL, Schulz KF, Jüni P, Altman DG, Gluud C, Martin RM, Wood AJ, Sterne JA. Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta-epidemiological study. BMJ. 2008;336:601–5. doi: 10.1136/bmj.39465.451748.AD. [DOI] [PMC free article] [PubMed] [Google Scholar]

Articles from BMC Medical Research Methodology are provided here courtesy of BMC

RESOURCES