Effect of regression to the mean on decision making in health care

Short abstract

Knowledge of regression to the mean can help with everything from interpreting test results to improving your career prospects. All healthcare professionals should be aware of its implications

Regression to the mean is a widespread statistical phenomenon with potentially serious implications for health care. It can result in wrongly concluding that an effect is due to treatment when it is due to chance. Ignorance of the problem will lead to errors in decision making. We discuss the importance of the issue and its effects on many common clinical, public health, and managerial decisions.

What is regression to the mean?

Regression to the mean occurs whenever a nonrandom sample is selected from a population and two imperfectly correlated variables are measured, such as two consecutive blood pressure measurements. The less correlated the two variables, the larger the effect of regression to the mean. Also, the more extreme the value from the population mean, the more room there is to regress to the mean. It occurs whenever a group is selected with extreme values for one variable and another variable is then measured. ¹,²

Francis Galton documented the phenomenon in 1886. Galton measured the height of 930 adult children and their parents and calculated the average height of the parents. He noted that when the average height of the parents was greater than the mean of the population, the children tended to be shorter then the parents. Likewise, when the average height of the parents was shorter than the population mean, the children tended to be taller than their parents. Galton called this phenomenon regression towards mediocrity, and it is now known as regression to the mean. ³

Ignorance of this phenomenon is widespread. Pilot instructors noted that when a trainee pilot was praised for a good landing they invariably made a subsequent poor landing. This was misinterpreted as praise lulling pilots into complacency when the real explanation was regression towards the mean. ⁴ All healthcare professionals need to be aware of regression to the mean as it has wide ranging effects.

Diagnostic tests

Clinicians use diagnostic tests to target and monitor treatment. Regression to the mean can confound this strategy. The preliminary test has a high probability of giving an abnormal result through chance, and initial treatment may be unnecessary. Because of this chance effect, there is a high probability that subsequent measurements will spontaneously regress towards the mean value. This misleads clinicians and patients into thinking that treatment has been effective when the treatment was either not required or ineffective.

Summary points

Regression to the mean affects all aspects of health care

Any intervention aimed at a group or characteristic that is very different from the average will appear to be successful because of regression to the mean

In clinical practice, the phenomenon can lead to misinterpretation of results of tests, new treatments, and the placebo effect

Public health interventions are often aimed at sudden increases in disease and thus vulnerable to the effects of regression to the mean

The figure shows the effect of regression to the mean in women treated for osteoporosis. Some women continue to lose bone at the first follow up measurement despite effective treatment. ⁵ It is tempting to assume that treatment is ineffective in those women who are losing bone. However, because of regression to the mean, most patients (> 80%) who lost bone in the first year of treatment went on to gain bone in the second year despite no change in treatment.

Figure 1.

Percentage changes in bone mineral density among women treated with alendronate or raloxifene after 12 and 24 months, showing regression to the mean at 24 months. Women are grouped according to the percentage change at 12 months, and values for 24 months are percentage change from value at 12 months

The solution to this problem depends on the cost and complexity of the test. Firstly, monitoring may be unnecessary. In the case of bone density measurements, one solution is to forego monitoring and change a patient's treatment only on clinical grounds, such as intolerance. When monitoring of treatment is less expensive, such as measuring blood pressure, the effect of regression to the mean can be reduced by taking serial measurements and calculating the average change.

New treatments

When new treatments become available, some clinicians may yield to the temptation of trying out the treatment on the patients who are most ill. This understandable desire to treat clinical outliers will usually produce a gratifying and sharp response to treatment because of regression to the mean. Furthermore, if the relevant clinical trials excluded patients who were resistant to treatment and clinicians use the treatment outside the licensed conditions, they may get the mistaken impression that the new treatment is even better among such patients.

Placebo effect

Trials of hormone replacement therapy show a strong placebo effect on menopausal symptoms. ⁶ This implies that menopausal symptoms are susceptible to placebo treatment. However, a recent systematic review of placebo versus “open” no treatment found little evidence for the placebo effect. ⁷ A more likely explanation is that the placebo effect is simply regression to the mean. Women recruited to trials of hormone replacement therapy typically score highly on a symptom index. Because the trialists are identifying women with relatively extreme menopausal symptoms, once treatment starts, improvement will occur in both the placebo and active treatment groups because of regression to the mean. An indication that regression to the mean is occurring is that patients with the worst clinical scores have the biggest placebo effect.

Public health

Public health interventions are often driven by unexpected increases in incidence of disease. A classic example is the response to a sudden rise in traffic incidents. Because a sudden peak in road crashes is often due to chance, changes in policy, such as more rigorous enforcement of speed laws, will reduce crashes because of regression to the mean. The policy of vaccinating children against meningitis was introduced at a time of heightened incidence (see bmj.com). The headline benefit of a 75%-90% reduction in cases ⁸ is an overestimate as most of the reduction would have been due to regression to the mean.

If public health physicians wish to prove their worth, our advice is that they focus their efforts on a group of problems that are much worse than the national average or have shown an unexpected increase as there will usually be an improvement. It is important, however, to focus on a group of outliers to guarantee an effect because there is a chance that regression to the mean will not affect the results of a single outlier as it is a group phenomenon.

Healthcare management

Regression to the mean can justify league table initiatives for improving poorly performing hospitals. When poor hospitals are helped by allocating them more resources, regression to the mean will ensure that most will suddenly climb the league table. In contrast, hospitals at the top of the league table who are rewarded with increased resources for their efforts will fall in the table. If governments want to justify any initiative, it is better to target those at the bottom of the league than those at the top. For the individual hospital manager, the problem is more complex. Those who manage the worst hospitals are likely to see an improvement and thus enhance their careers. However, because regression to the mean is a group phenomenon, the improvement is not certain, and some hospitals will move in the opposite direction.

Clinical audit

An audit might identify patients that were operated on by a particular surgical team and had unexpectedly poor results, such as increased postoperative infections. Implementation of a policy of aggressive procedures to control infection will again often seem to work because of regression to the mean.

What are the solutions?

Understanding the phenomenon is a first step to overcoming the problems caused by regression to the mean. Whenever possible, policy should be based on evidence from trials. The effectiveness of management league tables, for example, could be tested by randomising poorly performing hospitals to new management or extra resources. This would tell us which intervention was most effective. In clinical practice, sequential testing to get an average value, which most doctors would do for blood pressure, is a solution for some tests.