Common pitfalls in statistical analysis: Absolute risk reduction, relative risk reduction, and number needed to treat

Abstract

In the previous article in this series on common pitfalls in statistical analysis, we looked at the difference between risk and odds. Risk, which refers to the probability of occurrence of an event or outcome, can be defined in absolute or relative terms. Understanding what these measures represent is essential for the accurate interpretation of study results.

INTRODUCTION

A crucial question when interpreting the result of a clinical trial is whether the size of observed effect of an intervention is clinically important. Several measures are used to quantify the “effect size” of an intervention – absolute risk reduction (ARR), relative risk reduction (RRR), and number needed to treat (NNT). A clear understanding of these terms helps one make rational clinical decisions. In a previous article, we looked at the meanings of risk and relative risk.[1] Let us consider the same example of comparison of two treatments, ligation versus sclerotherapy, for the treatment of acute hemorrhage from esophageal varices [Table 1].

Table 1.

A randomized trial of sclerotherapy versus ligation for esophageal varices (hypothetical data)

ABSOLUTE AND RELATIVE RISK REDUCTION

In this hypothetical study, the risk of death in the ligation group was 28% (18 deaths among 64 patients) or 0.28, and that in the sclerotherapy group was 44% (29 deaths among 65 patients) or 0.44. This means that the absolute difference in risk of death in the two groups is 0.44–0.28 = 0.16 or 16%. In other words, ligation reduces the “absolute” risk of death by 16% as compared to sclerotherapy. This is also known as the “Absolute risk reduction (ARR)” or “risk difference,” and represents “the proportion of patients who are spared the adverse outcome as a result of having received the experimental rather than the control therapy.”

ARR has to be interpreted in the context of baseline risk. In the above example, the baseline risk of death was 44% and ligation with an ARR of 16%reducedthis risk to 28%, which is nearly two-third (=0.28/0.44 = 0.636 or 64%) of the baseline risk. This is a medium-sized “relative” effect. However, if the baseline risk of death had been 20%, then a 16% ARR would bring the risk down to 4%, i.e. to about one-fifth of the original risk, a much bigger change in relative terms. This interpretation of risk reduction in the context of baseline risk is termed as “relative risk reduction (RRR).”

RRR = ARR/risk in control group (baseline risk).

RRR is an estimate of the percentage of baseline risk that is removed as a result of the new therapy. The problem with using RRR is that we cannot assess the actual effect size if the event rate in the control group is not known. A particular RRR may thus imply very different ARRs, depending on the baseline risk. For instance, a 50% RRR may represent an ARR of 40% (if the absolute risk comes down from 80% to 40%), a major effect, or of only 1% (if the absolute risk comes down from 2% to 1%), probably an inconsequential effect size) [Table 2].

Table 2.

Relationship between absolute risk reduction, relative risk reduction, and number needed to treat depending on different baseline risks and risk reduction

NUMBER NEEDED TO TREAT

NNT = 1/ARR.

In the above example, if 100 patients are treated with sclerotherapy, then 44 of them are likely to die. However, if the same 100 patients are instead treated with ligation, then only 28 of them will die. Therefore, for every 100 patients treated with ligation instead of sclerotherapy, 16 patients will benefit (not experience an adverse outcome). Therefore, to benefit one patient, one needs to treat 100/16 = 6.25 patients with ligation instead of sclerotherapy. This is called the “number needed to treat (NNT).”

A study looks at several outcomes, each with its own ARR, RRR, and NNT. For instance, if our hypothetical study also looked at the number of patients having recurrent bleeding, one could have reported separate ARR, RRR, and NNT values for that outcome too.

RELATION OF NUMBER NEEDED TO TREAT WITH ABSOLUTE RISK REDUCTION AND RELATIVE RISK REDUCTION

NNT varies for different interventions depending on ARR. Obviously, the larger the ARR, the smaller the NNT, i.e., fewer the number of patients who have to be treated with the new intervention for one patient to benefit. On the other hand, knowledge of RRR does not allow one to estimate NNT directly. The concept of NNT has gained popularity in recent years, since it is simple to compute and easy to interpret clinically.

ABSOLUTE RISK REDUCTION OR RELATIVE RISK REDUCTION: WHICH SHOULD BE USED?

Physicians tend to over-estimate the efficacy of an intervention when results are expressed as relative measures rather than as absolute measures.[2] ARR (expressed along with baseline risk) is probably a more useful tool than RRR to express the efficacy of an intervention.[3] Thus, reporting of absolute measures is a must. The CONSORT statement for reporting of results of clinical trials recommends that both absolute and relative effect sizes should be reported.[4] It helps to report NNT in addition, since this is an easily-interpreted single indicator of clinical utility of an intervention.

BEYOND ABSOLUTE RISK REDUCTION, RELATIVE RISK REDUCTION, AND NUMBER NEEDED TO TREAT

When implementing the results of a trial in clinical practice, one needs to go one step further, i.e. look at ARR, RRR, and NNT in the context of cost-effectiveness (cost of the intervention versus severity of the outcome prevented). For example, if the cost of a complete sclerotherapy treatment is Indian Rupees International Normalized Ratio (INR) 20,000 per patient and the cost of ligation is INR 30,000 per patient, then if 6 patients need to be treated with ligation to prevent one adverse outcome, the additional expense for saving one life is INR 10,000 per patient × 6= INR 60,000, a justifiable expense.

Shermock looked at NNT and the cost of recombinant human erythropoietin to avoid one transfusion-related adverse event in critically ill patients.[5] They found that NNT was 5246 to avert one transfusion-related adverse event, 28,785 to avert a serious transfusion-related adverse event, and 81,000 to prevent one fatal transfusion-related adverse event. With the cost of each transfusion being around US dollars 900, this translated to costs of US dollars 4.7 million, 25.6 million, and 71.8 million, to prevent a transfusion-related adverse event, a serious transfusion-related adverse event, and a likely fatal transfusion-related adverse event, respectively, leading the authors to conclude that “erythropoietin does not appear to be an efficient use of limited resources for routine use in critically ill patients.”

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.