Interpretation of Treatment Effects in Periodontal Research: A Note on the Number Needed to Treat

In clinical research, investigators are interested in determining whether new interventions are safer and more effective than standard therapies. In clinical practice, dental practitioners must interpret research findings to determine whether new therapeutic approaches should be incorporated into practice. The effect of a new treatment, relative to that of the standard of care or a placebo control, can be presented in many different ways using a variety of summary statistics. For example, in periodontal research, the effect of an intervention relative to that of a control may be based on differences between the study groups in the proportion of sites with ≥ 2 mm attachment loss (risk difference), ratios of the proportions of sites with ≥ 2 mm attachment loss (relative risk), or differences in mean attachment loss. Another measure of treatment effect, used to summarize the clinical benefit of a treatment, is the number needed to treat (NNT).^1,2 In the context of dental research, NNT is defined as the number of sites that must be treated with the intervention to avoid one additional site with progressive disease compared to the control. The statistical and clinical significance of the estimated treatment effect, the safety profile and the feasibility of delivering the intervention are all used to determine whether an investigational treatment should be adopted into practice.

Many patients with periodontitis will have only a small number of sites with active disease demonstrating disease progression over the study period and hence only a small number of sites that may be responsive to treatment. In such patient populations, rates of disease progression and mean changes in measures such as probing depth and clinical attachment level over the treatment period are very low. It is important to understand how such low rates of disease progression influence estimates of treatment effects. This paper builds on the existing NNT literature by illustrating the influence of low disease-progression rates on calculations of NNT in periodontal research.

NNT in Periodontal Research

The NNT to avoid one additional site with progressive disease under the intervention compared with the control arm has been described as a useful summary of the clinical benefit of a treatment.^1,2 Greenstein and Nunn³ have presented details about the calculation and interpretation of NNT in periodontal research, and the meta-analysis literature⁴ has discussed the influence of low progression rates on calculated values of NNT. The discussion below further illustrates the influence of low progression rates on NNT in the setting of periodontal research, a topic touched on only briefly by Greenstein and Nunn.³

If P_C denotes the proportion of sites in the control arm demonstrating progression and P_T the proportion of sites in the treatment arm demonstrating progression, NNT is calculated as the inverse of the difference in disease-progression rates (the risk difference) between the control group and the treatment group:

$$\mathrm{NNT}=\frac{1}{P_C-P_T}$$

As an example, consider a study by Caton and others,⁵ who compared the use of subantimicrobial-dose doxycycline (SDD) in adult (chronic) periodontitis as an adjunct to scaling and root planing (SRP) with placebo plus SRP, as discussed by Greenstein and Nunn.³ Study end points included progression of periodontitis (defined as ≥ 2 mm loss of clinical attachment) over a 9-month treatment period. Among sites with an initial probing depth of at least 7 mm, the reported risk of attachment loss ≥ 2 mm was 0.3% for the SDD plus SRP group and 3.6% for the placebo plus SRP group. The risk difference is 3.3%, which results in a number of sites needed to treat of 31, after rounding up. Therefore, 31 sites on average would need to be treated with the combination of SDD and SRP to avoid periodontitis progression at one additional site relative to treatment with SRP plus placebo.

Influence of Low Periodontitis Progression Rates on NNT

As noted by Hujoel and others,⁶ the choice of statistical measure to summarize a treatment effect is important in periodontal research, given the low rates of periodontitis progression. NNT is based on the difference in progression rates between the treatment and control arms. In patient populations with low progression rates, differences in progression rates between the treatment and control arms will be small, and the NNT will necessarily be large. Figure 1 summarizes the association between the progression rate in the control group (P_C) and the NNT with various treatment effect sizes, identified by percent (risk) reductions with treatment. For example, if the proportion of sites demonstrating disease progression is 0.10 in the control group and 0.08 in the treatment group (a 20% relative reduction in the risk of progression), the NNT is 50. As shown in Fig. 1, the NNT increases as the progression rate in the control group decreases for a given relative reduction. In patient populations where the treatment reduces disease-progression rates relative to control, the minimum value of NNT is the inverse of the progression rate in the control group. The minimum NNT, shown in Fig. 1, is observed when the treatment reduces the disease-progression rate to 0 (a 100% risk reduction). For example, if the progression rate in the control group is 5%, the NNT must be at least 20.

Figure 1.

Relation between the number of sites needed to treat and the probability of disease progression in the control arm under different sizes of treatment effect. The treatment effect sizes are expressed in terms of the reduction in risk of progression for the treatment group relative to the risk of progression for the control group.

Relative Rates of Periodontitis Progression

As Hujoel and others⁶ have noted, the relative risk (the ratio of progression rates in the treatment and control groups) is a useful summary of treatment effect and is not influenced by disease-progression rates in the same way that the risk difference (and hence NNT) is influenced. If the progression rate is low in the control group and is even lower in the treatment arm, the NNT will necessarily be large. The estimate of relative risk, on the other hand, can take on any value greater than or equal to 0, regardless of the progression rate in the control group.

For example, in the study by Caton and others,⁵ discussed previously, the relative risk was 0.08, which implies that the risk of attachment loss was 92% lower for the SDD plus SRP group than for the placebo plus SRP group. The relative risk reduction (92%) should be interpreted in light of the estimated disease-progression rates (0.3% versus 3.6%) to judge clinical significance. For example, reduction of the disease-progression rate from 10% to 0.8%, also corresponding to a relative risk reduction of 92%, may be more important clinically than a reduction from 3.6% to 0.3%.

Conclusions

When interpreting the treatment effect of a particular intervention in a setting with low progression rates, it is important to keep in mind the influence of those low progression rates on the values of the summary measures. In particular, the NNT will necessarily be large when progression rates are small. Relative risk estimates, on the other hand, are not similarly influenced by the magnitude of the rates and therefore should be an integral part of the analysis of treatment effect.⁶ Overinterpretation of the treatment effect on the basis of relative risk estimates can be avoided by also reporting the progression rates in each group.

Biographies

Dr. Stoner is an associate professor of biostatistics in the department of biostatistics and epidemiology, University of Oklahoma Health Sciences Center College of Public Health, Oklahoma City, Oklahoma.

Dr. Payne is associate dean for research, F. Gene and Rosemary Dixon Endowed Chair in Dentistry, and professor in the department of surgical specialties, University of Nebraska Medical Center College of Dentistry, Lincoln, Nebraska.