Network Meta-Analysis of Studies Included in the Clinical Practice Guideline on the Nonsurgical Treatment of Chronic Periodontitis

Abstract

Aim

The recent ADA-commissioned Clinical Practice Guideline on the nonsurgical treatment of chronic periodontitis has provided the most exhaustive library of clinical trials on scaling and root planing (SRP) with or without adjuncts. This network meta-analysis compared the adjuncts against each other.

Materials and Methods

A star-shaped network meta-analysis was performed based on 36 indirect comparisons of clinical attachment level (CAL) gains among nine adjuncts in 74 studies from the Clinical Practice Guideline.

Results

All pairwise differences were accompanied by wide confidence intervals, and none of the adjuncts was statistically significantly superior to another. Local doxycycline hyclate and photodynamic therapy with a diode laser had the highest probabilities for ranking first and second, respectively. Publication bias was evident, with fewer than expected studies with small effects. The lack of these studies inflated the treatment effects by an estimated by 20%.

Conclusions

Adjuncts improve CAL gain by about a third of a mm over 6-12 months compared to SRP alone, but no significant differences were found among the adjuncts. The patient-perceived benefit of this gain is unclear because CAL is a physical measure made by the clinician and not a patient-oriented outcome. Publication bias inflated the observed treatment effects.

Background

The American Dental Association published “Evidence-Based Clinical Practice Guideline on the Nonsurgical Treatment of Chronic Periodontitis by Scaling and Root Planing with or without Adjuncts” (Smiley et al. 2015a, Smiley et al. 2015b), heretofore referred to as the “Guideline.” This Guideline is an evidence-based approach to the provision of care for chronic periodontitis patients.

While the Guideline provides recommendations regarding the use of various adjuncts to scaling and root planning (SRP), it does not compare the adjuncts to each other. However, many patients and dental practitioners face this exact question: which is the best treatment among the adjuncts? A network meta-analysis is a method to build on findings from a conventional meta-analysis. Whereas a typical meta-analysis compares two treatments in a particular population, the network meta-analysis compares more than two treatments, ideally all relevant treatment alternatives. Consequently, network meta-analyses have been recommended as the highest level of evidence for treatment guidelines.(Leucht et al. 2016) Therefore, comparing the nine adjunct treatments included in the Guideline to each other in a network meta-analysis can generate clinically meaningful comparative efficacy data.

In this study we aimed to provide comparative treatment effect estimates for non-surgical treatments of chronic periodontitis based on scaling and root planing with adjuncts utilizing a network meta-analysis of studies that were included in the recent Guideline.

Materials and Methods

Studies

To develop the Guidelines, potentially relevant articles from two electronic databases and selected systematic reviews were reviewed independently and in duplicate by multiple ADA panellists following a two phase approach according to PRISMA guidelines (for additional information on the search strategy, see Smiley et al. 2015 (Smiley et al. 2015a, Smiley et al. 2015b)). Change in clinical attachment level (CAL) was considered as the primary outcome measure. Only randomized controlled trials if they were published after 1960 through July 2014, written in English, and reported changes in CAL at least 6 months after randomization were included. Both parallel-arm and split-mouth studies were included, but studies of aggressive periodontitis were excluded. Experimental adjuncts, adjuncts not currently available in the United States, non-prescription (over-the-counter) adjuncts, adjuncts administered more than one week after SRP, and adjuncts reapplied to progressing (worsening) tooth sites were also excluded.

Of the original 72 studies included in the Guideline, we excluded 11 that compared SRP against an untreated control group. The remaining 61 randomized clinical trials contained 74 pairwise comparisons between SRP plus an adjunct versus SRP alone (i.e., some trials contained multiple treatment arms that were considered separately in the original Guideline document (Smiley et al. 2015a, Smiley et al. 2015b)). Other studies than trials included in the Guideline were not considered in this network meta-analysis because of a conceptual reason. We intended to make our network meta-analysis results complementary to the Guideline findings by using the same primary studies.

“Adjuncts” were defined in this article as a nonsurgical periodontitis treatment in addition to SRP and including systemic antimicrobials, systemic sub-antimicrobial dose doxycycline, locally applied antimicrobials (chlorhexidine chips, doxycycline hyclate gel, and minocycline microspheres), and non-surgical use of lasers (diode, both photodynamic and non-photodynamic therapies; Nd:YAG; and erbium).

Investigation of several treatment groupings representing different effects

The Guideline investigated nine adjuncts:

1. systemic host modulation (sub-antimicrobial dose doxycycline) (SDD)

2. systemic antimicrobials (ANTI)

3. chlorhexidine chips (CHX)

4. doxycycline hyclate gel (DH)

5. minocycline microspheres (MM)

6. photodynamic therapy [PDT] with a diode laser (PDT)

7. diode laser [non-PDT] (DL)

8. Nd: YAG lasers (NDL)

9. Erbium lasers (ERL)

Studies were investigated in a star-shaped (Figure 1) network meta-analysis (NMA) using the published data of the Guideline project (Smiley et al. 2015a, Smiley et al. 2015b)).

Figure 1.

Network plot of 74 studies in nine adjuncts (plus SRP) versus a common comparator SRP alone included in the Guideline Project (nodes and edges weighed according to the number of studies involved in each comparison of nine adjuncts versus SRP alone)

Here, nine adjuncts (plus SRP) were compared to a common comparator (SRP alone.) Because these 9 interventions are all connected in the network (i.e., each pair has a path from one to the other), 9^*8/2 = 36 indirect comparisons can be performed among the 9 interventions. The NMA was performed in a frequentist framework. Although the Guideline panel considered nine adjuncts, we further grouped them based on similarities in treatment approaches. This allowed us to provide an evolving picture how treatment effects change when progressively more treatments are combined until all are considered together. The following groupings were investigated:

1. Nine groups suggested by the expert panel of the Guideline. This categorization represents a grouping based on clinical application of the treatments.

2. Four groups based on the mechanism of action: 1. Host modulating agents; 2. Systemic antimicrobials; 3. Topical antimicrobials (chlorhexidine chips, doxycycline hyclate gel, minocycline microspheres, photodynamic therapy [PDT] with a diode laser); and 4. Lasers (diode laser [non-PDT], Nd: YAG lasers, Erbium lasers)

3. Three groups based on a more general mechanism of action: 1. Host-modulating agents; 2. Topical and systemic antimicrobials; and 3. Lasers

4. Two groups based on an even more general mechanism of action: We combined all antibiotics, antimicrobials and host modulating agents versus all laser treatments together.

5. All treatments combined, representing the non-specific treatment effect of adjuncts in general compared to SRP alone.

Data analysis

We first calculated treatment effects for the nine adjuvants (compared to SRP) based on the NMA and compared them to the treatment effects of the Guideline project. We anticipated that NMA treatment effects should be very similar to Guideline meta-analyses results because computations would only use different variance estimators to derive the summary estimates. We performed pairwise comparisons of treatments and computed, for each of the five grouping above, a CAL difference and 95% confidence intervals. Based on the 9^*8/2=36 treatment effects for the nine groups, i.e., grouping 1, we also ranked the interventions. In our frequentist approach to network meta-analysis (White 2011a, Lu & Ades 2004, Salanti et al. 2008, White 2011b), we used a parametric bootstrap procedure using 50,000 repetitions to allow for parameter uncertainty. (White et al. 2012) Ranking probabilities are presented in a cumulative bar chart, showing the probability that each of the 9 treatments rank first, second, third, and so on out of all nine interventions compared. We also calculated the Surface under the Cumulative RAnking (SUCRA) curve. SUCRA estimates the area under the curve underneath a cumulative ranking line starting with rank 1. Therefore, higher probabilities for better treatment ranks would result in a steeper ranking line and, hence, a large area under the ranking line. Larger SUCRA values represent better treatments on a scale from 0 to 1. SUCRA is a transformation of the mean rank of treatments, taking into account magnitude and variability of all relative treatment effects. (Salanti et al. 2011)

After comparing all nine treatments with each other (grouping 1), we derived treatment effects (CAL differences) and their 95% confidence intervals for groupings 2-5 (see above) with the intend to study whether larger groups of related treatments had different effects. Our approach was to combine treatments unless clinically relevant and statistically significant effects emerge that prohibit further combination.

We also investigated statistical heterogeneity when the number of studies was sufficiently large. The Cochrane Collaboration(Higgins & Green 2011) suggests that an I² value of 0% to 40% represents heterogeneity that might not be important. I² values of 30% to 60% may represent moderate heterogeneity, 50% to 90% substantial heterogeneity, and 75% to 100% considerable heterogeneity. We expected to find a moderate level of heterogeneity (I² of 60% or less) because of variations in baseline disease extent and severity among the studies, and because they were conducted over three decades and in 20 countries on 5 continents.

We also investigated whether publication bias exists. We used a contour-enhanced funnel plot(Palmer et al. 2008) to visually assess bias. Here, contours of statistical significance are overlaid on the funnel plot, presenting information whether studies appear missing in areas of low statistical significance. This situation would indicate that funnel asymmetry is likely due to publication bias. If publication appears to be present, as a next step, the missing studies could be imputed creating an unbiased data set and then perform the meta-analysis again talking apparently missing studies into account. The “trim and fill” method accomplishes this.(Duval & Tweedie 2000) For statistical assessment of funnel plot asymmetry, we also performed Egger’s test (a test for the Y intercept = 0 from a linear regression of normalized effect estimate against effect precision.)(Egger et al. 1997)

We used the statistical software Stata (Version 14, StataCorp, College Station, TX, USA) and user-written network meta-analyses programs including mvmeta (White 2011b, White 2011a) and network graphs(Chaimani et al. 2013, Chaimani & Salanti 2015).

Results

Comparison between NMA-based treatment effects and Guideline-presented treatment effects for nine adjuncts

Comparing the adjuncts’ estimates from the network meta-analysis (74 studies simultaneously analyzed) with those derived from the original random-effects model (74 studies analyzed in the 9 adjuncts groups (Smiley et al. 2015a, Smiley et al. 2015b)) showed small differences (≤0.09 mm CAL) and indicated that both meta-analytic approaches provided similar results when applied to the same research question (unshaded area of Table 1).

Table 1.

Pairwise comparisons (N=36) of 9 adjuncts (three largest differences in bold) included in the Guideline Project and summary estimates for the treatments + SRP versus SRP alone shown in the shaded area and summary estimates for the nine adjuncts treatments based on the network meta-analysis in comparison to the Guideline meta-analysis (unshaded area)

	1. Systemic host modu- lator	2. Systemic antimicro bials	3. Chlorhexi dine chips	4. Doxycycli ne hyclate gel	5. Minocycli ne microsph eres	6. Photody namic therapy [PDT] with a diode laser	7. Diode laser [non-PDT]	8. Nd: YAG lasers	9. Erbium laser
	CAL difference between treatments in mm (95% confidence interval)
2. Systemic antimicrobials	0.06 (−0.29, 0.41)
3. Chlorhexidine chips	0.08 (−0.39, 0.54)	0.02 (−0.37, 0.40)
4. Doxycycline hyclate gel	−0.22 (−0.91, 0.47)	−0.28 (−0.92, 0.36)	−0.29 (−1.00, 0.41)
5. Minocycline microspheres	0.09 (−0.44, 0.62)	0.03 (−0.44, 0.50)	0.01 (−0.54, 0.57)	0.31 (−0.45, 1.06)
6. Photodynamic therapy [PDT] with a diode laser	−0.13 (−0.56, 0.30)	−0.20 (−0.54, 0.15)	−0.21 (−0.67, 0.25)	0.08 (−0.61, 0.77)	−0.22 (−0.75, 0.31)
7. Diode laser [non-PDT]	0.22 (−0.34, 0.77)	0.15 (−0.34, 0.65)	0.14 (−0.44, 0.72)	0.43 (−0.34, 1.21)	0.12 (−0.51, 0.76)	0.35 (−0.20, 0.90)
8. Nd: YAG lasers	0.01 (−0.60, 0.62)	−0.05 (−0.61, 0.50)	−0.07 (−0.70, 0.56)	0.23 (−0.59, 1.04)	−0.08 (−0.77, 0.60)	0.14 (−0.47, 0.75)	−0.21 (−0.91, 0.50)
9. Erbium laser	0.20 (−0.46, 0.85)	0.13 (−0.47, 0.74)	0.12 (−0.56, 0.79)	0.41 (−0.44, 1.26)	0.10 (−0.62, 0.83)	0.33 (−0.32, 0.98)	−0.02 (−0.76, 0.72)	0.19 (−0.60, 0.97)
Summary estimate for treatment vs. SRP based on network meta-analysis	0.42* (0.11, 0.73)	0.36* (0.19, 0.53)	0.34 (0.00, 0.69)	0.64* (0.02, 1.26)	0.33 (−0.11, 0.77)	0.55* (0.25, 0.85)	0.20 (−0.26, 0.67)	0.41 (−0.12, 0.94)	0.22 (−0.36, 0.80)
Summary estimate for treatment vs. SRP based on Guideline meta-analysis	0.35** (0.15, 0.56)	0.35** (0.20, 0.51)	0.40** (0.24, 0.56)	0.64 (0.00, 1.28)	0.24 (− 0.06, 0.55)	0.53** (0.06, 1.00)	0.21 (−0.23, 0.64)	0.41 (− 0.12, 0.94)	0.18 (− 0.63, 0.98)

^*Significantly better than SRP in the network meta-analysis

^**Significantly better than SRP in conventional meta-analysis

Pairwise comparisons across nine adjuncts

The 36 pairwise treatment differences varied in absolute size from 0.01 to 0.43 mm CAL (shaded area in Table 1). All differences were accompanied by wide confidence intervals, and none of the adjuncts was statistically significantly superior to another. These pairwise differences between adjuncts corresponded to differences between 0.20 to 0.64 mm for adjuncts versus SRP alone. These latter treatment effects were larger because SRP is less effective than the same treatment plus an adjunct. Five of the nine differences were statistically significant (unshaded area in Table 1).

Treatment ranking

Doxycycline hyclate gel (DH) was ranked the best treatment by the network meta-analysis. It had the largest effect of 0.64 mm CAL compared to SRP alone (Table 1). It had a probability of 45% of being the best treatment, 15% of being the second, and 10% of being the third best treatment (Figure 2, left panel). The respective figures were 20%, 28%, and 20% for photodynamic therapy with a diode laser (PDT). The cumulative probability of being ranked in the top 3 was similar for both adjuncts (70% for DH and 68% for PDT). The SUCRA values yielded similar rankings. These values were largest for DH (78.5%) and PDT (78%) (Figure 2, right panel). The third highest SUCRA value was 61% for the systemic host modulator (SDD). Thus, both DH and PDT were top-ranked adjuncts using each of the two statistical ranking methods.

Figure 2.

Treatment rankings. Bar chart (left panel) of the probability for each of 9 adjuvant treatments and SRP alone for being the best to the worst in terms of CAL gain. Plots of the surface under the cumulative ranking curves, SUCRA (right panel) for 9 adjuvant treatments and SRP alone. Larger areas under the curve indicate better treatments.

1= systemic host modulation (sub-antimicrobial dose doxycycline) (SDD); 2= systemic antimicrobials (ANTI); 3= chlorhexidine chips (CHX); 4= doxycycline hyclate gel (DH); 5= minocycline microspheres (MM); 6= photodynamic therapy [PDT] with a diode laser (PDT); 7= diode laser [non-PDT] (DL); 8= Nd: YAG lasers (NDL); 9= Erbium lasers (ERL)

When all possible differences between 4, 3, and 2 treatment groups were analysed, differences decreased. The largest differences were 0.18 mm CAL for the 4 groups (shaded area of Table 2) and 0.14 mm CAL for 3 groups (shaded area of Table 3). The two groups (all laser treatments versus all other treatments) showed a difference of 0.12 (−0.19 to 0.44) mm CAL. All these differences were judged to be clinically insignificant based on the Guidelines’ judgment that 0.0 to 0.2 represents a “zero effect”. While confidence intervals became narrower, none of the differences became statistically significant, i.e., regardless of whether adjuncts were analysed as individual treatments or as groups of related treatments. We interpreted these findings as evidence against substantial differences across treatments and as rationale to combine all adjuncts into a single group.

Table 2.

Pairwise comparisons (N=6) of four groups of adjuvant periodontal treatments (largest difference in bold) included in the Guideline Project and summary estimates for the treatments + SRP versus SRP alone

	1. Systemic host modulator	2. Systemic antimicrobials	3. Topical antimicrobials	4. Lasers
	CAL difference between treatments in mm (95% confidence interval)
2. Systemic antimicrobials	0.06 (−0.28, 0.40)
3. Topical antimicrobials	−0.04 (−0.39, 0.32)	−0.10 (−0.35, 0.15)
4. Lasers	0.14 (−0.28, 0.56)	0.08 (−0.26, 0.42)	0.18 (−0.17, 0.52)
Summary estimate for treatment vs. SRP based on network meta-analysis*	0.42 (0.12, 0.72)	0.36 (0.19, 0.52)	0.45 (0.27, 0.64)	0.28 (−0.02, 0.57)

^*Guideline meta-analysis did not provide an estimate for these groupings

Table 3.

Pairwise comparisons (N=3) of three groups of adjuvant periodontal treatments (largest difference in bold) included in the Guideline Project and summary estimates for the treatments + SRP versus SRP alone

	1. Systemic host modulator	2. Antimicrobials	3. Lasers
	CAL difference between treatments in mm (95% confidence interval)
2. Antimicrobials	0.02 (−0.31, 0.34)
3. Lasers	0.14 (−0.28, 0.56)	0.12 (−0.20, 0.44)
Summary estimate for treatment vs. SRP based on network meta-analysis*	0.42 (0.12, 0.71)	0.40 (0.27, 0.52)	0.28 (−0.02, 0.57)

^*Guideline meta-analysis did not provide an estimate for these groupings

Combining all 74 studies considers the effect of a “global” adjunct (plus SRP) group versus SRP alone. The summary estimate was 0.35 (95% CI: 0.30 - 0.40) in a fixed effects model and 0.38 (95% CI: 0.28 - 0.48) mm in a random effects model. In the latter model, the standard deviation of the true effects was 0.33 mm CAL and the I² was 70%, representing “substantial” heterogeneity.

To identify sources of heterogeneity, we searched for outlier studies, i.e., studies that, when deleted, would decrease heterogeneity substantially. Deleting two studies (Pradeep and Ginanelli) reduced the I² to 49%, representing only “moderate” heterogeneity. Excluding these two studies, the summary estimates were reduced to 0.28 mm (95% CI: 0.23-0.33) in the fixed effects model and 0.32 mm (95% CI: 0.24-0.40) in the random effects model. In the latter model, the summary estimate was reduced in absolute terms by 0.06 mm CAL and in relative terms by 15% after excluding the two outlying studies.

Publication bias

The magnitude of publication bias for all adjuncts combined (but excluding Pradeep and Ginanelli to achieve moderate heterogeneity) is depicted in the funnel plot (Figure 3). Overall, there were fewer than expected small studies with negative effects (P value for Egger’s test for small study effects = 0.01). Following imputation of “missing” small studies using a trim-and-fill analysis, the overall treatment effects from the fixed and the random effects models were similar: 0.25 mm (95% CI: 0.20-0.30) and 0.26 mm (95% CI: 0.17-0.35), respectively. In the latter model, the mean estimate decreased 0.06 mm CAL in absolute terms and 20% in relative terms following imputation.

Figure 3.

Contour-enhanced funnel plot of the 72 studies (all studies of the Guidelines shown, but without Pradeep and Ginanelli to achieve moderate heterogeneity) plotting each study’s CAL effect versus its standard error as well as “trim and fill” imputed studies

Discussion

This meta-analysis led to three major findings; 1) adjuncts to scaling and root planning improved the response to scaling and root planning, measured by CAL gains, by about a third of a mm over 6 to 12 months; 2) there were no significant differences in CAL gains and acceptable study heterogeneity (after the two most influential studies were deleted) across the nine adjuncts evaluated in the recent ADA non-surgical periodontal therapy guidelines paper, suggesting that effects on CAL across disparate adjuncts may be similar; and 3) there was evidence for publication bias in the related literature. The magnitude of this bias is not trivial and has the potential to affect clinical decision making because the “corrected” overall treatment effect is about 20% smaller than the average effect reported in the literature.

Our first main finding is the magnitude of the overall effect of 0.32 mm CAL for adjuncts. How can the magnitude be interpreted? First, considering the effect in its original metric “mm”, the magnitude can be compared with the SRP effect alone. The adjuncts’ effect is about a third smaller than the effect of SRP alone, which was 0.49 mm. (Smiley et al. 2015a, Smiley et al. 2015b) Based on the expert-opinion clinical relevance scale for interpreting mean differences in CAL(Smiley et al. 2015a, Smiley et al. 2015b) SRP as a monotherapy yields a moderate effect (0.49 mm), while adjuncts contribute an additional small effect (0.32 mm). This is commensurate with the clinical practice guidelines that support employing SRP as the first-line treatment for chronic periodontitis. Nonetheless, adjuncts can yield a clinically impactful additional change in CAL. The clinical impact of this effect size can be further substantiated via its comparison with known modifiers of treatment outcome, such as smoking. For example, the 0.32 mm mean CAL gain following use of SPR adjuncts favourably compares to the negative effect of smoking on CAL gain both after non-surgical (0.11 mm)(Labriola et al. 2005) and surgical periodontal therapy (0.35 mm)(Kotsakis et al. 2015). Second, considering the treatment effect in a relative metric of its magnitude to its variability, an effect size (defined as mean effect divided by standard deviation of the effect) can be calculated that can be compared to guidelines. This standardized effect size was 0.92, which means the effect is almost as large as standard deviation of that effect. According to commonly applied guidelines from Cohen, an effect size >0.8 is considered a “large” effect.(Cohen 1998) Although the adjunct effect is statistically significant, its magnitude might have been substantially underestimated due to the use of whole-mouth measurements by the majority of the included studies. Since SRP alone benefits the majority of sites with moderate baseline probing depths,(Badersten et al. 1981) the additional effect of an adjunct in such sites would be minimized. Thus, the overall adjunct effect should shrink in studies reporting whole mouth averages. Inclusion of subgroup analyses according to baseline probing depth thresholds in future studies seems necessary to characterize the true effect size of SRP adjuncts and to shape the indications for their use. Third, CAL is just one indicator of periodontitis severity and other measures such as pocket depth (PD) exist. For a more comprehensive assessment, CAL and PD together would provide a more detailed outcome measurement, especially when performed over a long follow-up. Fourth and most importantly, a disease-oriented outcome, but patient-oriented outcomes are preferred for clinical decision-making.(Sacristan 2013) The patient-oriented approach to outcome measurement would also measure the total effect on the patients’ perceived oral health, not only the CAL gain. Oral health’s most comprehensive measure of perceived oral health is oral health-related quality of life. From a conceptual point of view, based in periodontitis’ direct effect on the periodontium and the indirect effects on tooth loss, all four dimensions of Oral Health-Related Quality of Life (OHRQoL) Oral Function, Orofacial Pain, Orofacial Appearance, and Psychosocial Impact(John et al. 2014b, John et al. 2014a) - core areas where patients perceive oral conditions’ and dental interventions’ impact – should be improved by adjuncts, in particular through prevention of tooth loss. CAL is related to OHRQoL, e.g., CAL ≥ 4mm (% of sites) was correlated with r=0.34(Eltas et al. 2016) to the summary score of OHIP-14(Slade 1997), a widely used OHRQoL instrument. CAL is predictive of tooth loss as well as denture status, and OHRQoL was substantially correlated with tooth loss(Gerritsen et al. 2010) and was associated with denture status(John et al. 2004). Therefore, the expected patient-perceived treatment effect from adjuncts should not be trivial even if considerable uncertainty about actual magnitude of the impact exists. In previous studies, periodontitis had a substantial effect on OHRQoL(Durham et al. 2013) and treatment studies showed an OHRQoL improvement over shorter time periods(Shanbhag et al. 2012), but is currently not well understood over a longer time. From a methodological perspective, use of patient-oriented outcomes in general, and OHRQoL assessment in particular are strongly encouraged to complement traditional disease-oriented outcome to more completely understand a treatment impact. In particular, future randomized trials evaluating effects of periodontal treatments in general and adjuncts specifically should include OHRQOL measures. Clinically feasible and widely comparable instruments ranging from 5(Naik et al. 2016) over 14(Slade 1997) to 49(Slade & Spencer 1994) items exist, providing pragmatic options to measure the patient-perceived impact with the desired level of precision.

Our second main finding is the homogeneity of the included trials. When we excluded the two most influential studies on the meta-analysis, the remaining 72 studies reached an acceptable level of heterogeneity considering their clinical and methodological diversity,(Higgins & Green 2011) i.e., they represented a level of variability that is compatible with an underlying common effect with acceptable variability so that treatment can be summarized in an overall effect. Support for not too large differences across studies, came from our pairwise comparisons of individual treatment and treatment groupings. Also, our definition what is the “best treatment” – one of the major goals of a network meta-analysis – needs some interpretation. It is based on network meta-analysis derived probabilities of treatment rankings. Our results indicated that doxycycline hyclate gel had with 45% the highest probability to be the best treatment as compared to photodynamic therapy [PDT] with a diode laser with only a 20% probability. This is a notable difference. However, when all treatment rankings (best, second, third, etc. best) were incorporated in the analysis, photodynamic therapy [PDT] with a diode laser had a similar overall treatment ranking compared with doxycycline hyclate gel, and therefore, we considered both adjuncts as the best treatments.

The findings of relative homogeneous treatment effects have theoretical and practical implications. The former point to common mechanisms, i.e., nonspecific effects such oral health changes due trial participation or specific effects due to commonalities among adjuncts. The latter point of smaller effect differences among the treatments provides patients and dentists with a better and simpler understanding what they can expected from adjuncts and, in the absence of substantial CAL gain differences across treatment, putting more emphasis on the profile of other treatment characteristics for clinical decision-making.

Finally, when interpreting the magnitude of the effects, the precision of the estimates must be considered. Our meta-analysis included a large number of studies, more than most network meta-analyses in oral health. Nevertheless, even with 72 studies the precision of our estimates is mediocre. Lower limits of effect of confidence intervals translate into medium or even small effect sizes.

Our third main finding is of methodological nature. We found evidence for publication bias. Health care providers, patients, and policy makers will receive misleading information, i.e., the information is not as robust as they think. Publication bias per se is not a problem. Bias of trivial magnitude, i.e. bias that does not lead to relevant impact, is not a challenge even if the bias is statistically significant. In contrast, bias of non-trivial size, i.e., when the bias can lead to a change in decision-making. In our study we found evidence for a possibly clinically relevant publication bias effect because the treatment effect may not be “large” anymore and consequently the patient-perceived impact considerably lower.

While the presence of publication bias presents a methodological challenge to systematic reviews/meta-analyses, it offers also a potential for more insight into mechanisms of treatments. On one hand, specific types of selection bias, poor methodological quality, sampling variation, and chance can cause publication bias. On the other hand, true heterogeneity among studies, i.e., the size of the effect differs according to study size, e.g., differences in the treated disease population or in the interventions applied in smaller studies can also cause ‘small study effects’ (Egger et al. 1997) In the latter case, such small study effects would be informative because they point to clinically interesting differences among patient populations. These effects should be explored further because of true differences in the effect of treatment in specific populations and/or specific interventions.

Strengths and limitation

We used available data for the network meta-analysis. The design of the Guideline project led to a star-shaped network presented here because all adjuncts were compared to SRP. Some original randomized clinical trials included in the Guideline had more than two arms with two or more of the nine investigated treatment. These data could have been incorporated in the present study and would have led to a more informative network meta-analysis. However, the number of the not used, but available evidence was small because only one study would contain a direct comparison among the nine groups of adjuncts. Likely, results of treatment ranking and existence of publication bias would not have changed much in the data set of 74 studies if this additional evidence would have been incorporated. More importantly than this technical aspect, we choose not to include these studies because of conceptual reasons. We intended to make our findings complementary to the Guideline by using the identical data.

Our NMA-based effect estimates for the pairwise adjuvants comparisons could have derived with a simpler approach. In a star-shape NMA design (and under certain assumptions), the difference (d) between a treatment B and a treatment C (dBC), would be dBC = dAC – dAB when both treatments B and C are compared to the common comparator A (dAC as well as dAB are their individual treatment differences with the comparator)(Jansen et al. 2011). This simple calculation of treatment effects illustrates the principles of deriving treatment effects in a NMA.

Like traditional meta-analyses, NMA also relies on the independence of data, i.e., that a particular study it not more informative about one study than another. Violations of the independence assumption occurred in the NMA, for example, when smokers and non-smokers from a particular trial or when partial and full-mouth recordings were both included as separate studies. Furthermore, some studies had more than two treatment arms and the individual comparisons were sometimes included in a particular adjunct group. As mentioned before, to be comparable, we analysed the identical data set as the Guideline project, but violations of the independence assumption would lead to slightly larger uncertainly around the effect estimates compared to what we reported.

We would like to emphasize that we deleted or combined different adjuncts purely based on statistical reasons. Our general approach was that we wanted to aggregate studies when there was no compelling reason not to do so. While authors of the Guideline grouped treatments according to clinical expertise, we started in this study with this grouping and then synthesized more and more studies because differences between groupings seemed not of substantial magnitude and could not be differentiated from chance. However, we emphasize that the patterns of findings – that individual adjuncts and groups of adjuncts showed consistently trivial differences among them – is a convincing argument for our strategy that was performed a reasonable large sample size (for a meta-analysis).

Conclusions

We interpret the summary estimate as the typical effect patients can expect when another treatment is added to the standard scaling and root planning.

Our network meta-analysis of adjuncts, i.e., treatments in addition to scaling and root planning, provided complementary information to the “Evidence-Based Clinical Practice Guideline on the Nonsurgical Treatment of Chronic Periodontitis by Scaling and Root Planing with or without Adjuncts.” Our finding that adjuncts typically provides a third of a mm or a quarter (when publication bias is considered) of a mm CAL gain of 6 to 12 months for patients with chronic periodontitis is a contribution to our understanding of mechanisms of periodontal treatment in general and a practical information for patients and health care providers. It further strengthens the recommendation that SRP is considered the primary therapeutic modality for non-surgical periodontal therapy. Further research is required to identify population subgroups that benefit from adjuncts. Our finding of publication bias for adjuncts is relevant for periodontal treatment literature in general. If the phenomenon exists for such an important and widely studied intervention exist, it is likely that it is also present for other treatments as well but it often remains undetected because of the usually small number of included primary studies.

We believe that the effect we identified in this study can be interpreted as what adjuncts have in common, i.e., the non-specific treatment effects from adding another treatment to address attachment loss due to chronic periodontitis. The patient-perceived benefit due to this gain is not clear because CAL is a physical indicator of periodontal health, a disease-oriented outcome.

Clinical Relevance.

• Scientific rationale for study

• Various adjuncts are recommended for use in conjunction with scaling and root planning (SRP). The comparison of these adjuncts to each other using a systematic framework can yield vital information on their effectiveness to inform clinical-decision making.

• Principal findings

• Network meta-analysis identified a local antimicrobial (doxycycline hyclate) and photodynamic therapy with diode laser as having the highest probabilities for ranking first and second SRP adjuncts in terms of clinical attachment gain, respectively.

• Practical implications.

• Our results complement results of the recent ADA-commissioned Clinical Practice Guideline on the nonsurgical treatment of chronic periodontitis and provide insights on the ranking of the adjuncts and publication bias in the existing literature.