Table 3. Summary of domain assessment for evaluating the quality of evidence from a network meta-analysis: Procedures for a pairwise effect estimate and overall ranking.

Evaluate the confidence in a specific pairwise effect estimated in network meta-analysis
GRADE domain	Domain assessment in NMA	Description of procedure	Instructions for downgrading
Study Limitations	Study limitations	Determine which directcomparisons contribute toestimation of the NMAtreatment effect1 and integraterisk of bias assessments fromthese into a single judgment.	Use standard GRADE considerations to inform judgment.
Indirectness	Joint considerationof indirectnessandintransitivity	Evaluate indirectness ofpopulations, interventions andoutcomes as in standardGRADE. Evaluate transitivityby comparing the distributionof known effect modifiersacross comparisons thatcontribute evidence toestimation of the NMAtreatment effect1.	If a priori assessment makes a transitivity assumption reasonable and suggests that effect modifiers are balanced, then do not downgrade. Otherwise downgrade (either if a transitivity assumption does not look reasonable or if there is insufficient evidence to judge).
Inconsistency	Joint considerationof statisticalheterogeneityand statisticalinconsistency	(a) Judge the extent ofheterogeneity, considering thecomparison-specificheterogeneity variance, theNMA estimate of variance, aprediction interval and/or otherrelevant metrics such as I2 . (b)Evaluate the extent to which thecomparison under evaluationis involved in inconsistentloops of evidence.	(a) If important heterogeneity is found, downgrade. If heterogeneity is low do not downgrade. (b) Power to detect inconsistency may be low; Downgrade in absence of statistical evidence for inconsistency when direct and indirect estimates imply different clinical decisions.
Imprecision	Imprecision	Focus on width of theconfidence interval.	Assess uncertainty around the pairwise estimate. Downgrade if confidence interval crosses null value or includes values favoring either treatment).
Publication bias	Publicationbias	Non-statistical considerationof likelihood of non-publication ofevidence that would inform thepairwise comparison. Plot pairwiseestimates on contour-enhancedfunnel plot.	Use standard GRADE to inform judgment.
Evaluate the confidence in treatment ranking estimated in network meta-analysis
GRADE domain	Domain assessment in NMA	Description of procedure	Instructions for downgrading
Study Limitations	Studylimitations	Integrate risk of bias assessmentsfrom each direct comparison toformulate a single overallconfidence rating for treatmentrankings.1	Use standard GRADEconsiderations to informjudgment.
Indirectness	Joint considerationof indirectnessand intransitivity	Evaluate indirectness of populations,interventions and outcomes as instandard GRADE. Evaluatetransitivity across network bycomparing the distribution of knowneffect modifiers acrosscomparisons.1	If a priori assessment of transitivity suggests effect modifiers are balanced across the network do not downgrade.Otherwise downgrade (either ifa transitivity assumption does not look reasonable or if thereis insufficient evidence to judge).
Inconsistency	Jointconsiderationof statisticalheterogeneityand statisticalinconsistency	(a) Judge the extent of heterogeneityconsidering primarily the NMAvariance estimate(s) used and othernetwork-wise metrics such as Q forheterogeneity in a network (b)Evaluate inconsistency in networkusing statistical methods (such as global tests of inconsistency, orglobal inconsistency parameter).	(a) If important heterogeneity is found, downgrade. If heterogeneity is low do not downgrade. (b) For overall treatment rankings, inconsistency should be given greater emphasis, since ranks are based on mean effects and the uncertainty they are estimated with. Downgrade in absence of statistical evidence for inconsistency when several direct and indirect estimates imply different clinical decisions.
Imprecision	Imprecision	Visually examine rankingprobabilities (e.g. rankograms) foroverlap to assess precision oftreatment rankings	If probabilities are similarly distributed across the ranks, downgrade for imprecision.
Publication bias	Publicationbias	Non-statistical consideration oflikelihood of non-publication foreach pairwise comparison. Ifappropriate, plot NMA estimateson a comparison adjusted funnelplot and assess asymmetry.	As asymmetry does not provideconcrete evidence of publication bias, downgrading should only be considered jointly with the non-statistical assessment.

¹When integrating assessments about direct comparisons into a judgement about an NMA treatment effect or the ranking, more weight should be given to assessments from direct comparisons that contribute more information. We recommend use of the contributions matrix to quantify how much information each direct comparison contributes to the estimation of the NMA treatment effect under evaluation or the ranking.