Perspective: Network Meta-analysis Reaches Nutrition Research: Current Status, Scientific Concepts, and Future Directions

ABSTRACT

Traditional pairwise meta-analysis (PMA) is a very useful method that pools evidence from one study design type if appropriate; its widespread use in nutrition research is an important phenomenon. Recently, a promising method for more advanced evidence-synthesis, called network meta-analysis (NMA), was introduced. NMA is an extension of PMA that enables simultaneous comparison of multiple interventions. NMA combines direct evidence (i.e., trials comparing 2 interventions directly) and indirect evidence (i.e., from a connected route via ≥1 comparators, e.g. placebo) in a network of studies. NMAs have the potential to advance knowledge in the field of nutrition as they provide insights that cannot be obtained by individual 2-arm randomized controlled trials or PMA. Thus, in this perspective paper, we aim to summarize the current (methodologic) status of published NMAs in nutrition research and emphasize advances and strengths in comparison with traditional PMA through specific examples, and highlight potential pitfalls and limitations. NMA is an emerging methodology in the field of nutrition research. A PubMed search identified only 23 nutrition research-related NMAs published since the inception of journals up to January 8, 2019 (61% of them published since 2017), compared with >5000 published PMAs. Moreover, we aim to highlight the scientific concepts and standards through the use of the following NMA example: “Which type of oils/solid fats offers the greatest impact on blood lipids?” In this regard, we discuss intervention definitions, transitivity/similarity, statistical methods, description and visualization of results, inconsistency, ranking, dissemination bias, assessing the certainty of evidence by Grading of Recommendations Assessment, Development and Evaluation, and reporting guidelines. We expect that rigorously conducted NMAs based on high-quality systematic reviews will become the new evidence synthesis benchmark in nutrition research. However, caution is warranted because abuse and misinterpretations of PMA and NMA findings could hamper the scientific field and possibly decision-making regarding public policy.

Introduction

During the last few decades, systematic reviews (SRs) have increased remarkably in number and continue to replace the narrative reviews previously used to summarize findings from multiple studies. Narrative reviews are often viewed critically for their lack of transparency in selection of evidence and therefore their inherent subjectivity (1); however, their key contribution is to deepen understanding of, for example, mechanisms (2). With the tremendous increase in scientific publications (3), the methodologic approach of narrative reviews is becoming less useful for the description of the consequences of an intervention or exposure, and systematic approaches to summarizing the scientific literature should become the preferred option (4). SRs aim to provide a comprehensive and objective summary of all relevant research evidence addressing specific questions according to prespecified eligibility criteria (5). Many SRs contain meta-analyses (MAs), a statistical method used to quantitatively summarize data from independent studies. In all fields of health sciences, including nutritional sciences (6), SRs and MAs have become an important tool for estimating the effects and associations based on intervention and observational studies. SRs are used to inform clinical practice guidelines and to contribute to health technology assessment reports, thereby supporting the transfer of research into evidence-based health care practice (5).

Pairwise meta-analysis (PMA) is a very useful method that pools effect estimates of randomized controlled trials (RCTs), observational studies such as prospective cohort studies, or other study design types (5) that compare 2 interventions/exposures directly (so-called direct evidence), i.e. intervention/exposure compared with control, or intervention/exposure compared with intervention/exposure.

The widespread implementation of PMAs is an important phenomenon, but the reporting and methodologic quality of SRs have to date often been inconsistent and flawed (7–10). The misuse of MAs in nutrition research has recently been criticized. By combining heterogeneous studies, including highly diverse participant demographics and study methods, the variability in findings that can reduce statistical power may increase, making “true” effects more difficult to identify (11). Although high-quality SRs and MAs may help to merge and ultimately enhance evidence-based dietary guidelines (12–17), some concerns have been expressed regarding their validity and application in nutrition research (18, 19). For example, RCTs in nutrition research are often prone to inherent methodologic constraints: sometimes they cannot be controlled with “true” placebos but rather by a limitation of certain aspects of nutrient compositions, food groups, or dietary patterns; other limitations include the lack of double blinding, poor compliance and adherence, crossover bias, and high drop-out rates (20). When conducting high-quality SRs and MAs, these study limitations need to be considered, assessed, and their impact on the findings of the SR/MA evaluated.

A promising, more recently introduced evidence-synthesis method is network meta-analysis (NMA) (also called multiple-treatment MA or mixed-treatment comparisons), which is an extension of PMA that enables a simultaneous comparison of multiple interventions. NMA combines direct (i.e., from trials directly comparing 2 interventions) and indirect (i.e., from a connected route via ≥1 intermediate comparators) evidence in a network of trials or studies. In this way, it enables inferences about every possible comparison between pairs of interventions in the network, even when some comparisons have never been evaluated directly in a trial or study. NMA offers the opportunity to synthesize large amounts of data relating to clinical outcomes and to rank interventions in terms of their relative efficacy, and might improve the precision of the effect estimates (21).

NMA has been applied widely in many medical fields, especially in psychiatry, focusing on the comparison of different pharmacologic treatments (22). In other medical fields, for example endocrinology, NMA methodology has also been applied successfully, e.g. to investigate the safety and effectiveness of long-acting compared with intermediate-acting insulin to treat patients with type 1 diabetes (23), and the resulting evidence has been implemented in the most recent “standards of medical care in diabetes” published yearly by the American Diabetes Association evaluating pharmacologic approaches to glycemic treatment (24). Findings from other NMAs (25, 26) investigating the health impact of different training modalities in patients with type 2 diabetes (T2D) have been included in the 2019 position paper of the European Association of Preventive Cardiology (27). In cases where multiple treatment options are available, guideline developers will increasingly consider NMA for establishing timely recommendations (28).

Therefore, high-quality NMAs have the potential to also give an insight into long-standing questions in nutrition research that have not yet been answered by individual trials or by PMA (29). Thus, in this perspective article, we aim to summarize the current (methodologic) status of published NMAs in nutrition research. In addition, we emphasize advances and strengths in comparison with traditional PMA through specific examples, highlighting also the pitfalls and limitations of NMA. Moreover, we will describe the scientific concepts of NMA and emphasize recent and future developments and the implications of this advanced evidence-synthesis method for nutrition research.

Current Status

Published NMAs in nutrition research

NMA is an emerging methodology in the field of nutrition research. A systematic search of PubMed was conducted, from the inception of journals up to January 8, 2019, with the following search-terms: (network meta-analysis[tiab] OR multiple treatments meta-analysis[tiab] OR mixed-treatment comparison[tiab] OR multiple treatments comparison[tiab]) AND (diet*[tiab] OR nutrition[tiab] OR food*[tiab] OR nutrient*[tiab]). This search yielded 92 references, whereas for traditional MA >5400 references were identified. Out of these 92 hits, only 23 NMAs (25%) dealt with a nutrition-related topic (30–52). An overview of the identified nutrition-related NMAs is given in Table 1. The identified NMAs were published between 2011 and 2018, 14 (61%) of them since 2017, and often in general medical journals. The number of included studies varied between 8 and 80 RCTs, and the number of participants between 840 and >17,000. The number of interventions (including placebo intervention and control/standard intervention) within an NMA varied between 3 and 20, and the number of outcomes ranged between 1 and 11. Five NMAs (22%) compared the effects of various dietary approaches (e.g. Mediterranean, low fat). Only 10 (43%) of the NMAs were based an a priori study protocol and 13 (57%) used frequentist methodology. Interestingly, it seems that the frequentist approach is used increasingly in more recently published NMAs, probably due to the availability of new software packages for NMA (e.g. network Stata, “netmeta” for R). The NMA methodologic approaches applied are summarized in Table 2 (53, 54).

TABLE 1.

Nutrition-related NMAs identified in PubMed¹

First author (reference)	Year	Journal	Studies included, n	Number and disease status of included participants	Number and type of included interventions/placebo/control (usual care)	Number and type of included outcomes	A priori protocol	NMA statistical approach
Schwingshackl (30)	2018	Am J Clin Nutr	66	3595 participants	10 food groups	10 LDL-C, TG, TC, HDL-C, FG, HbA1c, HOMA-IR, SBP, DBP, CRP	Y	Frequentist
Pan (31)	2018	J Evid Based Med	10	921 patients with T2D	5 dietary approaches	9 HbA1c, FG, TC, HDL-C, LDL-C, TG, BW, BMI, WC	Y	Frequentist
Schwingshackl (32)	2018	J Lipid Res	54	2065 participants	13 oils	4 TC, HDL-C, LDL-C, TG	Y	Frequentist
Liang (33)	2018	Medicine	17	1931 patients with AD or MCI	5 exercise, music therapy, nutrition therapy, computerized cognitive training, control	2 MMSE, NPI	N	Bayesian
Schwingshackl (34)	2018	Crit Rev Food Sci Nutr	67	17,230 patients with pre- and hypertension	13 dietary approaches, control	2 SBP, DBP	Y	Frequentist
Zou (35)	2018	Eur J Gastroenterol Hepatol	19	846 patients with NAFLD	6 diet, exercise, control	4 ALT, AST, BMI, HOMA-IR	U	Frequentist
Schwingshackl (36)	2018	Eur J Epidemiol	56	4937 patients with T2D	9 dietary approaches, control	2 HbA1c, FG	Y	Frequentist
Gutiérrez-Castrellón (37)	2017	Medicine	32	2242 children with Infantile colic	9 drug, diet, acupuncture, herbal, massage, reassurance/education, manipulative, L. retueri DSM17938, control	1 infantile colic	U	Frequentist
Muñoz Fernández (38)	2017	J Am Med Dir Assoc	21	2329 patients with AD	7 single antioxidants, complex antioxidants, polymeric formula, polypeptide, ω-3 fatty acid, B-vitamins, placebo	1 cognitive outcome	N	Bayesian
Ha (39)	2017	PLoS One	21	1865 pregnant women	7 dietary approaches	4 HbA1c, FG, HOMA-IR, FI	Y	Bayesian
Yu (40)	2017	Medicine	27	4649 preterm infants	6 food additives, placebo	5 ACM, NEC-incidence, NEC- related mortality, sepsis, hospitalization days	N	Bayesian
Song (41)	2017	Oncotarget	11	840 patients with gastric cancer	5 immunonutrition	3 IC, NIC, LOHS	N	Frequentist
Sekercioglu (52)	2017	PLoS One	29	8335 patients with chronic kidney disease	8 diet, phosphate binder, placebo	3 serum phosphate, calcium, PTH	Y	Frequentist
Iftikhar (42)	2017	Sleep Med	80	7882 patients with obstructive sleep apnea	5 continuous positive airway pressure, mandibular advancement devices, diet, exercise, control	5 AHI, ESS, ODI, sleep efficiency, O2 nadir	Y	Frequentist
Sekercioglu (43)	2016	PLoS One	28	8335 patients with chronic kidney disease	7 diet, phosphate binder, placebo	3 ACM, CVM, hospitalization	Y	Frequentist
Lehert (44)	2015	Climacteric	24	NA	11 dietary approaches, dietary supplements, social engagement, exercise	3 memory, general intelligence, screening cognition	N	Frequentist
Song (45)	2015	Medicine	27	NA patients with gastric cancer	4 immunonutrition, enteral nutrition	3 PIC, PNIC, PH	N	Frequentist
Stevens (46)	2015	Diabetes Res Clin Pract	30	NA patients with high risk of T2D	20 drugs, diet, exercise, usual care	1 progression of T2D	N	Bayesian
Mazaki (47)	2015	Ann Surg	74	7572 patients with gastrointestinal surgery	4 immunonutrition, enteral nutrition	9 AI, OC, ACM, WI, Pneumonia, IAA, AL, sepsis, UTI	N	Bayesian
Schwingshackl (48)	2014	Syst Rev	22	3521 patients with overweight or obesity	3 diet, exercise	11 BW, WC, FM, WHR, TC, LDL-C, HDL-C, TG, DBP, SBP, VO2 max	Y	Bayesian
Carter (49)	2014	J Hum Nutr Diet	8	NA patients with diabetes and non-diabetes	7 dietary approaches, usual care	3 HbA1c, FG, HOMA-IR	U	Bayesian
Dunkley (50)	2012	Diabetes Obes Metab	13	3907 patients with metabolic syndrome	12 exercise, diet, drug, usual care	2 T2D incidence, CVD, MS reversal	N	Bayesian
Wiebe (51)	2011	BMC Med	53	1126 patients with diabetes and non-diabetes	6 sweeteners	9 BW, EI, HbA1c, HOMA-IR, TC, HDL-C, TG	N	Bayesian

¹ACM, all-cause mortality; AD, Alzheimer's disease; AHI, apnea hypopnea index; AI, any infection; AL, anastomotic leak; ALT, alanine transaminase; AST, aspartate transaminase; BW, body weight; CVD, cardiovascular disease; CVM, cardiovascular mortality; DBP, diastolic blood pressure; EI, energy intake; ESS, Epworth Sleepiness Scores; FI, fasting insulin; HbA1c, glycosylated hemoglobin; HDL-C, high-density lipoprotein cholesterol; FG, fasting glucose; FM, fat mass, IAA, intra-abdominal abscess; IC, infectious complications; LDL-C, low-density lipoprotein cholesterol; LOHS, length of hospital stay; MCI, mild cognitive impairment; MMSE, Mini-Mental State Examination; MS, metabolic syndrome; N, no; NA, not applicable; NAFLD, nonalcoholic fatty liver disease patients; NEC, necrotizing enterocolitis; NIC, noninfectious complications; NMA, network meta-analysis; NPI, neuropsychiatric inventory; OC, overall complications; ODI, oxygen desaturation index; PH, postoperative hospitalization; PIC, postoperative infectious complications; PNIC, postoperative noninfectious complications; PTH, parathyroid hormone; SBP, systolic blood pressure; T2D, type 2 diabetes; TC, total cholesterol; TG, triacylglycerols; U, unclear; UTI, urinary tract infection; VO₂max, maximum oxygen uptake; WC, waist circumference; WHR, waist-to-hip ratio; WI, wound infection; Y, yes.

TABLE 2.

Methodological features applied across the included NMAs¹

	NMA methodology
					Ranking
First author (reference)	Risk of bias2	Network plot	Similarity/ transitivity	Contribution matrix	SUCRA or P score	Other ranking	Inconsistency testing4	Subgroup or sensitivity analyses	Funnel plot5	Certainty of the evidence	PRISMA NMA checklist cited
Schwingshackl (30)	Y	Y	Y	Y	Y	N	Y	Y	Y	Y (CINeMA)	Y
Pan (31)	Y	Y	N	N	Y	N	Y	Y	N	N	Y
Schwingshackl (32)	Y	Y	Y	Y	Y	N	Y	Y	Y	Y (CINeMA)	Y
Liang (33)	Y	Y	N	N	Y	N	N	N	N	N	N
Schwingshackl (34)	Y	Y	Y	Y	Y	N	Y	Y	Y	Y (CINeMA)	Y
Zou (35)	Y	Y	N	N	Y	N	N	N	N	N	Y
Schwingshackl (36)	Y	Y	Y	Y	Y	N	Y	Y	Y	Y (CINeMA)	Y
Gutiérrez-Castrellón (37)	Y	Y	N	Y	Y	N	N	N	Y	N	Y
Muñoz Fernández (38)	Y	Y	N	N	N	Y	Y	N	N	N	N
Ha (39)	Y	Y	Y	N	Y	N	Y	N	N	Y (GRADE)	Y
Yu (40)	Y	Y	N	N	Y	N	N	Y	Y	Y (GRADE)	N
Song (41)	N3	Y	N	N	Y	N	N	N	Y	N	N
Sekercioglu (52)	Y	Y	Y	N	Y	N	Y	Y	Y	Y (GRADE)	Y
Iftikhar (42)	Y	Y	N	N	Y	N	Y	N	Y	N	Y
Sekercioglu (43)	Y	Y	Y	Y	N	N	Y	Y	Y	Y (GRADE)	N
Lehert (44)	N	N	N	N	N	N	Y	N	N	N	NA
Song (45)	Y	Y	N	N	Y	N	Y	N	Y	N	NA
Stevens (46)	N3	Y	N	N	N	Y	Y	Y	N	N	NA
Mazaki (47)	Y	Y	N	N	Y	N	Y	Y	Y	N	NA
Schwingshackl (48)	Y	N	Y	N	N	Y	Y	Y	N	N	NA
Carter (49)	N3	N	N	N	N	N	Y	Y	N	N	NA
Dunkley (50)	Y	Y	N	N	N	Y	N	N	NA	N	NA
Wiebe (51)	N3	Y	N	N	N	Y	Y	Y	NA	N	NA
% Y	78%	87%	35%	26%	87%		74%	57%	57%	35%	67%

¹NA, not applicable (methodology not yet published); CINeMA, Confidence In Network Meta-Analysis; GRADE, Grading of Recommendations Assessment, Development and Evaluation for NMA; N, no; NMA, network meta-analysis; PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-Analyses (54); SUCRA, surface under the cumulative ranking curves; Y, yes.

²Risk of bias assessed by the tool of the Cochrane collaboration (55).

³Study quality assessed with Jadad scale (53).

⁴Inconsistency: loop-specific approach (56), side-splitting approach (57), design-by-treatment approach (58).

⁵Application of the comparison-adjusted funnel plot (59).

Examples of PMAs compared with NMAs

Dietary approaches in the management of T2D

The most recent nutrition recommendation position paper of the American Diabetes Association concluded that evidence suggests that there is not an ideal percentage of calories from carbohydrate, protein, and fat for all people with diabetes (60). This position paper included PMAs (61, 62) that dealt with the question of which dietary approach (low fat, Mediterranean, vegetarian, high protein, moderate carbohydrate, low carbohydrate, low glycemic index/load) offers the greatest benefits in the management of T2D (63). However, because no RCTs have ever been conducted comparing, for example, the Mediterranean diet with a vegetarian diet, or a high-protein diet with a low-carbohydrate diet, no conclusions could be drawn from the use of the PMA methodology. This important question has recently been addressed through the use of NMA methodology: for reducing HbA1c, the low-carbohydrate diet was ranked as the most effective dietary approach, and for reducing fasting glucose, the Mediterranean diet was ranked best. The NMA also revealed that all included dietary approaches significantly reduced HbA1c (−0.82% to −0.47% reduction) and fasting glucose (−1.61 to −1.00 mmol/L reduction) compared with a control diet (no intervention) (36). However, for most of the comparisons the certainty of evidence was rated low, limiting our confidence in the effect estimates.

Dietary approaches in the management of elevated blood pressure

A recent PMA compared several dietary approaches (e.g., Dietary Approaches to Stop Hypertension [DASH] diet, Mediterranean, low sodium) and a control diet (usual diet). The authors concluded that some dietary patterns might be more effective than others, without ranking the dietary approaches based on blood pressure–lowering impact (64). However, one of the most important questions that remains to be answered is which dietary approach offers the greatest effect in the management of elevated blood pressure. In a recent NMA of 67 RCTs (>17,000 patients with prehypertension and hypertension), the effects of 13 different dietary approaches (e.g. low carbohydrate, DASH, low fat) were compared. For systolic blood pressure and diastolic blood pressure the DASH diet was ranked the most effective dietary approach. Compared with a control diet, the DASH, Mediterranean, low-carbohydrate, paleolithic, high-protein, low-glycemic index, low-sodium, and low-fat dietary approaches were more effective in reducing systolic blood pressure (−8.73 to −2.32 mm Hg) and diastolic blood pressure (−4.85 to −1.27 mm Hg) (34). However, the findings were limited by very low to moderate certainty of evidence, with the exception of the DASH compared with the low-fat dietary approach, for which the certainty of evidence was rated high (34).

Impact of oils and solid fats on blood lipids

A traditional PMA showed that ω-3 (n–3) and ω-6 fatty acid–rich plant oils showed more pronounced LDL cholesterol– and total cholesterol (TC)–reducing effects compared with olive oil (65), whereas palm oil showed negative effects on LDL cholesterol compared with vegetable oils low in saturated fats (66). An important question that still remained to be answered was: which type of oil/solid fat offers the greatest effect on blood lipids combining direct and indirect evidence? This question has recently been addressed by investigating the effects of 13 oils and solid fats (safflower, sunflower, canola, hempseed, flaxseed, corn, olive, soybean, palm, and coconut oil, and beef fat, lard, and butter) on blood lipids (32) (Table 3). Safflower oil was ranked as the most effective oil for reducing LDL cholesterol and TC, followed by canola oil and sunflower oil; for reducing triacylglycerols soybean oil was ranked highest, followed by corn oil and palm oil; lard and butter were ranked lowest for improving LDL cholesterol and TC; for increasing HDL cholesterol, coconut oil was ranked highest, followed by palm oil and beef fat (32). Compared with butter, all vegetable oils were more effective in reducing LDL cholesterol (−0.42 to −0.23 mmol/L). The certainty of evidence for LDL cholesterol was rated mostly low or moderate where both direct and indirect evidence was available; whereas for comparisons based only on indirect evidence, the certainty of evidence was mostly rated low or very low.

TABLE 3.

Example of application of the PICOS criteria regarding the research question: which oils/solid fats offer the greatest improvements in blood lipids?¹

Criteria	Description
Participants	Participants aged ≥18 y
Interventions (comparator)	Eligible types of intervention/comparison of ≥2 of the following oils/solid fats: • safflower oil • sunflower oil • canola oil • hempseed oil • flaxseed oil • olive oil • corn oil • soybean oil • palm oil • coconut oil • lard • beef fat • butter
Outcomes	Primary outcome: LDL cholesterol; Secondary outcomes: TC, HDL cholesterol, triacylglycerols
Study design	Randomized parallel or crossover intervention trials with minimum intervention period of 21 d

¹PICOS, participants, interventions, comparisons, outcomes, and study design; TC, total cholesterol.

Future nutrition-relevant NMA topics

With the emerging methodology of NMA other pertinent questions in nutrition research could hopefully be answered in the future, such as the following:

• Which type of dietary sugar (glucose, fructose, sucrose, or starch) is the most harmful for cardiometabolic risk factors?

• Which dietary approach offers the greatest benefits in the management of obesity?

The findings of high-quality NMAs are important for deriving future dietary guidelines by answering pertinent questions that could otherwise not be answered with traditional MAs. For example, the USDA Dietary Guidelines for Americans 2015 answered the following question: what is the relation between dietary patterns and measures of body weight or obesity? (67) Findings from a future NMA investigating the effects of dietary patterns on measures of body weight or obesity would be of importance to provide additional indirect evidence answering this question.

Scientific Concepts

Essential steps before conducting an MA

Before conducting an MA, a high-quality SR needs to be undertaken, providing the most comprehensive qualitative synthesis of evidence. SRs are a form of observational research, and the methods for the SR should be agreed upon before the review commences. An essential part of good scientific practice for conducting an SR is the availability of a detailed protocol of each SR explicitly defining the participants (P), interventions (I) or exposures (E), comparisons (C), outcomes (O), and study design (S) (PICOS or PECOS criteria; see Table 3) and prespecifying analytic plans including subgroup and sensitivity analyses (68). High-quality SRs result in reduced bias and random error through transparent, explicit, reproducible, comprehensive, and rigorous processes that examine all of the available evidence for a specific research question (5). A comprehensively conducted systematic search for all available studies is a key element of each high-quality SR, followed by aspects of study selection and data extraction, which should be independently conducted by ≥2 people (5). Another key element is the risk-of-bias (RoB) assessment of each individual primary study by applying suggested assessment tools (55, 69).

Authors should comply with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) checklist guidelines that ensure high-quality results when evidence-synthesis methods are used (70). Transparent reporting of the PRISMA checklist includes, for example, the presentation of the full electronic search strategy for ≥1 database, presentation of the study selection process and the data extraction process, as well as evaluation of the RoB.

A meta-analysis should probably NOT be conducted when...

According to the Cochrane Handbook, 3 major reasons exist where meta-analyzing primary studies could be more of a hindrance than a help (5).

• Clinical or epidemiologic primary studies that are too diverse: the most important type of diversity is in relation to the comparisons/exposures within the primary studies. Sometimes, it might not make sense to combine all identified primary studies in a PMA. It might be more appropriate to consider them, for example, in separate subgroups. NMA can actually serve to explain heterogeneity by refining the definition of treatments or exposures: exposures that seem to be too heterogeneous (for example different doses) can be regarded as different nodes in the network. Decisions about whether or not an MA should be conducted are inevitably subjective and are not amenable to statistical solutions, but require profound discussion and clinical or epidemiologic knowledge, or a combination, and judgment. In some cases, consensus may be hard to reach.

• High RoB in primary studies. If bias is present in the majority of the primary studies, an MA will simply compound the errors and generate an “invalid” estimate that may be mistakenly interpreted as having more certainty.

• Evidence of serious publication or reporting bias. Hence, meta-analyses are at high risk of producing invalid summary estimates.

In the following paragraphs we will briefly describe the scientific concepts of NMA by using as an example the aforementioned NMA research question (i.e., impact of oils and solid fat on blood lipids [32]).

How to define the interventions

One of the first steps when planning an NMA is the consideration of which interventions should form the nodes of the network and how to define them. There is a broad spectrum between “splitting,” i.e., making very fine distinctions between interventions, and “lumping” interventions together that are roughly similar (71–73). Taking the example provided in Table 3, one possibility is to make a very fine distinction between oils/solid fats (comparison of all 13 oils/solid fats as shown in Figure 1), or to lump these oils/solid fats into 4 major categories (sources rich in saturated fat, monounsaturated fat, or ω-6 or ω-3 fatty acids).

FIGURE 1.

Network diagram taking into account 13 different oils and solid fats. The size of the nodes is proportional to the total number of participants allocated to the oils or solid fats, and the thickness of the lines is proportional to the number of studies evaluating each direct comparison. The line width is proportional to the number of studies that provide direct evidence for the respective comparison. The size of the blue points corresponds to the number of observations receiving the respective treatment and is also printed next to the treatment label. The treatment sequence on the circle was determined automatically such that a graph with a small number of line crossings is generated.

Transitivity, similarity

An important assumption of the validity of an NMA, often called the transitivity assumption, is that trials comparing different sets of interventions are similar in terms of important characteristics that may influence the outcome of interest (21, 74, 75). To evaluate transitivity/similarity, the distribution of potential effect modifiers across the available trials should be compared (for example, in Figure 1: if large differences in energy intake across trials exist, the transitivity assumption may be violated). However, only 8 out of the identified 23 NMAs (35%) evaluated transitivity/similarity (Table 2).

Statistical methods and software

Recent years have brought an expansion of statistical methods (76). Whereas in the first years Bayesian methodologic approaches dominated the field (77, 78), more recently standard regression methods based on a frequentist approach have also become widespread. This has been supported by the emergence of specialized software modules in popular statistical packages such as R (79) and Stata (80). Although Bayesian methods are in general more flexible, frequentist methods are often easier to apply for nonstatisticians (81, 82). They are also computationally less time consuming. Nevertheless, an expert should be consulted to make sure the complex assumptions are not violated and methods are appropriately used.

WinBUGS is the main resource for Bayesian network meta-analysis and is a comprehensive, albeit rather technical, implementation (83–85). The R package gemtc (86) also uses BUGS or alternatively JAGS for statistical analyses and provides a graphic front end for the user. Frequentist methods are available in Stata (80) and several R packages including netmeta (87) and metafor (88). An overview of R packages for network meta-analysis is given by Neupane et al. (89).

Description and visualization of results

The network structure is typically visualized by a network plot showing treatments as nodes and existing pairwise comparisons as edges between corresponding treatment nodes. Typically, a circular presentation of nodes is used. However, other presentation types are sometimes preferable (90). The network plot in Figure 1, which was generated with the R function netgraph of the R package netmeta, shows the common presentation with treatments on a circle. The graph shows a highly connected network where olive oil was the main comparator for the other interventions. It was recently shown that such well-connected network structures lead to a greater gain of precision when indirect evidence is added (91).

A forest plot can be used to summarize the results of the network meta-analysis if a common comparator exists, e.g., a placebo group. The forest plot in Figure 2 shows all treatment comparisons with lard as a reference. The comparisons are sorted by decreasing P score which is described below in the subsection on ranking of treatments. The forest plot shows that only safflower (mean difference: −0.95 mmol/L; 95% CI: −1.81, −0.08 mmol/L) and sunflower oil (mean difference: −0.81 mmol/L; 95% CI: −1.49, −0.13 mmol/L) are more effective in reducing TC compared with lard. All other oils/solid fats, with the exception of butter (which slightly increases TC), slightly reduce TC.

FIGURE 2.

Summary effect estimates for the comparison of different oils and solid fats on TC (mmol/L). Lard is defined as the reference treatment. P scores are defined such that they are between 0 and 1, where 0 means that a treatment is always worst, and 1 means that a treatment is always best compared with the other treatments in the network. Safflower oil (P score: 0.90) was ranked best to improve TC, followed by sunflower oil (0.85) and canola oil (0.78). MD, mean difference. TC, total cholesterol.

Another common approach for presentation of NMA results is a league table, which shows all pairwise network comparisons in a square matrix. Typically, different information is provided in the lower and upper triangles. As shown in Table 4, all network estimates and corresponding 95% CIs are shown in the lower triangle and available estimates from direct treatment comparisons are shown in the upper triangle. For example, in the comparison of beef fat with canola oil (in the top left corner), estimates are rather similar when taking the width of the 95% CI into account: the network estimate is 0.43 mmol/L (95% CI: −0.27, 1.13 mmol/L) and the estimate from the direct pairwise comparison is 0.61 mmol/L (95% CI: −0.41, 1.62 mmol/L).

TABLE 4.

League table with network and direct treatment estimates (95% CI) for TC (in mmol/L)¹

Beef fat

0.61 (−0.41, 1.62)

−0.34 (−1.26, 0.58)

0.61 (−0.41, 1.62)

—

—

—

—

0.32 (−0.69, 1.33)

—

0.36 (−0.56, 1.28)

—

—

0.43 (−0.27, 1.13)

Canola oil

—

0.00 (−0.97, 0.97)

—

—

—

—

−0.34 (−0.85, 0.18)

−0.10 (−1.07, 0.87)

—

—

−0.20 (−0.82, 0.41)

−0.02 (−0.67, 0.63)

−0.45 (−1.01, 0.11)

Coconut oil

—

−0.20 (−1.23, 0.83)

—

—

—

0.20 (−0.49, 0.90)

0.05 (−0.65, 0.74)

0.60 (−0.08, 1.28)

−0.29 (−1.34, 0.76)

—

0.06 (−0.61, 0.72)

−0.38 (−0.84, 0.09)

0.07 (−0.43, 0.58)

Corn oil

−0.90 (−1.58, −0.21)

0.38 (−0.67, 1.43)

—

—

−0.16 (−0.58, 0.25)

—

—

0.15 (−0.51, 0.82)

1.34 (0.63, 2.05)

−0.49 (−1.19, 0.21)

−0.92 (−1.42, −0.41)

−0.47 (−0.98, 0.04)

−0.54 (−0.96, −0.13)

Butter

—

—

—

0.35 (−0.08, 0.79)

—

—

0.81 (0.23, 1.39)

1.04 (0.04, 2.04)

0.32 (−0.41, 1.05)

−0.11 (−0.70, 0.48)

0.34 (−0.24, 0.92)

0.26 (−0.26, 0.79)

0.81 (0.23, 1.38)

Flaxseed oil

0.03 (−0.95, 1.01)

—

−0.11 (−1.06, 0.84)

—

0.55 (−0.21, 1.30)

—

−0.10 (−0.87, 0.66)

0.35 (−0.87, 1.57)

−0.08 (−1.22, 1.06)

0.37 (−0.77, 1.51)

0.29 (−0.81, 1.40)

0.84 (−0.29, 1.97)

0.03 (−0.95, 1.01)

Hempseed oil

—

—

—

—

—

—

−0.30 (−1.19, 0.60)

−0.73 (−1.47, 0.01)

−0.28 (−1.03, 0.48)

−0.35 (−1.06, 0.36)

0.19 (−0.53, 0.91)

−0.62 (−1.42, 0.18)

−0.65 (−1.91, 0.61)

Lard

0.24 (−0.77, 1.25)

0.31 (−0.40, 1.03)

—

0.63 (−0.40, 1.66)

—

0.01 (−0.63, 0.65)

−0.42 (−0.81, −0.03)

0.03 (−0.41, 0.47)

−0.04 (−0.37, 0.28)

0.50 (0.15, 0.85)

−0.31 (−0.80, 0.18)

−0.34 (−1.43, 0.75)

0.31 (−0.35, 0.97)

Olive oil

−0.13 (−0.57, 0.32)

—

0.25 (−0.45, 0.94)

0.67 (0.25, 1.09)

0.04 (−0.62, 0.71)

−0.39 (−0.82, 0.04)

0.06 (−0.39, 0.51)

−0.01 (−0.40, 0.37)

0.53 (0.12, 0.94)

−0.28 (−0.80, 0.25)

−0.31 (−1.41, 0.80)

0.34 (−0.30, 0.98)

0.03 (−0.26, 0.32)

Palm oil

—

0.11 (−0.37, 0.58)

0.42 (0.03, 0.81)

0.65 (−0.06, 1.36)

0.22 (−0.46, 0.90)

0.67 (0.10, 1.24)

0.59 (−0.04, 1.23)

1.14 (0.48, 1.80)

0.33 (−0.26, 0.91)

0.30 (−0.84, 1.44)

0.95 (0.08, 1.81)

0.64 (0.04, 1.23)

0.61 (−0.01, 1.22)

Safflower oil

—

—

0.18 (−0.50, 0.87)

−0.25 (−0.73, 0.24)

0.20 (−0.28, 0.69)

0.13 (−0.26, 0.52)

0.67 (0.27, 1.07)

−0.14 (−0.70, 0.42)

−0.17 (−1.29, 0.96)

0.48 (−0.19, 1.15)

0.17 (−0.17, 0.51)

0.14 (−0.20, 0.48)

−0.47 (−1.11, 0.18)

Soybean oil

—

0.51 (−0.15, 1.18)

0.08 (−0.33, 0.49)

0.53 (0.05, 1.01)

0.46 (0.08, 0.83)

1.00 (0.59, 1.41)

0.19 (−0.31, 0.69)

0.16 (−0.93, 1.25)

0.81 (0.13, 1.49)

0.50 (0.21, 0.78)

0.47 (0.18, 0.76)

−0.14 (−0.75, 0.48)

0.33 (−0.06, 0.71)

Sunflower oil

¹Network estimates are shown in the lower triangle comparing the treatment in the column with that in the row. The upper triangle shows available direct pairwise comparisons (treatment in the row compared with the column). The effect estimates presented in this table are based on our recently published NMA (32). Whereas the effect estimates in the original NMA (32) were reported as a 10% isocaloric exchange (effect estimates of each primary study have been standardized) between the different types of oils and solid fats, in this table no standardization has been conducted for total cholesterol. Instead, postintervention values with the corresponding SD (see Supplementary Table S2 of the corresponding NMA [32]) from each primary study were used to calculate effect estimates. NMA, network meta-analysis; TC, total cholesterol.

Inconsistency

To evaluate the presence of local statistical inconsistency (i.e., disagreement between the direct and indirect evidence), the loop-specific approach (detecting loops of evidence that might present important inconsistency) (56), as well as the side-splitting approach (57), should be applied. In the loop-specific approach, the inconsistency is tested for each loop. Loops are closed connections of direct evidence (56). For example, a closed triangular network is illustrated in Figure 1 for the comparison of palm oil with olive oil, palm oil with soybean oil, and olive oil with soybean oil. The side-splitting approach (side = separating indirect and direct evidence) tests for discrepancies between direct evidence (PMA) and indirect evidence for each individual comparison available in the overall network. In the side-splitting approach, one direct comparison at a time will be excluded (i.e. olive oil compared with canola oil in Figure 1) and the NMA will obtain the indirect relative effect for olive oil compared with canola oil.

Global methods, such as the design-by-treatment interaction model, investigate the presence of inconsistency jointly from all possible sources of evidence in the entire network simultaneously (58). Different relative effects of interventions in studies can therefore often be plausibly assumed depending on the design. For example, the effects of a very-low-calorie diet and a low-calorie diet in a 3-arm study (including only patients with obesity) that also investigates intermittent fasting may differ from those in a 2-arm comparison (including only patients with normal weight). The design-by-treatment interaction model aims to explain inconsistency by allowing effects to be different in studies with different designs.

Sources of inconsistency may be explored by subgroup (e.g., age) or sensitivity analyses (e.g., excluding high RoB studies) or meta-regression. High-quality NMAs draw conclusions only from analyses that are prespecified before inspecting the study findings, but even these findings should be interpreted cautiously (92).

Ranking

By conducting NMA, it is possible to derive a relative ranking of the different interventions for the given outcome through the use of the distribution of the ranking probabilities and the surface under the cumulative ranking curves (SUCRA) (93). SUCRA values are a concept developed in the context of Bayesian statistics. Their value is between 0 and 1, where 0 means that a treatment is always worst and 1 means that a treatment is always best compared with the other treatments in the network. In practice, the researcher has to specify whether small values (such as for HbA1c) or large values (HDL cholesterol) are desired for the specific clinical outcome. SUCRA values can also be obtained in a frequentist framework, either through the use of resampling methods (80) or an analytic approach (P score), with corresponding interpretation and identical rankings (94). Taking our example from Figure 2, safflower oil (P score: 0.90) was ranked best to improve TC, followed by sunflower oil (0.85) and canola oil (0.78).

Dissemination bias

In order to evaluate dissemination bias, a funnel plot can principally be created for each direct pairwise comparison. However, a disadvantage of this approach is that funnel plots are often uninformative as only a small number of studies are available in each pairwise comparison. Instead, Chaimani et al. (95) introduced a comparison-adjusted funnel plot which shows all pairwise comparisons in a single plot, which typically contains the recommended number of at least ten estimates for the evaluation of funnel plot asymmetry (96, 97). Advanced methods to adjust for publication bias and related biases in NMA were also suggested by Chaimani and Salanti (59). The crucial assumption of comparison-adjusted funnel plots, however, is that treatments have been ordered in a “meaningful way” (95). For instance, treatments can be ordered from oldest to newest treatment if newer treatments are expected to be favored in smaller trials, i.e., if small studies showing a favorable effect for the newer treatment have a larger probability of being published than small studies showing the opposite effect.

Grading the certainty of evidence

Evaluating the certainty of evidence in an NMA is a very important part of every type of MA. The Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach considers the following items: study limitations, imprecision, inconsistency, indirectness, and publication bias; the approach has been expanded for NMAs to assess the certainty of evidence (98–101). The Confidence In Network Meta-Analysis (CINeMA) framework, which is an improvement of a previously suggested approach (102), can facilitate judgments about the outcome-specific certainty of evidence in an NMA. CINeMA is an adaptation of the GRADE approach and considers the following items: RoB, indirectness, imprecision, heterogeneity, publication bias, and inconsistency (102). CINeMA has been recently implemented in a web application available at http://cinema.ispm.ch. Grading the certainty of evidence according to GRADE or CINeMA leads to judgments based on 4 levels of evidence certainty: high, moderate, low, and very low.

Reporting guidelines

Authors of NMAs of health care interventions are encouraged to follow the PRISMA NMA statement (103). In congruence with the standard PRISMA guidelines for PMAs (104), authors of NMAs should state explicitly the study questions being addressed with reference to PICOS or PECOS criteria. The inclusion criteria for the PICOS or PECOS criteria should be defined in the light of transitivity/similarity with respect to an NMA. One main difference between PMA and NMA is that the number of interventions is likely to be larger and the distinction between intervention and comparison is often not obvious (in Figure 1 and Table 3, for example, olive oil might be used as the experimental treatment in one trial and as the control comparison in another trial). Table 3 shows an example of our previously published NMA that used PICOS criteria regarding the research question (32): which oil/solid fat is most effective in improving blood lipids?

Compared with the standard PRISMA statement for PMAs (70), study protocols require extensions for NMAs, i.e., potential effect modifiers need to be defined to evaluate transitivity (92). Moreover, a network plot should be provided and characteristics of the network plot should be described (Figure 1). Regarding summary measures, treatment rankings and SUCRA values should be described and reported. For the assessment of inconsistency, statistical methods used to detect disagreement between direct and indirect evidence should be described, and inconsistency should be explored with various models.

It is recommended that authors of NMAs report and compare the findings of both NMA and PMA and be aware of the potential impact of including treatments mainly based on indirect evidence in a network (i.e., star networks) (91).

Future Directions

Component NMA

Treatments in NMA can be complex—for example, combinations of ≥2 treatments or of common components. Whereas a standard NMA handles all included treatments (single or combination therapy) as different nodes in the network, component NMA models assume that the effect of a combined treatment is the sum of the effects of its components (such that equal components cancel each other out, e.g., olive oil [25 g/d] + canola oil [25 g/d] compared with olive oil [50 g/d] = canola oil [50 g/d]). These assumptions are inevitably much more problematic in nutrition research than in other medical fields such as psychiatry, because the example of combining olive oil with canola oil treatments could result in added calories (i.e., if the dose of olive oil or canola oil is ignored) or restricting other nutrients (very difficult to assess in nutrition research), and thus the credibility of an additive model as conferring benefit is limited. At best, some interactions could be added to an additive model in a synergistic or antagonistic sense if they are biologically plausible (105).

Component NMA models allow the effects of treatment components to be estimated in the context of multicomponent interventions, borrowing strength from studies with common components and comparing the estimates to the standard NMA. These models even allow estimates of effects across disconnected networks if the parts of the network have enough common components (106).

Living NMA

Updating SRs is generally more efficient than starting again from the beginning when new evidence emerges (107). Prospectively planned living MA, i.e., NMA that is continuously updated as soon as new evidence becomes available, can facilitate timely recommendations (108) and contribute to reducing research waste by providing strong evidence against the null hypothesis earlier than living PMA (109). Cochrane hosts and encourages living SRs (https://community.cochrane.org/review-production/production-resources/living-systematic-reviews). A recent study that considered 77 NMAs showed that performing living NMA seems to be feasible. Most NMAs (∼75%) had <4 new trials included per year (110). In a specific example, Crequit et al. (111) spent ∼2 mo on a 1-y NMA update, which corresponds to 11% of the initial workload (18 mo).

Use of NMA for design of future studies

The design of new studies should ideally be based on all existing evidence about the underlying clinical or health-related research question. If there are several alternative interventions, an up-to-date NMA is the optimal basis for planning a new study. NMA has therefore been proposed as a tool to optimally plan the design and estimate the required sample size of a new trial (112–115). As shown in the example in Figure 1, future intervention trials may compare the effects of hempseed oil with those of coconut oil or butter, or may also compare the effects of canola or flaxseed oil with those of butter. By taking into account Figure 1, we can additionally show that the sample size for several treatment groups was very low (<100 participants per intervention arm for hempseed oil, safflower oil, and beef fat) which should be considered when planning future trials.

Salanti et al. (112) recently introduced a 4-step framework:

1. Perform NMA on a research question of interest.

2. Define targeted comparison or comparisons between the treatments of interest (i.e., olive oil),

3. Decide whether NMA answers the research question (i.e., hempseed oil compared with olive oil).

4. Estimate the features of a future trial that will update the network to answer the research question (calculation of sample size within a conditional framework considering available evidence).

This framework aims to identify the optimal design for a new trial that will both update the existing evidence and minimize the required sample size.

NMA of observational studies

If studies are suitable, it is technically possible to conduct NMAs of observational studies. Overall, NMAs of observational studies have been only rarely applied across scientific disciplines. To date, NMAs of observational studies have not been directly applied in the field of nutrition research, but in the adjacent field of obesity research. In a recent NMA of 19 observational studies Wiebe et al. (116) investigated whether vitamin B-12 concentrations are lower in people with higher BMI. The authors observed no association between serum or plasma B-12 concentrations and BMI. In another NMA the different definitions of metabolic health and risk of T2D were investigated (117). It was shown that metabolically unhealthy patients have a 2- to 4-fold higher risk of T2D compared with patients classified as healthy across all BMI categories.

In summary, we highlight in this perspective article that NMA is a highly attractive evidence-synthesis method that has recently reached the field of nutrition research. NMAs have the potential to advance knowledge in the field of nutrition because they provide insights that cannot be obtained by individual trials or PMA, and provide an important basis for the design of novel trials. We expect that rigorously conducted NMAs based on high-quality SRs will become the new evidence-synthesis benchmark in nutrition research. However, caution is warranted since abuse and misinterpretations of both PMA and NMA findings would hamper the scientific field and possibly decision-making in public policy.

Notes

Perspective articles allow authors to take a position on a topic of current major importance or controversy in the field of nutrition. As such, these articles could include statements based on author opinions or point of view. Opinions expressed in Perspective articles are those of the author and are not attributable to the funder(s) or the sponsor(s) or the publisher, Editor, or Editorial Board of Advances in Nutrition. Individuals with different positions on the topic of a Perspective are invited to submit their comments in the form of a Perspectives article or in a Letter to the Editor.

The authors reported no funding received for this study. GR is funded by DFG (German Research Foundation), grant RU1747/1-2.

Author disclosures: LS, GS, GR, and JJM, no conflicts of interest.

Abbreviations used: CINeMA, Confidence In Network Meta-Analysis; DASH, Dietary Approaches to Stop Hypertension; GRADE, Grading of Recommendations Assessment, Development and Evaluation; MA, meta-analysis; NMA, network meta-analysis; PECOS, participants, exposures, comparisons, outcomes, and study design; PICOS, participants, interventions, comparisons, outcomes, and study design; PMA, pairwise meta-analysis; PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-Analyses; RCT, randomized controlled trial; RoB, risk of bias; SR, systematic review; TC, total cholesterol; T2D, type 2 diabetes.