Application of Systematic Review Methodology to the Field of Nutrition^,

Abstract

Systematic reviews represent a rigorous and transparent approach to synthesizing scientific evidence that minimizes bias. They evolved within the medical community to support development of clinical and public health practice guidelines, set research agendas, and formulate scientific consensus statements. The use of systematic reviews for nutrition-related topics is more recent. Systematic reviews provide independently conducted comprehensive and objective assessments of available information addressing precise questions. This approach to summarizing available data is a useful tool for identifying the state of science including knowledge gaps and associated research needs, supporting development of science-based recommendations and guidelines, and serving as the foundation for updates as new data emerge. Our objective is to describe the steps for performing systematic reviews and highlight areas unique to the discipline of nutrition that are important to consider in data assessment. The steps involved in generating systematic reviews include identifying staffing and planning for outside expert input, forming a research team, developing an analytic framework, developing and refining research questions, defining eligibility criteria, identifying search terms, screening abstracts according to eligibility criteria, retrieving articles for evaluation, constructing evidence and summary tables, assessing methodological quality and applicability, and synthesizing results including performing meta-analysis, if appropriate. Unique and at times challenging, nutrition-related considerations include baseline nutrient exposure, nutrient status, bioequivalence of bioactive compounds, bioavailability, multiple and interrelated biological functions, undefined nature of some interventions, and uncertainties in intake assessment. Systematic reviews are a valuable and independent component of decision-making processes by groups responsible for developing science-based recommendations and policies.

Introduction

Systematic reviews represent a rigorous approach to synthesize and evaluate scientific evidence (1). This approach to summarize available data minimizes potential reporting bias through comprehensive and reproducible searches using clearly defined and described selections and reporting protocols. The systematic review approach enhances rigor by assessing the methodological quality of the included studies and overall strength of the body of evidence. Transparency of the process is ensured through detailed documentation of the decision-making process. An analytic framework helps to clarify key questions and delineate the connecting logic between them. The tables used to summarize study characteristics and findings stand alone as independent scientific publications that can be used to document the state of the scientific evidence, provide input into program and policy decision-making processes, identify knowledge gaps and research needs, and serve as the foundation for later updates as new data emerge. The objectivity of systematic reviews comes from the approach used to review the literature with its requisite documentation and also from the involvement of individuals trained in systematic review methodologies who are unlikely to have a vested interest in the particular nutrient/disease relationship outcome and predefined procedures for ensuring independence of the scientific review decisions from persons who may carry preconceived ideas or personal biases into the process. Examples include investigators whose studies may be considered in the systematic review process or persons and groups who may have vested interests in the outcome of the review such as sponsors, users, consumer advocacy, and industry groups.

There is a long history for the use of systematic reviews in the medical community to develop clinical and public health practice guidelines (2,3), set research agendas (1), and formulate scientific consensus statements (4,5). The use of systematic reviews to address nutrition-related issues is more recent (6–10). Nevertheless, there is a wide range of nutrition applications for which a systematic review process has been used or is being considered (Table 1). Although many of these applications are similar to those used in the areas of medicine and public health, characteristics unique to nutrition-related topics (e.g. essentiality, habitual exposure) necessitate the development of a more complex set of research questions and approaches to the decision-making process than have traditionally been encountered in other fields (11). It should be noted that as systematic reviews are increasingly being performed and published for nutrition-related topics, the term “systematic review” has been subjected to various modifications to include evidence-based review, systematic evidence-based review, and evidence-based systematic review. In this article, we use the term systematic review, which is the common usage in medicine and other disciplines.

TABLE 1.

Examples of current and potential uses of systematic reviews in nutrition applications

Nutrition application	Examples of applications	Citation
Identify research needs and priorities	(n-3) Fatty acids and cardiovascular disease	(39)
	Multivitamin/mineral supplements and chronic disease prevention	(40)
Formulate dietary guidelines	Vitamin D and bone health	(41)
	2005 Dietary Guidelines for Americans	(42)
	2008 American Diabetes Association Nutrition Recommendations	(43)
Establish nutrient reference intakes	Derive estimates of average requirements and acceptable upper levels of intake.	(11)
		(44)
Formulate clinical practice guidelines	Screening for iron deficiency anemia, including iron prophylaxis	(45)
	Counseling for a healthy diet	(46)
	Pediatric weight management	(47)
Formulate community practice guidelines	Multi-component, school-based nutrition programs	(48)
Evaluate applications for food and supplement label health claims	Tomatoes, lycopene, and cancer	(49)
	Lutein and zeaxanthin intakes and risk of age-related macular degeneration and cataracts	(50)

Understanding the basic components of the systematic review approach and how it can be adapted to address a wide range of nutrition-related questions is critical to maximizing its utility and gaining wider acceptance. It is important to appreciate that the systematic review approach is flexible and can accommodate unique challenges posed by questions related to food and nutrition. It is equally important to understand that the focus of a systematic review is to provide answers to specific questions. These questions may be just a few among many needed to address an overarching topic. The answers to these questions do not constitute recommendations. Users of systematic reviews (e.g. government agencies, expert panels) must combine the results of a systematic review with other information and expert judgment to formulate clinical or public health policies. The intent of this article is to describe the steps used to perform systematic reviews and measures to ensure the integrity of the reviews to minimize bias, identify areas unique to the discipline of nutrition that should be factored into an evidence review process prior to undertaking the task, and discuss the strengths and limitations of systematic reviews for users of these reviews in setting recommendations and guidelines and other nutrition applications. We also identify areas for future consideration.

Examples of recent systematic reviews of nutrition-related topics

Three examples of systematic review applications are summarized for nutrition-related topics: effectiveness and safety of vitamin D in relation to bone health (10), effects of soy on health outcomes (12), and health effects of (n-3) fatty acids on arrhythmogenic mechanisms in animal and isolated organ/cell culture studies (Table 2) (13). These examples were selected because they illustrate the comprehensive and flexible nature of the systematic review process. Although similar steps were followed, they were conducted by 2 evidence-based practice centers (Tufts Medical Center Evidence-Based Practice Center and University of Ottawa Evidence-Based Practice Center). The inherent flexibility of the systematic review methodology is illustrated by the topics that address issues related to a single nutrient, vitamin D and bone health, complex nutritional interventions, soy protein/isoflavones and health outcomes, and multiple experimental models and outcomes, (n-3) fatty acids, and animal/isolated organ/cell culture. They also include study design foci that address issues related to animal/in vitro, (n-3) fatty acids combination of observational and intervention human studies, soy and health outcomes, exclusive reliance on randomized clinical trials, and several questions for vitamin D and bone health.

TABLE 2.

Systematic review steps and examples from nutrition-related topics¹

	Examples
Systematic review step	Vitamin D and bone health (10)	Soy and health outcomes (12)	(n-3) Fatty acids and arrhythmogenic mechanisms (13)
Form multidisciplinary research team (in addition to systematic review methodologists and systematic review sponsors)	Domain experts in nutrition, endocrinology, pediatrics, and biochemistry	Domain experts in soy research and relevant health areas	Domain experts in (n-3) fatty acids research and cardiac electrogenesis and arrhythmia outcomes
Develop analytic framework	Related intakes, serum 25(OH)D, active form [1,25(OH)D] and bone health	Not available	Different for whole animal, intact animal isolated organ and cell, and cell culture studies
Develop and refine key questions	Serum 25(OH)D and bone health	Soy formulations, doses, and purposes in trials	Evidence from whole animal studies that (n-3) fatty acids affect arrhythmogenic outcomes
	Intake or sun exposures and serum 25(OH)D	Whole soy or soy constituents and heath outcomes
	Vitamin D intakes and BMD, fractures, or falls; variation with age, ethnicity, geography, BMI	Dose-response of soy forms or constitutions	Evidence from cell culture and tissue studies that (n-3) fatty acids directly affect cell organelles involved in electrogenesis
	Sunlight and 25(OH)D without ↑ skin cancer	Frequency and type of adverse effects
	Intakes related to toxicities	Dose-response of whole soy and constituents on safety
Define eligibility criteria	Some questions limited to RCT	Inclusions: subjects ≥13 y; RCT, cohorts, cross-over and nonrandomized comparison studies; ≥5 subjectsin soy arm; any health condition; quantification of soy intake; outcomes of interest; ≥4-wk duration	Inclusions: experiments of (n-3) fatty acids and arrhythmia, intermediate mechanisms of arrhythmia, and electrogenesis
	Some questions included prospective cohorts, case-control, and before-after studies.
	One question restricted to existing systematic reviews		Exclusions: letters or abstracts; mechanisms related to eicosanoids, enzymes, receptors, membrane composition, fluidity, or phospholipids; nonmammalian animals or cell lines; no relevant outcomes; no (n-3) fatty acids intervention; reviews; safety assessments
	Included studies that assessed vitamin ergocalciferol or cholecalciferol with or without calcium supplementation
	Excluded studies that used calcium with vitamin D as a control arm unless a placebo was available as a comparator; vitamin D preparations calcitriol or alphacalcidol; studies on the efficacy of vitamin D for the treatment of secondary causes of osteoporosis or for treatment of vitamin D-dependent rickets	Exclusions: soy mixed with other ingredients; soy enteral feedings; reviews; nontrial observational studies; animal or in vitro studies; ingested soy not quantified; insignificant amounts of soy; no intake data
Identify search terms and strategy	130 key terms	33–63 key terms per search	64 search terms
Screen abstracts according to eligibility criteria	6566 unique records	∼4800 abstracts	1807 abstracts
Articles retrieved for evaluation	1447 reports	599 full text articles	274 articles
Extract data from articles which met inclusion criteria	167 articles	178 articles	89 articles
Construct evidence and summary tables	18 summary tables	86 summary tables	31 summary tables
Assess methodological quality, applicability	Jadad scale for RCT	3 categories (A,B,C) of methodological quality	4 categories for fatty acid and/or level of fat in the comparison diet
	Good, fair, or consistent rating for observational studies	3-category applicability grade
Perform meta-analyses, as appropriate	Meta-analysis of RCT that assessed interventions, populations, and outcomes	Meta-analysis for several cardiovascular outcomes	Meta-analyses for whole animal studies
		Meta-regression of differences across studies and dose-responses	For isolated organ and cell studies, qualitative data summary
Synthesize results, write report, have report reviewed	Report written by EPC; peer review by TEP members and external reviewers	Report written by EPC; peer review by TEP members and external reviewers	Report written by EPC; peer review by TEP members and external reviewers

¹EPC, evidence-based practice center; FA, fatty acid; RCT, randomized controlled trial; TEP, technical expert panel; 25(OH)D, 25-hydroxyvitamin D; 1,25(OH)D, 1,25-dihydroxyvitamin D.

Systematic review methodology

Thousands of systematic reviews on healthcare topics have now been published and standards for reporting of systematic reviews have been proposed (14–17). Several organizations such as the Cochrane Collaboration (18) and the Agency for Healthcare Research and Quality (19) have established guidance for conducting systematic reviews. Here, we describe the common principles of conducting a systematic review. A systematic review should include a detailed description of the approach and parameters used to ensure completeness in identifying the available data, rationale for study selection, method of critical appraisal of the evidence, and method of analysis and interpretation. As will become apparent, depending on the question of interest or on the basis of new data, there are opportunities to revisit and refine decisions made at certain points. Critical to the integrity of the process is thorough documentation at all steps. The approach presented assumes that persons well versed in systematic review methodologies will be part of the research team and the product will be used by other groups as one component of a decision-making process.

Identify staffing.

The actual work and associated decisions of conducting the systematic review are the responsibility of a multidisciplinary research team. However, at appropriate times in the review process, it is also desirable to solicit input from external experts, sponsors, and users. The process of obtaining external inputs needs to be defined before starting the project to ensure independence of the review from vested interests and potentially biased perspectives while ensuring that the research team has the information needed to achieve subject matter appropriateness and usefulness of the review.

Form multidisciplinary research team.

Once the topic has been defined, the initial step in starting the systematic review process is to form a multidisciplinary research team. The research team is responsible for all of the activities and decisions involved in the conduct of the systematic review and must be free of actual or apparent biases relative to the particular topic area under review. The research team should include systematic review methodologists. In addition, depending on the nature of the topic and how the results will be used, the research team will generally also include, but not be limited to, domain experts (e.g. nutrition scientists), clinicians, epidemiologists, and statisticians. In forming the research team for a nutrition-related topic, it is important to include nutrition scientists and at least 1 scientist with a wide rather than narrow range of views and expertise on the topic under review. A broad-based research team works together to identify search terms, develop an analytic framework, answer technical questions, clarify relationships among related topics, and provide input during the peer review process.

Plan for outside inputs.

Outside inputs can enhance the quality and usefulness of the review. However, these inputs need to be carefully managed to avoid the potential introduction, or appearance, of bias and vested interests into the review process. Ideally, this is achieved through a prior definition of the roles and responsibilities of the multidisciplinary research team relative to the outside inputs. In all cases, the outside inputs are advisory in nature with the ultimate decisions related to the conduct and decisions involved in the review solely in the hands of the research team. In those cases where a review project has identified sponsors and/or users, an early consultation among the research team and the sponsors or users to ensure a common understanding of the scope of work and user needs can help to ensure the usefulness of the review. Specific subject matter experts and/or an advisory committee representing a wide range of expertise that also often includes persons with varying perspectives may be convened to provide comment on the analytic framework, research questions, eligibility criteria, and search terms. Finally, the rigor of the review can be enhanced by the use of external peer reviewers for the final draft review.

Develop analytic framework.

An analytic framework assists in the synthesis and interpretation of the study results and in some cases serves as a guide for the integration of information from multiple types of data. In general, the framework is developed by the systematic review by a collaborative effort of the domain experts and the methodologists and reviewed and refined by other members of the research team. The analytic framework is used by the systematic review methodologists as they review and summarize the data. It has been used successfully by the U.S. Preventive Services Task Force for many years to help formulate research questions (20). Analytic frameworks provide visual maps outlining specific linkages among the populations of interest, exposures, modifying factors, biological role of a nutrient, and outcomes of interest. These frameworks depict the chain of logic that evidence must support to link the exposure to clinical outcomes and should be identified a priori. Defining these relationships can be helpful in further refining the key questions and study eligibility criteria prior to starting the literature search and in interpreting relevant studies once they are identified. In the case of nutrition, the analytical framework reflects the known biological mechanisms of the nutrient and guides in integrating the various types of information available into a coherent picture. An example of the analytic framework used for a systematic review addressing the area of (n-3) fatty acids and cardiovascular disease is provided (Fig. 1) (21).

FIGURE 1 .

Analytic framework for (n-3) fatty acid exposure and cardiovascular disease. This framework concerns the effect of (n-3) fatty acid exposure (as a supplement or from food sources) on cardiovascular disease. Populations of interest are noted in the top rectangle, exposure in the oval, outcomes in the rounded rectangles, and effect modifiers in the hexagon. Connecting lines indicate associations and effects. CVD, cardiovascular disease; WBC, white blood cell (leukocyte); RBC, red blood cells (erythrocytes). Adapted from (21).

Develop and refine research questions.

Developing and refining the research question(s) is a collaborative effort between the research team and, when appropriate, sponsors and intended users of the systematic review. Frequently, there is an overarching question that needs to be broken down into smaller questions that can be addressed. Well-formulated question(s) are critical in ensuring that the systematic review will be useful in addressing the intended goals and needs of the project. The question(s) define the scope of the project, determine the search terms, inform the literature selection and evaluation, and dictate the approach to data synthesis. The types of key questions can vary widely depending on the purpose of the systematic review. Multiple questions are typically needed to address even narrowly defined topics, which are subsequently combined to form conclusions. The diverse types of questions developed for the example reviews reflecting both the sponsor interests and the available literature are presented (Table 2).

The PICO approach is commonly used to formulate research questions. The acronym PICO stands for population (participants), intervention (or exposure for observational studies), comparator, and outcomes (22). Thoughtfully and unambiguously specifying the parameters for each of these attributes allows for research questions to be created that will yield the intended outcome. Various combinations of these parameters form potentially useful questions. In formulating each question, it is necessary to consider the tradeoffs between the desire for ideal knowledge and the reality of limited data, study designs, and available resources. An example of a question and component parts is “What is the overall 5-y mortality in various populations taking 1 g of fish oil daily compared with those taking a placebo?” (Table 3). Alternately, a different question can be generated by selecting an entry from each of the components of the PICO approach (columns of the table) and applying modifiers of interest. For example, “What is the 5-y overall mortality in general populations taking 1 g of fish oil supplement daily compared with those taking a isocaloric fat placebo?”

TABLE 3.

Example of the PICO method of formulating a systematic review question¹²

Population	Intervention	Comparator	Outcome
General population (primary prevention)	Fish	Isocaloric fat placebo	All cause mortality
History of myocardial infarction (secondary prevention)	Fish oil (EPA, DHA) supplement	No placebo	Cardiac death

¹Entries in the table are shown for illustrative purpose and are not meant to be exhaustive.

²Shown in the table are possible choices (not exhaustively populated in this illustrative example) under each of the PICO elements. A question could be formulated by combining items selected under each of the PICO categories. For example, by selecting “general population,” “fish oil supplements,” “isocaloric fat placebo,” and all cause mortality" and adding appropriate modifiers, one would produce the question; “What is the 5-y overall mortality in general populations taking 1 g of fish oil supplement daily compared with those taking a isocaloric fat placebo?”

Define eligibility criteria.

The PICO components define much of the eligibility criteria for selecting the studies. Additional criteria include study design, minimum/maximum dose levels (plausibility at dietary or pharmacological level), minimum number of subjects per study arm, background diets, baseline nutritional status, minimum intervention period, minimum information for characterizing the intervention (placebo, active intervention), outcome measures of interest, and statistical analysis. Additional topic-specific criteria are often necessary. In the rare instances where many more potentially relevant articles may be available than feasibly can be reviewed within resources and time available, one might limit the review of the literature to larger and more recent studies. It is important that these decisions be made in consultation with domain experts knowledgeable about the topic of interest. In some cases, limiting the review to, for example, more recent studies can result in the loss of unique data that due to resources, ethics, or other reasons have not been duplicated recently. Examples of eligibility criteria for the 3 example reviews show the diverse types of data used to answer the range of questions that reflect the different interests and needs of the sponsors of these systematic reviews (Table 2).

Identify search terms.

The list of search terms, developed by the multidisciplinary team, must be adequate in scope to capture all of the relevant literature but narrow enough to avoid capturing so much extraneous literature that an undue burden is placed on the research team. To be comprehensive, multiple databases (e.g. Medline, CAB Abstracts, and Cochrane Library Central) as well as citations of relevant retrieved articles should be searched, supplemented by contributions of domain experts. The number of key search terms used in the 3 example reviews ranged from 33 to 130 (Table 2).

Perform literature search.

At this point, the domain experts step back from the review process and the methodologists conduct the literature search and summarize the findings. This division of labor ensures a level of objectivity unencumbered by potential biases of domain experts. Clear documentation of the search strategy used and bibliographic databases searched is an inherent part of a systematic review. It facilitates the ability of other groups to reproduce the systematic reviews, allows comparisons across reviews so users can assess their similarities and differences, and serves as a foundation for an efficient updating of the systematic reviews as new findings emerge. In addition, this documentation also facilitates other uses of a systematic review by clarifying both its breath and boundaries.

Evaluate search results.

Systematic reviews of nutrition topics typically evaluate a diverse body of literature that can be diffuse and voluminous. For this reason, screening abstracts guided by eligibility criteria for potentially relevant articles in a consistent, comprehensive, and efficient manner is critical to the integrity of the systematic review. Once potentially relevant literature is identified, full-text articles are retrieved and reviewed for inclusion on the basis of the predetermined criteria. For one topic, effects of soy on health outcomes (12), the initial literature search yielded ∼4800 citations (Table 2). A total of 599 potentially relevant full-text articles were retrieved for further evaluation and 178 articles met the inclusion criteria and were included in the final report. A flow diagram depicting the process of literature evaluation and a rejection log of retrieved full-text articles along with the reasons for exclusion should be provided to enhance transparency.

Construct evidence and summary tables, and extract data.

Data need to be extracted that will identify information that is important in evaluating the quality and relevance of a study using nutrient-specific criteria in addition to those criteria commonly used. Nutrient supplement information might include intake/dose, source of supplement and chemical analysis, chemical form, mode of delivery, route and duration of delivery, and measures of prior nutritional status. Additional types of information might include the level of the nutrient in the background diet, method used to estimate intake, analytical methods used to assess nutrient status, and whether a nutrient biomarker or other approach was used to validate the dietary data. An evidence table is a comprehensive compilation of a priori-defined data elements extracted from the primary studies that are judged to be important in the interpretation of the evidence. A summary table is a distillation and synthesis of information from evidence tables. It is typically used to succinctly present study characteristics and results in a report or manuscript to support the interpretation of the evidence addressing a specific question. Although a study will usually be found only once in evidence tables, the same study may appear in multiple summary tables addressing different questions. Construction of evidence and summary tables is critical to ensure that all relevant data are extracted and tabulated in a format that will lend itself to subsequent uses. The actual extraction, depending on the nature of that available, may involve data derived from different types of study designs (observation studies, randomized controlled trials, and animal and in vitro studies). Consistent with the different study designs, the format of evidence and summary tables can be adapted to accommodate the types of relevant information important to extract from the full-text articles. The type of acceptable study design and needed information to be included in evidence and summaries tables must be specified a priori.

Assess methodological quality and applicability of studies.

Studies included in a systematic review have different protocols, are conducted with different levels of rigor, and their results are reported in a variety of manners. These variations may be manifested as discrepancies of results across studies. Thus, it is important to assess studies for potential bias due to methodological deficiencies and to assess how variations of study conduct (e.g. population enrollment) may influence the results. A critical appraisal of the studies helps to interpret the effects of methodological and clinical/biological heterogeneity on the results. Certain features of study design and conduct such as randomization and blinding in randomized controlled trials, when poorly executed, could result in biased estimates. The effect of these factors, however, is difficult to predict in a specific study (23). Thus, although critical appraisal of studies is guided by certain principles, there are some inevitable subjective components that reviewers and readers should be aware of. Numerous approaches to appraise evidence have been proposed emphasizing different aspects of study design, conduct, and reporting (24). An example of the assessment of methodological quality and applicability of individual studies is depicted (Table 4). The Cochrane Collaboration (25) and the U.S. Preventive Services Task Force (26) and an international group, the Grading Recommendations Assessment, Development and Evaluation working group (27), propose a next step, which is to rate the overall strength of the body of evidence. This step integrates an estimation of the overall risk of bias of evidence based on methodologic study quality as described above with estimations of the directness, consistency, and precision of the evidence. Rating the applicability of the evidence to the target population is also done at this step. The applicability of these approaches to nutrition has not as yet been evaluated.

TABLE 4.

One approach to assessing methodological quality and applicability of studies

Methodological quality

A. Least bias; results are valid

B. Susceptible to some bias, but not sufficient to invalidate the results

C. Significant bias that may invalidate the results

Applicability

I. Sample is representative of the target population. It should be sufficiently large to cover both sexes, a wide age range, and other important features of the target population (e.g. diet).

II. Sample is representative of a relevant sub-group of the target population, but not the entire population.

III. Sample is representative of a narrow subgroup of subjects only, and is of limited applicability to other subgroups.

Perform meta-analysis as appropriate.

Meta-analysis uses statistical methods to combine 2 or more studies addressing the same question. It is often part of a systematic review and can identify significant results when individual studies are inadequately powered. Most meta-analyses combine results across studies to arrive at an overall estimate. When data are available, meta-regression can be performed to explain discrepancies across studies and to explore variations of effects such as dose-response relationships. Sometimes, meta-analyses may shed new insights that studies examined individually may fail to reveal (28). Statistical methods to perform meta-analyses have advanced in the past 2 decades and the strengths and limitations are well understood. A key issue in performing a meta-analysis is the appropriateness of combining studies. This decision should be weighted in the context of the nature of the data and how the results will be used. Because several meta-analyses addressing similar questions may result in dissimilar conclusions due to differences, at times small, in the questions asked, the inclusion criteria applied, and the method of assessing methodological quality and applicability of studies used, it is important in interpreting the results to carefully understand the questions and eligibility criteria.

Synthesize results.

It bears remembering that answers obtained from systematic reviews address only the identified questions. Users of the systematic review (e.g. government agencies, expert panels) must then integrate results from the systematic review with other information to form their practice recommendations or public policies. Sometimes a systematic review may find no or only poor quality evidence or identify inconsistencies among study results. These data would suggest areas where future research needs to be conducted.

Unique considerations when conducting nutrition-related systematic reviews

There are a number of issues that need to be factored into systematic reviews of nutrition-related topics that do not normally arise when systematic reviews of pharmaceuticals and related topics are conducted. These should in no way hamper the process. However, information relative to these issues often need to be captured in systematic reviews to facilitate interpretation of study results and the overall quality, applicability, and strength of the evidence. By accounting for them, their potential influence can be factored into the review.

Baseline exposure.

In contrast to pharmaceutical trials, in nutrition-related studies, for the most part all persons have some level of background dietary exposure to the nutrient of interest, either from food and/or supplement intake or, in certain cases, endogenous synthesis (e.g. vitamin D, vitamin K). Background levels of exposures can be difficult to accurately determine due to limitations in currently available assessment methodologies of food intake, incomplete nutrient databases with which nutrient intake estimates are calculated, and temporal changes in exposure. Therefore, information on background intakes and the methodologies used to assess them should be captured in the systematic review so that this level of uncertainty can be factored into data interpretation.

Nutrient status.

Nutrient status of an individual or population can affect the response to nutrient supplementation. An accurate approach to evaluate nutrient status is unique to each nutrient and dependent on the availability of nutrient-specific tissue for sampling and homoeostatic mechanisms regulating plasma concentrations via storage depot accretion and release. For some nutrients, a relatively good assessment of nurture can be made; in other cases, the level of uncertainty of nurture is great because of uncertainties about the biological interpretation and/or methodological errors in measuring the indicator of interest that it is necessary to incorporate this information into the systematic review conclusions to facilitate appropriate data interpretation.

Bioequivalence of different chemical forms of nutrients.

Many nutrients occur in multiple forms that differ in biological activity. The general approach to address this issue is to calculate nutrient equivalents as was done when setting the recommended dietary allowances for vitamin A (preformed vitamin A, carotenoids), folate (folate, folic acid), vitamin K (phylloquinone and menaquinone), and niacin (preformed niacin, tryptophan) (29–31). The challenge of determining accurate conversion factors for the calculation of nutrient equivalents has recently been demonstrated for β-carotene (29). Capturing information on nutrient forms of baseline diets and intervention products in summarized studies is therefore often essential for appropriate data interpretation.

Bioavailability of nutrients.

There are a number of factors that can alter the bioavailability of individual nutrients. These differences must be considered when estimating dietary intake and comparing response to dietary supplementation. Briefly, these include the chemical form of a nutrient (e.g. heme and nonheme iron), nutrient/nutrient interactions (e.g. vitamin C and nonheme iron), nutrient/drug interactions (e.g. isoniazid and vitamin B-6; coumadin and vitamin K; folate and metformin), nutrient/food interactions (e.g. fat-soluble vitamins and dietary fat, zinc- and phytic acid/oxalic acid-containing foods), form of inorganic mineral (e.g. calcium carbonate, citrate, or malate), biological response to single compared with multiple daily doses (e.g. calcium), and habitual intake effect on efficiency of absorption and excretion (e.g. iron, vitamin C). Other factors that may alter nutrient bioavailability include biological status (e.g. iron and pregnancy, achlorhydria and vitamin B-12), food processing (e.g. particle size and dietary fiber; lye-treated corn and trytophan; heat treatment and carotenoids), and, for dietary supplements, factors that alter completeness or rate of release (e.g. coatings, excipients, and surfactants). Bioavailability also differs among nutrients from biological stores. For example, vitamin A has a relatively high bioavailability from liver only when protein status is adequate. Release and deposition of nutrients from storage depots can be unrelated to biological needs. For example, fat-soluble vitamin deposition or release from adipose tissue is altered by weight gain or loss, respectively. Again, capturing relevant information on baseline diet and intervention product bioavailability may be necessary for interpreting summarized results included in a systematic review.

Multiple and interrelated biological functions of a nutrient.

Most nutrients have multiple biological functions. A critical point during the research question(s) development and refinement phase of the systematic review process is to clearly define the nutrient-specific scope of the review. This often entails narrowing the range of the work. Some biological functions of nutrients are dependent on multiple nutrients (e.g. folate, vitamin B-12, and vitamin B-6; vitamin D and calcium). These relationships must be defined early in the review process and putative factors incorporated into formulating the questions.

Undefined nature of nutrient intervention.

Food-based nutrient interventions, in contrast to nutrient supplement-based interventions, present unique challenges in accurately quantifying the absolute change in intake. For example, one approach to increasing very long-chain (n-3) fatty acid [eicosapentaenoic acid (EPA) and docosahexaenoic acid (DHA)] intake is to instruct study subjects to increase fish intake. However, there is considerable variability in the levels of EPA and DHA in different fish, within species of fish (32), time of year the fish were caught, and animal husbandry practices for farm-raised fish. Similarly, assessing EPA and DHA intakes from nutrient supplement data is not without challenges due to the wide variability in fatty acid contents of available fish oil supplements and potential changes in supplement potency during prolonged storage or exposure to heat. Documentation of nutrient intake assessment is important to record.

Uncertainties in assessing dose-response relationships.

Measurement and assay procedures can alter apparent dose-response relationships between nutrient intake or dietary pattern and health outcomes. This can be particularly important for systematic reviews where absolute intake/response relationships rather than relative intake response relationships are needed to assess the public health importance of a particular intervention or to identify dose-response relationships to inform the establishment of recommendations. In general, dietary intake methodologies underestimate energy and protein intakes with greater biases for food frequency than 24-h recall methodologies (33). Potential biases for other nutrient intake estimates are not adequately documented but likely exist. Assay procedures for biomarkers of nutritional status can also significantly affect the mean and distribution of reported values and need to be factored into data interpretation (34,35).

Strengths and limitations of systematic review approach for nutrition applications.

The systematic review approach brings a number of strengths to the evaluation of evidence in nutrition applications. One of the most compelling strengths is the transparent, objective, and rigorous nature of the process. A clearly defined and unambiguous system is put in place to define the scope of the review, refine the question(s) to be addressed, and identify and select studies prior to reviewing the data. Evidence available to address each question is summarized and critically appraised. This transparency is particularly critical when the systematic reviews are subsequently used by expert panels in developing program or policy guidelines and recommendations.

The ability to combine small studies with meta-analyses increases the statistical power available to address specific questions. This is particularly useful for systematic reviews of nutrition topics where the availability of large trials is relatively limited or lacking. Meta-analyses may be potentially useful in simulating dose-response curves across intervention studies that individually evaluate only 1 or 2 intake levels.

Inherent in the systematic review process is its flexibility in addressing wide variations in the nature of the questions of interest and available amounts and types of data to answer them, while simultaneously ensuring a consistency among topics. This has been particularly challenging for the nutrition community, because the scope of issues has gone beyond those traditionally addressed (from making recommendations for preventing deficiency to minimizing risk of developing chronic disease or nutrient excess). The methodologies of systematic reviews ensure an objective assessment of the available body of literature and minimize biases often encountered in narrative reviews.

When systematic reviews are conducted for the purpose of informing policy and program decisions, an important by-product of a systematic review is the identification of gaps in available data. This information can be used to assist the formulation of research agenda and funding priorities. Equally important is the ability of systematic reviews to identify needed improvements in the quality and nature of reporting. For example, a commonly identified problem in nutrition-related systematic reviews has been that even for topics for which there are a number of published trials, incomplete reporting of basic study design and conduct, as well as poor characterizations of baseline, placebo, and intervention characteristics limits the ability to make definitive conclusions about the outcome of interest. To avoid commonly observed study documentation deficiencies, consolidated standards of reporting trials guidelines for the reporting of randomized trials (16,17) and trials of complex herbal interactions (36) have been proposed. Their use by publishers of nutrition studies is encouraged.

Lastly, the detailed documenting of search strategies and summarizing of the data associated with generating systematic reviews facilitates the updating/revising process as new data become available by providing a comprehensive foundation on which to build. This has the benefits of maximizing the use of limited resources and decreasing the time necessary for generating topic updates.

Notwithstanding these strengths, there are clear limitations of using the systematic review approach in the field of nutrition. By definition, the systematic review process is most effective when limited to addressing targeted questions of limited scope. This may include the population of interest (e.g. age, sex, health status), intervention, comparator, outcome measure, and duration of intervention. Questions that require a broad-based exploratory search approach would better be served by using the systematic review approach after an initial literature search has been conducted and domain experts have narrowed and refined the questions of interest.

Systematic reviews are limited by the quality and availability of data. No approach to analyzing the data can adjust for poor study design, missing data, or publication bias in the area of interest. Multiple systematic reviews addressing what appear to be the same topic can result in different conclusions, causing considerable confusion (37,38). For the most part, discordant results are due to differences in study inclusion and/or exclusion criteria, temporal evolution of available data, and subtle differences in the actual questions addressed that are not initially obvious. By clearly documenting review decisions, comparisons of different reviews can be made and the reasons for differences become apparent.

Using the systemic review process when applied to the field of nutrition allows for considerable flexibility with regard to the types of questions evaluated, studies included, and information captured, as well as the nature of summary statements. Confidence in the results of systematic reviews occurs at a number of levels. These include the transparent nature of the process and involvement of a broad-based research team free of potential biases and vested interests. Confidence also derives from the involvement of trained systematic review methodologists, and, a priori formulation of key questions, search criteria, study evaluation criteria, and information captured for evidence tables, and a priori procedures for obtaining appropriate outside inputs from subject matter experts, sponsors and users while precluding the potential biases and conflicts of interest. Within these boundaries, the conclusions are comprehensive in nature and objective in the assessment of the available information without exceeding the limits of the data. The recognition of a number of challenges not necessarily encountered in other disciplinary areas can enhance the quality and usefulness of nutrition-related systematic reviews. Lastly, it is important to keep in mind that systematic reviews are a tool to be used by expert panels, funding agencies, and other groups and cannot serve as a replacement for expert deliberations and organizational policy development. Users of systemic reviews often need to augment the reviews with other sources of information and where uncertainties exist, with the application of expert scientific judgment. Systematic reviews are a valuable and independent component, but not the end, to decision-making processes by groups responsible for developing science-based recommendations and policies.