New Guidelines for Publishing Review Articles in JPP: Systematic Reviews and Topical Reviews

Review articles serve an important purpose in synthesizing knowledge in the field of pediatric psychology, highlighting new topic areas, innovative concepts, and contemporary issues in the field. In the arena of treatment, a high-quality systematic review may answer questions about the efficacy of psychological treatments for specific pediatric problems by synthesizing quantitative results from relevant studies about specific interventions. As new treatments are developed in the field and new conditions are being tackled with pediatric psychology interventions, efficacy will need to be established. Such results may inform pediatric psychologists and other health providers about which interventions to use. Moreover, such results may guide decision making concerning whether the strength of the evidence warrants that a specific intervention for a particular childhood medical condition be made available to patients and financed. Another important benefit of careful systematic reviews is that they may reveal gaps in the evidence base or challenging methodological problems that may lead to advances or improvements in future research. JPP has published systematic reviews of clinical interventions in pediatric psychology (e.g., Cushing & Steele, 2010; Uman, Chambers, McGrath, & Kisely, 2008) as well as reviews of the effects of pediatric conditions on children’s adjustment (e.g., Pinquart & Shen, 2011), and of the evidence base for assessments used in pediatric psychology (see Cohen et al., 2008). JPP will continue its emphasis on evidence-based practice in pediatric psychology through publication of review articles.

However, as mentioned by Lavigne (2008), the process of translating research into practice in pediatric psychology is difficult. In particular, we recognize that our processes need to become increasingly sophisticated to deal effectively with the tremendous change that has occurred over the past decade in evidence-based medicine and the scholarship of review articles. Systematic reviews, defined below, are considered the highest form of evidence. But, systematic reviews must be performed using rigorous review methodology to deliver on this promise. Following the lead taken by many major medical and social science journals, JPP will accept only systematic reviews beginning this year and discontinue its practice of publishing narrative literature reviews. JPP will also introduce a new type of submission called a Topical Review.

Definition of Terms

The scholarship of reviews is complicated by inconsistent use of terminology to describe review methodology. For the purposes of these author instructions, a narrative literature review is defined as an up-to-date literature review of a specific topic or theme from a theoretical and contextual point of view. However, this type of review neither describes the methodological approach that would permit reproduction of data nor answers specific quantitative research questions.

As defined by the Cochrane Collaboration (Higgins & Green, 2011), a “systematic review is a review of a clearly formulated question that uses systematic and explicit methods to identify, select, and critically appraise relevant research, and to collect and analyze data from the studies that are included in the review. Statistical methods (meta-analysis) may or may not be used to analyze and summarize the results of the included studies. Meta-analysis refers to the use of statistical techniques in a systematic review to integrate the results of included studies.” Therefore, the term “meta-analysis” refers to this last stage of quantitative synthesis, and is considered a component of a systematic review.

JPP prefers and encourages authors to include meta-analysis in their systematic reviews. Recognizing, however, that there are circumstances in which data from the included studies in a systematic review do not lend themselves to quantitative synthesis, authors should provide justification if meta-analysis cannot be performed. In this circumstance, a qualitative summary is performed instead; for an example of a systematic review without meta-analysis in JPP, see Stinson, Wilson, Gill, Yamada, and Holt (2009).

The purpose of a systematic review is to summarize the best available research on a specific question. Often the research question concerns the evidence for a particular treatment; however, a systematic review may also summarize descriptive studies such as children’s outcomes related to illness. One distinguishing feature of a systematic review (in contrast to a narrative review) is that it uses transparent procedures to find, evaluate, and synthesize the results of relevant research, allowing for a process that can be reproduced. In addition, researchers who conduct systematic reviews use explicit methods aimed at minimizing bias, to produce more reliable findings that can be used to inform decision making. Studies included in a review are screened for quality, so that the findings of a large number of valid studies can be combined using qualitative or quantitative methods (e.g., meta-analysis).

Reporting Standards

While there are differences in how authors approach systematic reviews and there is not universal consensus on one methodology for conducting a review, there are a core set of elements concerning the reporting of systematic reviews that authors should adhere to. Over the past decade, standards have evolved for the reporting of systematic reviews, emerging from an early statement called the QUOROM guidelines to an updated widely accepted statement called the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) (Moher et al., 2009). JPP will require authors to use these guidelines, which suggest a standardized way to ensure a transparent and complete reporting of systematic reviews.

The PRISMA Statement is now required for systematic reviews and meta-analyses by >170 medical journals worldwide. The PRISMA Statement includes a 27-item checklist of items that are deemed essential for transparent reporting by authors in a systematic review, and a four-phase flow diagram depicting the numbers of identified records, excluded articles, and included studies. Authors can find the PRISMA checklist and flow diagram in downloadable templates that can be re-used at this URL, http://www.prisma-statement.org/statement.htm. We highly encourage authors to read the accompanying PRISMA Explanation and Example document (Liberati et al., 2009) that explains each item in the checklist and provides useful background along with practical examples of reporting of each review element.

Each submission to JPP of a systematic review should include with it the PRISMA checklist and flow diagram. Authors will be required to attach these files as supplementary material at the time of the submission of the manuscript.

Conducting Systematic Reviews

As movement has been made toward evidence-based practice, an understanding of systematic reviews is critically important for all professionals involved in the delivery of health care. Few pediatric psychologists, however, have received advanced training in the methodology of systematic reviews or in performing meta-analytic techniques. Certain components of conducting reviews have been made more accessible such as increased ease and capacity to perform electronic searches and the availability of software to perform meta-analysis. This does not, however, immediately translate into an understanding of how to conduct or report a high-quality systematic review. The PRISMA Statement is limited in scope to the reporting of systematic reviews and is not intended to train individuals in how to conduct a systematic review. Fortunately, there are other resources available to provide education and instruction in best practices in systematic review methodology.

One of the best resources available on the conduct of systematic reviews of interventions is the Cochrane Collaboration. This is a group of over 28,000 specialists in health care who systematically review randomized trials of the effects of prevention, treatments, and rehabilitation as well as health systems interventions. Cochrane Reviews are published in The Cochrane Database of Systematic Reviews section of The Cochrane Library. The Cochrane Collaboration has a comprehensive handbook for authors, the Cochrane Handbook (Higgins & Green, 2011), that is publicly available at http://www.cochrane-handbook.org/. This is an excellent resource that is used by all authors preparing reviews for the Cochrane Collaboration. A main part of the Handbook is eight core chapters about the general steps for preparing a systematic review including (1) Defining the review question and developing criteria for including studies, (2) Searching for studies, (3) Selecting studies and collecting data, (4) Assessing risk of bias in included studies, (5) Analyzing data and undertaking meta-analyses, (6) Addressing reporting biases, (7) Presenting results and “summary of findings” tables, and (8) Interpreting results and drawing conclusions. These chapters provide the fundamentals of rigorous systematic review methodology, geared toward reviews of interventions. In addition, a special topics section covers issues such as including non-randomized studies, incorporating economics evidence, reviews in public health and health promotion, and overviews of reviews. Because the Cochrane Handbook undergoes regular updates, this is an excellent way for authors to stay apprised of new knowledge and conceptual advances in systematic review methodology. Although the Cochrane Handbook focuses on reviews of interventions, many of the methods may also be used for reviews that serve other purposes such as description of patient or family outcomes (e.g., child depression in pediatric populations). Thus, although some of the description below pertains primarily to reviews of interventions, authors of other types of systematic reviews should also be cognizant of the guidelines and resources available for further information on reporting of reviews.

Guidelines are helpful for standardizing the reporting of systematic reviews; however, at the same time, increased emphasis is needed on training in the conduct of systematic reviews. We also strongly encourage authors to make use of local institutional resources that focus on this topic. Educators may also consider curriculum enhancements to ensure that pediatric psychology students receive appropriate training in how to conduct systematic reviews and to evaluate the evidence from systematic reviews.

A Note on Errors in Systematic Reviews and Risk of Bias

Authors should also be aware of common problems in systematic reviews so as to proactively use strategies to avoid making the same mistakes. Systematic reviews can be misleading because of common errors, such as failure to assess the risk of bias of individual studies. Attention has been raised about the standards used for conducting and reporting systematic reviews in health psychology and behavioral medicine. For example, Coyne, Thombs, and Hagedoorn (2010) critically examined four recent meta-analyses published in Health Psychology of behavioral interventions for adults. They identified substantial problems with transparency and completeness of the reporting in each review, as well as a dependency on small underpowered trials of generally poor quality. This is an interesting article that is highly recommended to potential authors of systematic reviews. Although an analysis of systematic reviews and meta-analyses published in JPP has not been conducted, it would be a reasonable assumption that our authors and reviewers also have similar limitations in training to understand recent conceptual and statistical developments in systematic review and meta-analysis methodology.

Errors in systematic reviews are highly likely when poor-quality studies have been included. Systematic review and meta-analyses are not designed to control for the influence of poor-quality studies (Eccleston et al., 2010). In particular, it has been recommended that authors and reviewers need to consider carefully including only studies that meet acceptable definitions of study quality, including only studies of sufficient sample size in any meta-analysis, and making a priori definitions of the sample and primary outcome measurements.

In the PRISMA Statement, Item 12 is risk of bias in individual studies. This refers to an assessment of the validity of studies included in the review and risk of bias in the results, that is, the risk they will overestimate or underestimate the true effect. Given that the PRISMA Statement does not provide a specific recommendation on how to assess risk of bias and that this area of systematic review methodology continues to undergo refinement, we thought it was important to provide some additional direction. At this time, there are many available tools for examining the quality of studies (e.g., Jadad et al., 1996; Yates, Morley, Eccleston, & Williams, 2005). But, a number of limitations have been noted with these tools and checklists, including that they often do not assess internal validity, may be less appropriate for behavioral interventions (as compared with pharmacological interventions), rely heavily on author reporting, and that summary or weighting of items on a tool may not be appropriate. At this time, the Cochrane Handbook recommends against tools or checklists and rather suggests using a domain-based evaluation where authors examine the supporting information and make a judgment about risk of bias that falls into categories of low risk, high risk, or unclear risk of bias. The six domains include selection bias (random sequence generation, allocation concealment), performance bias (blinding of participants and personnel), detection bias (blinding of outcome assessment), attrition bias (incomplete outcome data), reporting bias (selective reporting), and other sources of bias. Authors can review Chapter 8: Assessing Risk of Bias in Included Studies, in the Cochrane Handbook for further information and examples of tables used to describe supporting information and criteria for making judgments about risk of bias, including in behavioral interventions specifically.

Researchers have put forth calls for pediatric psychologists to comprehensively report on study quality (Uman et al., 2010). Increased attention to risk of bias will elevate the rigor of our studies, and our confidence in findings of reviews of pediatric psychology interventions.

Topical Reviews

In contrast to a systematic review, a Topical Review is meant to provide an up-to-date overview of the latest hot topics in the field. Topical Reviews may present areas that are still developing rapidly and may provide an indication of the future direction of the field. Thus, we would like to encourage the submission of Topical Reviews to summarize contemporary findings, suggest new conceptual models, or to highlight noteworthy or controversial issues in pediatric psychology. A topical review is not intended to serve as an exhaustive review of the literature and thus will use a briefer format. Although JPP has used commentaries and editorials in the past to showcase innovative topics, these types of articles have lower impact than regular articles in the journal. Topical reviews will be regular original submissions. We believe that this new type of submission will expand JPP’s effort to summarize and synthesize the important work and ideas being developed in the field of pediatric psychology.

Topical Reviews may be submitted at any time and may also be invited (commissioned by the Editors). The instructions for Topical Reviews are that they are limited to 2,000 words, contain no >2 tables or figures, and have an upper limit of 30 references. Supplementary online material (e.g., additional tables) may be considered on a case by case basis. An effective topical review will contain a highly focused scope with clear aims and a summary or set of recommendations that pushes thinking forward in a particular research or practice area. Several medical and social science journals publish similar types of topical reviews (see for examples, Palermo & Chambers, 2005; Sibille, Witek-Janusek, Mathews, & Fillingim, 2012).

Updated instructions for submission of review articles are posted on the JPP website. Please consult the instructions before submitting review articles. We encourage submission of review articles (systematic reviews or topical reviews) on any area in pediatric psychology. Feel free to ask questions or seek guidance from the editors about your ideas for review articles.

Funding

National Institutes of Health.