Enhancing Public Health Systematic Reviews With Diagram Visualization

Systematic reviews provide a critical summary of a body of knowledge that links research to decision making, whether to inform public health, clinical medicine, medical education, system-level changes, or advocacy. Good reviews are accessed by a wide range of audiences, including health service users, health service providers, and policy decision makers. Because the topics studied, the thinking behind the review questions, the analytical plan, and the review’s interpretation in the broader policy context are often complex, diagrams can play an important role in communicating the review to the reader. Indeed, graphic design is increasingly important for researchers to communicate their work to each other and the wider world.¹ Visualizing the topic under study facilitates discussion, helps understanding by making complexity more accessible, provokes deeper thinking, and makes concepts more memorable.² Higher impact scientific articles tend to include more diagrams, possibly because diagrams improve clarity and thereby lead to more citations or because high-impact articles tend to include novel, complex ideas that require visual explanation.³

Merriam-Webster’s Dictionary defines a diagram as “a graphic design that explains rather than represents, especially: a drawing that shows arrangement and relations (as of parts).”⁴ Established standards exist for visualizing the flow of studies through a review,⁵ risk of bias, and individual study and meta-analysis results in forest plots; these are not the subject of this editorial. We consider diagrams that communicate the conceptual framework underpinning reviews.

Diagrams include “logic models,” “framework models,” or “conceptual models”—terms that are often used interchangeably and inconsistently in the literature.⁶ We examine how diagrams can help review authors and readers and offer guidance for presenting information diagrammatically. We based our work on a purposive search for diagrams from the Cochrane Library and sources of reviews more likely to illustrate conceptual frameworks. Drawing on the data and our own experience, we adapted rapid appraisal methods⁷ for analyzing documents, taking an iterative, inductive approach to understand what enhances the clarity and utility of diagrams. We then related this learning to methodological articles of systematic reviewing and science communication (Appendix A, available as a supplement to the online version of this article at http://www.ajph.org). We built on our collective experience of diagrams in reviews and helping others to develop them.

We first describe diagrams’ various purposes. Then we discuss what we recognized, as systematic review readers, authors, and editors, as important steps to creating a good diagram. Next, we consider how diagrams can enhance the review process for authors. We discuss these findings in relation to methodologies that routinely integrate diagrams into structure systematic reviews: framework synthesis⁸ and logic models of illness or treatment pathways, where principles and agreed good practice are emerging.⁹ Finally, we discuss theories underpinning science communication.¹⁰

WHAT DO DIAGRAMS ILLUSTRATE?

In our rapid appraisal (Appendix A), we found three categories of diagrams illustrating the context and baseline understanding, the review question and scope, and the results. Almost all of them comprised boxes and arrows to indicate causal relationships. This simple design aligns with systematic reviews generating or testing theories about causal relationships. Typically, the authors gave little or no description of how diagrams were developed, unless they had adapted an existing model. Those developed at the protocol stage illustrated components of the background or review question. Diagrams presenting results were developed at any stage in the review process. For each of the three categories, we describe what the diagrams illustrated or explained and signpost the best examples identified.

DESCRIBING THE CONTEXT

Diagrams visualized important psychological, social, systems, and contextual factors that influence particular behaviors, experiences, or views and the relationships among them. These were predominantly part of qualitative evidence syntheses, in which the diagram illustrates a theory of the phenomenon being reviewed, which may then be updated in light of the findings from the analysis. Factors may be represented visually in such diagrams as opposing forces that influence a chain of events¹¹ or in ecological hierarchies illustrating at which level factors influence experiences.¹²

For example, one diagram showed potential threats and expectations of engaging in physical activity for those with bipolar disorder; it also showed the modifying factors and behavioral cues that influence the decision to participate.¹³ The review authors developed the diagram from existing literature, published it in a protocol, and plan to use it for an ongoing framework synthesis. At the review stage, findings will be mapped to the existing diagram, and when findings do not fit the diagram they will be refined.

DESCRIBING THE REVIEW QUESTION AND SCOPE

In our sample, this was the main purpose of diagrams. Diagrams commonly clarified the review question, although wide variation can be seen in the complexity, depth, and scope of these examples. These diagrams were generally developed as part of comparative effectiveness reviews.

Simpler diagrams depicted the review’s participants, intervention, comparison, and outcomes. They tended to be descriptive and display a bird’s-eye view of the review question and inclusion criteria using standard headings and formatting. For example, one diagram outlined participants, intervention, comparison, and outcomes for hypertension screening to reduce the burden of disease¹⁴; another illustrated participants, intervention, comparison, and outcomes for interventions to reduce air pollution and the interventions’ effects on respiratory conditions.¹⁵ The researchers described details of the eligible participants, intervention, and expected outcomes in separate boxes that comprised the full diagram.

More advanced diagrams were explanatory; they typically illustrated and explored one aspect of the participants, intervention, comparison, and outcomes in depth, delineating relationships between diagram components. For example, they depicted a pathway of disease progression and manifestation, the development of a series of direct and intermediary outcomes as a result of the intervention, or the components or steps of an intervention.

Some diagrams merged two or more purposes. One showed both the progressive clinical manifestations and the consequences of dementia.¹⁶ The authors then used the disease pathway to map points where the intervention (animal-assisted therapy) may help. Other diagrams illustrated how similar interventions may vary, such as different forms of peer support to improve health literacy¹⁷ or alternative forms of taxes on unprocessed sugar or sugar-added food to tackle obesity.¹⁸

In addition, we identified three diagrams that combined the two approaches.^19–21 They displayed all elements of the participants, intervention, comparison, and outcomes in a standardized format, with a more explanatory depiction of the series of outcomes resulting from the intervention.

SHOWING RESULTS

For meta-analyses, pathway diagrams may be overlaid with the quantitative results.²² For qualitative syntheses, diagrams arrange findings into an image of the emerging theory, offering explanations or relationships between or among observations.²³ Diagrams sometimes combine quantitative and qualitative results from paired or mixed studies to generate an integrated understanding.²⁴

For example, a diagram that displayed the results of a qualitative synthesis identified factors influencing adherence to antiretroviral therapy in HIV patients.²³ The multiple external and internal influences on an individual, identified through the synthesis, were grouped to demonstrate how they drive engagement and disengagement, as well as good and poor adherence, in a dynamic manner.

WHAT MAKES A GOOD DIAGRAM?

We suggest steps inferred from our analysis and experience as being particularly helpful for developing clear diagrams:

• Choose the purpose of the diagram, whether it is to describe the context, illustrate the question and scope, or show results of a systematic review, before starting to assemble it.

• Identify the key information to be communicated, and acknowledge the complexity of the review while helping the reader make sense of it. Comprehensive diagrams often obscure the message with too much detail. Instead, focus on the point that is being illustrated, rather than incorporating too many ideas. Combining multiple diagrams in one usually reduces clarity.

• Work as a team to capture and share understanding from various perspectives.

• Start simply and expect at least a few iterations. Using a pen and paper or even a flipchart to draft the initial versions of the diagram, rather than doing this electronically, helps clarify and compile thoughts from team members. Keeping all the draft versions captures the evolution of thinking.

• Give the diagram a clear starting point to help readers navigate the diagram more easily.

• Use visual conventions such as reading from left to right, top to bottom, or both to offer a clear flow of ideas.

• Limit the number of arrows to guide the readers’ gaze. Avoid the distraction of multiple, intersecting arrows at various angles. Simplify multiple or complex routes with a topology that allows the reader to pick out pathways clearly.

• Group related information in columns or rows with headings, colors, or shapes to draw attention to key parts, such as activities or outcomes. Use these features selectively to avoid obscuring key relationships with too many layers. For example, employing colors and shapes, rather than colors or shapes, can complicate the picture.

• Use plain language and fewer words without a long legend, key, or acronyms so that the diagram can be understood intuitively.

• Ask others for feedback, including peers and the intended audience, while the diagram is developing.

SIMPLE, CLEAR EXAMPLES

We present three examples showing different sorts of content: (1) the context of a review, (2) the scope and question of a review, and (3) the results of a review. These examples are simple enough to be developed by systematic reviewers without the support of a graphic designer and published without additional color reproduction costs.

Figure 1 demonstrates how diagrams can portray the context of the review. As noted in “Diagrams Describing the Context,” context can be presented in a variety of ways. Here it takes the form of a typical logic model that describes a chain of events. It was created during protocol development for a qualitative review exploring factors influencing physical activity in people with bipolar disorder.¹³ Related factors are grouped in rows, and the diagram is organized into a hierarchy, with biological and social factors at the start (top) influencing complex psychological factors that subsequently lead to behavior change. Again, the diagram reads top to bottom, and, although there are multiple routes through the diagram, the topology has been simplified and arrows are kept to a minimum. Although there is some detailed information, bold text is used to highlight the key message of each box.

FIGURE 1—

Factors That Influence Participation in Physical Activity for People With Bipolar Disorder

Source. McCartan et al.¹³

As depicted in Figure 2, a diagram of the effects of mass deworming²⁴ is easy to interpret, as it has a clear starting point at the top and only three arrows—all of which point downward to indicate a top-to-bottom flow. It can be classed as an example of diagrams that elucidate the review question and scope, as it shows the range of potential outcomes of an intervention (see “Diagrams Describing the Review Question and Scope”). The outcomes are grouped into main effects, mediating pathways, and impacts. These categories are clearly organized in three rows under the appropriate subheading. Language is kept simple, and there is one outcome per box and a maximum of three outcomes per row. Each of these features helps to ensure that the diagram is easy to interpret at first glance, while conveying comprehensive information about intervention effects.

FIGURE 2—

The Effect of Public Health Programs That Regularly Treat All Children With Soil‐Transmitted Helminth Infection in Endemic Areas

Source. Taylor‐Robinson et al.²⁴

Figure A (available as a supplement to the online version of this article at http://www.ajph.org) depicts a theoretical model of the influences on engagement and adherence to antiretroviral therapy.²³ It is an example of a diagram that displays the results (see “Diagrams Showing Results”). In this case, the review’s in-depth qualitative findings were consolidated into one visual image that demonstrates how factors are interrelated. Individual factors are presented in separate boxes, and arrows indicate whether this may lead to engagement or disengagement in the care pathway. Although there are many arrows in the diagram, the authors have ensured that they do not intersect and that the logical flow of the diagram is maintained.

ENHANCING REVIEW DEVELOPMENT

When considering reviews that we have authored or edited, we recognized how the process of constructing a diagram can be useful for developing the review: conceptualizing the problem or findings, advancing thinking, and framing the analysis. Constructing diagrams as a team can help to develop a common language and understanding of the review.

A review of interventions to improve the involvement of older people with multimorbidity in decisions in primary care provides a good example with its Figure 2.²⁵ This diagram evolved during the review. Visualizing the range of interventions and processes provided an opportunity to distinguish three main strategies and identify different aims of different components. Later, outcomes were pictured as intermediate or ultimate endpoints. Gradually, likely pathways linked involvement in decision making to outcomes and effects, such as changes to behavior and health.

Recognizing distinct purposes for variations or components of interventions helped authors to group and analyze the interventions in terms of the wider theoretical context of capability, motivation, and opportunity for behavior change.²⁶ Importantly, the diagram enabled articulation of the links between the different strands of the interventions and the range of outcomes assessed, including those for different actors (i.e., patients, carers, providers, health systems) and reflecting different parts of the pathway between intervention and outcome (e.g., engagement in decision making, health outcomes, treatment burden, evaluation of care, attitudes, resource use, and quality of care).

CONCLUSIONS

We found that diagrams help the reader go straight to the essence of a systematic review. They may illustrate the context and initial understanding as a review begins, the review scope and questions, or the review’s findings. Diagrams from Cochrane more often illustrated the review scope and questions (“Diagrams Describing the Review Question and Scope”), whereas diagrams of context and findings generally came from elsewhere (“Diagrams Describing the Context” and “Diagrams Showing Results”), perhaps reflecting the smaller body of qualitative or mixed-methods research currently available in the Cochrane Library. Good examples simplified complexity and variation, facilitated readers’ navigation of that complexity, and portrayed a coherent picture. Developing diagrams together also helped authors develop a common understanding and guide the review’s development. Good diagrams can, therefore, function as tools for enhancing understanding and for developing reviews.

Authors frequently used diagrams to illustrate their conceptual framework, but they rarely acknowledged or illustrated how diagrams can evolve during the review—a finding that reflects a similar analysis of diagrams in the Cochrane Library and the International Initiative for Impact Evaluation database of systematic reviews.⁹ Nevertheless, visualization of conceptual frameworks is common during the development of framework syntheses.⁸

Our rapid appraisal of systematic review diagrams aligns well with good practice and theory of visual communication of science. Whether diagrams are designed for fellow scientists, policy decision makers, or the wider public, principles of good practice from using diagrams in the form of logic models in reviews—and human-centered design theory more broadly—encourage developing diagrams as a team and inviting feedback from the target audience.^9,10 Depicting essential components and relationships, and grouping related concepts, is achieved by keeping the diagram’s audience in mind while editing and simplifying, as seen when developing diagrams for systematic reviews, and are fundamental graphic design approaches.¹⁰ Appendix B (available as a supplement to the online version of this article at http://www.ajph.org) distils from our analysis practical tips for a broad range of diagrams to enhance systematic reviews.

Guidance specifically for constructing logic models for systematic reviews is available from the Cochrane Infectious Diseases Group²⁷ and in the academic literature.⁹