Visualizations throughout pharmacoepidemiology study planning, implementation, and reporting

Abstract

Transparency is increasingly promoted to instill trust in nonrandomized studies using real‐world data. Graphics and data visualizations support transparency by aiding communication and understanding, and can inform study design and analysis decisions. However, other than graphical representation of a study design and flow diagrams (e.g., a Consolidated Standards of Reporting Trials [CONSORT] like diagram), specific standards on how to maximize validity and transparency with visualization are needed. This paper provides guidance on how to use visualizations throughout the life cycle of a pharmacoepidemiology study—from initial study design to final report—to facilitate rationalized and transparent decision‐making about study design and implementation, and clear communication of study findings. Our intent is to help researchers align their practices with current consensus statements on transparency.

Key points.

• Graphics and data visualizations can support transparency and confidence in real‐world evidence by clearly communicating various aspects in protocol development, implementation, and results dissemination.

• This paper provides guidance on using visualizations throughout the life cycle of a comparative pharmacoepidemiology study.

• Traditional examples, such as directed acyclic graphs and forest plots, as well as lesser known data visualizations and the future role of interactive visualization are discussed.

• Recommendations for the visualization of results are provided.

1. INTRODUCTION

The generation of real‐world evidence (RWE) for regulatory, payer, and clinical decision‐making constitutes a series of complex choices surrounding methodology and interpretation. Increasing promotion of transparency is intended to instill trust in nonrandomized studies using real‐world data. ¹ , ² , ³ Effective graphics and data visualizations support transparency by demystifying decisions about study design choices and interpretation of findings. A precise graph tells a story. ⁴ Good figures are intuitive, reader‐friendly and illuminating.

Although the title of the classic 1984 Weiner article “How to display data badly,” is tongue‐in‐cheek, the sincere purpose is to promote good graphics, those which “display data accurately and clearly.” ⁵ Weiner classifies distinct purposes for data visualization including (among others): data exploration where the data contain a message that visualization can help uncover; communication to illustrate a finding for ease of understanding by others; and decoration to enliven what might otherwise be dull content.

We share the goal of accurate and clear display of data visualization in pharmacoepidemiology research, but we believe good visualization extends beyond reporting results. Visualizations can aid communication during the entire study lifecycle, and reduce the risks of using data inappropriately, implementing a poor study design, misinterpreting results, and losing the reader's attention. Visualization and graphical representation facilitate reproducibility and transparency, enabling trust by clearly communicating operational decisions, rationale, and study findings.

A number of well‐accepted pharmacoepidemiology guidelines, templates and tools exist that, used together, facilitate the planning, conduct, interpretation and transparency of research. ⁶ , ⁷ , ⁸ , ⁹ , ¹⁰ However, specific standards on how to make and document decisions with visualizations are needed.

This paper aims to provide guidance on how to use visualizations throughout the life cycle of a pharmacoepidemiology study, from initial study design to final report, to facilitate rapid, evidence‐based and transparent decision‐making (and therefore, credible RWE generation) (Figure 1). Our paper aligns with existing recommendations for transparent reporting of the conduct and reporting of pharmacoepidemiologic studies. While we focus on comparative hypothesis testing pharmacoepidemiology studies designed to facilitate a causal conclusion about the effects of a therapeutic product, many of these visualization tools can aid descriptive analyses and other types of studies (e.g., refer to Schachterle et al., Aakjær et al., and Zhou et al.). ¹¹ , ¹² , ¹³

FIGURE 1.

Visualizations throughout the life cycle of a pharmacoepidemiology study. Visualizations included in Figure 1 are simple illustrative examples of visualizations included in the paper and Appendix S1.

2. PROTOCOL DEVELOPMENT

For studies that use real world data to evaluate the causal effect of treatments, development of a protocol is an iterative process. Visualization at the protocol development stage can be used to help clarify thinking about design and data issues, avoid biases, and assist protocol reviewers in quickly grasping decisions made. In this section, we will discuss three visualization frameworks that can facilitate rationalized and transparent decision‐making in protocol development: causal structures, graphical depiction of study design, and identifying fit‐for‐purpose data (i.e., data reliability and relevancy which are defined by FDA as “data accuracy, completeness, provenance, and traceability” and “the availability of key data elements [exposure, outcome, covariates] and sufficient numbers of representative patients of study,” respectively).

2.1. Visualizing causal structures

In the absence of randomization, identifying variables that contribute to confounding and require measurement and control is key to valid study design, as well as assessing data fitness (i.e., systematically assessing whether data are fit for a particular purpose/research question). Visualization of the hypothesized causal relationships for the primary treatment‐comparator‐outcome objective—and often for other key relationships specified in secondary objectives—helps the researcher identify variables that should be controlled (e.g., a common cause of treatment and outcome) and those that should not (e.g., a mediating variable). Directed acyclic graphs (DAGs) are visualizations of these theorized causal relationships with rules about which variables should be controlled. ¹⁴ , ¹⁵ , ¹⁶ These graphs allow researchers to distinguish the relationships between the exposure, outcome, and potential confounders, intermediate causes, and colliders. For example, to conceptualize the causal structure of prenatal selective serotonin reuptake inhibitor (SSRI) medication (exposure) and attention deficit/hyperactivity disorder (ADHD) (outcome), Wood et al. ¹⁷ identified potential confounders warranting conditioning, such as “other psychiatric illness” (Figure S1A). The authors also identified other factors including gestational age (mediator) and live birth (collider) to avoid introducing bias by inadvertently controlling for these factors. In DAG construction, variable identification and potential relationship with the treatment and outcome are typically based on literature review and input from clinical subject‐matter experts. One limitation of the DAGs is the potentially subjective and speculative identification of variables based on background knowledge. In a systematic review of DAGs, Tennant et al. also observed variability, including oversimplification, of DAG reporting in the literature. Despite this, DAG construction is an important first step and following these rules enables identification of the minimally sufficient set of covariates, avoids model overfitting, and targets the assessment of whether a particular dataset is fit‐for‐purpose. While these diagrams can be quite complicated (e.g., see Reference [18]; Figure S1B) this critical step can be simplified by using tools such as DAGitty (http://www.dagitty.net), a free web‐ or R‐based tool for creating and analyzing causal diagrams.

2.2. Visualizing study design

The role of time in a pharmacoepidemiology study is most clearly communicated through visualization. Following the target trial framework, ¹⁹ database studies are carried out as closely as possible to a randomized trial, with the index date (i.e., “time zero”) defined by treatment initiation rather than set by a randomized intervention. As such, clear definitions of time zero, patient history, and follow‐up are critical to understanding the database study design, and visualization can greatly improve understanding of these concepts. Visualization can also clarify the relationship between calendar time and patient event time, which is time relative to a specific event, such as the first prescription of a drug of interest (Figure 2A). Importantly, visualizing the study design and aligning patient eligibility and treatment initiation to the start of follow‐up avoids introducing biases, such as immortal time bias. ²⁰

FIGURE 2.

Calendar time, patient event time, and study design diagrams. A, Patient events for four different patients are visualized, arranged according to calendar time (left) or patient event time (right). Patient event time is defined relative to an index date, in this case the first prescription of an ACE‐inhibitor (ACEI) or angiotensin receptor blocker (ARB). Events recorded as spanning intervals of time (e.g., hospitalizations and drug prescriptions) are represented as boxes, while events recorded in a single moment of time (e.g., outpatient visits) are represented as lines. B, An example study design diagram. Source: Adapted from www.repeatinitiative.org/projects.html

2.2.1. Visualizing time anchors and time windows with study design diagrams

A recent framework for graphical depiction of study design summarizes the temporality of key study parameters implemented to create an analytic dataset anchored on patient event time. ²¹ , ²² This graphical framework defines several types of study time anchors. Base anchors are defined in calendar time and describe the source database. Primary anchors are defined in patient event time and are used to specify time zero and other key anchor points. Second order anchors, such as windows to measure inclusion–exclusion, covariates, washout or follow up are defined relative to the primary anchor(s).

The study design diagram is intended to be read from top to bottom, showing the sequence of actions taken to create the analytic study population (Figure 2B). A vertical gray arrow is used to represent time zero, the point at which the study entry defining criterion is met and the patient enters the cohort. The size and location of the horizontal bars visually show the assessment windows for inclusion–exclusion criteria, covariates and follow up relative to time zero. In addition to the visual representation, bracketed numbers within each bar specify the exact window length. Schneeweiss et al. provide examples of diagrams using different study designs, which are available in the appendices of their article. ²¹ This graphical framework for study design has recently been expanded to include visual representation of data observability – an issue that is particularly salient for studies that make use of EHR data within a fragmented healthcare system. ²² Open‐source PowerPoint templates ²¹ and an R shiny application can be used to develop study design diagrams for protocols and publications.

2.2.2. Visualizing study measurements

Measurement is the identification of specific events, dates, or other values of interest in a longitudinal dataset in order to generate time anchors, covariates, outcomes, and other components of an analytic dataset. Although measurements require precise definitions in order to preserve transparency and reproducibility, textual definitions of measurements are often ambiguous or do not consider nuances of the underlying data. For example, a time anchor measurement defined as “follow‐up starting after the first usage of an ACE‐inhibitor” is ambiguous, as it is unclear if follow‐up starts on the first or last or other day relative to the use of a drug, and it is unclear if “usage” refers to a single dispensing of a drug or a complete course of treatment including multiple dispensings. Furthermore, if a “course of treatment” is being measured, it is unclear how that is defined, such as with respect to how overlapping drug dispensings or gaps between drug dispensings are handled. To gain insight into improving measurement definitions, it can be helpful to visualize case examples of patient events and how measurement is applied to those cases. For example, visualizing how time anchor measurement aligns against drug usage patterns in the data highlights the need to more explicitly define the choice of which day to anchor on (Figure 3).

FIGURE 3.

Measurement visualization. Definitions of study measurements such as time anchors may be made ambiguous by insufficiently detailed textual descriptions or may be complicated by nuances of underlying data. For example, consider the definition “Follow‐up starts after the first use of an ACE‐inhibitor (ACE‐I).” Such a definition is ambiguous because “after the first usage” is not well‐defined, as shown here. Furthermore, periods of drug usage recorded in the data may overlap in time or have gaps between them, requiring the researcher to make assumptions about how a drug is used by a patient. Examples of this are shown here, depicting drug data for a single patient and how different interpretations of the follow‐up start date may be applied to it. Illustration of case examples of patient data and how a measurement definition accommodates them can help the researcher be more precise and take into account data nuances during protocol development

2.3. Visualizing data fitness

Rationalizing a chosen data source for a given comparative observational study is crucial to ensure actual and perceived objectivity. Visualization can help stakeholders understand the researcher's decision‐making process, and why a particular data source meets or does not meet the needs of the study. In September 2021, the FDA issued draft guidance on assessing real‐world data for regulatory decision‐making which discusses considerations for using data derived from existing electronic health records and administrative claims data, including data reliability and data relevance. ²³ Assessing whether the data are fit‐for‐purpose is contextual and contingent on the study's research question. ²³ Following the steps laid out in Gatto et al. ²⁴ for identifying fit‐for‐purpose data, a CONSORT‐like diagram ²⁵ , ²⁶ can show considered data sources, those included/excluded based on pre‐specified criteria (evaluated without assessing associations among treatments and outcomes of interest), and those chosen for more detailed feasibility assessment.

To summarize the assessment findings, a heat map can demonstrate how each dataset performed on key study elements and visually communicate the datasets' fitness‐for‐purpose or lack thereof. ²⁴ The heat map is made by ranking each element assessed for each data source using colors to indicate if the data, for example, “fully meets”, “partially meets,” or “does not meet” the needs of the study (Figure S2). A quick visual scan then allows the reader to see where “fully meets” assessments are concentrated, and judge if a data source is sufficiently fit. A table containing the underlying information can be supplied for further detail.

3. IMPLEMENTATION: DIAGNOSTICS

The implementation stage of a study can be segmented into diagnostic and inferential phases. The diagnostic phase refers to the checks performed to evaluate key assumptions and fitness of the proposed study design, while the inferential phase refers to the conduct of planned analyses following locking of the dataset and protocol. Visualizations of diagnostic checks can help researchers refine study design decisions prior to conducting any inferential analysis and demonstrate the fitness of the design to stakeholders. This section provides examples of graphical diagnostic checks to evaluate these assumptions, as well as assumptions underlying statistical models.

3.1. Visualizations to check positivity and confounder balance in studies of causal effects

The positivity assumption—the assumption that each person under study is eligible for each treatment of interest—requires that there be a non‐zero probability of treatment receipt within all confounder strata. ²⁷ , ²⁸ , ²⁹ , ³⁰ One practical way to check this assumption across the range of confounders is to inspect propensity score distributions by treatment group; propensity scores close to zero or one suggest possible violations or near violations of positivity (Figure 4A). Trimming (excluding) patients with nonoverlapping propensity scores in the tails of the distribution helps address violations of the positivity assumption. Trimming of patients with extreme propensity scores would be prespecified in the protocol and transparently reported. For example, in a study of mortality and corticosteroid versus non‐corticosteroid use among hospitalized COVID‐19 patients, this check of positivity was pre‐specified in the protocol and performed before implementing any inferential analysis (study protocol is available on ClinicalTrials.gov, NCT04926571).

FIGURE 4.

Visualizations to assess for positivity and confounder balance. A, A propensity score density plot, adapted from Webster‐Clark et al. ²⁵ B, A plot of standardized differences, adapted from Austin PC ²⁷

The conditional exchangeability assumption requires that, conditional on confounders, the counterfactual outcome risk in treated individuals, had they not received the treatment, would have been the same as the outcome risk in the untreated/comparator group and vice versa. In other words, potential outcomes and treatment status are independent conditional on the confounders. A typical check of this assumption is the examination, after weighting, matching or other methods, of absolute standardized differences between treatment groups in the distribution of potential confounding variables measured at baseline. As a rule of thumb, an absolute standardized mean difference of 0.1 or less suggests adequate balance between comparison groups. ³¹ Residual imbalance may indicate that confounding control and/or model specification is insufficient. Plotting standardized differences is a useful visualization tool to evaluate and demonstrate covariate balance (Figure 4B). Additional visualizations may be used to assess the robustness of confounding control. Tarzare et al. ³² summarize helpful visualization tools when using high‐dimensional propensity score (HdPS), which use automated methods to select covariates. Rassen et al. ³³ suggest sequentially plotting treatment effect estimates with increasing number of empirically identified variables controlled to determine if controlling more variables helped with confounding control or if a parsimonious model was equally effective.

3.2. Visualizations to check statistical assumptions

Depending on the statistical methods employed, the model's assumptions and goodness‐of‐fit can be graphically displayed. For example, the Cox proportional hazard regression model used for time‐to‐event data requires that the hazard functions for different groups are proportional over time. Testing for proportionality is often based on visual inspections of Schoenfeld's residuals (Figure S3). ³⁴ , ³⁵ Violation of the proportional hazard assumptions will lead to invalid inference (i.e., the standard errors will be incorrect). This limitation can be addressed by computing confidence intervals using bootstrapping techniques. ³⁶ As with any analytic diagnostics, whether for models or other statistical approaches, reporting these visual diagnostics helps the investigator to justify their final choices.

4. RESULTS DISSEMINATION

Pharmacoepidemiology reporting standards mainly focus on the textual information required in study reports and manuscripts. ⁷ , ⁹ , ¹⁰ , ³⁷ Tables are foundational since interpretation and decision‐making require exact numerical estimates and confidence intervals. However, representing results graphically can convey distinct ideas or patterns, facilitating quicker comprehension by readers. Results readily lend themselves to both tabular and visual presentation. In this section, we discuss common practices for creating these visualizations. We also provide general recommendations and considerations for developing such reporting visuals (Table S1).

4.1. Cohort attrition: Flow diagrams

A core element of most reporting guidelines, flow diagrams are used to describe cohort attrition, which is the movement of patients in and out of the study (Figure S4). ⁹ , ²⁵ , ²⁶ , ³⁸ , ³⁹ , ⁴⁰ These diagrams are intended to convey the total number of patients included in the study, the number of patients removed at each exclusion step, the number of patients lost to follow‐up, and the final number of patients available for analysis, with information reported separately for each treatment arm as applicable. This information is a key element supporting study transparency, as it can help reveal potential sources of bias or generalizability limitations.

4.2. Effect estimates: Forest plots

Forest plots are used to illustrate effect estimates (e.g., odds ratios, hazard ratios, or mean differences) when more than one estimate is being reported. This allows effect estimates derived from different cohorts, subgroups, or statistical models to be visualized and compared in a single plot. Core elements of these plots include point estimates and their confidence intervals, and a vertical reference line indicating null effect (Figure 5). Ratio results should be plotted on a log scale so that, for example, risk ratios of 0.5 and 2.0 (half and double the risk, respectively) appear at an equal distance from the null.

FIGURE 5.

Example and recommendations for forest plots

4.3. Time‐to‐event analysis: Kaplan–Meier plots

Kaplan–Meier plots are used to visualize time‐to‐event analyses, where the event is the outcome of interest such as death, disease occurrence, or disease recovery. The vertical axis of these plots typically shows either survival probability (with curves starting at 1 and descending as time progresses) or its converse, the cumulative incidence (with curves starting at 0 and ascending), while the horizontal axis shows time (Figure 6A). In general, including zero on axes helps ground the reader in the magnitude of the effect being plotted; however, in survival plots of rare outcomes, including zero may mask differences of effect between the groups (since most values will be close to the maximum of 100%) (Figure 6B). In such cases, it may be preferable to plot cumulative incidence, or include both a survival plot with zero (to indicate the rarity of the outcome) and a “zoomed in” plot clearly showing any difference between the survival curves.

FIGURE 6.

Example and recommendations for Kaplan–Meier plots

A limitation of Kaplan–Meier curves is that they can only depict unadjusted survival probabilities. Alternatively, survival probabilities can be derived and plotted from an adjusted Cox Proportional Hazards Model. This approach requires specifying values for all the covariates used in the model. Therefore, the shape of the survival curves will be a function of the values assigned to each of the covariates. Methods such as the corrected group prognosis method have been developed to address this limitation. ⁴¹

4.4. Repeated measures: Line charts and bar charts

Pharmacoepidemiology studies may include outcomes of repeated measures, such as lab values or numerical disease scores measured at different points in time. As such data naturally capture trends over time, they lend themselves well to visualization through multiple chart options, including line charts, bar charts, box plots, or spaghetti plots (Figure S5A). Although such plots are generally straight‐forward, consideration must be given to issues such as representing multiple dimensions in a single plot, or dealing with overlapping plot elements (Figure S5B).

4.5. Changes of state: Sankey diagrams

Changes of the state of a patient over time, such as medication switching, disease stage progression, or movement through treatment pathways, can be visualized using Sankey diagrams. ⁴² These diagrams involve nodes arranged along the vertical axis, and another dimension (typically study time or a designation of a phase, such as line of therapy) arranged on the horizontal axis, with node height proportional to the number of patients at that node, and flows that depict state transition probabilities (Figure 7). State transition probabilities should be estimated using appropriate methods for censored data to account for disenrollment and other forms of censoring. Given the highly combinatorial nature of these types of analyses, Sankey plots can be an efficient way to represent data that otherwise might be difficult to interpret in tabular format. Sunburst diagrams can also represent some of the same information, with an inner ring showing the first phase and outer rings showing consecutively later phases but are less common.

FIGURE 7.

Sankey plots. A, An example of a Sankey plot, with colors representing transitions between all groups. B, The same plot but simplified, where colors are used only to highlight specific transitions of interest. In this case, transitions from Drug A to any other drug are highlighted. Other options to focus the presentation are also possible, such as highlighting transitions to higher lines of therapy (e.g., Drug A to B, B to C, A to C, etc.)

5. FUTURE OF VISUALIZATION

5.1. Animated and interactive visualizations

In recent years, there has been rapid development and dissemination of easy‐to‐use tools for creating web‐based, interactive visualizations and tables. These technologies allow a tremendous amount of information to be displayed and efficiently explored. Interactive output can be created by most modern statistical analysis systems, including open‐source software such as R.

There are many potentially valuable applications of these technologies in pharmacoepidemiology. For example, many comparative effectiveness studies involve multiple treatment contrasts, different cohorts, multiple outcomes, many subgroups of interest, and associated sensitivity analyses. Each analysis may involve estimation of cumulative incidence functions in addition to the previously mentioned diagnostics, such as inverse probability of treatment weighted table 1 information, and propensity score histograms. Frequently these results end up being included in online supplementary material or discussed in the paper but not presented. However, interactive data visualization tools can allow the user to easily browse a large volume of results from the many combinations of analyses that are performed. For example, in a negative control outcome study of lipid‐lowering therapies, ⁴³ results from multiple outcomes, cohorts, and subgroups are easily explored using simple interactive graphs and tables. ⁴⁴ Similarly, the Observational Health Data Science Initiative has created R packages to allow results from pharmacoepidemiologic studies to be explored via R/Shiny applications. ⁴⁵ In addition, the US FDA Sentinel Initiative will be releasing Sentinel Views, a dashboard with interactive visualizations, to help increase transparency and aid interpretation of post‐marketing safety data (https://views.sentinelsystem.org).

It is also becoming increasingly feasible to create a wide range of animated visualizations. These can be used to help convey movement and dynamism. Animation can also help a presenter tell a story to communicate findings. For example, in a classic TED Talk, animations were used to effectively convey information about country‐level changes in health and economic development from 1960 to the present day. ⁴⁶

6. CONSIDERATIONS

6.1. Cautionary considerations

Pharmacoepidemiology visualizations—whether traditional static or more novel interactive visuals—seek to explain the why and how a given study was performed, aid interpretation of results, and provide full transparency to support RWE credibility. However, unintended adverse consequences may result from graphics for several reasons. First, the best way to display multidimensional data in a clear and unambiguous manner may not always be obvious. Developing visualizations is often an iterative process; trying various techniques and soliciting feedback are important to ensure the reader will accurately interpret the visual as intended by the authors. Poorly constructed graphs can inadvertently confuse or mislead the reader. Overly complicated diagrams run the risk of subjective interpretation. Second, with interactivity, the use of the visualization may require careful consideration of how the user can interface with the tool to ensure visualization does not lead to varied interpretation. Third, while well‐crafted, easy‐to‐interpret visualizations can aid in transparency, a potential pitfall is critical information loss from attempts to simplify the graphs for readability; researchers must ensure visualizations appropriately balance reader‐friendliness with the amount of information needed for correct interpretation. Lastly, certain visualizations may require a reader to fully understand underlying assumptions to accurately interpret complex information; the researcher must be careful to articulate these assumptions.

6.2. General style recommendations

While some choices in plot design, such as whether to include 0 on graph axes, are purely functional, in general, a focus on design can improve a figure's visual appeal, attract and retain reader interest, and communicate ideas more clearly (Figure 8 and Figure S6). ⁴⁷ , ⁴⁸ The default graphical output provided by SAS, R, and Excel tends to be lackluster and seen as “unformatted,” so adjusting colors, weights and fills, and text is often time well‐spent. Inspiration for visually compelling content can be found in many sources, such as medical journals with dedicated graphics departments like the New England Journal of Medicine and JAMA, more general publications like The Economist, or digital publications like FiveThirtyEight or the online New York Times.

FIGURE 8.

Improving the visual style of figures. A, Plots can be Simplified by removing design elements that do not contribute to understanding of the data, such as superfluous shading or line borders, as shown here. B, Redundant information should be removed from plots, such as repeated percentage signs on axis labels, or over‐abundant tick mark labels. C, Careful consideration should be given to the use of space in a plot. In this example of a forest plot, there is an over‐abundant amount of spacing between rows, columns, and the scale of the X‐axis on the forest plot. D, An example of the same plot but with improved use of space

7. CONCLUSION

Increased use of visualizations can improve the efficiency and understanding of non‐randomized pharmacoepidemiology study design and results by clinicians, treatment guideline committees, regulators, industry, and payers. We have described a series of visualizations that serve decision‐making and transparency throughout the study lifecycle, focused on study design, study implementation, analytic strategy, study population, and findings and aligned with current consensus statements on transparency. ¹ , ² , ³ , ⁶ , ¹⁰ , ⁴⁹

ETHICS STATEMENT

The authors state that no ethical approval was needed.

CONFLICT OF INTEREST

Dr. Gatto, Ms. Mattox and Dr. Rassen are employees of Aetion, Inc., with stock options or existing equity. Dr. Gatto also owns stock in Pfizer Inc. Dr. Wang receives salary support as PI for investigator‐initiated grants to Brigham and Women's Hospital from Novartis, Johnson & Johnson, and Boehringer Ingelheim and is a consultant to Aetion for unrelated work. She is also PI on grants from the National Institute of Aging, Laura and John Arnold Foundation and the FDA Sentinel Initiative. She co‐directs the REPEAT Initiative, a non‐profit program with projects focused on increasing transparency and reproducibility of database studies. Dr. Murk is a contractor of and owns shares in Aetion, Inc., a technology company that provides analytic software and services to the healthcare industry. Dr. Brookhart serves on scientific advisory committees for Amgen, Astellas/Seagen, Atara Biotherapeutics, Brigham and Women's Hospital, Gilead, Kite, NIDDK, and Vertex; he receives consulting fees and owns equity in Target RWE. Dr. Bate is a full‐time employee of GSK and holds stock and stock options at Pfizer and GSK. Dr. Schneeweiss is participating in investigator‐initiated grants to the Brigham and Women's Hospital from Boehringer Ingelheim and UCB unrelated to the topic of this study. He is a consultant to Aetion Inc., a software manufacturer of which he owns equity. He is an advisor to Temedica GmbH, a patient‐oriented data generation company. His interests were declared, reviewed, and approved by the Brigham and Women's Hospital in accordance with their institutional compliance policies.

Supporting information

Appendix S1: Supporting information.

Gatto NM, Wang SV, Murk W, et al. Visualizations throughout pharmacoepidemiology study planning, implementation, and reporting. Pharmacoepidemiol Drug Saf. 2022;31(11):1140‐1152. doi: 10.1002/pds.5529