A Statistical Analysis Plan Template for Observational Studies: Promoting Quality and Rigor in Research

Abstract

Rigorous and transparent research practices are essential for trustworthy scientific findings, particularly in observational studies where data-driven analyses carry risks of questionable research practices. This paper introduces a statistical analysis plan (SAP) template specifically designed for observational research, an area where guidance on SAP development is crucially lacking. The template offers clear guidelines for prespecifying key aspects of the analysis. The guidance encompasses essential SAP components, including study objectives, measures and variables, and analytical methods, as well as administrative details to support documentation and reproducibility. Designed for broad useability, the template is intended to support researchers, statisticians, students, and interdisciplinary teams across clinical, academic, industry, and government sectors. By adopting this template, researchers can strengthen study integrity, reduce ad hoc analytic modifications, demonstrate the avoidance of questionable research practices such as p-hacking, and contribute to robust and reliable findings in observational research.

1. Introduction

Trustworthy scientific findings rely on rigorous, objective, and transparent practices. One valuable practice for enhancing research integrity is the development of a statistical analysis plan (SAP). A SAP is a comprehensive document that prespecifies the statistical methods for a study and serves as a blueprint for data analysis and interpretation (International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use, 1998; PSI Professional Standards Working Party, 1994). This paper addresses a critical gap in the literature by providing a SAP template tailored for observational studies. The template aims to promote research quality and reproducibility across disciplines, including medicine, social sciences, and public health. This template may also be relevant to other disciplines, although it is primarily grounded in the author’s experiences in medicine and psychology research.

For observational studies, the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) guidelines provide widely recognized standards for transparent reporting (von Elm et al., 2007). However, STROBE focuses on completed studies, offering limited guidance on prespecifying statistical analyses before data inspection. The SAP template presented here complements STROBE by supporting transparent, preplanned analysis in observational research, facilitating alignment with open science practices and improving reporting clarity.

1.1. Growing Importance of SAPs

The adoption of SAPs has gained significant momentum across various scientific contexts, including research settings, peer-reviewed journals, funding bodies, biobanks, and consortia. In the United States, SAPs have been required by law for drug trials since 1997 (“Food and Drug Administration Modernization Act of 1997,” 1997), and study preregistration became a requirement for publishing clinical trial results in leading medical journals in 2005 (DeAngelis et al., 2004). Historically, many researchers conducted analyses without SAPs, leading to flexible and ad hoc approaches that raise the risk of questionable research practices (QRPs), such as p-hacking, selective reporting, and HARKing (i.e., hypothesizing after results are known) (Banks et al., 2016; Hardwicke & Wagenmakers, 2023; Stefan & Schönbrodt, 2023). QRPs not only undermine scientific integrity but can also form a basis for scientific misconduct (Stefan & Schönbrodt, 2023).

While study protocols often outline planned statistical methods, they typically lack the technical detail required to document an unambiguous analytic plan (Yuan et al., 2019). SAPs bridge this gap, offering a rigorous and transparent document that enhances reproducibility and safeguards against biases and QRPs (Cressman & Sharp, 2022). The growing adoption of open science practices – such as study protocol registration, data and code sharing, collaborative platforms, and the growing availability of open access journals – further underscores the importance of SAPs in enhancing public trust, accelerating discovery, and supporting evidence-based decision-making (Ng et al., 2024; Onukwugha, 2013).

1.2. SAP Guidance for Observational Studies

While SAPs are widely promoted as essential to rigorous and transparent research, their development and writing often lack specific guidance, especially for observational studies (Cressman & Sharp, 2022; Hiemstra et al., 2019). Clinical trial SAPs benefit from more established conventions and requirements, driven by ethical and scientific standards and regulatory oversight from sponsors, regulatory agencies, and publicly funded research bodies. Authors of clinical trial SAPs frequently work in industry or contract research organizations, and particularly in North America, may have routine access to internal guidelines and templates. In contrast, researchers in academia or clinical healthcare settings, in resource-limited environments, and those conducting observational research, often lack comparable resources and support.

To address the lack of guidance in clinical trials, Gamble and colleagues (2017) developed a minimum set of essential items for inclusion in clinical trial SAPs through consultation with stakeholders, such as funders, regulators, statisticians, previous guideline authors, journal editors, and the pharmaceutical industry. However, this guidance was not designed for observational studies, leaving an unmet need in this area. Although these items have been evaluated for applicability to observational studies (Hiemstra et al., 2019; Yuan et al., 2019), the framework’s technicality and scope may render it unsuitable or impractical in applied contexts—particularly in clinical, medical, non-profit, or academic settings with limited research resources. Excessively stringent and technical requirements can inadvertently increase administrative burden, posing barriers to study feasibility and advancement of science (Olken, 2015).

Nonetheless, researchers conducting observational studies have been increasingly encouraged to develop and use SAPs, including by journal editors advocating for their adoption (Eisenach et al., 2016; Hiemstra et al., 2019; Islam et al., 2022; Onukwugha, 2013; Thomas & Peterson, 2012; Thor et al., 2020). Despite this increasing support, there remains a critical shortage of published guidance (Eisenach et al., 2016; Hiemstra et al., 2019; Onukwugha, 2013; Yuan et al., 2019) and templates (Cressman & Sharp, 2022) tailored specifically to observational studies. Given the complexity and variability of these study designs, there is a pressing need for SAP guidance and templates that strike a balance between methodological rigor and real-world feasibility. Closing this gap is essential for reducing QRPs and improving the transparency of statistical procedures.

2. Template Development Process

The template was developed from the author’s two decades of professional experience as a research psychologist and biostatistician across healthcare, nonprofit, and academic settings in the US and Australia, drawing on experience preparing SAPs for various studies. It incorporates best practices from observational research and clinical trial literature (Gamble et al., 2017; International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use, 1998, 2019; Stevens et al., 2023; von Elm et al., 2007). Additionally, the template aligns with the STROBE guidelines (von Elm et al., 2007) to support efficient preparation of the study manuscript for publication. Practicality and user-friendliness are emphasized over technical complexity, making it suitable for researchers, statisticians, and students across diverse fields.

3. Practical Guidance for Developing and Implementing SAPs

3.1. Who Can Benefit from Using This Template?

The SAP template is designed for a wide audience, including statisticians, researchers in clinical, academic, industry, government, and non-profit sectors, and students completing dissertations. It may also assist study management teams, biobanks, and consortium groups involved in study approvals and data access. The template is intended for use by teams with statistical expertise, either formally through a statistics degree or through interdisciplinary training (e.g., psychology). The template is primarily designed for biomedical and social sciences but can be adapted for other disciplines.

3.2. When to Develop the SAP

The SAP should be developed during the initial stages of research planning, ideally concurrently with the study protocol (Kahan et al., 2020; Stevens et al., 2023). Early development allows for a thoughtful design and helps mitigate bias from ad hoc changes. Finalizing the SAP prior to accessing or analyzing data is ideal.

3.3. Who Should Be Involved

The SAP is typically co-developed by the principal investigator (PI) and the statistician/data analyst, with the statistician taking primary responsibility for drafting the document. The PI provides the statistician with key information on the study’s research questions, hypotheses, population details (e.g., data sources, inclusion/exclusion criteria, recruitment methods), study design, measures, and any relevant background (e.g., study protocol, grant application, pilot findings, or related prior studies). The statistician drafts the SAP, plans the appropriate analysis methods, advises on study design, and clearly outlines methodological procedures (e.g., variable derivation from raw data, case-control matching procedure). The PI writes the background and rationale section. Other team members critically review the SAP to ensure analyses are well-aligned with study aims and address meaningful scientific questions. They may also apply domain-specific or clinical expertise, for example, to refine key variables, identify confounders or covariates, and flag design limitations or opportunities to strengthen the planned analyses. In some cases, the PI and data analyst are the same person, for example, a student or early-career investigator conducting their own research under the guidance of a supervisor or mentor.

4. Content of the Statistical Analysis Plan

A well-developed SAP includes various elements essential to a study’s analysis. The SAP template developed for this work, with accompanying guidance, is provided in the Online Supplement and permanently archived on the Open Science Framework (OSF) (https://osf.io/xrhva/), which also hosts examples of completed SAPs and additional resources. An overview of the SAP template is outlined in Table 1.

Table 1.

Structure and sections of the statistical analysis plan template.

Section	Title
1.	Administrative information
2.	Introduction
2.1	Background and rationale
2.2	Aims, objectives, and hypotheses
3	Study methods
3.1	Study design
3.2	Study population
3.3	Inclusion and exclusion criteria
3.4	Recruitment
3.5*	Data sources
4	Measures and variables
5	Statistical analysis
5.1	Selection of sample size/statistical power
5.2	Primary analysis(es)
5.3	Sensitivity analysis(es)
5.4	Missing data
5.5	Analysis populations
5.6	Confidence intervals and P values
6 *	Preregistration and study reporting
6.1	Preregistration
6.2	Study reporting
7	Revision history
8	References

Note.

^*Section 6 and 3.5 are optional; all other sections are considered essential.

The following subsections provide a brief description of each major section of the template, beginning with administrative information.

4.1. Administrative Information

The first section of the SAP template is 1. Administrative Information. This section includes information such as the study title and the roles and responsibilities of the individuals involved. These roles can be tailored to the project’s needs. Signatures from key individuals, including the statistician and principal investigator, may be included, and the SAP’s date and version. A list of abbreviations is recommended for clarity and readability. These administrative items have been outlined in existing SAP literature (Gamble et al., 2017; Janzen & Michler, 2021; Stevens et al., 2023)

4.2. Introduction

The 2. Introduction section of the template consists of two subsections: 2.1 Background and Rationale, and 2.2 Aims, Objectives, and Hypotheses. The Background and Rationale provides context for the study and may range from one to several paragraphs in length. The Aims, Objectives and Hypotheses clearly define these elements, which are typically drawn from the study protocol or grant application (Doody & Bailey, 2016; Farrugia et al., 2010). An aim provides a broad statement of the study’s main purpose, while research objectives are specific and testable, and break the aim down into detailed, actionable steps. Pre-specifying hypotheses helps avoid the QRP of altering hypotheses after data analysis (Andrade, 2021).

A well-structured SAP typically includes one overarching aim supported by multiple related objectives. If a study includes entirely distinct aims, such as those addressing different populations, data sources, or research questions, separate SAPs are recommended.

Each objective should reference the population being analyzed, which should be described in the Analysis Populations subsection of the 5. Statistical Analysis section. When writing objectives and hypotheses, consistency with established frameworks, such as PEO (P: Population, E: Exposure, O: Outcomes) or PICO (P: Population, I: Intervention, C: Comparison, O: Outcome) is crucial (Hosseini et al., 2024). These frameworks provide structured approaches to clearly define and report the key components of research questions, and improve clarity and reproducibility in study design and reporting.

For secondary studies linked to existing projects, such as biobanks, consortium studies, or longrunning pregnancy or birth cohort studies, tailor the Introduction section to reflect the specific focus of the secondary study, rather than those of the primary study, which may have addressed broader or different aims. This helps the research team work together to develop the study’s aim, objectives, and hypotheses by sharing important background information during the SAP drafting.

4.3. Study Methods

The 3. Study Methods section includes subsections on 3.1 Study Design, 3.2 Study Population, 3.3 Inclusion and Exclusion Criteria, 3.4 Recruitment, and 3.5 Data Sources. Although typically documented in the study protocol, including study methods ensures the SAP serves as a standalone document, and enhances clarity for external readers. For secondary analyses without a study protocol, the Study Methods section keeps the design prominent in study team member’s minds, ensuring details are readily accessible when refining the aim, objectives, and hypotheses.

To help prepare the Study Design subsection, the template includes a list of study designs (e.g., “nested matched case-control study”) to support STROBE-compliant reporting of the study design. The Study Population subsection provides an overview of the target population, such as a specific diagnostic group or other relevant characteristics (e.g., inpatients, adolescents). If diagnoses are involved, the nomenclature and methods are included here or in the next subsection. The Inclusion and Exclusion Criteria subsection specifies the conditions under which participants are eligible or excluded from the study, such as specific diagnoses, age ranges, gender, or other relevant characteristics. The methods used should be clear enough to ensure reproducibility. Recruitment includes information about recruitment procedures and identifies dates or periods for recruitment, exposure, follow-up or data collection, and geographical location and settings (e.g., medical clinic, social media). Data Sources documents data origins, such as data freeze and data release dates, biobank names, cohort study titles (i.e., for secondary studies), or registers used in register-based studies.

4.4. Measures and Variables

The next section, 4. Measures and Variables, details the measures and variables to be used in the analysis. Customized subheadings enhance clarity and for efficiency, can be crafted by SAP authors to align seamlessly with those planned for the Measures section of the study manuscript. Each study measure is detailed, including citations for the development and scoring of psychometric scales, with clarity on which scales and subscales are used. The timing of assessments should be clearly outlined throughout this section.

The section may begin with the reporting of the variables that will be used to describe the sample characteristics (e.g., age, gender, illness severity). Ensuring reproducibility requires explicit documentation of all variables, including primary and secondary outcomes (if applicable), independent variables (exposures/predictors), dependent variables (outcomes/responses), and other types (e.g., effect modifier, mediator, moderator) and criteria used to define cases versus controls, if applicable (International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use, 1998). The variables in this section should align with those in the Aims, Objectives, and Hypotheses subsection of the 1. Introduction and 5. Statistical Analysis section. Vague variable and outcome definitions, incomplete prespecification, inconsistencies, and failure to prespecify —such as omitting a variable mentioned in the hypothesis from the Measures and Variables section—can undermine research credibility (Kahan et al., 2020). Clarity and consistency across the SAP are essential.

Observational studies, which lack the controlled experimental procedures of clinical trials, are particularly prone to confounding bias (Hiemstra et al., 2019). Hence, this section should clearly identify covariates and confounds. Prespecification demonstrates the avoidance of QRPs. Without prespecification, covariate selection can become a p-hacking strategy, whereby the researcher flexibility selects, reports, and analyzes covariates to achieve significant p-values (Stefan & Schönbrodt, 2023).

Variable features, such as the measurement level (i.e., ratio, interval, ordinal, nominal) can be outlined here, particularly for measures that are not widely recognized in the subject matter area. Both raw and calculated variables (synthetic variables) should be described, including calculation methods. If continuous variables will be categorized, provide the boundaries and definitions, which are required for STROBE-compliant reporting in the manuscript (International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use, 1998; von Elm et al., 2007). Including a data-dictionary style table can improve clarity and usability by summarizing variables and calculations. For studies involving diagnoses or selection procedures (e.g., identifying a high-risk group in a prevention study), include details here of the nomenclature (e.g., Diagnostic and Statistical Manual of Mental Disorders 5th edition), measures, and thresholds if these were not provided in the Study Methods.

4.5. Statistical Analysis

The 5. Statistical Analysis section includes the following subsections: 5.1 Selection of Sample Size/Power Calculation; 5.2 Primary Analysis(es); 5.3 Sensitivity Analysis(es); 5.4 Missing Data; 5.5 Analysis Populations; and 5.6 Confidence Intervals and P values. Ideally, the analysis should be specified in enough detail such that an independent data analyst could reproduce the statistical analysis (Kahan et al., 2020). All of the subsections described next are documented in prior guidance (Gamble et al., 2017; International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use, 1998, 2019; von Elm et al., 2007).

The Selection of Sample Size subsection explains the rationale for the sample size, such as whether it was based on a statistical power calculation with details of the calculation. Other rationales, such as the use of an existing dataset or pilot study funding, might be specified otherwise.

The Primary Analysis(es) section is fundamental to the SAP. It specifies the statistical models and methods that will be used to analyze the data. This includes outlining specific analyses (e.g., linear regression, mixed-effects model, survival analysis) such that each analysis aligns clearly with each research objective and hypothesis in the 1. Introduction. Covariates and confounding variables that will be included and any procedures for their inclusion (e.g., principal component analysis, stepwise procedures) are outlined in this section. Both unadjusted and confounder-adjusted analyses are important for STROBE-compliant reporting (von Elm et al., 2007). Statistical assumptions and remedies for violations can also be addressed in this section. It will need to be clear in this section which analysis population will be used for each analysis, referencing the information developed for the Analysis Populations subsection for continuity.

The Sensitivity Analysis section prespecifies sensitivity analyses that will examine the robustness of the findings; if none are planned, this is noted.

The methods for handling missing data are prespecified in the Missing Data section. This helps to prevent against bias, as different approaches to managing missing data (e.g., imputation methods, complete case analysis) can influence study results. Clear protocols for addressing missing data ensure transparency and demonstrate avoidance of QRPs involving missing data handling that could constitute a p-hacking strategy (Stefan & Schönbrodt, 2023).

The Analysis Populations section describes all participant groups to be analyzed. For example, it might specify whether all participants in a single-arm intervention study will be analyzed, regardless of treatment adherence, and/or if subgroups will be analyzed (e.g., based on treatment completion, gender, or age). Each analysis population is clearly defined, including how the groups are operationalized. Clearly defining analysis populations prevents bias from QRPs, such as selective reporting of a statistically significant result found in one population and non-disclosure of nonsignificant results in another (Andrade, 2021; Janzen & Michler, 2021).

The Confidence Intervals and P values section outlines the confidence interval and P thresholds to be used, and whether adjustments will be made for multiple comparisons (International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use, 1998). Without such adjustments, researchers can risk inflating the Type I error rate (i.e., falsely detecting an effect). Strategies such as the Bonferroni correction or the Benjamini-Hochberg procedure are commonly used to account for multiplicity and should be outlined clearly in the SAP.

4.6. Preregistration and Study Reporting

The 6. Preregistration and Study Reporting section includes two subsections: 6.1 Preregistration and 6.2 Study Reporting. This is the only optional section of the template. While not traditionally part of SAP guidance, this section helps the PI and statistician discuss and agree on preregistration plans, and helps the statistician consider reporting guidelines to ensure planned analyses meet requirements.

The Preregistration subsection outlines whether the study protocol and/or SAP will be preregistered and identifies the registration repository. For already registered protocols, such as in secondary studies, include the registration details (e.g., ClinicalTrials.gov number, DOI); alternatively, these can be placed in the Study Methods section for more efficient integration into the study manuscript later. Preregistration demonstrates a commitment to the planned methodology and analysis, reducing QRPs and enhancing the rigor and credibility of the findings (Mathur & Fox, 2023).

Study Reporting includes a commitment to adhere to reporting guidelines, such as STROBE for observational studies (particularly cohort, case-control, or cross-sectional studies) and others (Otzen et al., 2020; von Elm et al., 2007). The EQUATOR network (Simera et al., 2009) (https://www.equator-network.org), and journal editorial guidance (e.g., Misiak & Kurpas, 2022) are valuable resources for selecting appropriate frameworks. STROBE extensions designed for specialized designs or fields (e.g., genetic association studies) offer more specific reporting guidance; information on these can be found on the STROBE Statement website under “Links” (https://www.strobe-statement.org).

4.7. Revision History

This section includes the revision history of the SAP, presented as a table identifying the version, date, comments or rationale, and timing in relation to key milestones, especially data access. This allows for flexibility in the analysis and responsiveness to changes in the study and factors influencing statistical procedures and deviations from the original plan. This transparency helps readers assess potential for bias in the study’s publication.

4.8. References

This section lists all citations referenced in the SAP.

4.9. Other

While not included in this template, another addition to consider is the use of table shells, which outline the planned structure and content of key tables in the results section of the study manuscript. Table shells can help clarify how each objective or hypothesis will be addressed, ensure alignment between the planned analyses and reporting, and support efficient and transparent presentation of findings.

5. Discussion

The contribution of a SAP template for observational studies offers an essential step forward in promoting research integrity and rigor on a topic lacking in formal guidance (Hiemstra et al., 2019). This template aims to bridge the gap between current SAP practices, largely established for clinical trials, and the needs of observational research. Through its structure, this SAP template fosters clarity, transparency, and reproducibility in the analysis of observational studies.

Moreover, the use of SAPs aligns with broader efforts to promote transparency in research (Onukwugha, 2013). The prospective development of SAPs and pre-registration in repositories such as ClinicalTrials.gov and the Open Science Framework allows other researchers to verify that analyses are conducted as planned, increasing public trust in science (Onukwugha, 2013). The practice of pre-specifying analysis procedures is not only a practical and technical tool but also an ethical commitment to reducing bias and enhancing scientific integrity.

Observational studies are crucial for exploring associations that experiments cannot capture. However, the flexibility inherent in statistical practices poses risks of QRPs, such as p-hacking and selective reporting (Stefan & Schönbrodt, 2023). These QRPs can compromise research integrity, inflate false-positive rates, and undermine public trust in science (Onukwugha, 2013). By emphasizing the development and use of a SAP, this template helps researchers document their analytic decisions in advance, reducing bias and mitigating QRPs.

A crucial contribution of this template, compared to existing templates, is its emphasis on practicality and its potential to improve the efficiency of reporting in study manuscripts (Hiemstra et al., 2019; Thomas & Peterson, 2012; Yuan et al., 2019). By integrating STROBE items within the SAP, this template allows researchers to document methods systematically and facilitates a seamless transition from the SAP to the study manuscript. This alignment could increase the likelihood of meeting journal publication standards.

Despite the benefits, the growing support for the use SAPs of has prompted a crucial debate on drawbacks, especially for observational studies (Janzen & Michler, 2021; Olken, 2015). SAPs are administratively burdensome, which is a barrier to adoption especially in lower-resource settings. Although this template aims to be user-friendly and practical, future studies should investigate the barriers to SAP implementation in real-world contexts. It has been discussed previously that SAPs may be especially suitable for hypothesis-driven observational studies but less so for truly exploratory studies (Sleigh, 2019). Developing SAPs is more challenging in complex studies, where it is not possible to prespecify every statistical decision or anticipate all factors bearing on analysis (Olken, 2015; Tackett et al., 2017). There are many nuances to the debate regarding whether to use SAPs for observational studies and merits to both the concerns and advantages raised. While many challenges exist, they are beyond the scope of this paper (Olken, 2015; Onukwugha, 2013; Sleigh, 2019; Tackett et al., 2017). This template is designed to support researchers who wish to develop a SAP, and in a similar manner to Kahan and colleagues Pre-SPEC framework for clinical trials, is not an endorsement of mandating SAPs, but may help researchers design an analytic plan that prevents p-hacking and can be seen by others, such as editors and end-users, to prevent p-hacking (Kahan et al., 2020).

5.1. Conclusion

Despite their value, SAPs remain more of an exception than the norm outside the regulatory contexts of clinical trials (Thor et al., 2020). Training in SAPs is not routinely integrated into research and statistics curricula, even in specialized statistics programs. Moreover, accessible guidance on preparing SAPs is scarce (Gamble et al., 2017; Hiemstra et al., 2019; Thomas & Peterson, 2012; Yuan et al., 2019). To address this gap, this paper offers a user-friendly SAP template accompanied by guidance, empowering researchers to prespecify their analyses with the rigor and transparency that the scientific community increasingly demands. Continued development of SAP guidance, standards, and training, alongside efforts to improve methodological rigor, will strengthen observational research and enhance the value of findings across scientific disciplines (Eisenach et al., 2016; Hiemstra et al., 2019).

In conclusion, this SAP template offers a practical tool for researchers conducting observational studies, providing a foundation to enhance research rigor and transparency. While challenges to widespread adoption remain, the benefits of using a well-structured SAP—ranging from improved reproducibility to increased publication quality—underscore its value in the scientific community. By supporting thoughtful, prespecified analysis plans, this template contributes to the advancement of robust, reliable, and transparent research practices, ultimately strengthening the credibility of observational research and fostering trust in scientific inquiry.