ESMO Guidance for Reporting Oncology real-World evidence (GROW)

Highlights

• •Real-world evidence in oncology is evolving rapidly with many particularities.
• •This guidance provides key recommendations for reporting real-world evidence studies in oncology.
• •Recommendations are based on a review of current evidence and the authors’ collective expert opinion.
• •Authors are a multidisciplinary group of experts from different institutions and countries.
• •Guidance is provided for full article development, including title, introduction, methods, results, discussion, conclusion.

Introduction

The use of real-world data (RWD) for generating real-world evidence (RWE) to complement interventional clinical trial-based research is rapidly increasing. This evolving field is particularly prevalent in oncology with a growing number of publications and increased use of RWD in medicine regulation in recent years.1, 2, 3, 4 Improving the quality of RWE is crucial for patients, the scientific community and health care authorities.

Several guidelines have been developed in recent years that are relevant for reporting RWE studies, including: REporting of studies Conducted using Observational Routinely-collected health Data (RECORD)⁵; STtrengthening the Reporting of OBservational studies in Epidemiology (STROBE)⁶; STROBE-Molecular Epidemiology (STROBE-ME)⁷; STrengthening the REporting of Genetic Association studies (STREGA)⁸; and REporting recommendations for tumour MARKer prognostic studies (REMARK).⁹ Additionally, research organisations, regulatory agencies and health technology assessment (HTA) agencies have developed specific guidance for the design, submission and assessment of RWE.10, 11, 12, 13, 14, 15, 16

There are various particularities in oncology research, such as specific variables, biomarkers, therapies or outcomes, which are not sufficiently covered by the currently available reporting guidance. In addition, modern technologies such as artificial intelligence (AI), machine learning (ML) and deep learning (DL) have been recently introduced for different stages of the data analysis process in RWE studies. While recent guidelines are available for interventional studies involving AI,17, 18, 19, 20 similar guidance specific for RWE research is currently lacking.

Although the availability and use of multiple complementary guidelines can provide specific instructions, this approach is demanding and burdensome for both authors and journals21, 22, 23 and, most importantly, may not capture all the relevant oncology research-specific considerations. To address this, the multidisciplinary experts of the ESMO Real-World Data and Digital Health Working Group have developed the first specific guidance for reporting oncology RWE studies in peer-reviewed journals: the ESMO Guidance for Reporting Oncology real-World evidence (ESMO-GROW).

Methods

A comprehensive literature search of the existing guidance for reporting RWE studies confirmed the unmet need for specific oncology guidelines. A first draft of ESMO-GROW was developed by four authors based on the literature search. A second draft incorporated feedback from an expanded group of co-authors, including interdisciplinary experts in clinical oncology, statistics, AI, digital health, public health, pharmacology, research methodologies, health databases, guidelines development, health law and ethics, as well as journal editors and reviewers (see Supplementary Material Section 1, Table S1, available at https://doi.org/10.1016/j.annonc.2023.10.001). The manuscript sections were then divided among author groups based on their expertise. Authors reviewed the recommendations and developed supporting text, tables and figures between October 2022 and February 2023. Monthly consensus meetings provided an opportunity for the whole group to discuss and align on the recommendations and supporting content. The manuscript sections were then combined into a full draft by a core team of authors, in collaboration and agreement with co-authors.

All authors voted on the recommendations before a full-day meeting in March 2023 to discuss conflicting opinions and reach agreement on all aspects of the guidance. Recommendations were approved if >90% of authors agreed with the proposal. The manuscript was then circulated to selected external stakeholders, including patient advocates, industry representatives, the European Medicines Agency, publishers and individual experts in AI, cancer databases and health policy (see Supplementary Material Section 1, Table S2, available at https://doi.org/10.1016/j.annonc.2023.10.001) to gather external feedback, which was considered for adjustments to the recommendations. A final author vote was carried out and the final version of the guidance was approved by all authors.

A full description of the process used to develop ESMO-GROW is provided in Supplementary Material Section 1, Table S3, available at https://doi.org/10.1016/j.annonc.2023.10.001. A further detailed explanation of the development and characteristics of ESMO-GROW will be provided in a separate publication.

This project is registered with the Enhancing the QUAlity and Transparency Of health Research (EQUATOR) network.²⁴

Results

The author group developed 35 reporting recommendations relating to each section of an RWE publication, including (i) title, (ii) introduction, (iii) methods, (iv) results, (v) discussion and conclusions, and (vi) final considerations. These recommendations are summarised in Table 1 and explained below. To complement the recommendations, lists of the definitions of key terms, main variables for RWE research in oncology and common sources of bias in RWE studies are provided in Supplementary Material Section 2, available at https://doi.org/10.1016/j.annonc.2023.10.001, and a template flowchart for RWE study case selection is provided in Figures 1 and 2.

Table 1.

Summary of recommendations on reporting RWE studies

1. Title

1.1:Concisely include relevant key terms referring to the study type, study population, objectives, data sources and outcomes, depending on the study. Consider including the terms ‘real-world’ or ‘observational’

2. Introduction

2.1:Explain the scientific rationale for the research question(s), providing concise background information on previous core evidence from systematic reviews, meta-analyses, clinical trials and/or real-world evidence studies

2.2:Identify the gaps in evidence and explain why and how they can be suitably addressed by real-world evidence research. Specify the new evidence that is expected from the current study

2.3:Briefly introduce the aim(s) of the study

3. Methods

Study objective(s), design, data sources and variables

3.1:Provide the study research question(s) including a description of the patients or the object under study and the target outcome(s)

3.2:Provide the study objective(s) and consider classifying the type of research as descriptive and/or analytical (explanatory or predictive)

3.3:Provide relevant information to describe and classify the study design used to address the research question

3.4:Give a clear definition of the eligibility criteria used to select the patients or objects under study, particularly regarding cancer-related aspects

3.5:Report the specific type and purpose of real-world data source(s) used, providing a detailed description and the reason(s) why the source was considered appropriate for the study objectives

3.6:When multiple real-world data sources are used, provide details on interoperability, including identification of duplicated cases or data linkage from separate databases

3.7:Provide details and timings of source and study data management. Consider specifying methods of raw data collection, updates and completeness, data extraction, cleaning and/or quality controls and validation

3.8:Provide core details on database and/or study registration, governance, ownership, metadata and full data accessibility in the main text or supplementary material

3.9:Identify the data source of each core variable, its definition, if the variable was derived or coded, and describe how the derivation or coding was conducted and validated

3.10:Specify the time points of core variables in relation to the cancer disease trajectory

3.11:Provide a complete list of core variables included in the study. Variables can be grouped as baseline characteristics, exposure, and outcomes or endpoints

3.12:For biomarker-related studies, provide details on biomarker description, timing, and methods of assessment and analytical validation

Statistical analysis and artificial intelligence methods

3.13:Summarise the main aspects of the statistical analysis

3.14:When applicable, provide details on the pre-planned sample size requirements and power of the study

3.15:Specify the pre-planned strategies to identify and mitigate the main sources of bias

3.16:Clearly distinguish prespecified from post hoc analyses, especially for subgroup analyses

3.17:Provide information on internal and external validity, as well as any sensitivity analyses

3.18:For analytical studies, the full version of the statistical analysis plan should be provided in the supplementary material, including a brief explanation of any amendments

3.19:When applicable, specify which machine learning, deep learning or alternative artificial intelligence method has been used

3.20:When reporting real-world data analysis with artificial intelligence (e.g. machine learning and deep learning) algorithms, include comprehensive aspects on data pre-processing techniques, feature engineering strategies and model development

3.21:Address the artificial intelligence model explainability and interpretability, and present the plan for integration into clinical practice, if applicable

3.22:When applicable, briefly describe the multidisciplinary team required for the study and explain how these needs were met

4. Results

4.1:Provide the number of cases excluded or nonparticipating and reasons at each stage of sample selection, as well as numbers lost to follow-up. Compare the cases excluded with those included in the analyses. Illustrate this with a flowchart

4.2:Describe the baseline characteristics of the cases included (e.g. clinico-demographic and tumour characteristics). The baseline characteristics of different groups under analysis should be compared, if applicable

4.3:Report the results of the primary analysis of study outcomes. Briefly describe the results of exploratory analyses if relevant (prespecified and/or post hoc). Provide details of how readers can access the full results

5. Discussion and conclusions

Discussion

5.1:Summarise the core results that address the primary research question(s) and objectively discuss the data in relation to the best available evidence on the topic. Avoid a convenient selection of literature to support a point

5.2:Discuss the strengths and limitations of the current study, including the main biases, how the strategies applied contributed to bias avoidance or mitigation and, if applicable, in which direction the authors estimate that residual bias may influence the core results of the study

5.3:Discuss the generalisability of the study results and their potential implications for clinical practice, health policies or public health and for the generation of hypotheses for future research

Conclusions

5.4:Provide a balanced summary of core results relating to the primary research question and the main implications for clinical practice, health policies and/or public health. Suggest further research considering the remaining unmet needs and limitations from the reported study

6. Final considerations

6.1:Specify all relevant study sponsorship(s) as well as direct and/or indirect or in-kind funding

6.2:Specify all relevant acknowledgements, author disclosures, individual contributions and other final considerations as per journal regulations

RWE, real-world evidence.

Figure 1.

Template for ESMO-GROW flowchart illustrating the process of case selection for analysis.^a

Dashed lines represent possibilities for linkage between datasets, to be adapted in each study.

ESMO, European Society for Medical Oncology; GROW, Guidance for Reporting Oncology real-World evidence; RWE, real-world evidence.

^aThe ESMO-GROW flowchart can be used to illustrate the process of case selection in RWE studies. The flowchart should include the sequence of relevant steps from data sources(s), case selection (inclusion criteria, exclusion criteria and data cleaning) and the main aspects of the final data analysis. The number of cases excluded and main reasons for exclusion should be stated at each step. When different datasets are used, the process should be described for all datasets and the interoperability or merging points included. For comparative effectiveness studies, the authors should also illustrate the flow of included patients per different groups under analysis. The ESMO-GROW flowchart can be adapted for different types of RWE studies [e.g. patient- or population-level (for example, ecological), descriptive or analytical studies].

Figure 2.

Example of ESMO-GROW flowchart for effectiveness analysis of a ‘novel anti-HER2 medicine’ in HER2-positive advanced breast cancer.^a

Dashed lines represent linkage between datasets.

ER, estrogen receptor; ER+, estrogen receptor positive; ER−, estrogen receptor negative; ESMO, European Society for Medical Oncology; GROW, Guidance for Reporting Oncology real-World evidence; HER2, human epidermal growth factor receptor 2; HER2+, human epidermal growth factor receptor 2 positive; HER2−, human epidermal growth factor receptor 2 negative.

^aTheoretical example of ESMO-GROW flowchart for RWE study assessing the effectiveness of a novel anti-HER2 drug using complementary data from two datasets. In this case, data were merged before eligibility analysis (exclusion criteria applied) and the main selection criterion for final analysis was the prescription of a new specific anti-HER2 medicine for patients with advanced breast cancer. Final analysis was also stratified by tumour hormonal status (ER positive versus ER negative).

Recommendations

1. Title

1.1: Concisely include relevant key terms referring to the study type, study population, objectives, data sources and outcomes, depending on the study. Consider including the terms ‘real-world’ or ‘observational’.

The title should include relevant key terms to identify the type of research being reported, such as ‘real-world’ or ‘observational’ studies or evidence based on established definitions (see Supplementary Material Section 2, Tables S4 and S5, available at https://doi.org/10.1016/j.annonc.2023.10.001). The (sub)population under consideration, tumour (sub)type, setting, biomarkers, treatments being evaluated and uni/multicentricity of the study should also be clearly stated. The terms ‘prospective’ or ‘retrospective’ related to the study design or data analysis can also be considered.

All these should be prioritised depending on the type of research and in alignment with the journal title word limit. The key terms selected for inclusion in the title will facilitate searches in electronic databases and should capture the reader’s attention.

2. Introduction

2.1: Explain the scientific rationale for the research question(s), providing concise background information on previous core evidence from systematic reviews, meta-analyses, clinical trials and/or real-world evidence studies.

The introduction should include a summary of the best evidence available from previous studies (systematic reviews, meta-analyses, clinical trials or prior RWE studies) that can be used to support the study rationale. The purpose is to provide an overview of what is already known in that specific setting, which gives context for presenting the unmet needs and gaps in evidence that triggered the design and conduct of the reported study.

Oncology is a complex field with an increasing number of tumour subclassifications, biomarkers, novel treatment strategies and studies in different research settings with heterogeneous quality. Literature supporting the scientific rationale of the study should be selected carefully, prioritising the highest level of evidence. A convenient selection of literature should be avoided.

2.2: Identify the gaps in evidence and explain why and how they can be suitably addressed by real-world evidence research. Specify the new evidence that is expected from the current study.

An explanation of the unmet need and what is lacking in terms of quantity, quality and strength of evidence should be provided. When appropriate, the authors may describe why a clinical trial or other type of interventional study might not be feasible or relevant (e.g. due to ethical issues, difficulties related to patient population or recruitment or even financial aspects) or how the specific study is adding relevant information to currently available data. For example, RWE studies can provide evidence from patient subgroups that are commonly excluded from clinical trials in oncology, such as older patients and those with impaired performance status, poor prognosis, specific comorbidities or rare cancers.²⁵ RWD can also provide a basis for synthetic control arms in single-arm trials where randomisation in a multi-arm trial would be unfeasible (e.g. rare tumours) or unethical [e.g. dismal expected outcome and ineffective standard of care (SoC)].²⁶

It is essential to provide information on the evidence and additional research output that is expected from the reported study. This should consider the clinical evidence that can be derived from RWE studies, the type of research being conducted and the potential strengths and limitations. For example, RWE studies may provide complementary data to randomised controlled trials in terms of effectiveness and safety (e.g. broader study population, longer follow-up), potential predictive or prognostic analysis, or insights that could inform the design and conduct of future clinical trials.27, 28, 29

2.3: Briefly introduce the aim(s) of the study.

The aim(s) of the study should be stated at the end of the introduction to explain the purpose. Patient population, exposures and outcomes may be concisely reported, depending on the study design.

3. Methods

Study objective(s), design, data sources and variables

3.1: Provide the study research question(s) including a description of the patients or the object under study and the target outcome(s).

The research question(s), objective(s) and study design should always be reported. It is acceptable to list these in a different order than provided here.^30,31

To aid understanding of the study and interpretation of results, the research question should be clearly and precisely stated.^30,31 If useful at this stage, specific information relevant to the study design, such as demographic characteristics, tumour-related details (tumour type, histology, stage of disease, subgroups, biomarker-driven selection) and study setting [e.g. type of cancer centre(s) involved, national or international setting], could be mentioned. If the study objective is analytical, the patients or population, exposure, comparator and outcome (PECO) framework can be considered for reporting the research question.³²

In the case of explanatory research (see Supplementary Material Section 2, Table S6, available at https://doi.org/10.1016/j.annonc.2023.10.001, and recommendation 3.2), the exposure of interest should be reported. The term ‘exposure’ can be broadly applied to any factor that may be associated with an outcome of interest, such as access to a new diagnostic technology or treatment. If the study is comparative, the authors should clearly describe the comparator group. For exposure to treatments, comparators could be, for example, usual care, standard care, another active treatment or the same treatment but with a different posology or modality. For aetiological factors, presence is often compared with absence, although other comparisons are possible (e.g. different levels for continuous measures such as age). In the case of predictive research (see Supplementary Material Section 2, Table S6, available at https://doi.org/10.1016/j.annonc.2023.10.001, and recommendation 3.2), authors should state whether they intended to develop a new prediction model, validate an existing one or investigate the added value of a potential predictor. The evaluated predictors should also be reported.

The main outcome(s) should be clearly stated. If a distinction is made between primary and secondary outcomes, this should be specified.

3.2: Provide the study objective(s) and consider classifying the type of research as descriptive and/or analytical (explanatory or predictive).

Research objectives can often be classified as descriptive or analytical (explanatory or predictive) (see Supplementary Material Section 2, Table S6, available at https://doi.org/10.1016/j.annonc.2023.10.001).^33,34 Descriptive objectives are commonly used in epidemiological studies, such as assessing the prevalence or incidence of a certain cancer type or evaluating access to different treatment modalities over time. Descriptive studies do not require a comparator group and generally describe data from a population-based perspective. Analytical studies can be explanatory or predictive. In explanatory research, the study intends to identify and interpret causal effects. Predictive studies aim to provide accurate predictions for future patients, such as an individual prognosis or treatment effects, without giving those predictions (or the model) a causal interpretation.³⁴

3.3: Provide relevant information to describe and classify the study design used to address the research question.

The study design varies according to the study objective(s) (i.e. descriptive or analytical) (see Supplementary Material Section 2, Table S6, available at https://doi.org/10.1016/j.annonc.2023.10.001). Describing and justifying the study design allows accurate estimation of the level of evidence that may be expected, as well as clear interpretation of the study results and potential biases.35, 36, 37

Various study designs can be considered, including case reports or series, cross-sectional or cohort studies (for individual-level data analysis) and ecological studies (for population-level data analysis)^38,39; this information must be adequately reported. If innovative or specific study designs are used, they must be carefully described; for example, studies in which an exposure is ‘controlled’ without interfering in the subject’s allocation to an experiment (also referred to as ‘quasi experimental studies’).⁴⁰ RWD can also be used for experimental designs (e.g. post-marketing surveillance studies, synthetic control arms and pragmatic trials).²⁶ Recently, target trial emulation was proposed as a framework for designing observational studies that aim to estimate the causal effect of an intervention.⁴¹

The terms ‘prospective’ and ‘retrospective’ can be used to describe the time of data collection; however, their use and meanings are not universally standardised.⁴² Therefore, authors should provide a clear description of the applicable timeframes of the study, namely (i) the time period of included cases, (ii) the exact time the data source was assessed for extraction and (iii) the time point(s) of core variables. The importance of describing these time points is further explained below in recommendations 3.4, 3.7, 3.10 and 3.12. Finally, the authors may consider an illustrative diagram of the study design.

3.4: Give a clear definition of the eligibility criteria used to select the patients or objects under study, particularly regarding cancer-related aspects.

Eligibility criteria, including details on inclusion and exclusion criteria, as well as time frame, should be reported as they are important for assessment of the internal and external validity of the sample.⁴³

For patients with cancer (individual-based studies), details on demographics, tumour type, histology, stage of disease, possible biomarkers, previous treatments and comorbidities should be provided. If the object under study is a population (e.g. a population-based study where individuals who share a common characteristic are taken from the general population) or another type of data (e.g. events from a pharmacovigilance database), the eligibility criteria related to that object should be provided. The timing of the included cases should also be specified. If the study is comparative, the characteristics of the comparator group should be provided; for example, the comparator group may consist of the same individuals followed at different time points, or it may include other individuals compared at the same follow-up time point (e.g. the effectiveness of a new treatment between 2020 and 2022 compared with the previous SoC between 2017 and 2019). This is particularly relevant considering how rapidly clinical practices are evolving in oncology.

Importantly, eligibility criteria in RWE studies are more pragmatic and generally not as strict as those in controlled clinical trials. This commonly allows for broader representativeness of the population under study (higher external validity).^44,45

3.5: Report the specific type and purpose of real-world data source(s) used, providing a detailed description and the reason(s) why the source was considered appropriate for the study objectives.

The characteristics of the RWD source(s) should be provided (see Supplementary Material Section 2, Table S7, available at https://doi.org/10.1016/j.annonc.2023.10.001),⁴⁶ including the primary aim of data collection for the selected source(s) and its starting date, particularly with routinely collected data [e.g. electronic health records (EHRs), administrative claims, cancer registries]. The authors should explain why the selected data sources were considered appropriate to address the study objectives and research questions. A description of the general type of data collected should also be provided, including its completeness, patient eligibility for the database, details on the health care setting (i.e. primary, secondary or tertiary care), geographical details [i.e. country(ies) and region(s)] and/or if the database is population based (i.e. covering all patients or only a selection of hospitals or patients). Importantly, the general characteristics of the data source(s) should be distinguished from study specificities. For instance, if the data source is a breast cancer database with a starting date of 1990, but the study objective is focused on survival analyses for metastatic disease diagnosed after 2015, this difference should be clearly reported.

If data were collected by questionnaires (e.g. for patient-reported outcome measurements) it should be clear how those tools were selected, designed and validated (e.g. pilot test in the case of a new questionnaire). Similarly, the same details should be reported when using novel methods of data collection (e.g. from wearable devices).

3.6: When multiple real-world data sources are used, provide details on interoperability, including identification of duplicated cases or data linkage from separate databases.

The interoperability between different data sources is a heterogeneous and rapidly evolving field that is influenced by types of sources, study designs, research questions and variables. The main points can be reported in the manuscript, while more detailed aspects regarding interoperability could be considered for the supplementary material.

Common scenarios and issues to consider when reporting interoperability between datasets include the following:

• •When two or more data sources may contain overlapping populations, the method used to identify potential duplicate cases should be described.
• •When multiple datasets containing complementary data from the same patients are combined, the matching or encounter variables and methodology used to link the data should be reported (e.g. unique identifiers for individual-level data or hospital for regional-level data).
• •When two or more data sources contain variables that would be merged, but have different codes, authors should elaborate on their differences and the rationale for recoding.
• •When data for the same case are discrepant between datasets, handling of those differences should be clearly reported.

If considered, the method of transferring data from one source to another while maintaining the integrity and reliability of the original data should also be described, including how data protection regulation was respected.⁴⁷ If novel methods for data transfer and sharing are used (e.g. federated analysis or swarm learning), the details of that process should be described.

Standards used to transfer data, such as Substitutable Medical Applications and Reusable Technologies on Fast Health Interoperability Resources (SMART on-FHIR), should be reported.⁴⁸ When the Findable, Accessible, Interoperable and Reusable (FAIR) data principles⁴⁹ are consulted and applied, this should be reported. Finally, when a common data model is used to harmonise data from multiple sources, the template [e.g. the observational medical outcomes partnership (OMOP) common data model] should be provided or the source should be mentioned.

3.7: Provide details and timings of source and study data management. Consider specifying methods of raw data collection, updates and completeness, data extraction, cleaning and/or quality controls and validation.

Methods of data collection should be reported for transparency and quality, including how the different data sources were accessed.¹ Data can be routinely collected (e.g. registries and administrative databases) or specifically collected for the purpose of the study. Authors should specify who collected the data (e.g. trained data managers, physicians, researchers, students), the frequency of data collection and whether it was collected manually or digitally. The exact date on which the data were extracted for the purpose of the reported work (e.g. data cut-off date or specific access date; for example, 20 January 2022) should also be clearly specified to allow critical appraisal of data maturity, particularly when studying time-dependent outcomes. Similarly, the data cleaning methods and the data validation process should be described.

Authors should confirm if data were systematically curated according to common standards for the specific cancer setting and if established ‘curation manuals’, dictionaries or coding systems were used. It is important to report on quality controls, including outlier and missing value assessment, during the data collection process. In addition, any technology used by data abstractors during the variable extraction process should be specified [e.g. algorithms for natural language processing (NLP) or large language models (LLM)]. Finally, information on how outliers and missingness were handled in the data source should be included, with full data provided in the results section and/or supplementary material.

The strategy applied to keep data up to date should be reported. Firstly, the variables that were updated throughout the patient’s trajectory should be provided. This may include cancer-related outcome data (i.e. recurrence, progression, status at last follow-up, death) and toxicity-related or patient-related outcomes. Secondly, the process for updates, including timelines and the references considered, should be reported (e.g. periodic updates via clinic visits versus automatic updates via wearable devices), especially for studies that analyse the effectiveness of different anticancer treatments or survival. The most relevant information could be provided in the main text, with full information to be considered for the supplementary material.

3.8: Provide core details on database and/or study registration, governance, ownership, metadata and full data accessibility in the main text or supplementary material.

This information can be provided in the main manuscript, as supplementary material or as a link to an online repository. Depending on the type of RWD source and study, this may include: identification (e.g. study acronym and registry identification number); investigators and centres involved; full variable details (annotated case report form) and whether each variable is composed of identifiable, pseudonymised or fully anonymised data; full dataset or policy for access; institutional approval(s) details; ethics and data protection approval (following General Data Protection Regulation and local requirements or other applicable data protection law); contact person for providing data accessibility; and funding sources.

We strongly encourage inclusion of the research protocol in the supplementary material. This can help to increase transparency and clarify various dimensions of the study design and conduct, which are difficult to fully cover in the main manuscript.

If data were collected through international sharing mechanisms (e.g. European Health Data Space), this should be stated. Reference to good practices and available guidance for developing RWE protocols should also be considered.¹⁴ If the study is registered with a public repository [e.g. clinicaltrials.gov or the European Union electronic register of post-authorisation studies (EU PAS register)], this should be reported, providing a direct link, identification code or registration number.

A description should be included on how informed consent was obtained or waived, eventual differences between centres or countries, and what supportive documentation was used for that purpose.

3.9: Identify the data source of each core variable, its definition, if the variable was derived or coded, and describe how the derivation or coding was conducted and validated.

The definition of each variable and whether data were directly obtained from a given data source, derived or coded and eventually imputed (see section on Statistical analysis and artificial intelligence methods) should be clear. This could be summarised for all variables but in more detail for the main variables [e.g. exposure(s) and endpoint(s)].

Specific biases related to the quality of each data source might apply to different variables. For example, a systematic bias related to a data source, where the identification of the exposure to a given anticancer treatment can fail, might lead to misclassification as ‘not exposed’. The same applies to the extent to which a prognostic factor or biomarker was measured homogeneously.

When a variable has been derived, a brief explanation is expected on how the derivation was conducted, its validation and quality (e.g. sensitivity and specificity). For example, if data on cancer recurrence are derived from an algorithm defined from hospital resource use or treatment activity data, this should be clearly explained along with a reference to the validation work behind the derived algorithm.

When variables are coded (e.g. transforming a numerical variable into categories, such as prostate-specific antigen ≤10 ng/ml, 10-20 ng/ml or >20 ng/ml), the rationale for the cut-offs should be included (e.g. supporting evidence to use these limits based on risk stratification).⁵ In addition, clear justification should be provided if the same coding is not kept throughout the analysis (e.g. describing age as continuous in the baseline characteristics table and later as categorical with age groups for multivariate analysis). These details can be reported either together with the description of data sources or with the listing of variables, which can be provided in more detail (e.g. as a code book) in the supplementary material.

3.10: Specify the time points of core variables in relation to the cancer disease trajectory.

For each core variable or group of variables, the authors should specify the time point of cancer disease trajectory they refer to (e.g. age at cancer diagnosis versus age at initiation of the anticancer treatment to be analysed), including variations between cases or datasets, as this may influence interpretation of the results.

When cancer diagnosis is the study baseline time point, diagnosis-related aspects (e.g. cancer screening participation, access to care) before this time point should be described as they might influence the time period to diagnosis.^50,51 Similarly, when the study baseline is defined by the start of a systemic anticancer treatment, the criteria for starting this treatment and the line of treatment should be clear.

For endpoints, assessment time points should be defined in relation to the study baseline, reporting the (local) standards and frequency of assessment. In case of time-to-event variables, the baseline date for follow-up (index date) and when it ends (event date) should be reported. The end of follow-up should also be specified (e.g. death, loss to follow-up or a fixed period of follow-up).¹⁴

These details can be reported together with the listing of variables.

3.11: Provide a complete list of core variables included in the study. Variables can be grouped as baseline characteristics, exposure, and outcomes or endpoints.

Identifying the study variables provides valuable information for readers to understand the study design, conduct and results. Listing the variables considered for baseline sample characteristics is critical, as there is a high risk of cohorts being imbalanced; for example, for variables with prognostic value (e.g. confounders or effect modifiers).

Baseline characteristics are also important to evaluate selection bias and the external validity of the findings by allowing assessment of the sample representativeness of the source population.⁵ Additionally, when reporting epidemiology data in cancer studies (e.g. incidence, prevalence or mortality), granular details on cancer-related variables should be specified (e.g. primary cancer diagnosis, tumour subtype, disease stage, previous treatments), adjusted to the specific oncology context. The same is true for molecular cancer epidemiology studies reporting the prevalence or incidence of certain biomarkers (e.g. from novel omics research) in different disease settings.

Clear identification and details of the exposure variables are essential to allow scrutiny of potential misclassification biases that may jeopardise the conclusions of the study.⁵²

The authors should clearly define the primary endpoint(s) (e.g. real-world progression-free survival, quality of life scores, serious adverse events)⁵³ and any secondary and/or exploratory endpoints. A proposed classification for clinical variables is provided in Table 2 and expanded in Supplementary Material Section 2, Table S8, available at https://doi.org/10.1016/j.annonc.2023.10.001.

Table 2.

Summary of main variables and their specificities to be considered in RWE studies in oncology^a

Category	Domain	Specificities
1. Baseline characteristics
Demographic	Age	Continuous and/or categorical
	Sex, gender	Sex is biologically determined while gender is self-reported
	Race, ethnicity, ancestry	Use the terms race, ethnicity and ancestry appropriately
	Other social determinants of health	Education, income, employment, urban versus rural area, distance to reference versus community centre, etc.
Clinical (general)	Performance status	ECOG or Karnofsky
	Geriatric scales	G8 score, CARG score or others
	Anthropometric measures	Height, weight, body surface area, body mass index
	Relevant medical history	Comorbidities (e.g. diabetes) and/or score (e.g. Charlson Comorbidity Index)
	Concomitant medications	Name by INN and dosage if relevant
Cancer specific	Risk factors and/or hereditary predisposition to cancer	Type of data collection (e.g. self-reported) and duration of exposure to risk factors Familial cancer cases and germline mutations when relevant and/or available
	Previous anticancer treatments	If relevant, consider reporting the treatment-free interval and/or number of prior treatment lines
	Tumour (sub)type	Classification used (e.g. WHO) Histological and molecular subtype
	Pathology-specific criteria	Criteria (e.g. Gleason score: 4 + 3, surgical margin: R+), method of assessment and rationale for categorisation
	Staging at diagnosis	International classification used (e.g. AJCC TNM, eighth edition)
	Metastatic disease	Imaging used (e.g. magnetic resonance), localisations and burden of disease
	Biomarker analyses	Biosample (e.g. liquid biopsy), technique or assay, time point (e.g. before treatment), type of alteration (e.g. KRAS mutation) and value or scoring (e.g. PD-L1 >50%)
2. Exposure (and comparator, if applicable)
Anticancer treatment	Systemic therapy	Type (e.g. chemotherapy), name by INN and dosage
	Surgery	Categories and rationale for classification (e.g. mastectomy versus breast conserving)
	Radiotherapy	Type, field and doses per session or in total (e.g. stereotactic body radiotherapy, 8 Gy, 1 fraction)
	Local ablation	Type and localisation of treatment (e.g. radiofrequency ablation for liver metastasis)
	Theranostics	Target, radiolabel and other characteristics as relevant
	Supportive care	Type, name by INN and dosage
	General considerations for comparison	If comparative study, explain the ‘allocation’ to the exposure (e.g. before versus after access to a therapy)
3. Outcomes and endpoints
Investigator-assessed outcomes	Tumour related and survival	Definition of outcome(s) used, frequency and type of assessment (e.g. pathological, radiological and/or clinical) and heterogeneity between centres or databases Specify median follow-up and how censoring was applied to time-to-event endpoints (e.g. real-world progression-free survival)
	Treatment related and safety	Definition of outcome(s) used and methods of assessment Classification of adverse events (e.g. CTCAE v5) and data sources
Patient-assessed outcomes	Patient-reported outcomes and QoL	Tools used (e.g. digital for remote monitoring), timing of assessment, rationale for selection (e.g. EORTC QoL questionnaires) and validation

AJCC, American Joint Committee on Cancer; CARG, Cancer and Aging Research Group; CTCAE, Common Terminology Criteria for Adverse Events; ECOG, Eastern Cooperative Oncology Group; EORTC, European Organisation for Research and Treatment of Cancer; INN, International Non-proprietary Names; KRAS, Kirsten rat sarcoma virus; PD-L1, programmed death-ligand 1; QoL, quality of life; R+, tumour at the resection margin; RWE, real-world evidence; TNM, tumour–node–metastasis; WHO, World Health Organization.

^aA detailed version of this table with more specificities and examples is presented in Supplementary Material Section 2, Table S8, available at https://doi.org/10.1016/j.annonc.2023.10.001.

3.12: For biomarker-related studies, provide details on biomarker description, timing, and methods of assessment and analytical validation.

Studies using RWD to assess the prognostic or predictive value of biomarkers are of growing importance given the current evolution of precision oncology; therefore, it is important to standardise methods of reporting this information. For each biomarker, the numerator and denominator (e.g. proportion of the sample that was tested or proportion of patients testing positive within those tested) should be described, as well as predefined cut-offs. It should be clear whether the biomarker is accessible in routine practice or investigational, and the time point of assessment or collection relative to the cancer disease trajectory should be stated.

For immunohistochemical markers, the name and clone of antibody, platform and scoring system should be reported. For molecular testing, the type of material (e.g. formalin-fixed paraffin-embedded tissue, fresh tissue, liquid biopsies), collection and analytical conditions, and processes (e.g. gene amplification with FISH, PCR or next-generation sequencing) should be described. The name, panel size, specifications (e.g. tumour-informed liquid biopsy) and vendor of each assay should be reported, and authors should indicate if the assay has been validated and/or approved.⁵⁴ For classical imaging biomarkers and novel imaging-based technologies such as pathomics and radiomics analyses, authors should describe the type of image, its means of collection and storage, how and at which time point it was derived and how it was validated, particularly if AI techniques were used.²⁰

Statistical analysis and artificial intelligence methods

3.13: Summarise the main aspects of the statistical analysis.

For descriptive studies, raw data are often quantitatively summarised by measures of central tendency (e.g. mean or median), dispersion (e.g. standard deviation or interquartile range) and measures of frequency distribution. Details on these methods should be provided. For analytical studies, a more detailed description of the statistical analysis covering the study research questions should be reported in the main manuscript, and the full statistical analysis plan (SAP) should be provided in the supplementary material.

3.14: When applicable, provide details on the pre-planned sample size requirements and power of the study.

Based on the study objective(s), the pre-planned sample size requirements and/or the power of the study should be reported, as these may increase confidence in the findings. Components to be reported include the primary endpoint, estimate of control value, targeted detectable difference, effect size, minimum power, significance level (clarify whether one-sided or two-sided) and statistical test used.

3.15: Specify the pre-planned strategies to identify and mitigate the main sources of bias.

Authors should report all potential sources of bias and the statistical methodologies used to address and minimise them, including strategies for handling missing values and adjusting for confounding factors (see Supplementary Material Section 2, Table S9, available at https://doi.org/10.1016/j.annonc.2023.10.001).

Methods to control for population or treatment selection, such as matching, stratification or the increasing use of target trial emulation,⁵⁵ should be described, providing information on the matching and/or stratification factors used, the reasoning and the clinical meaning of such a choice. The standardised methods, protocols and devices used across different centres in order to address potential sources of heterogeneity and data reporting errors should also be listed.

Special attention should be applied when choosing and reporting techniques to mitigate or explore the effects of confounding, namely propensity score methods, marginal structural modelling, doubly robust methods, g-computation or negative/positive controls. Statistical methodologies should be appropriately reported, along with evidence supporting the authors’ choice of potential confounding factors. Methodology for univariable and multivariable analyses should be described, including details of selection and inclusion of variables in the latter.

Temporality or immortal time bias might be dealt with by using landmark or time-dependent analyses, or simply by clearly defining and standardising the timelines of assessments across various centres. In any case, the source of temporality bias should be mentioned, and the technique implemented to rectify this issue should be presented.

For missing data, the mechanism of missingness assumed and/or assessed for the variables included in the (main) analyses should be reported [i.e. missing completely at random (MCAR), missing at random (MAR) or missing not at random (MNAR)]. It should be clear whether deletion, imputation or other methods were used to deal with missing values for each variable.^56,57

3.16: Clearly distinguish prespecified from post hoc analyses, especially for subgroup analyses.

Priority should be given to prespecified analyses, while any post hoc analysis not stated in the initial SAP should be explicitly described as such in the methods section, stressing its exploratory purpose.

The number of subgroup analyses carried out versus those presented should be reported.⁵⁸ The endpoint of interest and the statistical method used to assess the heterogeneity of intervention or exposure differences should be clearly stated. Finally, the multiplicity adjustment when applied to explanatory studies should be reported.

3.17: Provide information on internal and external validity, as well as any sensitivity analyses.

Internal validity is vital for meaningful interpretation of results, whereas external validity is crucial for generalisability of study findings. Sensitivity analysis aims to determine the robustness of an assessment by examining the extent to which results might be affected by changes in methods, models, values of unmeasured variables or assumptions.⁵⁹

For testable assumptions (e.g. proportionality), checks on the available data and modifications to the modelling process should be reported. For untestable assumptions that may depend on unavailable information (e.g. informative censoring, missing baseline covariates and unmeasured confounders), quantitative bias analyses should be clearly reported to alleviate concerns about RWD robustness. In this case, the results may be presented in full in the supplementary material. The effect on the outcome under multiple different assumptions should be described, with clear explanation of the purpose of such analyses.⁶⁰

3.18: For analytical studies, the full version of the statistical analysis plan should be provided in the supplementary material, including a brief explanation of any amendments.

The SAP, which might include study methods, statistical principles, study population and analysis methods,⁶¹ should be included in the supplementary material. Any statistical analysis methods that differed from those specified in the SAP should be reported, as this may clarify the final outcomes of the study.

3.19: When applicable, specify which machine learning, deep learning or alternative artificial intelligence method has been used.

AI models can efficiently analyse data to identify patterns and correlations that may not be apparent using traditional statistical methods. This ability is particularly important in oncology, where RWD can be used to generate hypotheses and inform clinical decision making.^62,63

There is a need for general recommendations on reporting the use of AI tools applied to RWD research in oncology. There is common misuse and overuse of AI-related terms in RWE research; the use of general terms such as ‘artificial intelligence’, without further details of the methods, should be avoided. For example, although the definitions of ML and DL are still debated (as they are essentially based on classical statistical methods),⁶⁴ the AI term should be avoided with simple ML models such as lasso regression. In addition, it is recommended that authors document any modifications or adaptations that were made to the standard algorithms to create more robust and accurate models. It is also necessary to state if the ML or DL methods were combined with other analytical tools, such as survival analysis (including Kaplan–Meier, univariable Cox analysis, etc.); the methodology behind the integration of two methods that justify an AI label should be clearly explained. Comparison of AI-based algorithms with classical statistical methods is strongly recommended to document the incremental performance improvement that justifies adoption of more complex models.

3.20: When reporting real-world data analysis with artificial intelligence (e.g. machine learning and deep learning) algorithms, include comprehensive information on data pre-processing techniques, feature engineering strategies and model development.

Clinical AI applications may include NLP methods such as LLMs to support scalable data extraction from EHRs coupled with human abstraction and models to support data analysis (e.g. reading physician notes, imaging and cancer genomics results, patient-reported outcomes and other sources). All details of the data acquisition and pre-processing steps should be documented in a detailed and fully traceable manner. These steps include collection, preparation, cleaning, transformation, exploratory quality control, normalisation, imputation, annotation and coding standards. As with classical methods, inclusion and exclusion criteria to select patients should be clearly stated when using AI techniques.

When applicable, the authors should include a description of how a balanced training set was acquired by including various subgroups of patients (e.g. with comorbidities or rare tumours) or how synthetic data were generated for underrepresented populations. If datasets from different sources were used, authors should describe the technology platform or method used to bring data together. If data privacy preserving frameworks were adopted, such as federated or swarm learning, the type of deployment should be specified.

The steps for model development should be detailed to promote generalisability and robustness, including the architecture of training, validation and testing datasets, feature selection and editing, regularisation and measures to prevent overfitting and underfitting.

Model performance assessment is mandatory to support credibility, including discrimination and calibration plots. Shared methods for data collection and processing between training and testing datasets should be disclosed, which can impact the future context of use of the algorithm. External prospective evaluation is the most robust way to assess the impact of the tool in clinical practice.

3.21: Address the artificial intelligence model explainability and interpretability, and present the plan for integration into clinical practice, if applicable.

Model predictions should be interpretable by human experts so that confidence and reliability in the model can be evaluated and potential biases can be identified and minimised. Methods to achieve interpretability, such as explainable AI (XAI), should be considered and reported whenever applicable. This may include, for instance, graphical representation of the model’s feature relevance and/or contribution, or a decision tree. Any reasons to omit details of XAI-based model evaluation should be stated (e.g. intellectual property rights).⁶⁵

It is important to use clear terminology that is understood by the target audience (e.g. medical oncology community, other health professionals) and to avoid complex jargon when targeting medical journals. A current challenge is the variety of terminologies used across professional fields and the relative lack of awareness of AI domains within the medical community.

We highly recommend open disclosure of the prospects for clinical deployment and performance monitoring of the AI model; for example, whether it will remain a research tool or eventually become part of a decision support system, with anticipated timelines for integration in routine workflows and ecosystems. If so, aspects related to interoperability of the software application, testing, updating, safety, risks, user variability, human–machine interaction, the degree of influence of the AI technology and plans for certification and HTA (e.g. of medical devices) should be addressed.^66,67

3.22: When applicable, briefly describe the multidisciplinary team required for the study and explain how these needs were met.

The growing complexity of RWE research in oncology often demands multidisciplinarity (inclusion of different and complementary scientific backgrounds and knowledge), which can be important for the design, conduct and reporting of studies.

Depending on the type of research, RWE studies may include clinical oncologists, other health professionals, public health experts, epidemiologists, statisticians, methodologists, data scientists, AI experts and patient representatives, among others. For example, a study applying an AI algorithm to assess the association between risk factors and incidence or mortality from colon cancer in a specific region will require input from, at a minimum, experts in several oncology specialities, epidemiology, data science and AI.

This information could be concisely reported in the methods section or in the supplementary material; for example, as reported for this guidance (see Supplementary Material Section 1, Table S2, available at https://doi.org/10.1016/j.annonc.2023.10.001).

4. Results

4.1: Provide the number of cases excluded or nonparticipating and reasons at each stage of sample selection, as well as numbers lost to follow-up. Compare the cases excluded with those included in the analyses. Illustrate this with a flowchart.

A description of the sample selection according to the predefined eligibility criteria is essential. Specifying the number of cases (e.g. patients, tumours, adverse events) excluded or nonparticipating and the reasons for that at each stage of the sample selection serves three main purposes: (i) fosters the interpretability of results to show who the results apply to; (ii) provides insights into data quality and identification of potential biases⁶⁸; (iii) aids replication efforts and comparisons with other studies. The full process of case selection for analysis should be provided in a flowchart (Figures 1 and 2).

To assess the potential for selection bias and evaluate the generalisability of the results, it is also important to compare the cases excluded or the full population with those included in the study. This comparison could be in the form of a supplementary table summarising key baseline characteristics stratified by participation status. If patients are selected for the study, yet excluded from the main analyses [e.g. if patients with missing values are excluded (‘complete case analysis’)], additional comparisons may be needed.

4.2: Describe the baseline characteristics of the cases included (e.g. clinico-demographic and tumour characteristics). The baseline characteristics of different groups under analysis should be compared, if applicable.

If the study involves patients, a detailed description of the relevant baseline characteristics (e.g. clinico-demographic aspects, tumour biology, biomarkers or previous anticancer treatments) is essential to help readers judge the magnitude and generalisability of the findings.

These data could be presented in a table, preferably stratified by groups of interest based on the study design (e.g. exposure and comparator groups). When carrying out a comparative analysis, beyond reporting the statistical comparison of the groups’ characteristics, it is important to report their respective frequencies and proportions, or measures of central tendency and dispersion. This allows numerical interpretation of the differences and facilitates comparisons with cohorts from other studies. Considering that missing data could affect the validity and generalisability of study results due to the risk of introducing bias in some cases,⁶⁹ authors should carefully describe the proportion of missing values for each relevant variable and by study group. If a propensity score was applied, it is advisable to include an additional table (generally as supplementary material) describing standardised differences before and after analysis.

4.3: Report the results of the primary analysis of study outcomes. Briefly describe the results of exploratory analyses if relevant (prespecified and/or post hoc). Provide details of how readers can access the full results.

Authors should adequately describe the information available on missing data for all variables needed for the study endpoint. To assess data maturity in studies where the main endpoint is time dependent, it is critical to report the median follow-up time (from the index to the event or censoring date) with the respective measure of variability in each group under analysis, if applicable.

Detailed information should be provided about the number of events observed per study group, the number of patients at risk at different time points, if appropriate (e.g. under Kaplan–Meier curves), and the timing of events. Reporting the number of events increases the transparency of the results and facilitates future meta-analyses. Whenever possible, authors must report the summary measure and its confidence interval (CI) (e.g. odds ratio or hazard ratio with respective CI) and not only the P value. This provides the reader with an estimation of the numerical risk increase associated with a given variation of a variable.

Measures of association and respective CIs should be reported for univariable (if reported) and multivariable analyses of all baseline variables prespecified in the SAP. For multivariable analyses, the most relevant measures of association and CIs must be reported. The variables and the sample size included in the model(s) should be explicit and the detailed results of adjustments for missing data, potential imputations, multiple comparisons, confounding factors, effect modifiers, subgroup and sensitivity analyses should be clear.

The results from exploratory analyses may be included, but with a lower priority than the predefined main results. As such analyses may be underpowered or carried out using a dataset not powered to answer that particular research question, addressing these results separately will allow the authors to interpret them with more caution.

5. Discussion and conclusions

Discussion

5.1: Summarise the core results that address the primary research question(s) and objectively discuss the data in relation to the best available evidence on the topic. Avoid a convenient selection of literature to support a point.

The discussion should include the core results responding to the main objective(s) of the study, followed by probable justifications for those findings based on the best available evidence from other studies. Importantly, the overemphasis of nonsignificant or nonprimary results should be avoided.⁶ Unexpected but seemingly relevant results should be discussed.

It is imperative that the discussion of results is based upon the best available evidence, avoiding a convenient selection of literature to support a point. For example, if a meta-analysis or clinical trial reported a negative result in one specific setting, this information should be prioritised above lower-quality evidence that demonstrated more positive results. Selection of literature is challenging as the quality of some publications, particularly RWE studies in oncology, can be uncertain.^70,71 To address this, authors could briefly mention early in the discussion section how supporting literature was selected.

5.2: Discuss the strengths and limitations of the current study, including the main biases, how the strategies applied contributed to bias avoidance or mitigation, and, if applicable, in which direction the authors estimate that residual bias may influence the core results of the study.

The potential strengths of the study should be emphasised to support the claimed level of evidence; however, authors should also summarise the implications of the main biases (see Supplementary Material Section 2, Table S9, available at https://doi.org/10.1016/j.annonc.2023.10.001) on the study findings, providing a clear explanation of which direction they estimate the results could be influenced.

The authors should also describe the strategies applied to avoid or mitigate the risk for specific biases, including the use of available tools.^53,72 The discussion on mitigating strategies should be balanced, considering that none of the methodological strategies to avoid or mitigate bias in RWE studies can eliminate the inherent bias risk associated with the nature of observational studies.

5.3: Discuss the generalisability of the study results and their potential implications for clinical practice, health policies or public health and for the generation of hypotheses for future research.

The implications of the study should be described, balancing the interpretation of core results and limitations. Impressive results that are likely to be biased will not have a great impact and neither will seemingly valid results if these are highly imprecise (e.g. wide CIs).

Importantly, statistical significance should always be interpreted in light of clinical relevance, and details on the magnitude of (non)benefit should be provided rather than simply a qualitative and subjective interpretation.⁷³ For instance, when an RWE comparative effectiveness study reports a small but statistically significant net survival gain of 2 weeks, it is important to clearly discuss the clinical meaningfulness of those results.

When RWD studies provide new information that is relevant to clinical practice and the data are convincing, recommendations about the translation of results could be provided. Implications should be realistic, as specific as possible and aligned with study data (e.g. regarding tumour setting, treatment indications, dosage and patient subgroups).

Regarding the hypothesis-generating implication of RWE studies, the authors should outline the need for further research based on the findings of their study.⁷⁴

Conclusions

5.4: Provide a balanced summary of core results relating to the primary research question and the main implications for clinical practice, health policies and/or public health. Suggest further research considering the remaining unmet needs and limitations from the reported study.

Depending on journal requirements, the conclusions may be an independent section or included in the discussion section.

The conclusion should summarise the main parts of the discussion section without extensive repetition, beginning with the core results and considering the strengths and limitations of the study. Whenever feasible, a brief suggestion for further research using the lessons learned from the reported study and the existing unmet needs should be provided. In a broader perspective, each RWE study is theoretically a source for new knowledge on how to utilise RWD in clinical research and eventually for the optimal conduct of further clinical research; a conclusion related to this added knowledge can be provided.

6. Final considerations

6.1: Specify all relevant study sponsorship(s) as well as direct and/or indirect or in-kind funding.

The sponsor of the study should be reported and all sources of direct funding (e.g. from industry, academic or governmental grants or other sources) should be clearly stated, including information on the role of the funding source in terms of study design, conduct, analysis and reporting. Sources of indirect funding or in-kind contributions (e.g. access to technology infrastructure or technology support) should also be reported. If no sponsorship or funding was received, this should be stated.

Authors are encouraged to state if the core research team conducted the study within their dedicated time for work-related activities.^75,76

6.2: Specify all relevant acknowledgements, author disclosures, individual contributions and other final considerations as per journal regulations.

Each journal has specific rules for final considerations, which should be carefully reviewed by authors before final submission.

The acknowledgements section provides an opportunity to mention individuals who contributed to the study design, study conduct and manuscript development but who are not considered co-authors (e.g. health professionals, researchers, administrative team, medical writers). All authors should provide disclosures, which may vary from study to study. There is frequently a final section to specify each author’s contribution, and international guidance for this section might be considered.^77,78

When used, AI-assisted technologies for manuscript development or revision should also be stated, either in this section or in the methods.⁷⁷

Discussion

ESMO-GROW is, to our knowledge, the first expert-based guidance specifically for reporting oncology RWE studies. Although there are general guidelines for reporting observational studies^5,6 and other more specific guidance, such as for molecular-based RWE research,^7,8 ESMO-GROW incorporates several particularities of modern RWE research in oncology. This includes the rapid development of novel treatment strategies for subgroups of patients, recent trends for molecular-based epidemiology analyses, considerations on oncology-specific variables or outcomes, novel research designs and the increased use of AI, ML and DL for RWE research.

ESMO-GROW includes detailed recommendations for different RWE research scenarios, providing clear guidance to facilitate harmonised interpretation by the users (authors, editors and reviewers). An online tool will be provided with full recommendations for its use in practice. Considering the heterogeneity of RWE studies and journal rules, some recommendations might be integrated into the main manuscript or as supplementary material, on a case-by-case basis.

This guidance is intended for the reporting of descriptive (e.g. epidemiological) or analytical (e.g. explanatory, predictive) oncology RWE studies. It may be also useful for RWE cohorts used in novel pragmatic studies, such as ‘target trial emulation’ designs, although it was not developed for this specific purpose. ESMO-GROW is focused on optimal reporting of oncology RWE studies in peer-reviewed journals; it is neither designed to assess the quality of studies nor a guide for the submission of RWE abstracts to medical congresses. Further initiatives could be focused on those important unmet needs for RWE research in oncology.

Our work has limitations; it was developed based on expert opinion and may not necessarily apply to all types of RWE studies in oncology, which is a rapidly evolving field. Nevertheless, the interdisciplinarity of the contributors (see Supplementary Material Section 1, Tables S1 and S2, available at https://doi.org/10.1016/j.annonc.2023.10.001), numerous rounds of revisions, voting process to validate recommendations and extensive review of literature provide validity for its use across RWE oncology studies.

We anticipate that ESMO-GROW will be widely considered by authors while writing and submitting manuscripts, by journals for manuscript assessment and by readers for critical appraisal of RWE studies in oncology. We will regularly assess the utility of ESMO-GROW for the oncology research community.