Skip to main content
NCBI home page
ESMO Open logoLink to ESMO Open
editorial
. 2023 Jun 19;8(4):101596. doi: 10.1016/j.esmoop.2023.101596

Learning lessons from the COVID-19 pandemic for real-world evidence research in oncology—shared perspectives from international consortia

L Castelo-Branco 1,2, R Lee 3,4, M Brandão 5, A Cortellini 6,7, A Freitas 8,9, M Garassino 10, T Geukens 11, P Grivas 12,13, S Halabi 14, J Oliveira 15, DJ Pinato 16,17, J Ribeiro 18, S Peters 19, G Pentheroudakis 1, JL Warner 20, E Romano 21
PMCID: PMC10277850  EMSID: EMS212060  PMID: 37418836

Introduction

The coronavirus disease 2019 (COVID-19) pandemic caused a dramatic disruption in clinical practice and undermined the timely delivery of optimal care for patients with cancer worldwide. In order to address many of these challenges and to provide insights into how cancer treatment could be safely delivered to a clinically vulnerable group during the pandemic, several research groups created registries from real-world evidence (RWE). These aimed to understand features and factors associated with COVID-19 overall mortality and severity in patients with cancer, optimal management and the impact of the infection in terms of cancer care delivery.1, 2, 3, 4, 5, 6, 7 This collaborative process was conducted at high speed, with several adjustments required over time as new data, questions and challenges emerged, such as disruptions in the health care systems and patient care globally, new virus strains/variants, COVID-19 treatments and vaccination, and the demonstration of long-term effects of the infection.

During this dynamic period, many important lessons were learned in the design and conduct of RWE studies, spanning different countries, which could be used for future RWE research in oncology and pandemic-related research. Herein, a group of global experts from several registries (Figure 1; Table 1), who collaborated and studied the effects of COVID-19 in patients with cancer, share their experiences and provide perspectives on how these insights could inform and collaborate on future studies.

Figure 1.

Figure 1

Open in a new tab

Geography covered by the different collaborative real-world evidence groups working in cancer and COVID-19. BSMO, Belgian Society of Medical Oncology; CCC19, COVID-19 and Cancer Consortium; CORONET, the COVID-19 Risk in ONcology Evaluation Tool; COVID-19, coronavirus disease 2019; ESMO-CoCARE, European Society for Medical Oncology (ESMO) COVID-19 and Cancer Registry; OnCOVID.UK, Cancer and COVID-19 research initiative from the Imperial College London; SPO, Portuguese Society for Medical Oncology.

Table 1.

Details of the collaborative real-world evidence groups working in cancer and COVID-19

Name Starting date Geography Number of participating centers Number of patients included Collaborations established
ESMO-CoCARE March 2020 Europe
Asia
Africa		 43 2366 CCC19
BSMO
SPO
OnCovid.UK
CORONET
CCC19 March 2020 North America 120+ 19 275 ESMO-CoCARE, OnCovid
OnCovid (UK) February 2020 Europe 37 3820 ESMO-CoCARE
CCC19
NCI
BSMO-COVIDa March 2020 Belgium 19 928 ESMO-CoCARE
OnCovid (Portugal)/SPO March 2021 Portugal 10 276 ESMO-CoCARE
CORONET March 2020 Europe, North America 18 1968 ESMO-CoCARE; various individual groups
TERAVOLT March 2020 Europe
America, North Africa Asia		 92 1491 CCC-19
Open in a new tab

BSMO, Belgian Society of Medical Oncology; CCC19, COVID-19 and Cancer Consortium; CORONET, the COVID-19 Risk in Oncology Evaluation Tool; COVID-19, coronavirus disease 2019; ESMO-CoCARE, European Society for Medical Oncology (ESMO) COVID-19 and Cancer Registry; NCI, National Cancer Institute; OnCOVID.UK, Cancer and COVID-19 research initiative from the Imperial College London; SPO, Portuguese Society for Medical Oncology.

a

Retrospective registration of patients diagnosed between March 2020 and February 2021.

Rapid organization of RWE infrastructures for data collection and analysis

Most health care systems as well as registry/database infrastructures were not yet prepared to study the rapidly evolving landscape of the pandemic, and either significant adjustments were needed or new databases had to be created. Voluntary ‘crowdsourcing’ was broadly used to rapidly capture the continuously changing experience of COVID-19 and its management.

Some groups such as the COVID-19 Risk in ONcology Evaluation Tool (CORONET) opted to start with very basic data capture tools (DCTs) and as studies increased in ambition, more sophisticated DCTs were developed to ensure data homogeneity and high quality. Other groups such as the European Society of Medical Oncology (ESMO) COVID-19 and Cancer Registry (ESMO-COCARE), COVID-19 and Cancer Consortium (CCC19), Belgian Society of Medical Oncology (BSMO) and Sociedade Portuguesa de Oncologia (SPO) decided to start with more sophisticated DTCs and larger number of variables collected from the start of the pandemic. One of the most commonly used tools was the Research Electronic Data Capture (REDCap) system, which enabled rapid prototyping and rollout of sophisticated, secure survey instruments, often across different countries.8 This also allowed future data sharing to increase the number of patients into a larger database.

Having a large electronic case report form (eCRF) to capture different variables from a new disease entity, with adjustments for relevant clinical and biological parameters over pandemic phases and the addition of collaborative centers, created several challenges for data completeness, optimal quality and interoperability. Moreover, in order to add flexibility and recognize the variability of RWE, several groups added numerous free text data variables, such as for medication lists, strain/variant of the virus and/or disease symptoms. This increased the amount and potential utility of data; however, this brought its own challenges and tradeoffs between precision and recall.

Databases that favored selected options over free text were more effective in the delivery of studies at scale. Furthermore, the DCTs had to be flexible in order to add additional fields as the pandemic shifted in terms of new variants, treatments, vaccines and other relevant variables. Thus, professional DCTs became critical to optimize data collection and would be recommended for use in future RWE research.

A frequent obstacle observed during the pandemic was the use of different definitions, grading and timing for the capture of several clinical, pathologic and biologic variables and clinical endpoints/outcomes across different institutions. This was particularly true as new collaborations developed, which required standardization of data fields in order to ensure consistent collection and proper interpretation of data. For instance, the ESMO-CoCARE, CCC19 and the Portuguese ONCOVID initiatives cooperated from the outset to have nearly overlapping data dictionaries, anticipating collaborative analysis. Later when ESMO-CoCARE and BSMO started to collaborate, differences in the data dictionaries required additional manual data curation to merge the datasets. A homogenous metadata for at least core variables under collection or automatic conversion, when different units are used, could mitigate this challenge. In addition, early agreement and definition by key stakeholders of optimal terms and outcome measures can facilitate the acquisition of knowledge over time. One example of this challenge is the use of composite and/or ordinal outcomes, both of which can easily become particular to a singular effort, hampering inter-exchange of ideas and actual data.

Another challenge in an RWE study is managing missing, unknown or unreliable data and this was particularly true for the COVID-19 studies. For instance, CCC-19 chose to implement a universal ‘unknown’ option for all variables, such that missingness at the survey level could be distinguished from underlying missingness in electronic health records.9 To mitigate the incomplete data issue, regular eCRF revisions and the implementation of centrally determined ‘quality scores’, taking into account missingness of critical data elements, such as survival or key prognostic factors were utilized by different groups. Thus, in developing eCRFs, investigators should contemplate not only the variables and outcomes that could optimally address the main research questions under study, but also the feasibility to harmonize the process of data collection and possible adjustments needed over time.

It is also important to invest time interacting with different collaborating centers in order to identify potential pitfalls to data collection before commencing, and estimate the workload (e.g. average time per patient data collection). For example, the CCC19 survey took on average 30 min per case, while for ESMO-CoCARE it was around 1 h. Moreover, careful review of data variables for selection of the clinical relevance can help reduce such burden via exclusion of unnecessary information that may not contribute meaningfully to the data analysis.

Finally, artificial intelligence or machine learning methods, commonly used in different domains, were also applied during the pandemic to improve data analysis and the development of some prognostic or predictive scores.4,10,11 These methods also have huge potential to aid data collection, although in reality natural language processing remains challenging for practical implementation.12 In the context of clinical prediction efforts, particular emphasis should be placed on the reliability, robustness and fairness of the proposed models as well as its transparency, utility and acceptability for health care professionals.13,14

Various ongoing initiatives may contribute to improving the standards of RWE infrastructures, data sharing and research quality globally, such as the European Health Data Space (EHDS)15 or the Data Analysis and Real World Interrogation Network (DARWIN EU).16

Collaborative and multidisciplinary mindset

Frequently, oncology RWE studies are generated from small cohorts, often from single-center experiences and there is commonly parallel research being carried out at different institutions addressing similar questions. This results in a lower level of evidence and various biases, and increases research inefficiencies.17 During the pandemic many centers were motivated to jointly study the effects of COVID-19. Intriguingly, the collaborative process was not only between major institutions, but also with regional and smaller health institutions, even if they were able to provide only a few clinical cases. Thus, the pandemic highlighted the potential of collaborative efforts to generate larger, broader and therefore more powerful datasets. Furthermore, it revealed that when supported, smaller centers are enabled to contribute to a common research goal. This hub-and-spoke type model where larger centers provide an easy-to-use platform to enter data, the administration, logistical and statistical support, thereby facilitating smaller centers to contribute, could be more used in RWE oncology projects.

In addition, bringing together a multi- and inter-disciplinary team of oncologists, virologists, immunologists, public health specialists/epidemiologists, biostatisticians, informaticians and data analysts, among other experts, is essential to improve interpretation of COVID-19-related data. This could be considered for this and other types of RWE research in oncology, with the required set of skills clearly identified to optimally address the aims in each case.

Reducing bureaucracy and improving efficiency

Before the pandemic, bureaucracy was already a common obstacle affecting the ability to conduct research in a timely manner.18,19 During the pandemic there was variability in the approach of different regulators toward these processes. For example, in the UK the research ethics committee removed the requirement to apply for specific approval to use patient data without written consent.20 However, in continental Europe most of the institutions kept their procedures and timings to approve new studies, taking weeks or even months until data could be shared or published. In addition, many institutions required data-sharing agreements to be in place, resulting in the inevitable major delays from legal teams. Thus, it was complex or even impossible for many institutions to share data quickly, compromising the ability to carry out research at a fast pace. It is therefore critical that regulators, ethics committees and institutions review their processes, taking into account the views of all stakeholders, to achieve a better balance between ensuring optimal ethical conduct and data protection while reducing barriers to the conduct of research in a timely manner.

The pandemic also highlighted the value of central registries, which significantly reduced the barriers to data transfer. For instance, the ESMO-CoCARE and CCC19 initiatives had a unique eCRF and a centralized data management from one institution, which was essential to increase efficiency in all processes of data collection. For institutions that wanted to store additional local data, partial federation was also enabled and was for the most part, successful. Going forward, centralization is probably one of the most practical approaches to reducing bureaucracy. Recent progress in fully federated approaches such as ‘Swarm Learning’ in the biomedical domain is intriguing, although the practicality of such approaches remains unproven;21 sites must still agree to adopt a common data model in any case. The creation of trusted research environments with emphasis on data security is another approach. In addition, it is also important to have proper, dedicated administrative, regulatory and legal teams to efficiently handle the bureaucratic issues that may arise, especially in cross-national collaborations, enabling researchers to focus on addressing the research questions.

These COVID-19 collaborative efforts relied on health care professionals providing their time voluntarily, in the majority of cases unsupported by research funding. This was at a time when there were also huge demands on them to deliver care for patients and thus their dedication to research during this period should be applauded. However, this is not a sustainable model for the long term and many of the consortia reported waning of effort over time as COVID-19 data collection became de-prioritized and the ‘routine’ practice resumed. Thus, proper infrastructure supported by dedicated research funds and protected time for health care professionals within institutions are required in order to more effectively deliver RWE. This could range from using electronic health record (EHR) forms that are completed at critical steps along a patient’s journey, to valid wearable devices which constantly collect patient data or patient apps which are completed in real time.22,23 Artificial intelligence techniques, such as natural language processing, may support a more flexible data interpretation paradigm, where events of interest can be extracted from free text into a target schema. In this new age of data-driven decision making, establishing this infrastructure will be critical for automatic, comprehensive and high-quality data collection.

Timely publication of results and guideline updates

The COVID-19 pandemic has been one of the fastest changing areas of clinical knowledge that we have ever seen. From an unknown illness in December 2019 to an extremely well characterized disease in March 2023 with many approved treatments and vaccines, it has shown how research can be rapidly carried out and translated into clinical practice.24,25 Various clinical trials were launched to assess treatments and vaccines for COVID-19 largely in the general population; therefore, such evidence was complemented with prospective and retrospective RWE studies in specific oncology populations, providing additional evidence for clinical and public health decisions.

For instance, before 2020 the estimated time between the start of a vaccine development and approval was around 10-15 years, and that period was shortened to less than a year for COVID-19.24

This resulted in challenges for journals to rapidly disseminate findings in order for the research community to quickly learn from each other. Often COVID-19 articles were fast-tracked to enable rapid publication, and were made open-access at reduced or no cost to share the results with the widest possible audience. However, one of the major drivers of timely research dissemination was the utilization of pre-print servers, which had not otherwise gained much traction in the clinical domain before 2020. These enabled researchers to share their results for comment while in parallel going through the peer review process.26 However, it also created challenges in that many poor-quality studies with often contradictory results were also widely available.27,28 It is therefore even more important that health care professionals are educated in the critical evaluation of studies in order to ensure they base their decisions on high-quality evidence. The implementation of reasonable scientific rigor and principles is of utmost importance not only during standard care, but also during pandemics and other crises.

Guidelines are an opportunity to distil the huge amounts of available information and highlight high-quality findings with the input from experts to inform the best standards in patient management. However, as COVID-19 revealed, they can quickly become outdated. A possible solution is to have regular mechanisms to capture the highest level of evidence for specific clinical decisions, such as the ‘living guidelines’ considered by the World Health Organization for COVID-19 clinical decisions.29,30 ESMO is also embracing this approach and is currently developing online, interactive and regularly updated living guidelines for different tumor settings,31,32 and more recently the American Society of Clinical Oncology (ASCO) also moved into this innovative direction.33

The COVID-19 pandemic has been a catalyst for development of many research areas, but has also highlighted how barriers to RWE need to be overcome. It is now critical that we use this experience to improve the design and conduct of oncology studies in order to rapidly provide high-quality evidence to inform the best practice in oncology (Table 2). It is our hope that many of the lessons that we have learned during the COVID-19 years can be utilized to build rigorous and informative RWE registries at speed, and at a global level.

Table 2.

Key recommendations to start and conduct collaborative RWE studies in oncology

Before starting—preparedness
  • -

    Pre-define agreed aims and research questions of the project(s);

  • -

    Carry out a literature review to ensure there is no research duplication. In addition, consider discussing with other groups opportunities to collaborate;

  • -

    Build a multi- and inter-disciplinary team with relevant skills for the research proposed;

  • -

    Be inclusive with different research centers and early career-motivated members; include diversity of thought, experience and professional backgrounds;

  • -

    Involve patient advocates in the research design, whenever appropriate;

  • -

    Follow best guidelines for that research field;

  • -

    Establish a central (and/or regional) coordinating team and define tasks for each member. Carry out training and quality control to deliver objectives properly and accurately;

  • -

    Ensure sufficient funds are in place, including for infrastructure maintenance and administrative support;

  • -

    Anticipate and mitigate barriers for data sharing;

  • -

    Embrace compliance and ethics principles but remove unnecessary barriers to research.

The research protocol
  • -

    Develop a protocol to be approved in all geographies and participating centers, flexible to accommodate emerging research questions;

  • -

    Define the statistical plan considering guidelines for RWE studies;

  • -

    Engage different centers on protocol development;

  • -

    Define the essential data fields required to address the research goals, distinguishing mandatory from optional variables if relevant;

  • -

    Specify methods for data sharing, interoperability and metadata details. If needed, have a dedicated and qualified team working on interoperability;

  • -

    Consider the use of novel methods for data collection and analysis, use of open software for data collection, artificial intelligence and machine learning. Such methods may require validation before or during the study.

During the study and reporting findings
  • -

    Regularly assess all processes and eventual obstacles with team members; pursue quality control checks;

  • -

    Agree and prioritize goals and deadlines, estimating the time allocated for different tasks (e.g. full data collection per patient);

  • -

    Promote the project amongst the communities for broader collaboration;

  • -

    Reward centers and collaborative members accordingly to their contributions, such as with adequate authorship positions, early access to data, public recognition, congress participation and/or financial support;

  • -

    Be open to new research ideas led by different team members;

  • -

    Ensure that good governance (e.g. steering committee) is achieved and documented;

  • -

    Be ready to adjust priorities upon new important unmet needs;

  • -

    Follow available guidance on reporting RWE studies when submitting for publication;

  • -

    Follow scientific rigor and principles. Report ethical misconduct in a timely manner.

Open in a new tab

RWE, real-world evidence.

Acknowledgements

We would like to thank all centers, researchers, patients, research coordinators and administrative teams for their contribution in the development, monitoring, implementation, support and coordination of the different collaborative projects since the beginning of the COVID-19 pandemic.

Funding

None declared.

Disclosure

LC-B declares full time employment with ESMO as Scientific and Medical Division Fellow. RL reports speaker’s engagement from AstraZeneca; reports consultation/advisory role from Pierre Fabre and Lumisphere Technology; institutional funding from AstraZeneca, BMS and Pierre Fabre. MB reports speaker’s engagement from Janssen, Takeda and Roche; travel grant from Sanofi, Takeda and AstraZeneca; advisory board for Sanofi; research grants (my institution): Roche/GNE, AstraZeneca, Merck, Boehringer and iTeos. AC reports speaker’s engagement from AstraZeneca, and Eisai; consultation/advisory role from AstraZeneca, MSD and OncoC4; writer’s engagement from BMS and MSD. AF reports institutional funding from AstraZeneca. MG declares speaker’s engagement from AstraZeneca, ecancer, GrupoPacifico-Secretaria Tecnica ICAPEM/AstraZeneca, Medscape, MSD, MSD Italia, S.O.S S.r.l, WebMD, WebMD Oncology/Takeda; consultation/advisory role from AstraZeneca/MedImmune, Daiichi Sankyo, AstraZeneca Poland, AstraZeneca UK, Bayer Healthcare Pharmaceuticals, Blueprint Medicines, Daiichi Sankyo, Eli Lilly, GlaxoSmithKline, Incyte, Mirati Therapeutics, MSD, Pfizer, Regeneron Pharmaceuticals, Roche, Sanofi/Prex, Sanofi Genzyme corporation, Seagen International GmbH and Takeda; institutional funding as local PI for phase III studies from Amgen, AstraZeneca, Bluprint, BMS, GlaxoSmithkline, Incyte Corporation, IPSEN Bioscience, MedImmune LCC, Novartis, Roche, Sanofi; institutional funding as local PI for phase II studies from AstraZeneca, Celgene Corporation, Daiichi Sankyo Development, MedImmune LCC, Merck KGaA, Merck Serono, Spectrum Pharmaceuticals, Turning Point Therapeutics; institutional funding as local PI for phase I studies from Exelixis Inc., Janssen; institutional funding as local PI from Bayer, Janssen, Merck, Pfizer, Otsuka Pharmaceutical Italy; institutional funding as coordinating PI from AstraZeneca, MSD; steering committee member of the MK-3475 KN671 Steering Committee (KEYNOTE-671) from MSD, MK-3475 KN671 Steering Committee from MSD, Steering Committee ML41118 from Roche and steering committee of Pfizer Global Lung Cancer Educational Programme; personal financial interests as PACIFIC-R Scientific Committee from AstraZeneca, PACIFIC-R Global Scientific Committee from AstraZeneca, Pacific 6 International Coordinating Investigator from AstraZeneca, steering Committee member and Co-chair at the AstraZeneca Lung Cancer Summit 2019, GSK-Garassino- ZEAL Steering Committee 2020-23, GSK Lung Cancer Global Council, Jannesen Scientific Advisory Board and Therapeutic Area Steering Committee Meeting on Lung Cancer, Seattle Genetics Lung Cancer Platform Study; travel, accommodations and expenses from AstraZeneca, Pfizer and Roche; non-remunerated activities as principal investigator of AO Spedali Civili Brescia, Eli Lilly Studio TYME trial, European Thoracic Oncology Platform (ETOP) phase III trial, GOIRC Studio CHANCE trial, GOIRC RAME trial, GUSTAVE-ROUSSY PARIGI LIPI TRIAL, Istituto dei Tumori Pascale MILES 5 trial, Istituto dei Tumori Pascale IND227 trial, Istituto dei Tumori Pascale Beverly trial, Istituto Nazionale dei Tumori Progetto Timoma trial, Istituto Nazionale dei Tumori APOLLO trial, Istituto Nazionale dei Tumori Bando finalizzata Mesotelioma trial, Istituto Nazionale dei Tumori POST-ALK trial, Istituto Nazionale dei Tumori TERAVOLT trial, Istituto Nazionale dei Tumori FAME trial, MSD People, Pfizer STYLE trial, and Sant’Orsola Malpighi Creta trial; non-remunerated activities as member of AIOM, AIOT, ASCO, EMA Scientific Advisory Group, ESMO, IPOP, TUTOR, WCLC, Women for Oncology Italy; non-remunerated activities as Scientific Programme Committee of AACR. PG reports consultation/advisory role from 4D Pharma, Aadi Bioscience, Asieris Pharmaceuticals, Astellas Pharma, AstraZeneca, BostonGene, Bristol Myers Squibb, CG Oncology, Dyania Health, Exelixis, Fresenius Kabi, G1 Therapeutics, Genentech, Gilead Sciences, Guardant Health, ImmunityBio, Infinity Pharmaceuticals, Janssen, Lucence, Mirati Therapeutics, MSD, Pfizer, PureTech, Regeneron, Roche, Seattle Genetics, Silverback Therapeutics, Strata Oncology, QED Therapeutics, Merck KGaA, UroGen Pharma; research funding for Bavarian Nordic, Bristol Myers Squibb, Clovis Oncology, Debiopharm Group, Merck KGaA, G1 Therapeutics, GlaxoSmithKline, Gilead Sciences, MSD, Mirati Therapeutics, Pfizer,QED Therapeutics. SH reports member of DSMB for Aveo Oncology, BMS, J&J and Sanofi; institutional funding from ASCO. JO declares speaker honoraria from AstraZeneca, Bayer, BMS, GSK, Janssen, MSD, Novartis and Roche; declares advisory role from AstraZeneca, Eisai, Janssen, Novartis and Roche; Institutional funding from AstraZeneca for investigator-initiated clinical trial. DJP reports consultation/advisory role from AstraZeneca, Bayer Healthcare, DaVolterra, Eisai, Exact Sciences, H3B, Mina Therapeutics and MURSLA; speaker’s engagement from BMS, IPSEN and Roche; editor-in-chief role from Wiley; institutional research grant from BMS and MSD; principal investigator of Apollomics, H3B, Incyte, Ribon Therapeutics, Roche, Starpharma, Turning Point Therapeutics; charity trustee of Cancer Treatment and Research Trust. JR reports speaker honoraria and/or advisory board from AstraZeneca, eESO and Gilead Sciences. SP reports consultation/advisory role for AbbVie, Amgen, Arcus, AstraZeneca, Bayer, BeiGene, Bio Invent, Biocartis, Blueprint Medicines, BMS, Boehringer Ingelheim, Daiichi Sankyo, Debiopharm, Eli Lilly, F-Star, Foundation Medicine, Genzyme, Gilhead, GlaxoSmithKline, Illumina, Incyte, IQVIA, iTheos, Janssen, Merck Serono, Mirati, MSD, Novartis, Novocure, Pfizer, PharmaMar, Phosplatin Therapeutics, Regeneron, Roche/Genentech, Sanofi, Seattle Genetics, Takeda and Vaccibody; speaker’s engagement from AstraZeneca, BMS, Boehringer Ingelheim, Ecancer, Eli Lilly, Fishawack, Imedex, Medscape, Mirati, MSD, Novartis, OncologyEducation, PER, Pfizer, PRIME, RMEI, Roche/Genentech, RTP, Sanofi and Takeda; associate editor Annals of Oncology for Elsevier; coordinating PI for AstraZeneca; steering committee member of AstraZeneca, Beigene, BMS, iTeos, Mirati, MSD, Pharma Mar, Phosplatin Therapeutics and ALEX; trial chair for GSK and Roche/Genentech. GP reports full time employment from ESMO. JLW reports advisory board role for Melax Tech, consulting with Westat; deputy editor of JCO Clinical Cancer Informatics for ASCO; ownership interest in HemOnc.org LLC. ER reports investigator-initiated trial funds (paid to the institution) by AstraZeneca, BMS, Janssen and Amgen; coordinating PI of Light Chain Biosciences. TG has declared no conflicts of interest.

References

  • 1.Castelo-Branco L., Tsourti Z., Gennatas S., et al. COVID-19 in patients with cancer: first report of the ESMO international, registry-based, cohort study (ESMO-CoCARE) ESMO Open. 2022;7(3):100499. doi: 10.1016/j.esmoop.2022.100499. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Kuderer N.M., Choueiri T.K., Shah D.P., et al. Clinical impact of COVID-19 on patients with cancer (CCC19): a cohort study. Lancet. 2020;395(10241):1907–1918. doi: 10.1016/S0140-6736(20)31187-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Pinato D.J., Scotti L., Gennari A., et al. Determinants of enhanced vulnerability to coronavirus disease 2019 in UK patients with cancer: a European study. Eur J Cancer. 2021;150:190–202. doi: 10.1016/j.ejca.2021.03.035. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Lee R.J., Wysocki O., Zhou C., et al. Establishment of CORONET, COVID-19 risk in oncology evaluation tool, to identify patients with cancer at low versus high risk of severe complications of COVID-19 disease on presentation to Hospital. JCO Clin Cancer Inform. 2022;6 doi: 10.1200/CCI.21.00177. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Whisenant J.G., Trama A., Torri V., et al. TERAVOLT Thoracic Cancers International COVID-19 Collaboration. Cancer Cell. 2020;37(6):742–745. doi: 10.1016/j.ccell.2020.05.008. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Geukens T., Brandão M., Laenen A., et al. Changes in anticancer treatment plans in patients with solid cancer hospitalized with COVID-19: analysis of the nationwide BSMO-COVID registry providing lessons for the future. ESMO Open. 2022;7(6):100610. doi: 10.1016/j.esmoop.2022.100610. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Grivas P., Khaki A.R., Wise-Draper T.M., et al. Association of clinical factors and recent anticancer therapy with COVID-19 severity among patients with cancer: a report from the COVID-19 and Cancer Consortium. Ann Oncol. 2021;32(6):787–800. doi: 10.1016/j.annonc.2021.02.024. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Harris P.A., Taylor R., Thielke R., et al. Research electronic data capture (REDCap) - a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42(2):377–381. doi: 10.1016/j.jbi.2008.08.010. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Abidi M., Aboulafia D.M., Accordino M.K., et al. A systematic framework to rapidly obtain data on patients with cancer and COVID-19: CCC19 governance, protocol, and quality assurance. Cancer Cell. 2020;38(6):761–766. doi: 10.1016/j.ccell.2020.10.022. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Boddu R.S.K., Karmakar P., Bhaumik A., et al. Analyzing the impact of machine learning and artificial intelligence and its effect on management of lung cancer detection in covid-19 pandemic. Mater Today Proc. 2022;56:2213–2216. doi: 10.1016/j.matpr.2021.11.549. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Asada K., Komatsu M., Shimoyama R., et al. Application of artificial intelligence in COVID-19 diagnosis and therapeutics. J Pers Med. 2021;11(9):886. doi: 10.3390/jpm11090886. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Savova G.K., Danciu I., Alamudun F., et al. Use of natural language processing to extract clinical cancer phenotypes from electronic medical records. Cancer Res. 2019;79(21):5463–5470. doi: 10.1158/0008-5472.CAN-19-0579. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Wysocki O., Davies J.K., Vigo M., et al. Assessing the communication gap between AI models and healthcare professionals: explainability, utility and trust in AI-driven clinical decision-making. Artif Intell. 2023;316:103839. doi: 10.48550/arXiv.2204.05030. [DOI] [Google Scholar]
  • 14.Sovrano F., Vitali F. Explanatory artificial intelligence (YAI): human-centered explanations of explainable AI and complex data. Data Min Knowl Disc. 2022 doi: 10.1007/s10618-022-00872-x. [DOI] [Google Scholar]
  • 15.Horgan D., Hajduch M., Vrana M., et al. European Health Data Space - an opportunity now to grasp the future of data-driven healthcare. Healthcare (Basel) 2022;10(9):1629. doi: 10.3390/healthcare10091629. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.EMA EMA-. Data Analysis and Real World Interrogation Network (DARWIN EU) 2023. https://www.ema.europa.eu/en/about-us/how-we-work/big-data/data-analysis-real-world-interrogation-network-darwin-eu Available at:
  • 17.Booth C.M., Karim S., Mackillop W.J. Real-world data: towards achieving the achievable in cancer care. Nat Rev Clin Oncol. 2019;16(5):312–325. doi: 10.1038/s41571-019-0167-7. [DOI] [PubMed] [Google Scholar]
  • 18.Perez-Gracia J.L., Awada A., Calvo E., et al. ESMO Clinical Research Observatory (ECRO): improving the efficiency of clinical research through rationalisation of bureaucracy. ESMO Open. 2020;5(3) doi: 10.1136/esmoopen-2019-000662. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Perez-Gracia J.L., Penel N., Calvo E., et al. Streamlining clinical research: an ESMO awareness call to improve sponsoring and monitoring of clinical trials. Ann Oncol. 2023;34(1):70–77. doi: 10.1016/j.annonc.2022.09.162. [DOI] [PubMed] [Google Scholar]
  • 20.Health-Research-Authority N-. NHS COVID-19 research. https://www.hra.nhs.uk/covid-19-research/ Available at:
  • 21.Warnat-Herresthal S., Schultze H., Shastry K.L., et al. Swarm Learning for decentralized and confidential clinical machine learning. Nature. 2021;594(7862):265–270. doi: 10.1038/s41586-021-03583-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Beg S., Handa M., Shukla R., et al. Wearable smart devices in cancer diagnosis and remote clinical trial monitoring: transforming the healthcare applications. Drug Discov Today. 2022;27(10):103314. doi: 10.1016/j.drudis.2022.06.014. [DOI] [PubMed] [Google Scholar]
  • 23.Beauchamp U.L., Pappot H., Holländer-Mieritz C. The use of wearables in clinical trials during cancer treatment: systematic review. JMIR Mhealth Uhealth. 2020;8(11) doi: 10.2196/22006. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Kalinke U., Barouch D.H., Rizzi R., et al. Clinical development and approval of COVID-19 vaccines. Expert Rev Vaccines. 2022;21(5):609–619. doi: 10.1080/14760584.2022.2042257. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Murakami N., Hayden R., Hills T., et al. Therapeutic advances in COVID-19. Nat Rev Nephrol. 2023;19(1):38–52. doi: 10.1038/s41581-022-00642-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Otridge J., Ogden C.L., Bernstein K.T., Knuth M., Fishman J., Brooks J.T. Publication and Impact of Preprints Included in the First 100 Editions of the CDC COVID-19 Science Update: Content Analysis. JMIR Public Health Surveill. 2022;8(7) doi: 10.2196/35276. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Teixeira da Silva J.A. Adjusting the use of preprints to accommodate the ‘quality’ factor in response to COVID-19. J Taibah Univ Med Sci. 2021;16(4):477–481. doi: 10.1016/j.jtumed.2021.04.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28.Añazco D., Nicolalde B., Espinosa I., et al. Publication rate and citation counts for preprints released during the COVID-19 pandemic: the good, the bad and the ugly. PeerJ. 2021;9 doi: 10.7717/peerj.10927. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.World Health Organization Therapeutics and COVID-19: living guideline. WHO-2019-nCoV-therapeutics-2022.4. 2023. https://www.who.int/publications/i/item/WHO-2019-nCoV-therapeutics-2022.4 Available at: [PubMed]
  • 30.White H., McDonald S.J., Barber B., et al. Care for adults with COVID-19: living guidelines from the National COVID-19 clinical evidence taskforce. Med J Aust. 2022;217(7):368–378. doi: 10.5694/mja2.51718. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Curigliano G., Castelo-Branco L., Gennari A., Harbeck N., Criscitiello C., Trapani D. ESMO Metastatic Breast Cancer Living Guidelines, v1.0.0 May 2023. 2023. https://www.esmo.org/living-guidelines Available at:
  • 32.Gennari A., André F., Barrios C.H., et al. ESMO Clinical Practice Guideline for the diagnosis, staging and treatment of patients with metastatic breast cancer. Ann Oncol. 2021;32(12):1475–1495. doi: 10.1016/j.annonc.2021.09.019. [DOI] [PubMed] [Google Scholar]
  • 33.Giri V.N., Oliver T.K., Rumble R.B., Friedberg J.W., Miller K.D., Geisel R.L. ASCO Living Guidelines: The Next Frontier. J Clin Oncol. 2023;41(5):955–957. doi: 10.1200/JCO.22.02344. [DOI] [PubMed] [Google Scholar]

RESOURCES