Collaborative Real‐World Evidence Among Regulators: Lessons and Perspectives

Abstract

The International Coalition of Medicines Regulatory Authorities (ICMRA), comprising 38 global medicines regulatory authorities, collaborates on shared challenges, notably during the COVID‐19 pandemic. This article focuses on the ICMRA COVID‐19 Real‐World Evidence (RWE) and Observational Studies Working Group. The Working Group aimed to address challenges related to RWE and observational studies during the pandemic, resulting in impactful studies and ICMRA statements on international collaboration for RWE and COVID‐19 vaccine safety. Reflecting on 3 years of collaboration, the Working Group surveyed members for insights, and recommendations were formulated to enhance research preparedness, collaboration, and response to future public health emergencies. The lessons learned highlight the importance of global collaborations, governance structures for rapid decision‐making, and effective utilization of existing networks. Recommendations include the establishment of an international governance structure, a “coalition of the willing” for swift research collaboration, dedicated sub‐groups, periodic workshops, common protocols, joint timelines, and data model templates, leveraging existing infrastructure, and strengthening outreach for transparency and engagement. The Working Group envisions repurposing into an RWE strategic and operational entity, contributing to global public health emergency response mechanisms. In conclusion, the Working Group's success lies in effective communication, collaborative research, and leveraging existing infrastructure, with ongoing contributions to global emergency response mechanisms.

The International Coalition of Medicines Regulatory Authorities (ICMRA) is an assembly of leaders from medicines regulatory authorities worldwide, uniting to provide strategic direction for collaboration, improved communication, and shared approaches to address common challenges, including the COVID‐19 pandemic. Comprised of 38 regulators and with the World Health Organization (WHO) as an observer, ICMRA's mission is emphasized by promoting cooperation among regulatory bodies, facilitating the exchange of reliable information, and enabling the leveraging of resources. ICMRA actively addresses current and emerging regulatory and safety challenges in the realm of human medicines, aiming to fortify the global quality, safety, and efficacy of medicinal products. Central to its vision is the promotion of access to safe, effective, and high‐quality medicines for patients across the globe. ¹

ICMRA's strategic framework serves as a fundamental guide for its members in responding to both current and emerging global challenges in the regulation and safety of human medicines. At its core, the framework articulates three key pillars that embody the coalition's approach to strategic leadership, enabling and facilitating collaboration, and informing and engaging stakeholders. Under the first pillar, strategic leadership, ICMRA aims to identify shared regulatory challenges and leverage collective resources, forging effective channels of communication, and promoting coordinated multilateral responses to global issues. The second pillar, enabling and facilitating collaboration, underscores efforts to support global collaboration needs, enabling regulatory systems that enhance access to safe medicines, fostering innovation in regulatory science, and promoting the convergence of regulatory frameworks. This pillar also seeks to facilitate information exchange, enhance risk‐based allocation of regulatory resources, and coordinate training initiatives and tools. The third pillar, informing and engaging stakeholders, emphasizes transparent communication of ICMRA's goals and activities to stakeholders, leveraging existing initiatives, engaging industry and non‐governmental organizations to address regulatory challenges, and fostering the alignment of regulatory systems worldwide. Altogether, this strategic framework positions ICMRA as a proactive and collaborative force in advancing global regulatory efforts. ¹

The ICMRA COVID‐19 Real‐World Evidence (RWE) and Observational Studies Working Group was established in April 2020 and co‐chaired by the European Medicines Agency (EMA) and Health Canada (HC). This group brought together 28 regulatory authorities (Table S1 ) to address challenges and uncertainties related to RWE and observational studies during the COVID‐19 pandemic. The Working Group addressed critical areas such as the characterization of COVID‐19 disease, the safety and effectiveness of treatments and vaccines through observational studies, and the feasibility of collaboration to address specific research questions in this context.

The Working Group established three technical workstreams focusing on specific areas of interest: the COVID‐19 Vaccine Pharmacovigilance Network, Pregnancy Observational Studies, and Building International Cohorts. These sub‐groups allowed for a more targeted and effective approach to research and decision‐making. Each workstream has made significant progress, fostering information sharing, collaborative projects, and the publication of studies. These collaborations have resulted in several impactful studies and publications that have advanced our understanding of COVID‐19 treatment. ² , ³ , ⁴ , ⁵ , ⁶ , ⁷ , ⁸ , ⁹ , ¹⁰ , ¹¹ , ¹² , ¹³ , ¹⁴ , ¹⁵ Additionally, the group has issued key ICMRA statements on the importance of international collaboration in enabling RWE for regulatory decision‐making and on the safety and effectiveness of COVID‐19 vaccines so healthcare providers have accurate and reliable information to share with patients. ¹⁶ , ¹⁷ As the technical workstreams of the Working Group neared completion, they reflected on lessons learned to consider their future role beyond COVID‐19 (see Table 1 ). They aimed to build on the insights and successes from the pandemic to have a broader and lasting impact. To gather these insights, the Working Group conducted a survey (see Supplementary Item S2 for detailed methods and analysis), which led to the development of seven recommendations on how to prepare, collaborate, and respond internationally to new emergent public health issues (see Table 2 ). These recommendations draw on the collective wisdom and experience of the Working Group members in conducting international observational studies during COVID‐19.

Table 1.

Key lessons learned from the ICMRA working group experience

Uncertainty in the evidence and collaborations to address knowledge gaps

Limitations in individual studies and the significance of engaging in international collaborations for conducting large‐scale and higher‐powered studies Pooling resources, expertise, and data on an international scale is a strategic approach to enhance evidence robustness Essential for a more comprehensive understanding of complex issues, such as pregnancy and COVID‐19 vaccines

Rapid response and the necessity for a governance structure

The Working Group was established within 2 months of the pandemic's onset, leveraging existing structures for rapid regulatory decision‐making. Members highlighted the importance of maintaining “ever warm” collaborations and networks to avoid time loss during emergency setup phases. Three dedicated technical workstreams or sub‐groups with clear scopes and mandates proved beneficial for addressing specific topics. This approach ensured the dynamic and responsive nature of the working group, fostering efficient collaboration among agencies with varying levels of involvement

Project management

Improved the operational aspect of international collaborative observational studies through standardized protocols, timelines, and data model templates Highlighted the effectiveness of the joint master protocol approach and one‐on‐one meetings among regulatory authorities in fostering flexibility and consensus Successful instances like CONSIGN, coagulopathy in COVID‐19, and steroid utilization for COVID‐19 underscored the value of common protocols in enhancing project efficiency

Importance of information sharing among regulators and a broader audience

Emphasized the critical role of effective and timely information sharing, expertise, and data in addressing COVID‐19 challenges Comprehensive sharing of international knowledge for decision‐making, including safety signals, data governance, and ICH activities Periodic workshops and publication of Working Group reports on the ICMRA website played a pivotal role in keeping regulatory authorities informed, promoting transparency, and providing a structured avenue for sharing research activities and best practices Success such as the 2022 ICMRA RWE Workshop which resulted in the ICMRA statement on RWE and international coordination

Addressing challenges in international collaboration

Managing time differences proved difficult, hindering active contribution and sustained involvement Emerging challenges related to funding, resources, and prioritization Complexities were introduced by factors such as the lack of standardization of terminologies and different coding systems

Table 2.

Recommendations based on the lessons learned

1. Develop an international governance structure	Establish a governance structure for overseeing and coordinating international observational studies Identify agencies capable of providing oversight and coordination, emphasizing the facilitation of evidence generation
2. Coalition of the willing	Form a coalition of members ready to collaborate quickly on international observational studies Utilize common protocols and data models to streamline decision‐making and improve efficiency
3. Create dedicated sub‐groups	Establish sub‐groups with clear mandates, focusing on specific areas such as pregnancy or vaccine safety Ensure sustainability of infrastructures and networks beyond public health emergencies
4. Periodic workshops	Organize regular workshops for progress updates and information sharing Share research activities through a shared site to learn best practices and collaborate on common interests
5. Develop common protocol, timelines, and data model templates	Create common protocols, timelines, and data model templates to facilitate collaboration and data sharing. Harmonize data elements, standardize terminologies and develop templates to strengthen existing systems
6. Leverage existing infrastructure	Systematically use existing infrastructure and experience to answer queries on the benefits and risks of medicines at the international level
7. Strengthen outreach for transparency and engagement	Disseminate materials through individual communication channels to enhance transparency and engagement Utilize agency platforms, including social media, websites, and targeted mailings, to maximize visibility and impact

This review article shares the key lessons and recommendations from the ICMRA COVID‐19 RWE and Observational Studies Working Group. It emphasizes the importance of international collaboration, effective governance, and utilizing existing networks to enhance research preparedness and response to future health emergencies. The article details the Working Group's experiences, challenges, and solutions in using RWE and observational studies during the pandemic, including technical workstreams, key studies, and projects (see Supplementary Item S3 ). It concludes with recommendations to improve global health research quality and efficiency, ensuring the safety and effectiveness of treatments during health crises.

LESSONS LEARNED

This section is intended to provide valuable insights for a broad audience, such as regulatory agencies, public health entities, clinicians, academic researchers, and health technology assessment organizations involved in large‐scale collaborative research projects. By reflecting on the experiences and challenges faced by the Working Group, this section aims to highlight key lessons learned that can enhance future collaborative efforts. The shared knowledge and recommendations can serve as a guide to improving the quality, efficiency, and impact of research studies.

Uncertainty in the evidence and collaborations to address knowledge gaps

Recognizing the limitations inherent in individual studies and the necessity for a more expansive data perspective, the members underscored the importance of engaging in international collaborations. These collaborative efforts were essential for conducting large‐scale and high‐powered research studies that could collectively contribute to a more comprehensive understanding of complex issues, such as pregnancy and COVID‐19 vaccines. By consolidating resources, expertise, and data on an international scale, these global collaborations were seen as a strategic approach to navigate uncertainties and enhance the robustness of evidence. An example was the CONSIGN project, ² , ³ where the need for a broader data pool highlighted the importance of a collaborative approach in generating robust evidence. The demonstrated willingness to collaborate played an instrumental role in establishing a collaborative group of regulatory agencies, public health entities, clinicians, and academic researchers, all contributing valuable data to the collective effort.

Rapid response and the necessity for a governance structure

Recognizing the urgency of regulatory decision‐making during the pandemic, the Working Group was successfully established within 2 months of the pandemic's onset by leveraging existing structures. Members emphasized the importance of having established collaborations and networks that are “ever warm” to avoid time loss during the initial setup phase of emergencies. Collaborative efforts without a governance structure led to delays, hindering timely responses to emerging health threats. By accessing a network of regulatory bodies and connecting with academia, the Working Group efficiently addressed queries related to the benefits and risks of medicines at the international level. Utilizing established systems, such as the US FDA's Sentinel, ¹⁸ HC's Canadian Network for Observational Drug Effects Studies (CNODES) ¹⁹ , ²⁰ and CADTH Post‐Market Drug Evaluation (PMDE) ²¹ Program formerly the Drug Safety and Effectiveness Network, ²² and EMA's DARWIN EU, ²³ and others to address queries related to the benefits and risks of medicines at the international level.

Project management

Enhancing the operational aspect of international collaborative observational studies was crucial. Drawing insights from mature systems and harmonizing data elements addressed various challenges experienced during collaboration, including data time lag, resource constraints, lack of standardization of terminologies, and different coding systems. Members emphasized the effectiveness of the joint master protocol approach and one‐on‐one meetings between regulatory authorities. These strategies fostered flexibility, enhanced communication, and facilitated consensus on both scientific and administrative issues, ensuring seamless collaboration and optimizing data for analyses. Successful instances, such as the CONSIGN project, ² the natural history of coagulopathy in COVID‐19 study, and the steroids utilization for the treatment of COVID‐19 study, underscored the value of common protocols in ensuring seamless collaboration, optimizing data for analyses, and enhancing project efficiency. ²⁴ , ²⁵

Importance of information sharing among regulators and a broader audience

The effective and timely sharing of information, expertise, and data was highlighted as a critical aspect in addressing the challenges posed by the COVID‐19 pandemic. Sharing international knowledge for decision‐making was emphasized, particularly in learning about safety signals, data governance, technical workstream activities, and international standards for RWE. Periodic workshops and the publication of Working Group summary reports on the ICMRA website played pivotal roles in keeping regulatory authorities informed, promoting transparency, and providing a structured avenue for sharing research activities and best practices. The success of previous collaborative workshops, such as the ICMRA RWE Workshop in June 2022 co‐chaired by the EMA, the US FDA, and HC, exemplifies the dedication to advancing global RWE collaborations. The positive outcomes of this workshop, including the formulation of the ICMRA statement on international coordination to advance RWE, underscore the importance of sustaining these periodic gatherings. ¹⁶ , ²⁶

Various addressing challenges in international collaboration

The Working Group encountered several challenges that impacted the inclusivity and engagement of participating regulatory agencies. Managing time zone differences posed difficulties in ensuring active contribution and sustained involvement. Additionally, challenges related to funding resources and prioritization emerged, as well as complexities introduced by factors such as lack of standardization of terminologies and different coding systems. These challenges underscored the need for coordinated efforts to streamline collaborative processes.

These lessons learned highlight the importance of strategic collaboration, effective governance, robust project management, and comprehensive information sharing in enhancing the quality, efficiency, and impact of research studies. By addressing these aspects, future collaborative efforts can be significantly improved, contributing to better public health outcomes during emergencies.

RECOMMENDATIONS

The following recommendations were developed by the working group of regulatory agencies for future research projects, but could also be intended for a broader audience and benefit those interested in large‐scale collaborative research projects, such as health technology assessment organizations, public health entities, clinicians, and academic researchers. These recommendations collectively aim to enhance the project life cycle by fostering better research collaborations, improving transparency, and increasing efficiency in addressing global public health challenges, drawing on the lessons learned and experiences of the Working Group (see Table 2 ). They also align with the ICMRA statement on international collaboration on RWE such as harmonization, readiness, and transparency. ¹⁶

Recommendation #1 – Develop an international governance structure for oversight and coordination of international observational studies

To enhance preparedness for future public health emergencies, it is essential to establish a dedicated international governance structure for the oversight and coordination of international observational studies led by regulators. This structure should identify agencies with the capability to provide oversight and coordination, facilitating the efficient setup of collaborative efforts while working within the existing mandates of participating agencies.

Recommendation #2 – Form a “Coalition of the Willing” of regulatory agencies ready for swift collaboration

To enhance the agility and efficiency of international collaboration on observational studies, it is recommended to establish a “Coalition of the Willing” comprising regulatory agencies ready to swiftly engage in collaborative studies. This coalition aims to streamline decision‐making and foster rapid collaboration through the utilization of tools such as common protocols and data models, ensuring regulatory agencies are readily mobilized to generate evidence on a global scale, particularly in response to emerging public health issues.

Recommendation #3 – Dedicated sub‐groups for targeted focus areas

It is recommended to establish dedicated sub‐groups with clear mandates, well‐defined scopes focused on specific areas of interest. These sub‐groups allow for a more targeted and effective approach to research and decision‐making. The timing for forming such sub‐groups should align with the progress and objectives of the broader working group, leveraging the experience gained from existing initiatives.

Recommendation #4 – Organize periodic workshops for sustained collaboration and information exchange

To ensure ongoing collaboration and information exchange among participating agencies, and to enable a wider selection of ICMRA agencies to benefit, the Working Group should continue organizing regular workshops with extended participation in addition to monthly meetings. These workshops serve as valuable platforms for sharing progress, disseminating information, and fostering engagement, ultimately contributing to the advancement of initiatives.

Recommendation #5 – Develop common protocol, timelines, and data model templates for enhanced operational efficiency

To optimize collaboration, data sharing, and project alignment, it is recommended to develop common protocols, timelines, and data model templates. This approach aims to enable faster delivery of evidence into regulatory decision‐making by streamlining research processes, enhancing operational efficiency, and facilitating smooth collaboration among participating regulatory agencies. Standardized metadata should be adopted to streamline the characterization and discoverability of diverse RWD sources. Furthermore, it underscores the importance of creating universally applicable templates for study protocols and reports, aiming to enhance consistency and effectiveness across various regulatory jurisdictions.

Recommendation #6 – Leverage existing infrastructure for enhanced international collaboration

To bolster international collaboration, it is recommended to leverage existing infrastructure and experience, where available. Integrating existing infrastructures into the collaborative framework can be a more streamlined and effective process for evidence generation. This approach underscores the importance of optimizing and expanding upon existing resources to respond effectively to emerging public health challenges. By systematically utilizing well‐established infrastructures, regulatory agencies can contribute to a more coordinated approach to evaluating medicine benefits and risks on a global scale.

Recommendation #7 – Strengthen outreach for transparency and engagement

To enhance transparency, engagement, and the global impact of collaborative efforts, it is recommended that materials developed by ICMRA initiatives, and the Working Group be disseminated through ICMRA and each regulator's individual communications channels. This includes social media, agency websites, targeted mailings, and media distribution lists. Key elements of this recommendation included materials generated by ICMRA, including press releases, news announcements, and key messages. Another key element includes transparency mechanisms, such as reporting of periodic workshops and the registration of research protocols.

DISCUSSION

The Working Group members continued discussions on lessons learned, the seven recommendations, and the Working Group's evolution. While recognizing the importance of rapid response and global collaborations in pandemic management, members emphasized the need for an operational framework for successful and efficient international collaborative observational studies.

The future trajectory of the Working Group envisions a dynamic and strategic approach to international collaborations, extending beyond the immediate challenges previously posed by the COVID‐19 pandemic (see Table 3 ). The group aims to repurpose itself into an RWE strategic and operational entity, prioritizing public health emergencies by leveraging past experiences. This shift demonstrates the group's proactive commitment to preparedness, adaptive nature, and learning from past experiences. Key initiatives include establishing international governance principles, processes, and templates for collaborative studies to enhance response efficiency. The proposal involves continued collaborative projects, ensuring “ever warm” readiness and leveraging past experiences for swift action. Communication and transparency remain crucial, with periodic workshops and public registry publications. A continual evaluation process is integral for ongoing refinement, reflecting the group's dedication to global public health preparedness.

Table 3.

Key points for the future developments of the ICMRA Working Group on COVID‐19 real‐world evidence and observational studies

Establishment of governance principles	Repurposing into an RWE strategic and operational entity, prioritizing public health emergencies, and leveraging past experiences Prioritize the development of governance principles, processes, and templates for collaborative studies to enhance timeliness, data sharing, and project alignment
Optimized resource use	Prioritization of efficient resource utilization and members' time through optimized governance and existing infrastructure, guided by strong leadership.
Collaborative preparedness studies	Conduct collaborative preparedness studies to ensure “ever warm” readiness and leverage past experiences for swift action
Communication and transparency	Strengthen communication and transparency through periodic workshops to review study progress Publish study protocols and results in publicly available registries to improve outreach
Continual evaluation for efficiency	Continual evaluation process integral for ongoing refinement, reflecting dedication to global public health preparedness

In summary, these steps signify the Working Group's commitment to agility, effectiveness, and global collaboration in addressing public health emergencies through RWE and observational studies. The focus on adaptability, transparent communication, and continual evaluation reflects a dedication to fostering global collaboration and ensuring the group remains aligned with its strategic ICMRA goals.

CONCLUSION

In conclusion, this article provides a summary of the work done by the ICMRA Working Group on COVID‐19 real‐world evidence and observational studies, the lessons learned, and its recommendations for the future which aim at enhancing international preparedness and evidence‐based responses to emerging public health issues.

The success of the Working Group was evident in its communication strategies and coordinated research activities. The emphasis on collaboration and information sharing, demonstrated through efficient project management, periodic workshops, and collaborative research protocols, has proven pivotal. These practices have not only facilitated international connections but have also played a role in reducing research waste. The group's ability to leverage existing ICMRA infrastructure and expertise facilitated timely responses during the pandemic, albeit with potential improvements discussed in the recommendations. Despite notable achievements, challenges did persist.

In the face of dynamic and rapidly evolving public health emergencies, ICMRA and its Working Group recognized the need for immediate and collaborative action. The envisioned ongoing role of the group in emergency preparedness and proactive initiatives reflects a commitment to addressing evidence gaps and ensuring the safety, effectiveness, and quality of medicines globally. Looking ahead, the Working Group anticipates future developments that foster continued collaborative studies and real‐time information exchange among participating agencies. This commitment underscores the collective effort to uphold global standards in the face of unforeseen health crises. Through these endeavors, the Working Group strives to contribute to the ongoing improvement of emergency response mechanisms and the safeguarding of public health on a global scale.

FUNDING

No funding was received for this work.

CONFLICT OF INTEREST

The authors declared no competing interests for this work.

DISCLAIMER

The views expressed in this article are the personal views of the author(s) and may not be understood or quoted as being made on behalf of or reflecting the position of the regulatory agency/agencies or organizations with which the author(s) is/are employed/affiliated.

Supporting information

Data S1.