The National Collaborating Centre for Methods and Tools (NCCMT): Supporting evidence-informed decision-making in public health in Canada

Abstract

The National Collaborating Centre for Methods and Tools (NCCMT) is part of a network of six National Collaborating Centres for Public Health (NCC) created in 2005 by the federal government following the severe acute respiratory syndrome (SARS) epidemic to strengthen public health infrastructure in Canada. The work of the NCCMT, to support evidence-informed decision-making (EIDM) in public health in Canada, is accomplished by curating trustworthy evidence, building competence to use evidence and accelerating change in EIDM. Ongoing engagement with its target audiences ensures NCCMT’s relevance and ability to respond to evolving public health needs. This has been particularly critical during the coronavirus disease 2019 (COVID-19) pandemic, which saw NCCMT pivot its activities to support the public health response by conducting rapid reviews on priority questions identified by decision-makers from federal to local levels as well as create and maintain a national repository of in-progress or completed syntheses. These efforts, along with partnering with the COVID-19 Evidence Network to support Decision-Making (COVID-END), sought to reduce duplication, increase coordination of synthesis efforts and support decision-makers to use the best available evidence in decision-making. Data from website statistics illustrate the successful uptake of these initiatives across Canada and internationally.

Overview—National Collaborating Centres for Public Health

Funded by the Public Health Agency of Canada (PHAC) each of the six National Collaborating Centres for Public Health (NCCs) focuses on a specific public health area: Determinants of Health, Healthy Public Policy, Knowledge Translation Methods and Tools, Infectious Diseases, Environmental Health, and Indigenous Health. Each is hosted by an academic institution or government-based organization, which are geographically dispersed across the country (1–3). In 2019, PHAC renewed funding for the NCCs for an eight-year period (2020–2028), reaffirming their value in Canada's public health infrastructure.

The NCCs synthesize and disseminate high-quality evidence and knowledges, foster collaboration among diverse groups and support the use of the best available evidence in public health decision-making to improve health outcomes for Canadians. Expert advisory boards, comprising public health practitioners, senior decision-makers, policy makers and Indigenous leaders, provide advice to their respective NCC on goals and objectives and annual workplans, prior to their submission to PHAC for approval. The NCC priorities are established through national gatherings, participation in networks and committees and needs assessments. Detailed descriptions of the NCCs have been reported previously (3,4).

National Collaborating Centre for Methods and Tools—Mandate

This article, the third of six, describes the work of the National Collaborating Centre for Methods and Tools (NCCMT; the Centre) generally, and its response to coronavirus disease 2019 (COVID-19) specifically. The NCCMT (5) acts as an evidence intermediary, curating trustworthy evidence, and building capacity in public health for evidence-informed decision-making (EIDM) including finding, appraising, interpreting, adapting and implementing evidence into decision-making.

The Centre’s strategic direction and workplans are guided through extensive consultation with its advisory board members and target audiences, including front-line public health practitioners and decision-makers, policy makers at all levels of government, those working in other public health organizations, post-secondary institutions that provide education and training to future public health professionals, public health researchers, the Public Health Network and PHAC. Engagement with these audiences ensures NCCMT’s agility and responsiveness to evolving public health needs, as illustrated below in relation to COVID-19. Core objectives for 2020–2028 include developing methods and tools to facilitate synthesis of a wide array of evidence beyond research evidence, facilitating organizational change and supporting ongoing EIDM capacity development.

The NCCMT’s work and related resources and services can be categorized into three domains: access to the evidence on what works; capacity development in EIDM; and system change (Table 1).

Table 1. National Collaborating Centre for Methods and Tools’ work and related resources and services.

NCCMT resource	Description	Launch date	Audience use
Supporting access to evidence
Health Evidence™ (6)	A searchable repository of over 6,900 critically appraised systematic reviews evaluating the effectiveness and cost-effectiveness of public health interventions	2005	Annual average: 90,000 visits from 181 countries
Registry of Evidence-Informed Decision-Making Tools (7)	A curated, searchable repository of over 150 methods and tools in EIDM	2007	Annual average: 250,000 visits from 195 countries
Capacity development for EIDM
Online learning modules (8)	Twelve interactive modules focused on one or more steps in the EIDM process	2011	Completed over 35,000 times
Understanding Research Evidence videos (9,10)	Eleven short videos explaining research terms	2014	Viewed over 300,000 times
Evidence-Informed Decision-making Skills Assessment (11)	A 20-item tool of multiple-choice questions that assess EIDM knowledge and skill	2018	Completed over 3,000 times by 1,400 unique users
Knowledge Broker Mentoring program (12)	A 16-month training program to support organizational capacity development for EIDM	2014	Completed by 55 participants from 10 public health organizations
Workshops (13)	Half, full, and multi-day sessions to build EIDM capacity	2010	Delivered to 28 Canadian public health organizations
Webinars (14)	90-minute sessions to explore and practice EIDM competencies	2012	Annual average: 10 webinars; 1,500 attendees Over 90% agree participation increased understanding of EIDM
Systems change resources
Applicability and Transferability of Evidence Tool (15)	Assesses the feasibility and generalizability of evidence to public health practice in specific jurisdictions	2011	Accessed more than 5,500 times since 2017
Rapid Review Guidebook (16)	Step-by-step guide to the rapid review process	2017	Accessed over 10,000 times
Quality Assessment of Community Evidence (QACE) Tools (17)	Two tools that can be used to assess community evidence to ensure it is relevant, trustworthy and equity-informed	2020	Accessed over 2,300 times

Abbreviation: EIDM, evidence-informed decision-making

Access to evidence

To ensure access to evidence on what works in public health, the NCCMT maintains and continuously updates two curated repositories: Health Evidence™ (6) contains research evidence related to public health practice and The Registry of Evidence-Informed Decision-making Tools contains methods and tools for EIDM (7). The repositories are primarily visited by public health nurses, managers, project specialists and government representatives and policy makers, students and researchers.

Capacity development

A diverse suite of capacity development resources and services are available both online, through a skills assessment, learning modules, and videos, and face-to face, including self-paced and virtual workshops and mentoring programs. Briefly, the skills assessment (11) assesses individual and/or organizational knowledge and skill for EIDM. The learning modules (8) focus on one or more steps in the EIDM process, while the Understanding Research Evidence video series (9,10) explains regularly used research terms (relative risk, odds ratios). The NCCMT also provides education and mentorship through webinars (14), tailored workshops (13) and the knowledge broker mentoring program (12). The knowledge broker mentoring program is an intensive training program to build organizational capacity in EIDM.

Systems change resources

At the systems level, resources are available that can be embedded within decision-making mechanisms such as how to conduct a rapid review, assess community level evidence and assess the applicability and transferability of evidence to a jurisdiction. The Rapid Review Guidebook (16) outlines the steps of rapid reviews. The Quality Assessment of Community Evidence (QACE) Tool (17) assesses the quality of community evidence (local surveillance and contextual evidence, and societal and political preferences). Finally, the Applicability and Transferability of Evidence Tool (15), evaluates the feasibility and generalizability of evidence in different settings.

Measures of impact

Internal contact data illustrate the NCCMT has provided education and mentorship to over 435 organizations in every province and territory in Canada, and 40 governments in six countries, 42 public health organizations in seven countries, 54 health care organizations in seven countries, 281 post-secondary institutions in 23 countries and 145 countries with at least one person accessing the learning modules.

Personalized quarterly outreach, to 215 senior decision-makers (e.g. Medical Officers of Health, senior management) in Canada has led to 88 new projects in the last five years. Routine communication with 27 public health programs/schools in academic institutions resulted in the integration of resources into curricula, and 14 Master of Public Health student practicum placements with the NCCMT, contributing to students’ preparedness for a future in public health (18).

Results from embedded pre-post knowledge and self-efficacy assessments in the online learning modules show statistically significant increases in knowledge (p<0.0001) and self-efficacy (p<0.01) (C. Howarth, personal communication, March 21, 2018), and pre-post evaluation of the Understanding Research Evidence videos also show statistically significant increases in knowledge (p<0.001) (10). Statistically significant increases in knowledge have also been observed in pre-post evaluations of EIDM workshops (p<0.001) (M. Dobbins, personal communication, June 20, 2017) and the knowledge broker mentoring program (p<0.001) (J. Yost, personal communication, April 28, 2016), and the Rapid Review Guidebook has been adopted by several public health organizations.

National Collaborating Centre for Methods and Tools’ response to COVID-19

In March 2020, the NCCMT pivoted its work to support public health’s response to the COVID-19 pandemic by creating a rapid evidence service, developing a COVID-19 public health review repository, and partnering with others to increase coordination of evidence syntheses.

Rapid evidence service

In April 2020, in response to requests from the Pan-Canadian Public Health Network’s Special Advisory Committee on COVID-19, its supporting Technical Advisory Committee, and public health decision-makers at the local, regional and provincial levels, the NCCMT started conducting rapid reviews on public health topics in close collaboration with the Office of the Chief Science Officer within PHAC. Living reviews were registered with PROSPERO. The reviews informed public health measures related to re-opening of schools, and transmission in long-term care facilities, gyms and restaurants. Other reviews informed early thinking on surface transmission, the incubation period, wastewater as a surveillance strategy and COVID-19 re-infection risk. NCCMT’s Rapid Review Guidebook (16,19) guided such processes as discussions with the requestor to refine questions, appraising and GRADing the evidence, and identifying key messages and knowledge gaps. Reviews were completed in five to ten days, posted on NCCMT’s website (20) and widely disseminated. Public interest in “hot topics” generated substantial media uptake with news coverage by more than 30 media outlets. As of December 2020, the NCCMT had completed 43 full reviews or updates on 25 unique questions that are indexed in global databases and have been downloaded an average of 250 times per review from people in 82 countries (S. Neil-Sztramko, personal communication, December 15, 2020). Additionally, the NCCMT connected with Canadian and international evidence synthesis organizations to discuss duplication of reviews, rapid review methods and to share capacity building resources.

COVID-19 repository

The NCCMT created the COVID-19 public health repository for Canadian reviews in April 2020. Both currently underway and completed rapid reviews were eligible for inclusion. As of December 2020, 215 rapid reviews were included in the repository, and it had received over 43,000 page views from people in 108 countries (S. Neil-Sztramko, personal communication, December 15, 2020). Anecdotal evidence shows that duplication of effort was avoided when visitors to the site identified a review in progress or completed on a topic they intended to conduct a review on.

Systematic reviews to support public health system recovery

The NCCMT is completing two systematic reviews to 1) identify effective strategies to support the mental health of frontline workers responding to COVID-19 and 2) identify strategies for post-pandemic public health system recovery. Both reviews are registered with PROSPERO and follow methods outlined in the Cochrane handbook. Once completed, the reviews will be disseminated broadly.

Amplifying networks and collaborations

The NCCMT was an early partner of the COVID-19 Evidence Network to support Decision-making (COVID-END) (21), an international network of more than 50 evidence synthesis and knowledge translation organizations. COVID-END’s aim is to support decision-makers in finding and using evidence, while reducing duplication. Of the seven working groups, NCCMT’s Scientific Director co-leads the engaging workgroup, which supports those supporting decision-makers via an online discussion group and monthly webinar series by disseminating resources related to evidence synthesis (22). The NCCMT is also participating in COVID-END Canada, a Canadian Institutes of Health Research Operating Grant, which is conducting evidence syntheses to support health care, public health and health systems decision-making.

Challenges and next steps

The NCCMT’s quick pivot was not without its challenges. Some staff required training in rapid review methods, the pace of work changed dramatically from projects that generally took months to complete to projects needing to be completed in just days, the demand for reviews outweighed the Centre’s capacity to complete them, the evidence, particularly in the early months, changed almost daily, and the rapid review methods had to be modified as the evidence evolved (19). In addition, given the abundance of evidence there were challenges ensuring decision-makers were aware of the best available and up to date evidence, and it was impossible to stay abreast of all rapid reviews in progress, resulting in some duplication of effort.

As the sprint of the pandemic transitioned to a marathon, it was important to reduce the work pace to avoid staff burnout, and horizon scanning was important but challenging to do given decision-makers’ limited availability. As the anniversary of the COVID-19 pandemic passes, it is important to start planning for post-COVID, although many uncertainties remain as to what the needs of public health will be. In the days and months ahead, engagement with the Centre’s target audiences will assist the NCCMT to be ready to transition again to meet the evidence and capacity development needs of the public health sector.

Conclusion

Since its launch in 2007, the NCCMT has contributed to evidence-informed public health decision-making by ensuring resources and services that directly address the EIDM needs of the public health sector are readily available. Its extensive network and long-standing collaborative relationships with decision-makers at all levels contributed to NCCMT recognizing the need to quickly pivot its activities as the pandemic took hold. This experience highlights the important role autonomous organizations, at arms-length to government, with the flexibility to change their proposed workplans, can have in times of crisis.