Development of the Canadian COVID-19 Emergency Department Rapid Response Network population-based registry: a methodology study

Abstract

Background:

Emergency physicians lack high-quality evidence for many diagnostic and treatment decisions made for patients with suspected or confirmed coronavirus disease 2019 (COVID-19). Our objective is to describe the methods used to collect and ensure the data quality of a multicentre registry of patients presenting to the emergency department with suspected or confirmed COVID-19.

Methods:

This methodology study describes a population-based registry that has been enrolling consecutive patients presenting to the emergency department with suspected or confirmed COVID-19 since Mar. 1, 2020. Most data are collected from retrospective chart review. Phone follow-up with patients at 30 days captures the World Health Organization clinical improvement scale and contextual, social and cultural variables. Phone follow-up also captures patient-reported quality of life using the Veterans Rand 12-Item Health Survey at 30 days, 60 days, 6 months and 12 months. Fifty participating emergency departments from 8 provinces in Canada currently enrol patients into the registry.

Interpretation:

Data from the registry of the Canadian COVID-19 Emergency Department Rapid Response Network will be used to derive and validate clinical decision rules to inform clinical decision-making, describe the natural history of the disease, evaluate COVID-19 diagnostic tests and establish the real-world effectiveness of treatments and vaccines, including in populations that are excluded or underrepresented in clinical trials. This registry has the potential to generate scientific evidence to inform our pandemic response, and to serve as a model for the rapid implementation of population-based data collection protocols for future public health emergencies.

Trial registration:

Clinicaltrials.gov, no. NCT04702945

The coronavirus disease 2019 (COVID-19) pandemic is the largest public health crisis in over a century.¹ As of Jan. 15, 2021, COVID-19 has resulted in over 113 million infections and almost 2.5 million deaths globally. ² The global crude mortality rate among patients diagnosed with COVID-19 is about 3%, but some countries have reported rates that are up to 3 times higher.^2,3 Factors explaining these variations include population differences in demographics, health status and socioeconomics, as well as system factors such as the availability of testing, pandemic preparedness and response, with others yet to be uncovered. ^4,5 There is an urgent need for high-quality, population-level data to understand modifiable risks for disease severity and transmissibility, to evaluate therapies and vaccines and to develop evidence-based prevention, treatment and resource allocation strategies.

The emergency department is often the first point of contact for patients with severe COVID-19, and is the location where critical decisions regarding management and disposition are made.⁶ These decisions affect downstream health resource use and transmissibility. Early in the pandemic, emergency physicians were encouraged to intubate hypoxic patients early in their presentation to reduce aerosols, based on theoretical considerations without evidence of effectiveness.⁷ This and other strategies to manage hypoxia have since evolved, while maintaining good patient outcomes. ⁸ Similarly, early, poor-quality data showed a possible benefit of hydroxychloroquine, now proven ineffective.⁹

We developed the Canadian COVID-19 Emergency Department Rapid Response Network (CCEDRRN) to collect high-quality, population-based data from geographically distributed sites over the course of the pandemic. We will use this registry to derive and validate clinical decision rules to enable evidence-based emergency department decision-making. We will also use the registry to evaluate emerging therapies and vaccines, particularly among populations commonly excluded or underrepresented in clinical trials. Our objective is to describe the methods we have used to collect and ensure the data quality in our registry of patients suspected and confirmed to have COVID-19.

Methods

Study design and setting

This methodology study outlines the development of the CCEDRRN registry. We designed this population-based, multisite registry to enrol consecutive eligible patients presenting with suspected or confirmed COVID-19 to 50 emergency departments in 8 Canadian provinces, from Mar. 1, 2020 onward (Figure 1; for details on contributing sites and study investigators, see Appendix 1, available at www.cmajopen.ca/content/9/1/E261/suppl/DC1). We maintain a list of network investigators and hospitals on our website (https://canadiancovid19ednetwork.org).¹⁰

Figure 1:

Participating sites of the Canadian COVID-19 Emergency Department Rapid Response Network registry. This figure contains information licensed under the Open Government Licence – Canada (https://open.canada.ca/en/open-government-licence-canada).

Study population

We are enrolling patients presenting to participating emergency departments with suspected or confirmed COVID-19. We defined 2 periods for enrolment based on the availability of COVID-19 testing (Table 1). Research assistants use medical microbiology testing and discharge diagnoses to screen for potentially eligible patients (Table 2).

Table 1:

Screening date, province and inclusion criteria

Period 1	Criteria
Alberta: Mar. 1–Apr. 7, 2020 British Columbia: Mar. 1–Apr. 19, 2020 Manitoba: Mar. 1–Apr. 27, 2020 New Brunswick: Mar. 1–Apr. 12, 2020 Nova Scotia: Mar. 1–Apr. 5, 2020 Ontario: Mar. 1–May 13, 2020 Quebec: Mar. 1–May 3, 2020 Saskatchewan: Mar. 1–Apr. 2, 2020	Presenting to the ED meeting WHO clinical criteria for suspected COVID-19: Fever and Respiratory syndrome, including flu-like illness, shortness of breath or cough Presenting to the ED and tested for SARS-CoV-2 in the ED
Period 2	Criteria
Alberta: Apr. 8, 2020 onward British Columbia: Apr. 20, 2020 onward Manitoba: Apr. 28, 2020 onward Nova Scotia: Apr. 6, 2020 onward New Brunswick: Apr. 13, 2020 onward Ontario: May 14, 2020 onward Quebec: May 4, 2020 onward Saskatchewan: Apr. 3, 2020 onward	Tested for SARS-CoV-2 in the ED or within 24 hours of arrival Elective, non-ED admissions excluded Patient presenting to the ED within 14 days of a positive SARS-CoV-2 test and presenting with clinical symptoms of COVID-19.

Note: COVID-19 = coronavirus disease 2019, ED = emergency department, SARS-CoV-2 = severe acute respiratory syndrome coronavirus 2, WHO = World Health Organization.

Table 2:

Clinical screening criteria

Clinical screening criteria	Period 1	Period 2
Complaints	Fever Shortness of breath Respiratory distress Respiratory symptoms Cough Influenza or flu-like illness	Not applicable*
Discharge diagnoses	Anosmia ARDS Asthma Bronchitis Chronic obstructive lung disease Confirmed case of COVID-19 Confirmed COVID-19 Coronavirus Cough, NYD COVID COVID-19 FUO Fever, NYD Flu-like illness Influenza-like illness Pharyngitis Pneumonia Pulmonary edema/congestive heart failure Pulmonary embolism Respiratory distress Respiratory disease, NOS/NYD Sepsis, NYD SOB Sinusitis Suspected case of COVID-19 Suspected COVID-19 Upper respiratory infection Upper respiratory tract infection Viral pneumonia	Anosmia ARDS Confirmed case of COVID-19 Confirmed COVID-19 Coronavirus Cough, NYD COVID COVID-19 FUO Fever, NYD Flu-like illness Influenza-like illness Pneumonia Respiratory distress Respiratory disease, NOS Sepsis, NYD SOB Viral pneumonia

Note: ARDS = adult respiratory distress syndrome, COVID-19 = coronavirus disease 2019, FUO = fever of unknown origin, NOS = not otherwise specified, NYD = not yet diagnosed, SOB = shortness of breath.

^*In period 2, screening by chief complaint should be avoided, unless the site cannot screen charts by discharge diagnosis.

In Period 1, when testing for COVID-19 in each province was restricted to specific patient populations (e.g., health care workers, admitted patients), we included patients meeting the World Health Organization (WHO) criteria for suspected COVID-19 (i.e., fever and a respiratory symptom, such as shortness of breath) when they visited the emergency department, and those who tested positive for severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) in emergency departments.¹¹ Early case series showed that fever may be absent in many patients with COVID-19.¹² Therefore, we were liberal in our interpretation of fever, and included patients with self-reported or subjective fever. In addition, all sites screened for cases by reviewing the charts of patients with relevant discharge diagnoses or, when this was not available, presenting complaints (Table 2). Period 1 had no exclusion criteria.

Period 2 started on the date on which each province expanded testing criteria, allowing clinicians to test patients based on clinical suspicion or policy. In Period 2, we are including patients who were tested for SARS-CoV-2 in the emergency department or within 24 hours of arrival. We are also including patients presenting with a test that was confirmed positive for SARS-CoV-2 from the community or another facility, and those diagnosed with a complication related to COVID-19 (Table 2). We are excluding patients tested for SARS-CoV-2 in the context of an elective admission (e.g., planned hip revision) and those seen in the emergency department directly by another service (e.g., trauma team activation).

Data collection by site

By Sept. 21, 2020, only 4% of patients meeting inclusion criteria were testing positive for SARS-CoV-2 (Gelareh Ghaderi, CCEDRRN: unpublished data, 2020), thereby providing the registry with a high volume of controls testing negative for SARS-CoV-2. We thus redirected the network to accrue a larger sample of patients confirmed to have SARS-CoV-2 to provide us with greater power for longitudinal comparison studies, and to enable clinical decision rule development with risk stratification of patients with COVID-19.

We transitioned sites with high-volume data collection and low positivity rates for SARS-CoV-2 (< 2% test positivity) to instead collect data on consecutive COVID-19 cases only (COVID-19 data collection sites). These sites establish consecutive COVID-19 cases based on positive test results from specimens taken for the nucleic acid amplification test in the emergency department, within 24 hours of arrival, or the first 14 days of hospitalization. These sites also capture patients with clinical symptoms of COVID-19 presenting to the emergency department within 14 days of receiving test results positive for COVID-19.

At sites with adequate human resources and a higher incidence of SARS-CoV-2 (i.e., ≥ 2% test positivity rate), we continue to capture data on both patients with suspected (negative test results) and confirmed (positive test results) COVID-19 (full data collection sites).

Data sources

We developed standardized operating procedures for screening potentially eligible patients for COVID-19 and for full data collection sites, and standardized data entry and follow-up across the network.

A national research coordinator and provincial research coordinators drafted, piloted, refined and subsequently oversaw the implementation of standard operating procedures for data collection and the study conduct. The research coordinators all have extensive research experience in emergency medicine or related fields and expertise in data abstraction. The national research coordinator onboards all research assistants remotely and uses instructional videos to ensure consistency in data collection across the network. Provincial coordinators assist with data verification, quality checks and study coordination. Research assistants complete all institutional privacy training requirements.

Retrospective data source

Retrospective data is collected by research assistants, who abstract data from medical charts and enter it into a central, Web-based REDCap database (Vanderbilt University). They conduct data abstraction at 30 days after the index visit to the emergency department. This schedule captures additional emergency department hospital admissions and deaths. We keep current data dictionaries on our website (Appendix 2, available at www.cmajopen.ca/content/9/1/E261/suppl/DC1).¹⁰

Prospective data source

Physicians from emergency departments at the initial data collection sites (see Data quality section for details) completed a prospective data collection of 32 important clinical data points, enabling us to evaluate missing data and reliability of retrospective data collection (prospective data collection sheet available in Appendix 3, available at www.cmajopen.ca/content/9/1/E261/suppl/DC1).

Telephone follow-up

At 30 days after the index visit to the emergency department, we contact patients by telephone to obtain verbal consent for follow-up. We measure the Veterans Rand 12-Item Health Survey (VR12),^13–15 calculate the WHO Ordinal Outcome Scale¹⁶ and ask questions regarding culture, race, ethnicity, occupation, socioeconomics and gender, as well as self-isolation. We had developed these questions previously with input from people with lived experience with COVID-19. We repeat VR12 assessments at 60 days, 6 months and 12 months after the index visit. In January 2021, we added vaccination status to our follow-up questionnaires. We piloted the follow-up tool in British Columbia, Ontario and Nova Scotia (see follow-up data collection form at Appendix 4, available at www.cmajopen.ca/content/9/1/E261/suppl/DC1).

It is not feasible to follow up all suspected patients enrolled at full data collection sites. For every patient with COVID-19, we randomly select 4 control patients who tested negative for COVID-19 (suspected COVID-19) for every COVID-19 case from the same site and date for follow-up, as little gains in power occur for more than 4 controls per case.¹⁷ If a control patient cannot be reached, we randomly select control patients until we successfully reach 4 control patients.

National administrative data sources

The Health Data Research Network facilitated the development of a data flow of personal health identifiers and study identification numbers from each province allowing us to link registry data with national administrative data repositories (Figure 2).

Figure 2:

Data flow diagram for registry data. Note: CIHI = Canadian Institute for Health Information, COVID-19 = coronavirus disease 2019, DAD = Discharge Abstract Database, ED = emergency department, HCN = health care number, ID = identification number, NACRS = National Ambulatory Care Reporting System, UBC = University of British Columbia. *The Protocol Review and Publication Committee reviews the Registry Study manuscript proposal and recommends to the Registry Executive Committee (REC) that it is in scope. The Data Access and Management Committee reviews and recommends to the REC that all the necessary agreements and approvals are in place to access the data. The REC then approves all manuscripts and data access, including linkage, when it is required.

Data quality

We developed a data monitoring protocol to document the process of data verification and editing, and to outline targets for data completeness. We programmed internal logic and error checks in REDCap to ensure that nonsensical values could not be entered (e.g., an admission date preceding the visit to the emergency department). An analyst completes biweekly data quality checks to identify missing, incomplete and outlying data, and returns records to sites for completion and verification or editing. Missing data are measured monthly and continue to be less than 1% for highly important data points. We embedded free text fields into data collection instruments for when subjective assessments are required or no uniform data standard exists. A qualitative research assistant reviews text data iteratively to refine data fields and data dictionaries, and developed explanatory notes in REDCap to optimize data collection.

For the first months of enrolment, we measured the interrater agreement between variables collected prospectively and retrospectively on 811 cases (Table 3). We terminated prospective data collection after showing the reliability of retrospective data capture.

Table 3:

Interrater agreement between variables collected prospectively and retrospectively from 811 patients

Variable	Cohen κ coefficient (95% CI)
Living situation
Home, long-term care, homeless, other*	0.76 (0.69 to 0.84)
Symptoms
Cough	0.63 (0.57 to 0.68)
Shortness of breath	0.67 (0.61 to 0.72)
Fever	0.65 (0.60 to 0.71)
Headache	0.58 (0.51 to 0.66)
Nausea or vomiting	0.53 (0.45 to 0.61)
Diarrhea	0.63 (0.55 to 0.71)
Myalgias	0.40 (0.32 to 0.49)
Dysgeusia or anosmia	0.37 (0.11 to 0.64)
Infection risk
Travel	0.31 (0.04 to 0.58)
Institutional exposure	0.51 (0.36 to 0.66)
Health care worker	0.69 (0.59 to 0.80)
Household or caregiver contact	0.37 (0.19 to 0.56)
Other	0.24 (0.04 to 0.44)
Comorbidities
Congestive heart failure	0.71 (0.61 to 0.82)
Coronary artery disease	0.51 (0.39 to 0.62)
Hypertension	0.70 (0.64 to 0.76)
Asthma	0.80 (0.72 to 0.87)
Pulmonary fibrosis	0.39 (−0.15 to 0.94)
Chronic lung disease (not asthma or IPF)	0.72 (0.64 to 0.80)
Chronic kidney disease	0.73 (0.63 to 0.84)
Dialysis	0.58 (0.14 to 0.30)
Diabetes	0.69 (0.61 to 0.77)
Liver disease	0.43 (0.17 to 0.68)
Organ transplant	0.77 (0.51 to 1.00)
Chronic neurological disorder (not dementia)	0.17 (−0.01 to 0.34)
Dementia	0.51 (0.26 to 0.77)
Rheumatologic disorder	0.39 (0.18 to 0.60)
Active malignant neoplasm	0.55 (0.41 to 0.69)
Past malignant neoplasm	0.23 (0.07 to 0.38)
Obesity (clinical impression)	0.22 (0.05 to 0.39)
Respiratory distress
Respiratory distress	0.18 (0.12 to 0.25)
Other risk factors
Smoking (never, current, past use)*	0.73 (0.66 to 0.80)
Alcohol misuse (never, current, past use)*	0.53 (0.43 to 0.63)
Illicit substance use (never, current, past use)*	0.82 (0.75 to 0.89)

Note: CI = confidence interval, IPF = idiopathic pulmonary fibrosis.

^*Nonbinary variables with multiple categories.

Outcomes

In accordance with the WHO case definition at the time we created the registry, we define “suspected COVID-19” as a patient with fever and at least 1 symptom or sign of respiratory illness (e.g., cough, shortness of breath or flu-like illness); a patient with an epidemiologic link to COVID-19 infection, including travel to an affected area within the past 14 days, local community spread or contact with a confirmed or probable case of COVID-19; or a patient with no alternative diagnosis that fully explains the clinical presentation.¹⁸

We define “confirmed COVID-19” as any patient in whom a biological specimen tested positive for SARS-CoV-2 using the nucleic acid amplification test. The specimen had to have been drawn within 2 weeks of the emergency department visit, if the patient presented to the emergency department with a complication related to COVID-19. If the initial test result was negative, the patient had to have a specimen that tested positive within 14 days after the index visit, accounting for the longest possible incubation period.

Additional outcome variables include admission, mechanical ventilation, subsequent emergency department visits, readmissions, death, discharge from hospital, clinical recovery and quality of life. Each writing group will select the most appropriate outcome for its study question.

Data management

After assigning a unique study identifier, deidentified data are collected and stored using REDCap at the University of British Columbia. After verification, registry data are uploaded into CaraSpace, a secure, private cloud for storage and analysis of privacy-sensitive data (https://www.popdata.bc.ca/secure_data/CaraSpace). Analysts access the cloud space via an encrypted Virtual Private Network through a firewall and 2-factor authentication. This process enables remote data access and sharing without transfer of data, minimizing privacy risks.

Governance

The CCEDRRN steering committee consists of a chair, vice-chair, a site investigator from each participating site and patient partners with lived experience of COVID-19 (Figure 3). Member investigators can propose additional retrospective, prospective or follow-up data collection to answer emerging research questions. Members and external investigators can apply to CCEDRRN’s Protocol Review and Publication and Data Access and Management Committees for access to data.

Figure 3:

Governance of the Canadian COVID-19 Emergency Department Rapid Response Network. Note: CIHR = Canadian Institutes of Health Research, DAMC = Data Access and Management Committee, EC = Executive Committee, HDRN = Health Data Research Network, iKT = integrated knowledge translation, KT = knowledge translation, nPI = nominated principal investigator, PE = patient engagement, PDF = postdoctoral fellow, PRPC = Protocol Review and Publication Committee, SAC = Scientific Advisory Committee, UBC = University of British Columbia, VCHRI = Vancouver Coastal Health Research Institute.

Statistical analysis

Each manuscript writing group will develop an analytic protocol for its specific study question, which will be reviewed for appropriateness by CCEDRRN’s Protocol Review and Publication Committee. Each group will ensure that missing data and potential confounders be addressed for each study question. We use the Cohen κ statistic to measure the interrater agreement of variables collected both prospectively and retrospectively.¹⁹

Ethics approval

This protocol has been approved by the University of British Columbia Research Ethics Board (UBC REB H20-01015) and by participating sites. The protocol was approved with a waiver of informed consent for enrolment, retrospective data collection and storage of Personal Health Information for linkage with administrative data (Figure 2). We obtain verbal consent from patients at the time of first telephone follow-up to collect follow-up data at 30 and 60 days, and at 6 and 12 months.

Interpretation

Our network harmonized data collection for patients with suspected and confirmed COVID-19 to enhance research capacity in Canada. This has enabled rapid accrual of high-quality, representative data from across the country to answer priority research questions about COVID-19. As of Feb. 26, 2021, there were 59 197 suspected cases and 12 378 confirmed COVID-19 cases enrolled into the registry. We will continue enrolling cases as the pandemic evolves.

Registry data are currently being used to derive and validate clinical decision rules, evaluate diagnostic tests, determine the impact of new treatments and vaccinations and complete observational studies as the pandemic evolves. It is anticipated that the registry will enable observational cohort studies on COVID-19.

The creation of this large network was facilitated by rapid mobilization of funding through the Canadian Institutes of Health Research and other agencies, and by a commitment to an open and fair governance structure that enabled all investigators to participate in governance and authorship, guided by the International Committee of Medical Journal Editors criteria (http://www.icmje.org/). We endorsed the central tenets of the WHO Knowledge Translation and Open Science Frameworks to optimize collaboration and use of our data, including by external investigators.^20,21 This model of collaboration may be expanded to other disciplines and countries, and for other emerging health crises.

The registry continues to collect data on both suspected and confirmed cases of COVID-19, allowing us to develop and validate clinical decision rules and evaluate diagnostic tests and vaccinations over the course of the pandemic. By collecting data on patients with respiratory syndromes not caused by SARS-CoV-2, we limit attribution bias, which is a common problem in early case series.^4,7,8

Our broad inclusion criteria and the diversity of participating institutions enable us to collect data on patients commonly excluded from clinical trials, such as First Nations people, prisoners, people who are pregnant or children. Understanding COVID-19 in these vulnerable populations will be important for containing and mitigating the effects of the pandemic. ²² By collecting data on these patients, we hope to contribute to the body of evidence required to identify and address gaps in care and policy, and answer questions about COVID-19 in vulnerable patient groups who often seek care in emergency departments.^23,24

We faced substantial impediments to rapidly mobilize the network, given a nonharmonized approach to research ethics reviews, institutional differences in the interpretation of privacy laws, and a focus on protection of the individual institution. Some provinces opted out of the network or restricted the flow of health data into national repositories. These structural impediments are substantial and need to be addressed urgently through harmonization of provincial privacy law interpretation, a national ethics review process and standardized agreements for interinstitutional data sharing and transfer of funds.

Our integrated knowledge translation plan engages knowledge users and patient partners in defining, refining and prioritizing research questions and study outcomes, and in developing knowledge translation tools and strategies. Although our website (https://canadiancovid19ednetwork.org) is our central knowledge dissemination tool, we have partnered with knowledge translation specialists, including editors of open-access podcasts (e.g., EMcases, https://emergencymedicinecases.com/), infographics (e.g., CanadiEM, https://canadiem.org/) and COVID-19 town halls (hosted by the Canadian Association of Emergency Physicians) to ensure timely and broad dissemination of our research results.

Limitations

Registry data are based mainly on retrospective chart review and follow-up interviews with patients, which can be limited in quality and quantity. Follow-up interviews are also subject to recall bias. Although prospective and retrospective interrater reliability was moderate to high for most variables, some had poor agreement. We have maximized data quality through standardized procedures, data validation and data quality and logic checks. We have mitigated sources of error and bias by ensuring enrolment of consecutive cases. We anticipated difficulties in collecting follow-up data from disadvantaged populations; we mitigated this by having broad geographical coverage and linkage to national administrative databases to address gaps. Race and ethnicity could not be captured in retrospective data, and can only be collected from patients during follow-up.

Although our network does not include all Canadian provinces and territories, we have included 50 academic and nonacademic sites, including in rural and remote areas, across 8 provinces. Substantial delays in institutional reviews resulted in delays to deriving and validating clinical decision rules during the first wave of the pandemic. Fortunately, all sites were enrolling by Aug. 31, 2020, and our first research results are expected in early 2021.

Conclusion

This protocol describes the harmonized, high-quality collection of data from patients presenting to Canadian emergency departments with suspected and confirmed COVID-19 to enhance research capacity during the pandemic. This represents the latest and largest collaborative emergency medicine network in Canada. It has the potential to generate scientific evidence to inform our pandemic response, and to serve as a model for the rapid implementation of population-based data collection protocols for future public health emergencies.