COVID-19 Surveillance for Local Decision Making

Abstract

Objective

Data-informed decision making is valued among school districts, but challenges remain for local health departments to provide data, especially during a pandemic. We describe the rapid planning and deployment of a school-based COVID-19 surveillance system in a metropolitan US county.

Methods

In 2020, we used several data sources to construct disease- and school-based indicators for COVID-19 surveillance in Franklin County, an urban county in central Ohio. We collected, processed, analyzed, and visualized data in the COVID-19 Analytics and Targeted Surveillance System for Schools (CATS). CATS included web-based applications (public and secure versions), automated alerts, and weekly reports for the general public and decision makers, including school administrators, school boards, and local health departments.

Results

We deployed a pilot version of CATS in less than 2 months (August–September 2020) and added 21 school districts in central Ohio (15 in Franklin County and 6 outside the county) into CATS during the subsequent months. Public-facing web-based applications provided parents and students with local information for data-informed decision making. We created an algorithm to enable local health departments to precisely identify school districts and school buildings at high risk of an outbreak and active SARS-CoV-2 transmission in school settings.

Practice Implications

Piloting a surveillance system with diverse school districts helps scale up to other districts. Leveraging past relationships and identifying emerging partner needs were critical to rapid and sustainable collaboration. Valuing diverse skill sets is key to rapid deployment of proactive and innovative public health practices during a global pandemic.

The COVID-19 pandemic has affected all sectors of society. Schoolchildren have experienced changes in academics, social and emotional learning, and physical and mental health. ¹ Their families and guardians may have faced negative economic consequences while schools were closed (eg, need for childcare). Although some schools fully opened, for schools that were not open fully, instruction was delivered either remotely or in a hybrid learning mode (eg, alternating in-person learning for cohorts of students), which still requires additional resources for childcare. ²

To mitigate these negative effects, public health departments have implemented nonpharmaceutical interventions, such as timely identification of cases, testing, contact tracing, isolation and quarantine, social distancing, and face mask mandates. ³ In Ohio and across the nation, districts have used multiple learning modes during various phases of the pandemic to meet the needs of their families. ^4,5 Most school-reopening plans are currently based on guidance from federal, state, and local health departments and include data collection and reporting (eg, the number of COVID-19 cases among students and staff members) and protocols for social distancing and wearing face masks for in-person instruction, athletic events, and other activities. As the 2020-2021 school year comes to an end, most school staff members have been fully vaccinated, and most school districts in Ohio have fully opened for in-person learning.

Although nonpharmaceutical interventions by public health departments and school district administrators may reduce SARS-CoV-2 transmission and COVID-19 outbreaks in school settings, it is not yet known whether school-based COVID-19 surveillance can offer additional lead time to prevent outbreaks and active transmission in school settings.

We developed a tool for school-based COVID-19 surveillance. Like school-based influenza surveillance systems, ^{6 -8} the system we describe attempts to combine school data and traditional disease surveillance approaches. Our approach differs from studies on outbreak surveillance in schools, which mainly describe the epidemiological characteristics of outbreaks after they have occurred in school settings. ⁹ The Ohio Department of Health developed the Ohio Public Health Alert System (OPHAS), a statewide system that uses county-level data on disease transmission, emergency department visits, hospitalizations, and hospital capacity, to determine the risk level in each county. Although the data in OPHAS are updated weekly by the Ohio Department of Health, no information is available at the subcounty level. Therefore, OPHAS is not useful for school districts and public health departments for surveillance purposes. That said, SARS-CoV-2 is a novel virus, and little information is available on how to rapidly set up school-based surveillance systems for COVID-19. We describe the rapid planning and deployment of a school-based COVID-19 surveillance system in a metropolitan county in Ohio.

Methods

Setting

Franklin County is a metropolitan county in central Ohio with a population of approximately 1.3 million people. ¹⁰ Franklin County encompasses Columbus (the state capital) and smaller cities and suburbs that comprise the Greater Columbus area. Franklin County has 16 public school districts with a student population ranging from 1254 (Grandview Heights City School District) to 75 955 (Columbus City Schools).

Our tools and partnerships, the COVID-19 Analytics and Targeted Surveillance System for Schools (CATS), consisted of a planning phase (phase 1), which included 2 school districts and started in July 2020, and an implementation phase (phase 2), which included 12 additional school districts and started in September 2020. During phase 1, faculty and staff members from The Ohio State University (OSU) College of Public Health partnered with 2 local school districts, Columbus Public Health (CPH), and Franklin County Public Health (FCPH). We established these partnerships (1) to inform school leaders about emerging trends in COVID-19 cases in their school district, surrounding school districts, and county and (2) to inform students and parents about the rationale for changing learning strategies during the school year (eg, online, hybrid, or fully in person).

To achieve these goals, we established an operations team to manage processes for informed decision making and aligning goals, and we developed a technical team to collect, process, analyze, visualize, and interpret data.

Data Sources

We used several data sources to develop the CATS surveillance tool (Figure 1, Table 1). We extracted COVID-19 case data from the Ohio Disease Reporting System, which is used by the Ohio Department of Health to track infectious diseases for surveillance and reporting. We geocoded case data and aggregated cases using school district maps and attendance area maps for school buildings in each district. We excluded cases occurring in congregate settings (eg, nursing homes and prisons). We aggregated COVID-19 case counts and rates by multiple geographic units (eg, county, census tract, school district attendance area, school building attendance area) and demographic factors (eg, cases among school-aged people vs non–school-aged people). School-based indicators included percentage of student absences, percentage of staff absences, and number of student nurse visits for symptoms of COVID-19–like illness or influenza-like illness per 1000 students (Table 1). We obtained the list of symptoms for COVID-19–like illness and influenza-like illness from guidelines set by FCPH and the Centers for Disease Control and Prevention. ¹² Although we did not collect data on how many students were screened for in-person learning, we did ascertain from COVID-19 coordinators in several school districts that parents were allowed to call in if their student showed any symptoms of COVID-19 or COVID-19–like illness based on an algorithm provided to districts by local health departments (LHDs). We are not aware of any schools that conducted temperature checks or symptom screening after a student had arrived at school, although data on students visiting the nurse’s office were collected if a student showed symptoms while at school.

Figure 1.

Workflow for data collection, processing, analysis, and visualization in COVID-19 Analytics and Targeted Surveillance System for Schools (CATS) by programmers during the course of setting up and implementing collaboration with local health departments and school districts, Franklin County, Ohio, 2020.

Table 1.

Indicators, rationale, calculation method, data sources, and limitations used for school-based surveillance of COVID-19 for school districts with elementary, middle school, and high school buildings, Franklin County, Ohio, 2020

Indicator	Rationale	Calculation method	Data sources	Limitations	Notes
COVID-19 case count	For tracking rapid daily increases overall and by age group within a school district’s attendance boundary	Number of probable or confirmed cases of COVID-19	Ohio Disease Reporting System (ODRS) district attendance zone maps	Data lag of up to 3 d	Result date (the date a positive case is entered into ODRS) used for time series Residence addresses of cases aggregated by age group (0-5; school age groups vary by school district; 19-22, 23-29, 30-39, 40-49, 50-59, 60-69, 70-79, ≥80) Time series shown for each district attendance zone and school building attendance zone
COVID-19 case rate	For comparison of incidence across populations and measure of community spread	Numerator: number of probable or confirmed cases of COVID-19 in the previous 2 weeks. Denominator: total number of people in specific geography (eg, building attendance zone, census tract, county).	ODRS district attendance zone maps US census block groups 5-y population estimates, 2014-2018 11	Data lag of up to 3 d	Result date (the date a positive case is entered into ODRS) used for time series Converted to per 100 000 people Census blocks are not perfectly aligned with district and building attendance zones Should not be compared with rates reported by Ohio Department of Health and local health departments because of differences in the calculation of numerator and denominator and inclusion and exclusion criteria District attendance zone maps from school districts and/or validated by school districts using School Attendance Boundary Information System Time series shown for each district attendance zone
Student absences due to illness	Early indicator of increased community spread of disease among school-aged children; potential indicator of spread of disease within schools	Numerator: number of students expected to be physically in building for learning but are excused for any type of illness. a Denominator: total number of students expected to be physically in building for learning.	District student attendance system; data gathering and transfer is facilitated by META Solutions on behalf of school districts and some districts provide these data directly	Data lag of up to 3 d; some districts may not have historical data because they did not record the reason for student absences	Reported as percentage of student absences due to excused illness Updated daily or twice per week depending on school district Each district used a different student attendance software (ProgressBook, Powerschool, and Infinite Campus), which required coordinating 3 separate data ingestion workflows for data on student absences
Staff absences due to illness	Early indicator of increased community spread of disease among adults; potential indicator of spread of disease within schools	Numerator: number of staff members expected to be physically in building and interact with students (conversationally for at least a few minutes) who are excused for any type of illness. b Denominator: total number of staff members expected to be physically in building who interact with students (conversationally for at least a few minutes).	District employee/staff system; data transfer facilitated by the Education Services Center of Central Ohio c for most school districts from Frontline/Aesop	Not all districts have the same level of data specificity	Reported as percentage of staff absences due to illness Staff type and absence reasons vary by school district and are confirmed in meetings with each district prior to data analysis to ensure that the numerator is calculated properly based on each school district’s policies
Student health/nurse visits	Early indicator of increased community spread of disease among children; potential indicator of spread of disease within schools	Numerator: number of students seen by school nurse or other staff member for influenza-like illness and/or COVID-19–like symptoms. d Denominator: total number of students expected to be physically in building for learning.	District student nurse data system or META for those districts using Education Management Information System modules to enter these data	Not all districts have the same level of specificity for data, and not all districts recorded the reason for the visit in an accessible format	Number of visits per 1000 students

^aStudent’s absence may be due to the following reasons related to COVID-19: student is a suspected case; student has a confirmed case, including being out due to quarantine; or student is excused because of exposure to suspected or confirmed case at home.

^bStaff member absence may be due to the following reasons: staff member is a suspected case; staff member has a confirmed case, including being out due to quarantine; staff member is excused because of exposure to suspected or confirmed case at home; staff member is taking care of children or other family members for reasons related to the COVID-19 pandemic based on the Families First Coronavirus Response Act: Employee Paid Leave.

^cSymptoms of COVID-19–like illness include fever or chills, cough, shortness of breath or difficulty breathing, fatigue, muscle or body aches, headache, new loss of taste or smell, sore throat, congestion or runny nose, nausea or vomiting, diarrhea. Symptoms of influenza-like illness include fever or feeling feverish/chills, cough, sore throat, runny or stuffy nose, muscle or body aches, headaches, and fatigue (tiredness); some people may have vomiting and diarrhea, although these symptoms are more common in children than in adults.

^dThe Education Services Center of Central Ohio coordinates services across all school districts in Franklin County, such as substitute teachers, professional development, and research and evaluation.

During phase 1, we received daily data on each school-based indicator from the pilot school districts; these data were securely stored at the Ohio Supercomputing Center. Data on student absences contained information on demographic characteristics (age, race, ethnicity), socioeconomic status (receipt of free or reduced-price meals), and location (grade, school building, learning mode). We prepared the following materials for school districts to facilitate with initiation into CATS: (1) CATS instruction manual, which contained instructions for accessing secure dashboards, a data guide, and a tutorial on secure data; (2) an indicators document, which outlined the rationale, interpretation, data sources, assumptions, calculation details, and a description of data limitations for each indicator available in CATS; and (3) a training video for how to use the secure CATS web-based application (app) for surveillance and decision-making purposes.

We arranged for data use agreements between OSU and each partner school district and LHD. The OSU Biomedical Institutional Review Board reviewed this study and determined that it did not meet the federal definition of human subjects research.

Analysis

We used data from the 2019-2020 school year to estimate baseline values for school-based indicators. We used data from April 1 through May 31, 2020, to estimate baseline values for COVID-19 case counts, rates, and number of daily new cases for each school district, because a stay-at-home order was in place in Ohio from mid-March until May 1, 2020; therefore, the trends in daily case counts, rates, and number of new cases during this period reflected what would happen if the pandemic were under control. We calculated the expected number of cases in each school district by assuming that 500 new cases (based on trends from April through May 2020) were expected in Ohio and multiplied 500 new cases by the proportion of the school district population (ie, the number of people residing in the district boundary) compared with Ohio’s population. Baseline values for each indicator were used to identify anomalous trends based on the exponentially weighted moving average method. ¹³ This method allows for adjusted weights applied to more recent data, which was a known bias in both the disease and school data because of reporting lags. We used R version 3.6.3 (R Foundation for statistical computing) for all data processing, analysis, and visualization.

Each school district’s data on student absences, staff member absences, and visits to the school nurse underwent data processing, which involved aggregating data by school building and grade and applying inclusion criteria for each indicator based on how the district coded absences, staff member type, and visits to the school nurse. COVID-19 case data were aggregated by multiple geographies and by age range.

System Products

We developed several products for partners based on the data collected, processed, and analyzed in CATS. These deliverables met the needs of multiple groups of decision makers in CATS, including LHDs, school districts, school boards, and students and parents in the school district. First, we generated a weekly report that included the COVID-19 case rate per 100 000 population based on the previous 2 weeks of data for Franklin County and for each school district. This report was generated on a Friday and summarized by the CATS team during an all-superintendent meeting the following Tuesday. LHDs included this report on their websites.

Second, we created a public-facing dashboard for each school district participating in CATS (Figure 2). ¹⁴ We used the ShinyR package in R to create these dashboards. Each school district used a district-specific URL. We also developed a tutorial video for parents and other community members to help them interpret the data and charts in the public-facing dashboards.

Figure 2.

Screenshot of the public-facing web-based application (app) that was one of the deliverables of the COVID-19 Analytics and Targeted Surveillance System tool, Franklin County, Ohio, 2020. The screenshot is from a participating school district (Hilliard City School District). A current version of the public app is available at https://www.hilliardschools.org/20-21/covid-19-dashboard/. ^{1 4}

Third, we created a secure, password-protected, web-based app hosted on the Ohio Supercomputing Center that was accessible to approved users only. The secure app contained detailed information on each indicator at the district and building level, maps, surveillance charts, and additional analytical capabilities, such as adjusting parameters for the exponentially weighted moving average for each indicator, filtering capabilities by school buildings and age groups, and estimating the effective reproduction number for COVID-19 at the county level.

Fourth, we provided districts with an automated report that could be downloaded from the secure CATS web-based app as a table. The report listed values for each district-level indicator and a color-coded description (eg, “exceeded” in red, “warning” in orange, and “not exceeded” in green). An indicator was marked as “exceeded” if it was above a predefined threshold for the district or a school building in the district. These thresholds were determined in consultation with each school district.

We developed an algorithm called WHat Information helpS mitigate risK insidE ouR Schools (WHISKERS) to generate the indicator table. The purpose of this algorithm and the table was 2-fold. First, it gave school districts and LHDs a snapshot of districts and buildings within districts showing higher-than-baseline (anomalous) trends for multiple indicators in a single location on the dashboard. Providing this snapshot reduced the amount of time system users spent reviewing each indicator separately for each district and for buildings within districts. Second, the integration of the table in the secure app allowed users to view information on all indicators that were used by the WHISKERS algorithm. In addition, if school districts developed parameters for switching among learning modalities based on CATS data, districts could use their own thresholds for indicators to issue alerts (Table 2).

Table 2.

Generic decision tree for helping a school district make decisions about switching between learning modalities ^a

Ohio public health advisory level for Franklin County	District-specific 2-wk COVID-19 case rate per 100 000 population	District-specific school data indicator thresholds reached	Recommended district learning modality
Red	≥100	Yes	Virtual
		No	Hybrid
	<100	Yes	Virtual
		No	Hybrid
Orange	≥50	Yes	Virtual
		No	Hybrid
	<50	Yes	Hybrid
		No	In-person
Yellow	≥25	Yes	Hybrid
		No	In person
	<25	Yes	In person
		No	In person

^aThresholds for district-specific 2-week COVID-19 case rate per 100 000 population were calculated using COVID-19 Analytics and Targeted Surveillance System for Schools (CATS) data and may vary among school districts. The values listed here are for illustrative purposes only. District-specific school data indicator thresholds were also based on CATS data and may vary among school districts. School data indicators include percentage of student absences due to excused illness or COVID-19–related reasons, percentage of staff member absences due to illness or COVID-19–related reasons, and number of school nurse visits per 1000 students for symptoms of influenza-like illness or COVID-19–like illness.

Lastly, we held office hours every Friday afternoon, during which school district administrators from all participating districts could ask questions about CATS deliverables. These office hours served as a forum for reviewing data, providing community context for data trends, discussing appropriate actions to prevent disease transmission, and learning how to increase the sensitivity and specificity of our surveillance system.

Surveillance and Response

Based on the deliverables, the CATS operational team took the following steps. First, we held an initial meeting with school district staff members to provide an overview of CATS, conducted a live demonstration of CATS deliverables (eg, secure web-based app), and assessed the feasibility of providing data on school-based indicators. We also asked about contextual factors, such as social, economic, and political forces that may influence the interpretation of data, the willingness of school administrators to act based on CATS data, and the district’s goals for transparency in the decision-making process for switching among learning modalities. Second, we worked with a point of contact at the school district to set up workflows for providing data as outlined in the CATS instruction manual. Lastly, we held a second meeting with school district staff members and/or the medical advisory board (if one was established by the district for advisory purposes) and reviewed trends in multiple indicators. If asked by a district, we shared our framework for decision making about when to switch among learning modalities and what actions or responses would be taken if CATS sent an alert to the district based on the WHISKERS algorithm.

Results

In the CATS project, phase 1 began on July 1, 2020, and ended on August 31, 2020, during which 2 schools were initiated into the system. Phase 2 began on September 1, 2020, and is ongoing, as other regions in Ohio are interested in launching this system. Our results are based only on Franklin County school districts (15 of 16 school districts in the county), but as of March 2021, 21 school districts were participating in CATS. In addition, 3 LHDs outside of Ohio expressed interest in joining CATS but ultimately did not. Districts currently participating in CATS are heterogeneous in population size, demographic characteristics, and social and economic factors (Table 3).

Table 3.

Descriptive statistics for demographic, social, and economic factors among school districts in Central Ohio, 2020 ^a

District characteristic	Participating districts, mean (range) (n = 15)	Not participating districts (n = 1)
Resident population, no. b	77 794 (8222-550 146)	142 647
Student population, no. c	12 939 (1254-75 955)	26 406
Children in poverty, % b	14.1 (3.4-35.7)	21.1
Median household income, $ d ,e	78 342.53 (37 864-121 535)	55 085
Non-Hispanic White, % b	58.5 (22.2-90.1)	57.6
Non-Hispanic Black, % b	20.6 (1.0-53.0)	15.8
Hispanic b	7.9 (2.5-23.5)	17.5
Students receiving free or reduced-price meals, % f	37.3 (2.5-100.0)	57.9
Students on individual education plan, % f	14.2 (11.1-17.4)	17.6

Abbreviation: CATS, COVID-19 Analytics and Targeted Surveillance System for Schools.

^aDistricts that enrolled in CATS are not identified to maintain confidentiality.

^bData source: US Census Bureau. ¹⁵

^cData source: Ohio Department of Education. ¹⁶

^dData source: US Census Bureau. ¹⁷

^eMedian household income is less than the federal poverty threshold based on family size.

^fData source: Ohio Department of Education. ¹⁸

The web-based app gave students and parents an overview of the latest data on multiple indicators for their school district and allowed end users to interact with visualizations (eg, zooming in and out, copying pictures; Figure 2). We observed that some school districts were reluctant to publicly release their data because of local factors such as community politics, concerns about data literacy, and fear of receiving too many questions about how to interpret trends in the various indicators. One school district embedded the web-based app on its website, whereas other districts either used the app internally for decision making or took screenshots of the graphics and embedded selected graphics on their websites.

In early 2021, we made several changes to CATS. First, we partnered with a school district that was using a weekly survey on adherence to mitigation strategies. We created a web-based version of this survey, such that survey results were automatically submitted to CATS, analyzed, and then displayed along with the school indicators and COVID-19 data. We have since offered this survey to all school districts participating in CATS. Second, we completed initiation for the largest school district in Franklin County (Columbus City Schools) into CATS in February 2021. For Columbus City Schools, we made 2 modifications to the web-based app: (1) we designed a new version of the app that provided aggregated data on COVID-19 cases identified in the school rather than within the school district attendance boundary, which was done for other school districts, and (2) we added translation in 5 languages to the app to ensure equal access to surveillance data. Lastly, we conducted several ad hoc analyses to study the association between learning mode (remote, hybrid, fully in person) and overall and age-specific prevalence of COVID-19 at the school district level and the association between social distancing (3 ft vs 6 ft) and overall and age-specific prevalence of COVID-19 at the school district level.

The app provided LHDs with detailed information on school-based indicators and COVID-19 cases within school districts and building attendance areas. The combination of these data into a single app increased situational awareness at multiple subcounty levels. Although we did not formally evaluate the use of each app, anecdotal evidence suggests that school district administrators, board members, and parents found the district-level information to be more useful than county-level indicators reported by OPHAS. Six school administrators said that CATS data empowered them to be more transparent with parents, students, and staff members and to make data-informed decisions about transitioning among learning modalities (remote, hybrid, fully in person). One superintendent from the Hilliard City School District said this about the usefulness and potential of CATS:

School districts have generally engaged with our health departments for specific, infrequent issues. The partnerships we’ve developed with health professions as a result of this pandemic will serve everyone better in the future. Schools have never had these partnerships or analytics. This is a situation when good will come from crisis. We are better when we are connected.

Discussion

We learned several lessons from our development of a school-based COVID-19 surveillance system in collaboration with academic, school district, and public health partners. First, data-informed decision making is highly valued among school districts. However, LHDs that have multiple school districts may find it difficult to provide local data, especially during pandemics and other disasters. Second, piloting the surveillance system in multiple school districts with different population characteristics (eg, study poverty, family demographic characteristics, median household income) helped with expansion into additional school districts. Lastly, early alignment of values and objectives among collaborators allowed us to build value for and sustain CATS.

Data-informed decision making during pandemics and disasters can be challenging because of a lack of infrastructure to share and analyze data, limited resources to fulfill requests for local data (eg, neighborhood- or school district–specific reports) and data analysis, and lack of experience with surveillance at smaller scales than the county or city level. We leveraged the strengths of each partner to rapidly form and launch a collaborative effort. Existing relationships between academic partners (eg, OSU) and LHDs through work on an opioid epidemic surveillance project streamlined the process for quickly finalizing data use agreements. Because of these established agreements for other data-related projects, we were able to easily amend them to facilitate sharing of COVID-19 surveillance data.

Piloting the surveillance system in 2 or more school districts allowed our team to learn about the variability in data management systems for each school-based indicator, to consider various approaches for data transfer from districts’ limited information technology capabilities and resources, and to better understand the local politics of each school district’s community. Because the COVID-19 pandemic shows little signs of slowing down, we have noticed an increased desire to use local data for decision making, more requests from districts for an in-depth examination of their data, and a need for tracking trends in the proportion of cases among school-aged children compared with all cases in a school district.

The alignment of values between partners in the academic–school district–public health collaboration was critical because of legal, ethical, and regulatory ramifications of our collaboration. The academic partner (OSU) was not a public health agency and, therefore, school districts were hesitant to rely only on the advice or recommendation of the academic partner in passing school board resolutions related to school reopening and closing plans. Conversely, the LHD had the legal authority to close school buildings under Ohio law, but the risks and benefits of such a decision had to be weighed against equity considerations and potential economic effects of closing schools (eg, disproportionate effect on low-income essential workers). Therefore, a key moment during phase 1 occurred when each partner clearly specified its role. As a group, we determined that the role of OSU was to provide technical assistance for CATS and facilitate additional dialogue and relationship building between school districts and LHDS on CATS data. We determined that the role of the school district was to make decisions and take action in the school buildings based on CATS data and other data provided by LHDs. We also determined that the role of LHDs was to conduct outbreak investigations, perform contact tracing, and make recommendations to school districts based on CATS data and other available data.

Practice Implications

We demonstrated that timely, practical, data-informed decision making is possible through a school-based surveillance system during a pandemic. Planning and deploying proactive and innovative public health practices during a global pandemic is not an easy task, but it is possible with the right set of collaborators and conditions. One such condition is the willingness of collaborators to take informed risks and set aside organizational or professional goals and objectives for the sake of the greater and common good—in this case, the academic, social/emotional, and health needs of school-aged children.