Abstract
Inadequate attention to racial health equity is a common challenge to effective, reliable monitoring and mitigation of COVID-19 disparities. Efforts to monitor and mitigate COVID-19 disparities continue to be hampered by inadequacies in how surveillance systems collect, tabulate, and report COVID-19-related outcomes. We conducted environmental scans of existing public health surveillance systems and reporting standards, literature reviews, focus groups with surveillance experts, and consultations with the Centers for Disease Control and Prevention (CDC) and an expert panel on surveillance to identify and explore strengths, weaknesses, and gaps in how existing systems monitor COVID-19 and their implications for addressing disparities in related outcomes. We present recommendations based on these reviews and propose a core minimum set of health indicators and best-practice standards for reporting these indicators by COVID-19 surveillance systems to monitor racial/ethnic and other disparities in the pandemic. These recommendations are relevant to monitoring disparities in the ongoing COVID-19 pandemic and may inform monitoring of future epidemics. This discussion is part of an effort by Project REFOCUS to develop syndemic surveillance systems for monitoring the intersecting pandemics of COVID-19 and racism.
Keywords: COVID-19, Health Equity, Surveillance Systems, Health Disparities
Introduction
Persistent racial/ethnic inequities in US COVID-19 diagnoses, hospitalizations, deaths, and other outcomes underscore the critical need to improve the capacity of public health surveillance systems to detect risks early and inform intervention efforts. Surveillance systems vary in the COVID-19 indicators they include, the metrics used to assess each indicator, and the extent to which the data generated by these systems support the identification and mitigation of disparities.1 Many of the existing concerns stem from inconsistencies and inadequacies in both core data sources used for surveillance and processes used for data generation and exchange so that information across multiple systems can be compiled and integrated into reporting systems.
Efforts to monitor and mitigate disparities have been hampered by insufficient, inconsistent, and often incomplete information on COVID-19 outcomes by race/ethnicity and variability in how different surveillance systems collect and report on similar sociodemographic factors and outcomes. In general, these systems report trends in COVID-19 outcomes for the overall population, but provide little-to-no consistent information stratified by key sociodemographic characteristics. This obscures both the disproportionate impacts among socially marginalized groups and the types of interventions that are needed to address them.
Inadequate attention to racial health equity is a common challenge to reliable and effective monitoring and mitigation of disparities. Focused efforts to increase the comprehensiveness of data collection for COVID-19, to resolve upstream issues that create inadequacies in compiling data from multiple sources, and to develop best practices for data reporting will improve identification of COVID-19 disparities. Technological and communication advances have been brought about by the need for rapid reporting of COVID-19 trends. Critically evaluating and refining the systems that generate this information, compile, and report it can improve routine surveillance for COVID-19 as well other conditions, including morbidity and mortality associated with future disasters, thereby aiding policy makers, providers, and affected communities in addressing the inequities that lead to observed disparities.
Braveman defines health equity as social justice in health (ie, no one is denied the possibility to be healthy for belonging to a group that has historically been economically/socially disadvantaged) and health disparities as the indicators used to measure progress toward achieving health equity.2 Pandemic mitigation efforts that are based on an equity framework seek not only to track overall patterns but also to reduce disparities in new infections and related health outcomes. These efforts necessitate surveillance systems that: identify disparities in key disease indicators across diverse populations; indicate the populations among whom current burdens are greatest; identify the populations who are likely to experience future disparities; and assess the extent to which mitigation efforts are having the intended impacts.
To improve the reliability, validity, and utility with which health departments and other organizations monitor and report disparities, this article proposes a core minimum set of health indicators and best-practice standards in COVID-19 surveillance systems for monitoring the current pandemic and future ones. The standards that currently determine processes of data generation, integration, and exchange are externally developed, govern surveillance practices across many jurisdictions, and challenge granular monitoring of health disparities. Core data sources such as electronic laboratory reporting (ELR), vital records, and registries (eg, immunization and treatment) needed to populate reporting systems may not all follow the same standards. Differences in these standards and their lack of requirements for specific information limit what can be tabulated and reported by public health surveillance systems. Additionally, established standards may be adhered to more consistently for periodic population-based surveys, vital records, and registries compared with disease notification systems.3 Interoperability across many of the core data sources is also a challenge due to a lack of widespread adaptation of the standards needed to allow for optimal data integration across data sources.4
Although COVID-19 serves as a catalyst for these recommendations, they could be adopted to address other conditions, as well. We discuss the importance of each indicator for quantifying health inequities and highlight challenges and data gaps in existing systems that may hamper the achievement of these goals. The COVID-19 pandemic has also contributed to other negative health outcomes that warrant monitoring from a health equity standpoint, such as delays in care seeking or diagnosis due to persons being turned away from emergency health care settings for both COVID-19 and other emergent health issues; however, these are beyond the scope of this research effort.
Methods
The findings and recommendations reported here stem from a larger initiative that draws on existing surveillance approaches to inform the development of a novel, syndemic surveillance system to monitor the intersecting COVID-19 and racism pandemics. The methods, which are based on Public Health Critical Race Praxis (PHCRP), have been described elsewhere.1 Briefly, we conducted environmental scans of existing public health surveillance systems and reporting standards, literature reviews, focus groups with surveillance experts, and consultations with the Centers for Disease Control and Prevention (CDC) and an expert panel on disease surveillance to identify and explore strengths, weaknesses, and gaps in how existing systems monitor COVID-19 disparities. We examined the COVID-19 dashboards developed by public health agencies that have the primary responsibility of monitoring major threats to the health of populations and by other entities (eg, academic institutions, media) engaged in COVID surveillance by December 2021.
Indicators of COVID-19-Related Outcomes
This section outlines recommendations for improving the capacity of public health surveillance systems to contribute useful, timely data to policy makers and COVID-19 equity initiatives.
Monitoring COVID-19 Outcomes
To improve the utility, flexibility, and sustainability of surveillance systems to detect disproportionate disease burdens or risk early and monitor them across diverse populations, the data collection standards, data generation, and exchange processes should allow for reporting on a wide range of relevant indicators and stratification by key sociodemographic indicators. The information that public health officials need to respond to disparities changes over the course of a pandemic; therefore, equity-focused surveillance systems should be able to provide whatever data are most useful at different phases of the pandemic. At a minimum, a health-equity-focused COVID-19 surveillance system should include COVID-19 indicators that describe the epidemiology of the pandemic, which include rates of testing, diagnoses, hospitalizations, vaccinations, and deaths, and the implications of these data for specific populations and settings. They should also characterize the availability of key health care resources (eg, hospital capacity), the utilization of those resources (eg, vaccine uptake), and the impact of mitigation efforts (eg, contact tracing).5 Table 1 lists metrics recommended for capturing COVID-19 morbidity and mortality and the reach and adequacy of public health and health care resources. It also describes the types of health disparities each metric can help to identify.
Table 1. Public health significance, metrics, and timing of key COVID-19 pandemic indicators of morbidity, mortality, and pandemic stage.
Indicators & public health significance | Metrics | Timing | ||
Cumulative | Prior Month | 7-Day Avg. | ||
Testing/screening: Measures prevalence among testers, levels of testing uptake, & impact of mitigation efforts; helps account for impact of differences in levels of screening over time & between groups/areas on observed numbers & rates of newly diagnosed cases. | Positivity rate (pos. tests/tot. tests) | - | X | X |
# tests completed | - | X | X | |
# unique persons tested | - | X | X | |
Testing coverage (# unique persons tested/tot. population) | - | X | X | |
Cases: Measures burden, potential need for hospital beds, staffing & the impact of mitigation efforts based on diagnoses of COVID-19 disease. | # persons newly diagnosed | X | X | X |
Diagnosis rate (# new diagnoses/tot. population) | X | X | X | |
Hospitalizationsa: Measures burden, negative COVID-19 outcomes & potential disparities related to poor access to care, pre-existing conditions & environmental factors that may exacerbate COVID-19 disease; may indicate temporal changes in disease severity; forecasts mortality rates. | # COVID-19 patients newly admitted (% of all hospitalized COVID-19 patients) | - | X | X |
# COVID-19 patients hospitalized (total) | X | X | X | |
# COVID-19 patients in ICU (total) | X | - | X | |
Rates (# hospitalized/tot. population) | X | X | X | |
Deathsb: Measures worst COVID-19 outcome & potential disparities related to poor access to care, pre-existing conditions, structural determinants & environmental factors that may exacerbate COVID-19 disease; may indicate temporal changes in disease severity. | # deaths | X | X | X |
mortality rates (# COVID-19 deaths/tot. population) | X | X | X | |
case fatality ratesb (# COVID-19 deaths/# COVID-19 cases)b | X | X | X | |
Vaccination coverage & effectiveness: Measures uptake; helps identify populations & areas that experience barriers to uptake; indicates vaccine effectiveness at preventing hospitalization & death; indicates populations with high levels of protection or risk for most adverse COVID-19 outcomes. | # of doses delivered (total) | X | X | X |
#/% of population received ≥ 1 dose | X | X | X | |
#/% of population received 2 doses | X | X | X | |
#/% of population received 3 doses | X | X | X | |
# vaccines administered | X | X | X | |
#/% hospitalized by vaccination status | - | X | X | |
#/% deaths by vaccination status#/% hospitalized by vaccination status | - | X | X | |
#/% of contacts made with new cases#/% of contacts made with new cases | - | X | X | |
Contact tracing: Measures coverage & process; identifies potential disparities in coverage & in participation with contact tracing; can inform outbreak investigations & facilitate early detection of outbreaks. | #/% of new cases willing to share information | - | X | X |
Avg. # of contacts generated per case | - | X | X | |
#/% of contacts made with identified contacts | - | X | X | |
#/% of identified contacts tested for COVID-19 | - | X | X | |
#/% of identified contacts sharing their contact information | - | X | X | |
a. All hospitalization, death, & case-fatality rates should be adjusted for group differences in age distributions. Because of the sometimes-substantial differences in age distributions across racial groups, not controlling for age can grossly underestimate disparities in COVID-19 outcomes.
b. Deaths based on current month; cases based on prior month based on average time from diagnosis to death.
Data on COVID-19 among hospitalized patients has nuances and implications that are worthy of note. COVID-19 prevalence data among hospitalized patients, where it is available based on routine testing for SARS-CoV-2, provides a measure of disease risk in the community at-large as well as among other patients and staff within the hospital. Prevalence among hospitalized patients also depends on the underlying likelihood of hospitalization, which shifts during severe lockdowns and periods when hospitals experience bed and staffing shortages leading them to minimize admissions and postpone elective surgeries and procedures.
Ideally, surveillance systems should be able to distinguish patients who are hospitalized primarily for COVID-19 from those who, despite having COVID-19, were hospitalized primarily for some other reason. By linking data from vaccine, laboratory, and health care data systems together, the New York State Department of Health was able to estimate rates of COVID-19 infection and hospitalization by vaccine status. Between May and July 2021, an estimated 42.9% of hospitalized vaccinated adults with COVID-19 and 58.1% of hospitalized unvaccinated adults with COVID-19 were admitted for COVID-19-related symptoms.6 Rates of hospitalization for COVID-19 indicate both the burden of severe disease on the community and on the health system itself. Information on the rates at which vaccinated people with COVID-19 are hospitalized primarily for this condition can be used to help assess vaccine effectiveness, waning immunity, and the implications of newly emerging SARS-CoV-2 variants that may be more or less virulent than prior variants.6 By monitoring fatality rates among those hospitalized for COVID-19 over time, hospitals also can assess whether their approaches to care are improving rates of disease survival. While published studies examining racial/ethnic differences in hospitalization for, vs with, COVID-19 have not yet been published, an examination of excess deaths not assigned to COVID-19 across US counties did find differences by racial/ethnic composition.7
Most surveillance systems monitor and report metrics, which are the numerators of COVID-19 outcomes (eg, the number of new cases); however, denominators (eg, the populations at risk), proportions (eg, % of hospital beds currently available), and rates (ie, the number of new cases over a specified period out of the total population at risk) are less frequently reported, which can obscure the existence of racial/ethnic disparities or their magnitude. Population denominator information may not be readily available in the health or health care sources from which the numerator data are obtained; however, it can be estimated using the sources below:
- NCHS Bridged Race Files.8 National Center for Health Statistics. Vintage 2019 post-census estimates of the resident population of the United States (April 1, 2010, July 1, 2010-July 1, 2019), by year, county, single-year of age (0, 1, 2, ..., 85 years and over), bridged race, Hispanic origin, and sex. Prepared under a collaborative arrangement with the US Census Bureau.
- The American Community Survey,9 US Census Bureau. Data Profiles have the most frequently requested social, economic, housing, and demographic data in separate data profiles. Each profile summarizes the data for a single geographic area, both numbers and percentage with the information provided annually from 2014 – 2019. The year 2020 data are scheduled for release in March 2022.
These sources can also be used for age adjustment of COVID-19 rates. Age adjustment is particularly important in identifying health disparities, because age distributions vary across groups. For example, some immigrant populations are younger on average than native-born populations. Without age adjustment, the rates of hospitalization and death will not show the true impact of the disease on these populations, and their rates will not be comparable to populations with older age distributions.
Monitoring Vaccine Uptake and Effectiveness
As new vaccines become available, it is critical that vaccine uptake be monitored to identify and address racial/ethnic disparities in distribution, access, and inoculations. Early in the US effort to distribute vaccines, the vaccines were made available via phased rollouts to priority populations as determined at the state and local levels. Evaluation of vaccine uptake in this context requires jurisdictions to indicate their vaccine priority allocations in their reporting, as well as the target vs actual dates of the activities of each phase (and subphase). The recommendations (Table 1) include stratification by vaccine type – information that becomes particularly relevant as differences in the rates of waning immunity by vaccine type emerge and booster shots for specific regimens become available. The recommendations also include disaggregating rates of hospitalization and death by vaccination status to monitor vaccine impact by race/ethnicity and other sociodemographic characteristics. Vaccination reduces hospitalization and death effectively; therefore, monitoring this indicator allows information on vaccine effectiveness across all groups, and it detects disparities in waning immunity. In addition, reporting uptake by neighborhood or other geographic divisions may facilitate earlier detection of disparities in vaccine availability and ease of access.
Monitoring Contact Tracing
Monitoring and reporting of contact tracing efforts should include stratification by race/ethnicity of cases from which contacts are solicited. The stratified data are necessary to evaluate the success of these efforts, determine if effectiveness varies across groups, and generate evidence regarding specific populations that are over or under-targeted. Multiple contact tracing processes may exist in a given area (eg, efforts conducted by health department, mobile applications, and educational or health care institutions); therefore, surveillance systems should indicate which data are and are not captured in their reporting of COVID-19 contact tracing efforts. Mistrust continues to hamper responses to the pandemic, heightening the importance of assuring confidentiality during contact tracing. When public health agencies hire contractors to conduct contact tracing, testing, or vaccination, the data should be protected. It should not be permitted to be commercialized or utilized for any purpose other than the originally intended public health efforts. Elsewhere we discuss the potential for harm to communities through criminalization or disruption of services such as through data sharing with immigration and law enforcement.1 Given how contact-tracing data are generated locally, there may be fewer barriers to the implementation of these recommendations by local public health systems.
Monitoring Health Care Resource Availability and Utilization
Racial/ethnic inequities in health care access are well-documented, with areas of majority Black populations far more likely than majority White population areas to have primary care provider shortages, trauma deserts and segregation for both Black and Hispanic rural populations – all factors associated with lower access to a usual source of health care.10–12 During periods of COVID-19 surge, the waning capacity of hospitals, intensive care units (ICUs), and equipment (eg, ventilators) becomes a critical signal of potential increases in case fatalities. Hospital capacity may affect diverse groups differentially due to underlying structural determinants of health (eg, poverty, segregation) that in turn influence distance to facilities, transportation access, and health insurance coverage. These factors contribute to disparities in COVID-19 death rates by the median income of the ZIP code where people live.13 Mapping ICU bed availability or tracking it by subregions of a jurisdiction can help detect these disparities earlier and guide mitigation efforts in real-time. Though ICU bed availability is currently estimated based on adults, if the severity of COVID-19 in young people worsens as new mutations arise, it will be necessary to track the availability of ICU beds for both adults and children.
Key Characteristics and Indicators by Which to Stratify Data
Few COVID-19 surveillance systems routinely present COVID-19-related information by race/ethnicity making it difficult to monitor disparities. We recommend collecting and reporting data on key sociodemographic characteristics whenever possible, because populations marginalized due to racism and other social forces (eg, homophobia) may be uniquely or disproportionately impacted during public health crises. As of December 2021, key sociodemographic characteristics by which to stratify COVID-related data include age category, race/ethnicity, and gender (Table 2). CDC and others have also recommended collecting and reporting data on educational attainment and sexual orientation as part of routine COVID-19 data collection.14,15 Just five US states and the District of Columbia are currently collecting or attempting to collect COVID-19 data stratified by sexual orientation and gender identity (SOGI).16 As of December 2021, only one state, California, reports these data publicly.
Table 2. Public health significance, metrics, and timing of key COVID-19 pandemic indicators of health care resource availability and utilization.
Indicators & public health significance | Metrics | Timing | |
Daily | 7 Day Avg. | ||
Available adulta hospital beds: Measures capacity to care for patients with COVID-19, other conditions requiring hospitalization, & those with injuries or emergent conditions; indicates the current number of physical, staffed adult hospital beds. | # available adult hospital beds | X | X |
% of total capacity (# available beds/tot. # of adult beds) | X | X | |
% change from prior week | X | X | |
Available adulta ICU beds: Measures the capacity to care adequately for the sickest COVID-19 patients & others in need of intensive care; indicates the current number of physical, staffed adult intensive care beds, excluding surge, PICU, & NICU beds. | # available adult ICU beds | X | X |
% of total capacity (# available beds/tot. # of adult ICU beds) | X | X | |
% change from prior week | X | X | |
Available ventilators: Measures the overall capacity to adequately care for the sickest COVID-19 patients; based on routinely used ventilators, excluding anesthesia machines, portable/transport ventilators, BiPAP machines, & ventilators used exclusively for NICU or PICU patients. | # ventilators | X | X |
% of total ventilator capacity (# available ventilators/tot. # of ventilators) | X | X | |
% change from prior week | X | X |
a. Focus on adult resources may expand to younger patients if the serious impacts on this group increase with subsequent COVID-19 waves
Information on SOGI and educational attainment can inform outreach to populations for whom data are not routinely collected, despite their experiences of social stigma, health disparities, and low levels of health literacy. As Cahill has described, states that have attempted to collect SOGI information have encountered several implementation challenges, including a lack of political will.16 Stigmatization of sexual and gender minorities is also a concern. In interviews we conducted with COVID-19 contact tracers, several reported that when they asked persons who had tested positive for COVID-19 a standard questionnaire item about their sexual orientation, the question was often met with suspicion. This made it difficult for the contact tracer to elicit additional information on potential contacts. As recommendations for routine collection of SOGI data in a wide variety of settings are more widely implemented, concerns about the reasons for such questions will diminish.
Sociodemographics
Table 3 outlines key sociodemographic factors for COVID-19 surveillance systems to include and the categories by which to stratify them when monitoring specific outcomes (ie, cases, outbreaks, vaccination uptake, and contract tracing process/outcome measures, etc.). In general, the categories are similar across outcomes; however, several distinctions are notable. The recommendations for age group rely on what is known about age-related differences in risk for more adverse COVID-19 outcomes due to biological (eg, more co-morbidities with age) or social (eg, greater likelihood of residence in congregate housing with age) factors. Recommended vaccine uptake indicators account for the age groups for which vaccines were originally and subsequently approved. They also account for the age categories that are most likely to congregate in K-12 or university settings or workplaces.
Table 3. Key strata by which to collect and report COVID-19-related metrics.
COVID-19 Epidemiology | Contact Tracing | Vaccine Uptake | Outbreak Investigations | |
Sociodemographic Factors | Settingsa | |||
Age group, years | <18 | <5 | Occupational | |
18-29 | 18-29 | 5-11 | ||
30-39 | 30-49 | 12-17 | Correctionalb | |
40-49 | 18-49 | |||
50-65 | 50-65 | 50-64 | Educationalb | |
65-79 | 65+ | 65+ | ||
80+ | Nursing homeb | |||
Gender/sex | Male | Male | Male | Other health care settingsb |
Female | Female | Female | ||
Transgender Male/ Female-to-Male | Transgender Male/ Female-to-Male | Transgender Male/ Female-to-Male | ||
Transgender Female/ Male-to-Female | Transgender Female/ Male-to-Female | Transgender Female/ Male-to-Female | Other congregate living (eg, homeless shelters, drug treatment centers)b | |
Other, please specify | Other, please specify | Other, please specify | ||
Chose not to disclose | Chose not to disclose | Chose not to disclose | ||
Race/ ethnicity | Hispanic, any race | Hispanic, any race | Hispanic, any race | Information on race and preferred language should be included for large outbreaks & for summaries of data collected across setting types |
American Indian/ Alaska Native, non-Hispanic | American Indian/ Alaska Native, non-Hispanic | American Indian/ Alaska Native, non-Hispanic | ||
Black, non-Hispanic | Black, non-Hispanic | Black, non-Hispanic | ||
Asian, non-Hispanic | Asian, non-Hispanic | Asian, non-Hispanic | ||
Pacific Islander/ Native Hawaiian, non-Hispanic | Pacific Islander/ Native Hawaiian, non-Hispanic | Pacific Islander/ Native Hawaiian, non-Hispanic | ||
White, non-Hispanic | White, non-Hispanic | White, non-Hispanic | ||
Other, please specify | Other, please specify | Other, please specify | ||
Preferred language | - | English | - | |
- | Non-English, specify (translator available) | - | ||
- | Non-English, specify (translator not available) | - | ||
Sexual orientation | Lesbian/Gay/Homosexual | N/A | Lesbian/Gay/Homosexual | |
Heterosexual/Straight | Heterosexual/Straight | |||
Bisexual | Bisexual | |||
Something else | Something else | |||
Don’t know | Don’t Know | |||
Choose not to disclose | Choose not to disclose |
a. Outbreak investigations should be conducted in these settings; Include information on setting type, institutional name, & ZIP code or ZIP Code Tabulation Area (ZCTA)
b. Stratified by students vs. staff members; patients/clients/residents vs staff
The recommended racial/ethnic reporting categories are consistent with guidelines established by the US Office of Management and Budget’s (OMB’s) Standards for the Classification of Federal Data on Race and Ethnicity (Statistical Policy Directive No. 15) and with the standards for laboratory reporting outlined in CARES Act Section 18115 (hhs.gov), which require the use of a minimum of five racial categories and a separate question assessing whether respondents are of Hispanic ethnicity (Table 3).17,18 As discussed below, data from these two questions can then be combined into at least six race/ethnicity categories: (Hispanic/Latino and non-Hispanic/Latino: White, Black or African American, Asian, Native Hawaiian or other Pacific Islander, Native American or Alaska Native).19 The options for people to report multiple races yields more than 60 possible combinations of responses. Consistent with OMB, we recommend that non-Hispanic/non-Latino persons reporting more than one racial category be included in a seventh category, “two or more races,” but that Hispanic/Latino persons who report more than one racial category be categorized as Hispanic/Latino. This population continues to be disproportionately impacted by the pandemic; therefore, this inclusive strategy may enhance the ability to detect disparities affecting them. Several other approaches to categorizing people who report multiple races have been proposed and may be considered.20 Of necessity, different approaches may be used in different places to address local needs; however, the specific approach used should be explained clearly in any reporting. In addition to addressing local information needs, this will have the added benefit of improving the validity of comparisons made between different communities or over time.
Because the OMB race categories do not always resonate with individuals, the data are frequently missing in surveys, especially for Hispanics/Latinos. In the method currently used by CDC, persons indicating Hispanic ethnicity are grouped as Hispanic/Latino regardless of race, while persons indicating they are not of Hispanic ethnicity and reporting a race, are grouped as their selected race category, non-Hispanic, and those reporting their race but not their ethnicity are grouped as unknown, essentially excluding those with missing/unknown ethnicity even if they report a race. This approach can lead to uneven levels of missing data across race/ethnicities and substantial information loss.21 CDC recently compared two alternate approaches for addressing those with missing data on ethnicity using data from people who selected one race.21 Using the first approach, persons with missing/unknown ethnicity, but available data on race were coded as their selected race category, non-Hispanic, leaving fewer people uncategorized. Using the second approach, in addition to treating those with missing/unknown ethnicity in this manner, persons who reported both Hispanic ethnicity and a race were assigned to both groups (ie, both to the Hispanic ethnicity and to the racial category they specified). In other words, the same person could be categorized in both the Hispanic and race categories, as the latter are not limited to non-Hispanic individuals.21 All three methods exclude those with missing/unknown information on both ethnicity and race. Compared with the method currently in use, both of CDC’s alternative approaches produced substantially higher estimated numbers and rates of COVID-19 cases and persons fully vaccinated. A larger increase was observed with the second approach due to the overlapping categories. With both methods, the largest relative increases were observed for the number of COVID-19 cases among American Indian/Alaska Native people and the number of people fully vaccinated among Native Hawaiian/Pacific Islander people. This is a critical advantage given that data from these groups are often suppressed or distorted due to their small numbers.21 We recommend the first of these two alternative approaches because it increases the amount of information available without creating partially overlapping categories.
Although small population sizes may make it difficult to protect confidentiality for reporting on certain groups, most notably American Indians and Alaska Natives, people from the Pacific Islands and some Asian populations, the complete data may nevertheless be needed to guide health promotion and disease control efforts among them. One strategy for addressing the sparse data limitation is to report information for the subgroups over longer time intervals (eg, every 30 vs every 7 days). There are confidentiality rules that limit data reporting to national laboratories and vaccine surveillance systems; therefore, these data may need to be stripped of identifiers before aggregating them in order to protect confidentiality, even if aggregation across multiple jurisdictions or periods could protect confidentiality. Finally, groups that are not well represented in an area and non-Black racial/ethnic minorities are particularly vulnerable to racial/ethnic misclassification. Thus, it is important to underscore the need to train data collectors, educate the public regarding the routine collection and uses of these data, and use well-designed data collection forms that reflect the ways members of diverse racial/ethnic groups identify themselves.22
In addition to monitoring disparities based on race/ethnicity and age, we recommend that data also be stratified to monitor disparities based on sexual orientation and gender identity (SOGI) where feasible. Lesbian, gay, bisexual, transgender, queer, intersex, asexual, and allied (LGBTQIA) advocates have urged the inclusion of SOGI data in monitoring systems for years. Federal guidance and uniform standards on the labels to use for each category is urgently needed to ensure consistency across sources of these data and to reduce the use of categories and labels that stigmatize or alienate members of sexual and gender minority groups. Advocates for these groups have developed evidence-informed suggestions that should be considered in developing the categories and labels used within data collection instruments.23,24 For example, Electronic Library Reporting (ELR) follows Health Level Seven International (HL7) 2.5.125 and while this standard does include racial categories, it does not include any fields for sexual orientation. Additionally, the recommended values for gender (including categories for transgender and gender non-binary individuals) are not included.25 Not establishing this standard, and instituting the proper training to be able to apply it, can result in non-uniform coding and renders key populations and disparities experienced by them, invisible.
For other important factors such as race/ethnicity, unavailable and missing data are also a challenge.26 Missing data can be attributed to many reasons including issues of data collection and reporting.4 For complex systems, limitations in data exchange also can occur at multiple points of the process that involve both data acquisition and integration.27 Additionally, missing data can be attributed to varying reporting priorities of the states.28 For example, citing patient privacy laws, some states, such as California and Texas, have declined to share with the federal government the race/ethnicity information in their data on vaccine uptake. Other states, such as Idaho, have not collected race/ethnicity data at all or are not prioritizing this data collection, such as Connecticut, at the time of vaccination.28
Monitoring Place-Based COVID-19 Vulnerability
Place factors reflect not only certain social characteristics (eg, social cohesion) of the populations residing in a neighborhood, city or other geography, they also reflect how characteristics of the area itself (eg, its policies, the density of health care providers, etc.) may affect COVID-19 outcomes directly (eg, hazardous waste sites contributing to poor air quality) or indirectly (eg, segregation that contributes to mistrust of the healthcare sector may lead to delays in accessing care or reflect racial differences in the treatment of patients, both of which may also contribute to disparities). In short, place characteristics render some communities more susceptible to COVID-19 than others. To capture this, University of California, Los Angeles (UCLA) researchers and others have developed COVID-19 vulnerability indices. The UCLA index uses ZCTAs (ZIP Code Tabulation Area) as the geographic units of analysis and includes four indicators of medical vulnerability to COVID-19:29
- Preexisting Health Vulnerability
- Barriers to Accessing Services
- Built Environment Risk, and
- Social Vulnerability Index (a close replica of the 2018 CDC SVI).
The data sources for the indicators are the American Community Survey (ACS), California Health Interview Survey (CHIS), and California Department of Health and Recreation. The Pre-existing Health Vulnerability indicator is based on the CHIS, a population-based study of California with a similar design to NHIS, including ZCTA-level data on preexisting health status. Similar resources are not available in many states; however, this index is highly correlated with the Barriers to Accessing Services and the Social Vulnerability indices. Hence, in the absence of a data source like CHIS, states can approximate the COVID-19 vulnerability index using the other three indicators listed above.
Another index of vulnerability has been developed by Surgo Ventures Foundation, a privately funded “action tank” that employs diverse teams of data and behavioral scientists, as well as scientists from other disciplines to compile and analyze data from multiple sources to describe health problems and help to identify scalable solutions.30 Their COVID-19 Community Vulnerability Index (CCVI) uses data from an even wider range of sources involving seven different themes to quantify vulnerability down to the census track. It was designed to predict vulnerability to COVID risk and related outcomes and to help elucidate the reasons for disparities in these negative outcomes. The communities with high vulnerability scores on the CCVI have been shown to have higher rates of COVID-19 infection and death and lower rates of vaccination and testing compared with communities with low vulnerability scores. Differences in the distributions of CCVIs by race are stark, with Black Americans nearly twice as likely as White Americans to live in census tracks with high CCVIs (60% vs 34%).31 Systems that allow for rapid integration of reported data and indices like the CCVI will enable their use to increase understanding of the spatial distribution of other reportable conditions.31 Although vulnerability indices are susceptible to ecologic and other types of biases, these place-based indicators can provide a more nuanced view of health disparities, one with greater explanatory power than the limited individual-level sociodemographic information routinely collected for tracking reportable diseases and conditions like overdoses.
Model Example
In the early months of the COVID-19 pandemic in Alleghany County, PA, evidence of racial disparities in COVID outcomes led the Black Equity Coalition (BEC) to partner with the RAND Corporation, and Surgo Ventures to explore the state of COVID-19 testing, cases, and deaths in the county.32,33 This effort illustrates ways in which implementation of several of the recommendations discussed here led to changes in the community’s COVID-19 response.34 The tool demonstrated higher test positivity rates and rates of COVID-19 hospitalizations among Black and Asian people compared with White people over the course of the pandemic, and a closing gap in mortality rates between Blacks and Whites over time. Their visualizations of weekly case distribution helped describe geographic trends and emerging hotspots. The tool mapped the CCVI onto distributions of cases, test sites, and negative COVID-19 outcomes, and used this information to ensure that sufficient test sites were later made available in areas with high levels of vulnerability. The data sources they used were:
- Allegheny County Health Department, “Coronavirus: COVID-19 Testing—Information on COVID-19 Testing,”https://www.alleghenycounty.us/Health-Department/Resources/COVID-19/Information-on-COVID-19-Testing.aspx
- GISCorps, “GISCorps COVID-19 Resources: Apps and Data Created by GISCorps Volunteers,” webpage, undated-a. As of April 26, 2021: https://covid-19-giscorps.hub.arcgis.com/
- GIS Corps, “Locate COVID-19 Testing Sites,” data set, undated-b. As of April 26, 2021: https://www.giscorps.org/covid-19-testing-site-locator/
- Google Maps, “Web Services: Distance Matrix API,” webpage, updated April 20, 2021. As of April 26, 2021: https://developers.google.com/maps/documentation/distance-matrix/overview
- Kohler, Jared, and the BEC Data Working Group, “Weekly COVID-19 Case Positivity and Cases per 100K Population,” Tableau data, November 11, 2020. As of January 6, 2021: https://public.tableau.com/profile/jared6542#!/vizhome/AlleghenyCountyCovidWeeklyCases/WeeklyCOVID-19CasePositivity
- MapQuest, “Directions API,” webpage, undated. As of April 26, 2021: https://developer.mapquest.com/documentation/directions-api/
- SafeGraph, “Social Distancing Metrics,” webpage, 2021. As of January 22, 2021: https://docs.safegraph.com/docs/social-distancing-metrics
- Surgo Ventures, “The U.S. COVID Community Vulnerability Index (CCVI),” webpage, undated-b. As of January 6, 2021: https://precisionforcovid.org/ccvi
- U.S. Census Bureau, Explore Census Data (Both the decennial census and American Community Surveys)
- Western Pennsylvania Regional Data Center, Datasets - WPRDC https://data.wprdc.org/dataset
Monitoring Outbreaks in Key Settings
COVID-19 outbreaks may generate new disparities or exacerbate existing ones. Throughout the pandemic, the largest outbreaks have impacted populations made vulnerable because of issues related to race (eg, prisons and jails), immigration status (eg, detention facilities and industries primarily employing people who are not citizens), stigmatized conditions (eg, residential substance abuse treatment facilities), and advanced age (eg, nursing facilities). Per CDC, “Definitions for COVID-19 outbreaks are relative to the local context. A recommended definition is a situation that is consistent with either of two sets of criteria:”
“During (and because of) a case investigation and contact tracing, two or more contacts are identified as having active COVID-19.”
Or
“Two or more patients with COVID-19 are discovered to be linked, and the linkage is established outside of a case investigation and contact tracing (eg, two patients who received a diagnosis of COVID-19 are found to work in the same office, and only one or neither of them was listed as a contact to the other).”
The types of settings in which COVID-19 outbreaks should be monitored and reported are listed in Table 3. In monitoring these settings for outbreaks, existing data systems should track at least five indicators: the confirmed number of linked cases that have been identified; the number of suspected cases linked to the outbreak that have been identified; the number of people tested; the approximate date of the initial case; and, where possible, the estimated total number of employees, residents, or students in the setting and the number of outbreak-associated deaths. This level of detail may not be possible for every outbreak due to limitations brought about by underfunding and staff shortages or small numbers making it difficult to disclose information without violating individuals’ confidentiality rights. Under those circumstances, we recommend summarizing the data across setting types (eg, specific employment industries) to identify disparate impacts for workers, students, and residents in different types of occupational, educational, and residential settings. It is particularly important to include data on race/ethnicity given the disproportionate share of people of color who are designated “essential workers” during the pandemic and the clustering of specific racial/ethnic groups in these occupations (eg, nursing, custodial services, meatpackers). CDC provides definitions of each setting type and additional guidance on the criteria used to declare an outbreak in specific types of settings.35
Discussion
The COVID-19 pandemic has ushered in a new era of public health data gathering, reporting, and information sharing. In no other epidemic to date has the US public health community generated in real-time the amount and types of information about a health condition that is currently available to the public via local, state, and national dashboards. Both the gravity of the pandemic and technological advances in collecting, mining, and sharing information have made this possible; however, limitations and inadequacies exist at every level up to and including the reporting of these data. A national awareness about racism, stigma, and disparities is emerging alongside attention to the public health crisis. As the pandemic evolves, so too will the information needs of policymakers, public health departments, health care systems, and others seeking to address COVID-related inequities. Periodic assessments of the data collected and the summary information produced by these systems, in addition to the mitigation efforts undertaken based on them, will improve the ongoing utility of COVID-19 surveillance systems for monitoring disparities and identifying populations and places where targeted mitigation efforts are needed.
Recognition of the linkages between these two pandemics and expansion of these recommendations to other conditions, such as drug overdoses, can promote focused dialogue, targeted programs, and needed policy change. For example, evidence of disparities in access to COVID-19 testing and vaccines among Black, Indigenous, and other people of color (BIPOC) communities led to federal and local investments in expanding and shifting the approaches used to provide testing and vaccination in these communities. These include NIH initiatives like the Rapid Acceleration of Diagnostics (RADx®) initiative “to speed innovation in the development, commercialization and implementation of technologies for COVID-19 testing” and the NIH’s prevention-focused Community-Engaged Alliance (CEAL) Against COVID-19 Disparities, along with state-level initiatives like California’s Get Out the Vaccine effort that funded community-level campaigns modeled on successful get out the vote strategies.36–38
Many efforts to educate, test, and vaccinate the communities impacted most by COVID-19 involve community-based approaches, cultural humility, and structural competency. The strategies these approaches include are: engaging peer ambassadors; delivering services in non-medical locations or on weekends and evenings; and offering other (ie, non-COVID-19) needed services when working with under-resourced communities. To reach the communities who may distrust the government and public health messages most, it is important to rely upon trusted messengers and provide some services in non-medical spaces such as churches, schools, barbershops, and beauty salons. These community venues may be more inviting and have been shown to be effective points of delivery for health education and offering services like vaccination, screening, and testing for many conditions.39–43 Services offered in these settings can reach populations who might not otherwise seek care. These settings are also conducive to peer-based interventions, which are known to reduce some of the fear and suspicion associated with public health campaigns. Data with sufficient detail and nuance to identify racial/ethnic populations and areas at increased risk allow for the efficient deployment of such interventions. They also provide the public with the rationale for the targeting of certain populations and convey to policymakers the need for the investments. Addressing legitimate concerns about mistrust among racial/ethnic minority populations requires adequate training of public health professionals and improved health research literacy among the general public, along with assurances of confidentiality for those concerned about stigma.44
We agree with calls for more and better data to monitor disparities by SOGI and socioeconomic status in health surveillance systems. This should be part of a larger shift in the routine collection of medical and public health data across many systems, rather than a move that is limited to a small number of reportable disease conditions. Despite the limitations encountered to date, places that have initiated the collection of data from these populations represent important test cases. The inclusion of SOGI data in their routine reporting on COVID-19, along with descriptions of both the limitations and lessons learned from the collected data, will contribute to improved approaches and encourage more accurate, informative, and complete data collection.16,45
Conclusion
Current approaches to COVID-19 surveillance vary widely and do not capture health disparities reliably. Improving the capacity of public health agencies, elected officials, and communities to mitigate racial/ethnic and other disparities in public health pandemics requires standardizing how COVID-19-related information is collected by laboratories, testing sites, providers, and health systems; how it is transferred between systems; and how it is compiled by local agencies. Although our recommendations focus on the ways that these data should ultimately be summarized and reported by local and national surveillance systems, limitations and differences at each of these levels make fulfilling them aspirational, without consistent standards and resources for implementation. A national and broadly focused approach will provide a clear roadmap for obtaining data that can be reported from a COVID-19 equity perspective; ultimately, these changes should be extended to all conditions of significance that are routinely reported.
Acknowledgments
Funding for this project was provided in part by the CDC Foundation. The source of this information is the COVID-19 Stigma Project, a joint project of the CDC Foundation and Howard University/UCLA Center for the Study of Racism, Social Justice and Health.
References
- 1.Ford CL, Amani B, Harawa NT, et al. Adequacy of existing surveillance systems to monitor racism, social stigma and COVID inequities: a detailed assessment and recommendations. Int J Environ Res Public Health. 2021;18(24):13099. 10.3390/ijerph182413099 10.3390/ijerph182413099 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Braveman P. What are health disparities and health equity? We need to be clear. Public Health Rep. 2014;129(1_ suppl2)(suppl 2):5-8. https://doi. org/ 10.1177/00333549141291S203 PMID:24385658 [DOI] [PMC free article] [PubMed]
- 3.Kress AC, Asberry A, Taillepierre JD, Johns MM, Tucker P, Penman-Aguilar A. Collection of data on sex, sexual orientation, and gender identity by U.S. Public Health Data and Monitoring Systems, 2015-2018. Int J Environ Res Public Health. 2021;18(22):12189. 10.3390/ijerph182212189 10.3390/ijerph182212189 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Richwine C, Marshall C, Johnson C, Patel V. Challenges to public health reporting experienced by non-federal acute care hospitals, 2019. ONC Data Br. 2021;56. Last accessed February 3, 2022 from https://www.healthit.gov/data/data-briefs/challenges-public-health-reporting-experienced-non-federal-acute-care-hospitals.
- 5.Nash D, Geng E. Goal-aligned, epidemic intelligence for the public health response to the COVID-19 pandemic. Am J Public Health. 2020;110(8):1154-1156. 10.2105/AJPH.2020.305794 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Rosenberg ES, Holtgrave DR, Dorabawila V, et al. New COVID-19 cases and hospitalizations among adults, by vaccination status - New York, May 3-July 25, 2021. MMWR Morb Mortal Wkly Rep. 2021;70(34):1150-1155. https://doi.org/ 10.15585/mmwr. mm7034e1 PMID:34437517 [DOI] [PMC free article] [PubMed]
- 7.Stokes AC, Lundberg DJ, Elo IT, Hempstead K, Bor J, Preston SH. COVID-19 and excess mortality in the United States: A county-level analysis. PLoS Med. 2021;18(5):e1003571. https://doi.org/ 10.1371/journal. pmed.1003571 PMID:34014945 [DOI] [PMC free article] [PubMed]
- 8.National Vital Statistics System . U.S. Census Populations with Bridged Race Categories. 2021. Last accessed February 7, 2022 from https://www.cdc.gov/nchs/nvss/bridged_race.htm.
- 9.U.S. Census Bureau . Data Profiles. Last accessed December 19, 2021 from https://www.census.gov/acs/www/data/data-tables-and-tools/data-profiles/
- 10.Gaskin DJ, Dinwiddie GY, Chan KS, McCleary RR. Residential segregation and the availability of primary care physicians. Health Serv Res. 2012;47(6):2353-2376. 10.1111/j.1475-6773.2012.01417.x [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Tung EL, Hampton DA, Kolak M, Rogers SO, Yang JP, Peek ME. Race/ethnicity and geographic access to urban trauma care. JAMA Netw Open. 2019;2(3):e190138-e190138. 10.1001/jamanetworkopen.2019.0138 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Caldwell JT, Ford CL, Wallace SP, Wang MC, Takahashi LM. Racial and ethnic residential segregation and access to health care in rural areas. Health Place. 2017;43:104-112. 10.1016/j.healthplace.2016.11.015 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Sen-Crowe B, Lin IC, Alfaro R, McKenney M, Elkbuli A. COVID-19 fatalities by zip codes and socioeconomic indicators across various U.S. regions. Ann Med Surg (Lond). 2021;67:102471. https://doi.org/ 10.1016/j. amsu.2021.102471 PMID:34150208 [DOI] [PMC free article] [PubMed]
- 14.Krieger N, Gonsalves G, Bassett MT, Hanage W, Krumholz HM. The fierce urgency of now: closing glaring gaps in US surveillance data on COVID-19. Health Affaires Forefront. April 14, 2020. Last accessed February 8, 2022 from https://www.healthaffairs.org/do/10.1377/forefront.20200414.238084/abs/.
- 15.Heslin KC, Hall JE. Sexual orientation disparities in risk factors for adverse COVID-19-related outcomes, by race/ethnicity - Behavioral Risk Factor Surveillance System, United States, 2017-2019. MMWR Morb Mortal Wkly Rep. 2021;70(5):149-154. 10.15585/mmwr.mm7005a1 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Cahill SR. Still in the dark regarding the public health impact of covid-19 on sexual and gender minorities. Am J Public Health. 2021;111(9):1606-1609. 10.2105/AJPH.2021.306397 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Office of Management and Budget . Revisions to the standards for the classification of federal data on race and ethnicity. Fed Regist. 1997;62(210):58782-58790. https://www.govinfo.gov/content/pkg/FR-1997-10-30/pdf/97-28653.pdf. Accessed January 6, 2022. [Google Scholar]
- 18.US Department of Health and Human Services . COVID-19 Pandemic Response, Laboratory Data Reporting: CARES Act Section 18115.; 2021. Last accessed February 8, 2022 from https://www.hhs.gov/sites/default/files/covid-19-laboratory-data-reporting-guidance.pdf
- 19.Office of Management and Budget (OMB) Standards . NIH Inclusion Outreach Toolkit: How to Engaee, Recruit, and Retain Women in Clinical Research. Last accessed February 8, 2022 https://orwh.od.nih.gov/toolkit/other-relevant-federal-policies/OMB-standards.
- 20.Friedman DJ, Cohen BB, Averbach AR, Norton JM. Race/ethnicity and OMB Directive 15: implications for state public health practice. Am J Public Health. 2000;90(11):1714-1719. 10.2105/AJPH.90.11.1714 10.2105/AJPH.90.11.1714 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Yoon P, Hall J, Fuld J, et al. Alternative Methods for Grouping Race and Ethnicity to Monitor COVID-19 Outcomes and Vaccination Coverage. MMWR Morb Mortal Wkly Rep. 2021;70(32):1075-1080. 10.15585/mmwr.mm7032a2 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Grafova IB, Jarrín OF. Beyond Black and White: mapping misclassification of Medicare beneficiaries race and ethnicity. Med Care Res Rev. 2021;78(5):616-626. 10.1177/1077558720935733 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.UCLA School of Law , Williams Institute. Best Practices for Asking Questions to Identify Transgender and Other Gender Minority Respondents on Population-Based Surveys (GenIUSS). 2014. Last accessed February 8, 2022 from https://williamsinstitute.law.ucla.edu/publications/geniuss-trans-pop-based-survey/.
- 24.Institute of Medicine (US) Committee on Lesbian , Gay, Bisexual and Transgender Health Issues and Research Gaps and Opportunities. The Health of Lesbian, Gay, Bisexual and Transgender People: Building a Foundation for Better Understanding. 2011. Washington, DC: National Academies Press. Last accessed February 8, 2022 from https://www.ncbi.nlm.nih.gov/books/NBK64813/. [PubMed]
- 25.Center for Disease Control and Prevention . What is ELR? Electronic Laboratory Reporting (ELR). Last accessed February 2, 2022 from https://www.cdc.gov/elr/about.html.
- 26.Kaiser Family Foundation . Latest Data on COVID-19 Vaccinations by Race/Ethnicity. Last accessed February 2, 2022 from https://www.kff.org/coronavirus-covid-19/issue-brief/latest-data-on-covid-19-vaccinations-by-race-ethnicity/.
- 27.Council of State and Territorial Epidemiologists . Review of and Recommendations for the National Notifiable Disease Surveillance System: A State and Local Health Department Perspective. 2013. Last accessed February 8, 2022 from https://cdn.ymaws.com/www.cste.org/resource/resmgr/pdfs/nndss_report.pdf
- 28.Antonios C, Chatterjee M, Gee G, Kravitz D, Senese K. Why Some States Won’t Share Race and Ethnicity Data on Vaccinations with the CDC—and Why That’s a Problem. The COVID Tracking Project. February 16, 2021. Last accessed February 8, 2022 from https://covidtracking.com/analysis-updates/why-some-states-wont-share-race-and-ethnicity-data-on-vaccinations-with-the-cdc-and-why-thats-a-problem.
- 29.Ong PM, Pech C, Gutierrez NR, Mays VM. COVID-19 medical vulnerability indicators: a predictive, local data model for equity in public health decision making. Int J Environ Res Public Health. 2021;18(9):4829. 10.3390/ijerph18094829 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.The COVID-19 Crisis Leveraging data to tackle the greatest health crisis in a generation. Surgo Ventures. Last accessed February 3, 2022 from https://surgoventures.org/the-covid19-crisis.
- 31.Smittenaar P. Geography of vulnerability: COVID-19 Community Vulnerability Index. Fall Meet Geogr Sci Comm. 2020:1-26. https://www.nationalacademies.org/documents/embed/link/LF2255DA3DD1C41C0A42D3BEF0989ACAECE3053A6A9B/file/DFBAA10AAA336AF00E428EBBD921037C106900A05480
- 32.Black Equity Coalition [website]. Last accessed February 3, 2022 from https://www.blackequitypgh.org/.
- 33.Surgo Ventures [website]. Last accessed February 3, 2022 from https://surgoventures.org/.
- 34.RAND Corp . Vulnerability, Inequity, and COVID-19: A Portrait of the Pandemic in Allegheny County. May 24, 2021. Last accessed February 3, 2022 from https://www.rand.org/pubs/tools/TLA1080-1-v2/tool.html.
- 35.Centers for Disease Control and Prevention . Managing Investigations During an Outbreak. Last accessed February 8, 2022 from https://www.cdc.gov/coronavirus/2019-ncov/php/contact-tracing/contact-tracing-plan/outbreaks.html.
- 36.National Institutes of Health (NIH) . RADx [website]. Last accessed February 8, 2022 from https://www.nih.gov/research-training/medical-research-initiatives/radx.
- 37.National Institutes of Health . Community Engagement Alliance (CEAL) Against COVID-19 Disparities [website]. Last accessed February 8, 2022 from https://covid19community.nih.gov/.
- 38.UCLA Clinical and Translational Science Institute . CTSI-supported “Get Out the Vaccine” Campaign Among California Governor’s Initiatives. Last accessed February 8, 2022 from https://ctsi.ucla.edu/news/item?item_ id=867826.
- 39.Suthar AB, Ford N, Bachanas PJ, et al. Towards universal voluntary HIV testing and counselling: a systematic review and meta-analysis of community-based approaches. PLoS Med. 2013;10(8):e1001496. 10.1371/journal.pmed.1001496 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Briss PA, Rodewald LE, Hinman AR, et al. ; The Task Force on Community Preventive Services . Reviews of evidence regarding interventions to improve vaccination coverage in children, adolescents, and adults. Am J Prev Med. 2000;18(1)(suppl):97-140. 10.1016/S0749-3797(99)00118-X [DOI] [PubMed] [Google Scholar]
- 41.Linnan LA, D’Angelo H, Harrington CB. A literature synthesis of health promotion research in salons and barbershops. Am J Prev Med. 2014;47(1):77-85. 10.1016/j.amepre.2014.02.007 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.Palmer KNB, Rivers PS, Melton FL, et al. Health promotion interventions for African Americans delivered in U.S. barbershops and hair salons- a systematic review. BMC Public Health. 2021;21(1):1553. 10.1186/s12889-021-11584-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Perez LG, Arredondo EM, Elder JP, Barquera S, Nagle B, Holub CK. Evidence-based obesity treatment interventions for Latino adults in the U.S.: a systematic review. Am J Prev Med. 2013;44(5):550-560. 10.1016/j.amepre.2013.01.016 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44.Ford CL, Griffith DM, Bruce MA, Gilbert KL, eds. Racism: Science & Tools for the Public Health Professional. Washington, DC: American Public Health Association; 2019. 10.2105/9780875533049 [DOI] [Google Scholar]
- 45.Bowleg L, Landers S. The Need for COVID-19 LGBTQ-specific data. Am J Public Health. 2021;111(9):1604-1605. 10.2105/AJPH.2021.306463 [DOI] [PMC free article] [PubMed] [Google Scholar]