A Test-Negative Design with Additional Population Controls Can Be Used to Rapidly Study Causes of the SARS-CoV-2 Epidemic

Supplemental Digital Content is available in the text.

Abstract

Testing of symptomatic persons for infection with severe acute respiratory syndrome coronavirus-2 is occurring worldwide. We propose two types of case–control studies that can be carried out jointly in test settings for symptomatic persons. The first, the test-negative case–control design (TND) is the easiest to implement; it only requires collecting information about potential risk factors for Coronavirus Disease 2019 (COVID-19) from the tested symptomatic persons. The second, standard case–control studies with population controls, requires the collection of data on one or more population controls for each person who is tested in the test facilities, so that test-positives and test-negatives can each be compared with population controls. The TND will detect differences in risk factors between symptomatic persons who have COVID-19 (test-positives) and those who have other respiratory infections (test-negatives). However, risk factors with effect sizes of equal magnitude for both COVID-19 and other respiratory infections will not be identified by the TND. Therefore, we discuss how to add population controls to compare with the test-positives and the test-negatives, yielding two additional case–control studies. We describe two options for population control groups: one composed of accompanying persons to the test facilities, the other drawn from existing country-wide healthcare databases. We also describe other possibilities for population controls. Combining the TND with population controls yields a triangulation approach that distinguishes between exposures that are risk factors for both COVID-19 and other respiratory infections, and exposures that are risk factors for just COVID-19. This combined design can be applied to future epidemics, but also to study causes of nonepidemic disease.

Widespread testing is essential for monitoring the Coronavirus 2019 Disease (COVID-19) pandemic.^{1, 2} Most countries are focusing on testing persons with symptoms to identify patients with Severe Acute Respiratory Syndrome Coronavirus-2 (SARS-CoV-2) infections. Ideally, this should be coupled with random/representative population testing to follow the epidemic in the population.³ However, there is much that can be learnt about the causes of COVID-19, even if only symptomatic people are tested. Still more may be learnt by conducting formal test-negative design studies, with additional population controls, thus yielding three linked case–control studies. In this article, we describe how these combined study designs can enhance understanding of risk factors for symptomatic SARS-CoV-2 infection in the COVID-19 pandemic.

Essence of the Test-negative Case–Control Design

Test-negative case–control studies^4–9 are based on persons who undergo testing because they present with signs and symptoms of a particular disease. The cases are those who test positive for the disease, and the controls are those who test negative—the latter will have another reason for their signs and symptoms, most likely another disease.⁸ These “cases” and “controls” usually come from one geographic population, although not everyone in a particular area may present for testing (and some people may come from outside the area).

Test-negative case–control designs (TNDs) involve comparing the odds of a given intervention (e.g., vaccine receipt) or a given risk factor (e.g., oral contraceptives) among symptomatic persons who test positive compared to those who test negative. Given assumptions described in the literature,⁸ it can produce effect estimates (odds ratios) that are generalizable to the general population (See eAppendix A; http://links.lww.com/EDE/B716 for more detail). The approach is most commonly known for its use in assessing vaccine effectiveness,⁴ but has also been applied to study risk factors for antibiotic resistance,^{5, 10} and to estimate risk factors in circumstances in which diagnostic bias was suspected, for example, in studies on oral contraceptives and venous thrombosis, and on aspirin use and Reye syndrome.⁸

Test-negative designs allow us to obtain quick answers to important questions. Additionally, by design, they protect against some forms of bias which are otherwise difficult to control. People who are tested for a disease will not be a representative of all those who have the disease (unless everyone in the population is tested)—usually, they are more likely to have severe symptoms, and more likely to seek medical help. This help-seeking behavior is affected by many factors such as age, gender, socioeconomic status, access to healthcare, proximity to testing facilities, severity of symptoms, personality, and insurance coverage. In a test-negative design, the same selective forces that lead individuals to be tested will operate on both those who test positive and those who test negative. There is a substantial literature on this study design,^4–9 and it is generally agreed that it can produce valid effect estimates under the assumption of similar selection pressures for the test-positives and the test-negatives.

Reasons for Considering the Test-negative Case–Control Design in the Coronavirus 2019 Pandemic

Insights into risk factors for COVID-19 can be gained by collecting the same information on symptomatic individuals who test positive and those who test negative, that is, by performing a test-negative case–control study. Because the test-negatives belong to the same population (i.e., people who would come for testing if they had symptoms of COVID-19) as the test-positives, this may give timely and locally relevant insight into the causes of SARS-CoV-2 infection in different communities (urban and rural), in communities with many cases, or communities with few cases.

Direct comparisons of test-positives to test-negatives (comparison of TND in Figures 1 and 2) can yield insight into specific risk factors for becoming infected and symptomatic with SARS-CoV-2 (i.e., having COVID-19); these may include age, sex, race/ethnicity, socioeconomic factors (e.g., income, education), occupational exposures (e.g.. healthcare workers performing aerosol-generating procedures, delivery drivers, teachers), contact patterns (e.g.. household exposure to confirmed case, crowding, travel histories, childcare responsibilities), geographic residence (e.g.. urban vs. rural), behavioral factors (e.g., shopping locations and smoking), medical risk factors (e.g., immunodeficiency), and genetic factors (from the swabs or blood sample taken for viral diagnosis, which will also contain human cells).

FIGURE 1.

Test-negative design and case-control studies with accompanying persons.

FIGURE 2.

Test-negative design and case-control studies with random population controls.

Some of this information might already be routinely collected. If a sufficient number of test sites test large numbers of people, different types of additional information may be asked at different testing sites, so as not to burden the test sites and to be able to adapt questionnaires to evolving questions. Some risk factors may be immediately important for local decisions, others more widely or more theoretically. The data can be analyzed like any other case–control study, although consideration should be given to assessing possible interpretation issues arising because both the cases and controls are drawn from a subgroup of the general population (see also eAppendix A; http://links.lww.com/EDE/B716).⁸

An interesting variant might be to study risk factors for antibody seroprevalence (instead of new infections), which is an approach that investigates the cumulative risk of infection rather than incident infection. Some of its uses are discussed below.

Critical Reflections on the Interpretation and Feasibility of the TND in the COVID-19 Pandemic

The TND involves a comparison between persons who test positive for SARS-CoV-2 and persons who test negative but who have similar signs and symptoms. The test-negatives will have another reason for their similar signs and symptoms; most likely they will have another viral respiratory infection. Some exposures (e.g., overcrowding) will increase the risks of both COVID-19 and other respiratory infections. Thus, the TND can only identify those risk factors that are either totally distinct or clearly different in magnitude from the risk factors of illnesses that manifest with similar symptoms. For example, if living in crowded conditions equally increases the risks of both COVID-19 and other respiratory infections, then the proportions living in crowded conditions would be similar in the test-positives and the test-negatives. On the other hand, if male sex was a risk factor for COVID-19, but not for other respiratory infections, then more of the test-positives than the test-negatives would be male.

A second concern is about the sensitivity and specificity of the test. Although reverse transcriptase-polymerase chain reaction (RT-PCR) testing has a high specificity for SARS-CoV-2, the sensitivity can vary in relation to timing of symptom onset,^{11, 12} the bodily fluid tested,¹³ and the assay used.¹⁴ There will be misclassification of cases and controls. This can be expected to be “nondifferential” (whether the test works correctly on a particular person is unrelated to exposures such as crowding). Such nondifferential misclassification of exposure or disease is a known problem in case–control studies, and it usually results in bias of the effect estimate towards the null (an underestimation of effects). However, there is a major difference between the usual case–control study and the TND. In the standard population-based case–control study, the false-negatives remain part of the source population, and only a (small) fraction of them will be sampled and end up in the control group. In contrast, in the TND, it is certain that at a particular test site, all false-negatives will be included in the control group (the test negatives); similarly, all false-positives will be included in the case group—since all persons tested at the test site will be in the study. This may lead to stronger misclassification which has most consequences in situations wherein the proportion of COVID-19 relative to other respiratory disease among the persons tested is either very high or very low. There is an extensive literature on sensitivity analysis for standard case–control studies,^{15, 16} which essentially involves making assumptions about how large this misclassification would be, and these methods can be adapted to the TND situation. See eAppendix B; http://links.lww.com/EDE/B716 for further details.

An additional reflection is about seasonality; it is not known at the time of writing how the incidence of SARS-CoV-2 infections over the calendar year will evolve. There is the possibility that other respiratory viruses such as influenza might disappear during summer,¹¹ whereas the SARS-CoV-2 may continue to circulate; in the extreme, there may not be sufficient test-negative controls.

In the earliest phases of the COVID-19 epidemic, testing for acute infection may not have been done. The above-mentioned variant of a TND on seroprevalence of antibodies may be a solution. For example, in a hospital-based setting, serologic testing of healthcare workers for antibodies specific to SARS-CoV-2 may yield insights into exposure risks that could have been missed due to the initial lack of testing for acute infection. Of course, this will miss persons who did not survive.

As the epidemic progresses, risk factors for having had the infection might become negative risk factors for new infections. For example, bus drivers may have been frequently infected early on; if these infections conferred sufficiently strong immunity, bus drivers may turn up later in the epidemic mainly as seemingly test-negatives for acute SARS-CoV-2 infection when having signs and symptoms of respiratory disease (from another virus). In addition, measures might have been taken to shield bus drivers (passengers only entering via rear doors; obligatory unoccupied seats). While this muddles the estimates of risk factors, the immunity conferred by earlier infection as well as the preventive measures taken earlier are worthwhile goals of a TND study. This paradox ideally requires investigating the evolution of the magnitude of the risk factor associations over the course of the epidemic, with additional background knowledge beyond the data; as a minimum we need to look separately at the upward and downward phases of the epidemic curve. To verify a potential role of immunity in an epidemic, subsamples of test-positives and test-negatives for acute infection might be tested for SARS-CoV-2 antibodies.

Adding Standard Case–Control Studies with a Control Group Representing the Underlying Population

As noted, a TND can potentially identify risk factors for COVID-19 that differ from those for other respiratory infections, either in kind or in magnitude, but will not identify risk factors that the test-positives and test-negatives have in common. On the other hand, comparing test-positives with general population controls will tell us about risk factors for COVID-19, but does not tell us which factor is specific for SARS-CoV-2 infections rather than respiratory infections in general. Thus, the ideal situation is to also have a comparison of the test-negatives with the general population. This strategy has already been applied as an extension of TNDs of antibiotic resistance.^{5, 10}

Below, we outline two different strategies to obtain population controls: first, the use of friends or household members accompanying the case as matched controls, and second the use of a random sample from general population databases with healthcare and other registered information from that population. In a separate section, we will briefly discuss other possibilities for choosing controls that might be more useful in a diversity of situations. First, we discuss the benefits of population controls.

Benefits of Added Population Controls in Separate Case–Control Studies with Test-positives and Test-negatives

The importance of having population controls can be seen from Figures 1 and 2, which, respectively, refer to the situation with accompanying persons as controls, and to the more general situation of population controls. A comparison of the findings from the TND with a case–control comparison of the test-positives and their population controls (CC-POS comparison), and a separate comparison of test-negatives with the population (CC-NEG comparison), will enable us to assess which risk factors are specific to COVID-19 and which are risk factors for all respiratory infections (including SARS-CoV-2) in general. If these studies were all perfect, one would be able to calculate the results of any one contrast from the two others, for example, the results of comparison CC-POS should logically follow from combining the results of the TND comparison and comparison CC-NEG (if the odds ratio for male sex is 1.0 in the TND, but is 2.0 in comparison CC-NEG, then it also should be 2.0 in comparison CC-POS). In reality, there might be differences due to sampling and/or unknown selection biases. Thus, although it would be sufficient in theory to only conduct the TND and comparison CC-POS, it remains valuable to conduct also comparison CC-NEG. This enables “triangulation”¹⁷ with information about differences in risk factors between symptomatic test-positives and test-negatives, and two case–control studies of test-positives and test-negatives with their population controls.

Accompanying Persons as a Control Group

Symptomatic persons who go for SARS-CoV-2 testing may be accompanied by other persons, for example, household members, relatives, or friends. Thus, it may be expedient to ask an accompanying person to volunteer the same information (e.g., completing a questionnaire) at the time of testing the person with symptoms—this may be done before the test result is known. These persons are members of the source population which generated the cases and should not have COVID-19 symptoms. Note that for this design, it is not necessary to carry out the test on the accompanying person.

For both the test-positives (COVID-19 cases) and the test-negatives (controls with other respiratory infections), the accompanying person can be seen as a matched population control. Such approaches have been widely used in epidemiology, and the strengths and weaknesses have been extensively discussed.^{18, 19} Briefly, using friends, siblings, or spouses as matched population controls has the advantage of logistic convenience, and may indirectly match for various risk factors (e.g., socioeconomic status, availability of healthcare, health seeking behavior). As with any other pair-matched case–control study, this necessitates a pair-matched analysis. Essentially, the matched analysis focuses on the subgroup of case–control pairs where the case and control differ with respect to the exposure under study: a pair-matched analysis is an analysis of the differences that remain between cases and their controls despite them being made in effect more equal by the matching.

This strategy leads to the case–control comparisons represented in Figure 1: test-positives with their accompanying persons (CC-POS comparison), and test-negatives with their accompanying persons (CC-NEG comparison). Comparison CC-POS enables us to study directly the differences in risk factors between a person with COVID-19 and a control person without respiratory symptoms. Thus, in this analysis, all risk factors that increase the risk of COVID-19 (some of which will also be risk factors for other respiratory diseases) will be seen to differ between cases and controls. Comparison CC-NEG enables us to directly assess risk factors for the mixture of other respiratory pathogens (e.g., influenza virus and rhinovirus) that could be causing symptoms similar to those of COVID-19.

Critical Reflections on the Interpretation of Case–Control Studies with Accompanying Persons as Matched Controls

The use of friend controls leaves the choice of the control to the case and not to the investigator (see pp. 119–120 in Ref ¹⁹). Friend controls may be quite similar to the cases, which is an intended benefit of matching. However, they may have some possible inherent biases (pp. 119–120 in Ref ¹⁹), for example, popular persons and extroverts are more often mentioned as friends. We should stress that the problem is not that the cases and controls are made similar—this problem applies to all matched case–control studies and is addressed by taking the matching into account in the analyses.¹⁸ Rather, the problem is that they may be made similar in ways that the investigator cannot control, and certain types of persons might be more valued to be named as friends.

A second issue is that the accompanying persons of the test-positives in the CC-POS comparison may be as yet asymptomatic carriers of SARS-CoV-2. A common reflex might be to want to know this and to remove these persons from the analysis. However, apart from involving logistically difficult additional testing of the accompanying persons, it is not necessary. This is explained in detail in eAppendix C; http://links.lww.com/EDE/B716. Briefly, the studies are based on the source population which would come for testing if they develop symptoms; the cases are people who have actually developed symptoms and come for testing. The controls should be a sample of the source population which generated the cases.^20,21 Because the accompanying persons came with their index person for testing, it is reasonable to assume that they would also have come for testing at the same facility if they had developed symptoms.

Random Sampling From Country-wide General Population Healthcare and Other Databases

In regions or countries where all healthcare activities are registered (prescriptions, hospitalizations, test results, etc.) in digital databases, it may be possible to use a different type of control group, constituting a control population randomly sampled from the region or country as a whole. While analyses based on existing databases may lack the immediacy and flexibility of point-of-care data collection of persons who are tested, the advantage is that data are recorded prospectively in past time, and the epidemic can be analyzed, and reanalyzed, in its several stages (e.g., in relation to the implementation of social distancing and lockdowns).

The analysis of the COVID-19 epidemic would start with recorded data of test-positives and test-negatives for SARS-CoV-2 in the total administrative population of a country or region. While this limits information to healthcare data that are registered at a particular point in time, an advantage is that healthcare data that have been registered before (e.g., pre-existing diseases and prescriptions, prior hospitalizations etc.), can be added, as well as other data such as data on crowding, income, level of education, etc., from other databases.

A single control group can be used for both CC-POS and CC-NEG (see Figure 2). This allows one to randomly sample several times as many controls as there are test-positives and test-negatives combined. For efficiency purposes, the cases and controls, as well as the random population control might be limited to an age bracket, say age 15–74, as there will be few symptomatic SARS-CoV-2 cases below age 15, and persons above age 75 may not be tested nor hospitalized. Age and sex matching are undesirable in the context of COVID-19 as these may be determinants of infection and disease course. It is always possible to stratify on age and sex, as the numbers will be sufficiently large. Matching on being alive at the index date of the cases (i.e., the date of testing) might be considered; however, this might be replaced by a control group that is composed of persons being alive in the middle of the month in which persons were tested.

It might be objected that we use two different control groups for one case group, which is often frowned upon, because if the findings with the different control groups are different, the investigator has to make a judgment call about the most appropriate control group (see pp. 121–122 in Ref ¹⁹) However, in interpreting, the combination of the TND with population control groups neither is really the correct one, as both point to a different contrast. This can be learned from a test-negative case–control study on urinary tract infection with antibiotic-resistant bacteria in contrast to infections with sensitive bacteria, with added population controls to both groups.¹⁰ In this study, male gender proved a strong risk factor for antibiotic resistance in the test-negative design, while female gender was a strong risk factor for urinary tract infections in comparisons with the population. This seems like a reversing of usual risk factors, but is logical because men generally only acquire urinary tract infections at older ages, subsequent to prostate or other pathology which also puts them at risk of acquiring resistant bacteria.¹⁰ This analysis and reasoning are explained in eAppendix D; http://links.lww.com/EDE/B716, which shows how the triangulation of the test-negative design with population controls leads to identifying the right causal pathways.

Other Population Control Groups

Many alternatives for population control selection are possible, depending on the situation in different regions or countries. Some of these other options will be closer to the flexibility of the accompanying-persons control groups, and others will be closer to the advantages of using existing databases. The appropriate approach will depend on local considerations. For example:

• – Records from General Practitioner databases (e.g., in the United Kingdom, in some regions in Italy or Spain), or third-party payers and insurers (Medicare, Medicaid, health maintenance organizations in the United States), which will also allow for database-centered research.

• – If patients are presenting in a special setup organized by groups of general practitioners (GPs) (e.g., coronavirus testing sites managed by several GP practices where patients are referred for testing), control persons may be a sample from these general practices; this sample could be matched to the practice of the referring GPs, or weighted according to the size of the referring GP practices. This might facilitate the collection of specific new information relevant to local situations in individual practices (e.g., the use of local sports facilities).

• – If patients present to outpatient clinics or hospital departments, a control group of non-respiratory out- or inpatients might be constructed to represent the catchment population of the hospital; such patient controls used to be common in pharmacoepidemiology.²²

Critical Reflections on the Choice of Population Controls

It is imperative to consider to what extent the test-positives and test-negatives from a TND are representative of all cases in the general population, that is, whether the general population can really be seen as the source population for the tested persons. There are two considerations: patient selection and doctor’s preferences.

Patient Selection

Not all diseases present equally to healthcare facilities. In countries with universal access to healthcare and relatively standardized care, it is likely that, for example, almost all solid cancers (colon, lung, etc.) with onset before age 70 will ultimately be diagnosed and recorded. That is not the case for self-limiting diseases such as influenza-like illnesses or headache, with which many persons will just stay home. Only persons who worry or have more severe symptoms will present themselves to a primary care service. Still, in a country with universal access to relatively standardized care, the types of person who present themselves at several points of care will be roughly similar, and if testing is done for SARS-CoV-2, test-positives and test-negatives can be seen as drawn from the same underlying general population.

The type of person that is tested may differ between countries, however. During the initial wave of COVID-19 in February–March in Europe, testing for persons with minor symptoms was available in Germany; in The Netherlands, only persons with symptoms that were sufficiently severe to require hospitalization were tested. Because these were country-wide measures, in both countries, the general population may be seen as the source population. In this context, it should be noted that if (self) selection is based only on severity of disease, this will not create a bias in itself within one country.

Problems only arise if there are other factors which affect presentation for testing, given a particular level of symptoms. For example, consider private healthcare facilities that are only accessible to individuals who can afford them (these facilities exist in many guises, from standard private health care coming as an employee benefit to facilities only available to the very rich). A comparison of the test-positives and test-negatives from such facilities with a general population control group may not be warranted, because of inherent differences in socioeconomic status, medical care, and lifestyle. Among persons tested in private facilities, both test-positives and test-negatives are, for example, unlikely to live in very overcrowded conditions. Thus, a better control group, representing their own source population, might consist of other persons (or patients) who make use of the same healthcare facilities. This means that one may not be able to study all of the causes of the disease (e.g., if poverty is an underlying cause, but no one who accesses these healthcare facilities is low income). Also, if we suspect that there are differences in access to testing for persons with very mild symptoms or without symptoms (depending on type or health insurance or wealth), but few such differences for severe disease, then one might restrict the analysis to the subgroup of tested individuals with more severe symptomatic disease.

Doctors’ Preferences

Even in settings with relatively standardized care, there might be variation in testing strategies for a new disease like COVID-19 between (primary care) practices. The existence of physician preference has been studied in different countries.²³ Physician’s preference can be based on a different interpretation of the literature on topics where there is not yet consensus, or on implicit biases (regarding age, sex, and ethnicity). If this is suspected to have been the case, it might be better to select population controls from the GP practices of the individuals who underwent testing—and to approach this in the analysis as a form of matching. Matching by GP practice would, however, limit the ability to compare between catchment areas and lead to loss of information about regional differences. Once again, restricting the analysis to patients with severe disease (whether test-positive or test-negative) may suffer less from self-selection and testing preference by doctors. An analysis according to severity can also be added as a sensitivity analysis.

DISCUSSION

An ideal approach for identifying risk factors for COVID-19 would involve random or representative population sampling.³ However, in the surveillance efforts that are being developed when an epidemic is unfolding, population-based testing often is limited by laboratory capacity (i.e., due to unprecedented demands for reagents and trained technicians), funding, and political will. The first thoughts of decision makers are to facilitate testing for people with symptoms who became ill recently, either to isolate, or to know which treatment trajectory is necessary if symptoms worsen.^{1, 2}

The situation with COVID-19 remains urgent in many parts of the world, and it is important that the best possible use is made of information collected in the process of widespread testing of symptomatic persons. Therefore, there are research and public health benefits in employing a test-negative case–control design, combined with case–control studies with population controls added to it. Still, such collection of information has to be as unburdensome as possible, in order not to disturb the primary medical aim: to test people for their own benefit and for controlling the epidemic. The proposed data collection can be done with a minimal extra effort, it would roll along with the epidemic, and it can potentially yield important information at much less cost, and with greater ease, than doing genuinely random population repeated sampling and testing. In situations where extensive databases exist, data will have been collected as the epidemic unfolded, and then kept frozen in time in the databases. This allows investigators to return to the data and evaluate the course of the epidemic with new hypotheses. At the time of this writing, several efforts are underway to set up collections of types of questions and data of interest to study the COVID-19 pandemic.^{24, 25}

Adding general population controls yields linked case–control studies (the TND, CC-POS, and CC-NEG) and creates a triangulation situation¹⁷ for inferences about local as well as general factors that drive the pandemic. Follow-up of test-positives or test-negatives and other additional strategies will lead to better understanding of the course of the disease. In particular, follow-up starting from a test-negative design of acute disease may be a good starting point to provide information on the degree and duration of protection from (re)infection conferred by having had a SARS-CoV-2 infection previously and/or specific antibody titers, because such studies might need exact matching on date and place of testing.²⁶ Finally, having an infrastructure for a test-negative design already established in different settings may be a valuable base to evaluate the effectiveness of interventions such vaccines when they become available, or other measures to limit transmission.

The combined test-negative and population-based case–control design may be useful, not only for recurrent waves of this epidemic or other epidemics, but also to study causes of nonepidemic diseases, in circumstances where a test-negative design would have advantages but in which its inferences could be strengthened by comparing with population controls.

ACKNOWLEDGMENTS

We acknowledge our debt to comments on a previous version of this paper by Ivo Foppa, Mike Jackson, Daniel Westreich, Sheena Sullivan, Martie van Tongeren, several Twitter commentators (David Simons, Mark D. Simmons), and Marc Lipsitch; for comments on Appendix B, we thank Tim Lash and Avnika Amin. All opinions as well as remaining inadequacies are our own.