The Usefulness of the Test-Positive Proportion of Severe Acute Respiratory Syndrome Coronavirus 2 as a Surveillance Tool

Abstract

Comparison of coronavirus disease 2019 (COVID-19) case numbers over time and between locations is complicated by limits to virological testing to confirm severe acute respiratory syndrome coronavirus 2 infection. The proportion of tested individuals who have tested positive (test-positive proportion, TPP) can potentially be used to inform trends in incidence. We propose a model for testing in a population experiencing an epidemic of COVID-19 and derive an expression for TPP in terms of well-defined parameters related to testing and presence of other pathogens causing COVID-19-like symptoms. In the absence of dramatic shifts of testing practices in time or between locations, the TPP is positively correlated with the incidence of infection. We show that the proportion of tested individuals who present COVID-19-like symptoms encodes information similar to the TPP but has different relationships with the testing parameters, and can thus provide additional information regarding dynamic changes in TPP and incidence. Finally, we compare data on confirmed cases and TPP from US states up to October 2020. We conjecture why states might have higher or lower TPP than average. Collection of symptom status and age/risk category of tested individuals can increase the utility of TPP in assessing the state of the pandemic in different locations and times.

Abbreviations

CLI

coronavirus disease 2019–like illness

SARS-CoV-2

severe acute respiratory syndrome coronavirus 2

SEIR

susceptible-exposed-infectious-recovered

TPP

test-positive proportion

TSP

test-symptomatic proportion

The number of confirmed infections with severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) is determined by the true infection rate and the number and type of people who are tested for presence of the virus. Monitoring the pandemic in a location (or comparing locations) by the number of reported cases is confounded by the amount and type of testing being done; the number of cases is inherently limited by the number of tests performed. The test-positive proportion (TPP) is used as an additional indicator of case burden; a high TPP coupled with a large number of cases is seen as an indication that the reported cases represent the “tip of the iceberg” and that testing capacity should be increased to get a better understanding of transmission. Where capacity is limited, tests are given preferentially to those more likely to be positive, such as hospitalized patients, meaning that mildly symptomatic infections are likely to be undetected. On the other hand, a low TPP is viewed as an indication of a potentially effective surveillance and containment strategy (1) and implies that increased testing would not reveal a substantial number of undetected infections. The World Health Organization and US Centers for Disease Control and Prevention made TPP part of their guidelines for easing lockdown restrictions, indicating that it can be used to assess readiness for releasing restrictions and recommending that communities should be below various thresholds (5%–20%) for 14 days before they consider relaxing social-distancing measures (2, 3).

While these interpretations are broadly plausible, many aspects of testing strategies, including the rate of testing of symptomatic and asymptomatic individuals, the number of tests available, and the performance of the tests, could change TPP independently of the true incidence. We aimed to explore the relationship between TPP and testing parameters, and suggest additional metrics and data to aid interpretation of the TPP.

METHODS

Model for disease and testing

We modeled transmission of SARS-CoV-2 using a susceptible-exposed-infectious-recovered (SEIR) model (see Web Appendix 1 for further details, available at https://doi.org/10.1093/aje/kwab023). The state variables represent the number of individuals in each compartment, with . All individuals in the compartment are infectious and are symptomatic with coronavirus disease 2019–like illness (CLI) until they recover. A proportion of those in the and compartments are symptomatic with CLI from other causes (henceforth “non–SARS-CoV-2 CLI”).

We model the application of tests using compartments for available test kits and completed tests applied to SARS-CoV-2–positive/SARS-CoV-2–negative and symptomatic/asymptomatic individuals. We assume that an individual’s rate of testing differs by symptom status. Recovered individuals are assumed not to seek testing. The test has sensitivity . Exposed and infectious individuals are isolated upon testing (efficacy of isolation is assumed to be perfect, represented by moving those individuals to the compartment).

A schematic for the natural history and testing model is shown in Figure 1 (see Web Table 1 for a table of parameters). The demand for tests is equal to the number of individuals in each group multiplied by their rate of test-seeking. Let and be the daily rate of testing among asymptomatic and symptomatic individuals, respectively. For example, every day a proportion of symptomatic individuals will receive a test. Total demand can be expressed as (i.e., the total number of tests sought by the entire population each day). The demand from any specific group is denoted using subscripts (e.g., for exposed individuals with CLI). We assume that anyone with CLI seeks testing at the symptomatic rate .

Figure 1.

Schematic for disease natural history (susceptible-exposed-infectious-recovered) and testing. Administered tests to each compartment are determined by the proportion of individuals symptomatic with coronavirus disease 2019–like illness (CLI) in each compartment (proportion p_I in the S and E compartments caused by non–severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) pathogens, 1 in the I compartment caused by SARS-CoV-2) and by the rate of testing among symptomatic (d_S) and asymptomatic (d_A) individuals. Tests administered each day are limited by available tests T^*, and assigned proportionally to each compartment according to demand (T^*/D). Tests applied to individuals with CLI are “symptomatic” and vice versa. Tests applied to individuals in the S compartment are true negatives, while tests applied to individuals in the E or I compartment are true positives.

The number of tests performed per day is limited by a daily maximum number of tests that can be performed (i.e., laboratory capacity) and by the number of available test kits (i.e., test stockpile). We choose a simple functional form for the number of tests performed per day: .

If demand for testing exceeds the number of available test kits, the rate of testing for each compartment is normalized by the demand (e.g., rate of testing among symptomatic, exposed individuals is ). Thus, test kits are allocated to each type of individual proportional to demand.

Relationship between incidence and TPP

The transmission model gives rise to an equation for TPP as a function of infectious prevalence and the model parameters. We assume that early in the epidemic, there are few individuals in the compartment . The TPP is the number of positive tests over the total number of tests carried out on a given day, or

(1)

as when is very small. Rearranging gives

All components of the above equation are positive if . An upper bound for infectious prevalence is thus

We assume that test sensitivity for pre-infectious individuals is very low (4) so that the class contributes little to detected cases compared with the class, and the above inequality becomes an approximate equality.

Given the infectious prevalence at a point in time and assuming that the majority of detected cases are from the infectious compartment, the number of positive tests per capita (henceforth “confirmed incidence”) is estimated by

(2)

We write TPP as a function of the confirmed incidence rate,

(3)

If the relevant parameters were known, we could use equation (3) to “estimate” the confirmed incidence from the TPP; we refer to this as the “TPP-estimated” incidence.

Additional metrics to aid interpretation of TPP

If the number of individuals with CLI (due to SARS-CoV-2 and/or other causes) among those seeking testing is known, the test-symptomatic proportion (TSP) provides a similar relationship between confirmed incidence and TSP, assuming :

(4)

In addition to the presymptomatic exposed period, some individuals remain asymptomatic throughout the course of infection. To model asymptomatic infections, we assume that a fraction of infected individuals become infectious with symptoms, with the remaining not developing CLI due to their SARS-CoV-2 infection. We assume that infectiousness is the same regardless of symptoms. The relationship in (3) becomes

(5)

This structure allows testing demand to vary for certain target groups, defined either by frequent exposure to infection (e.g., essential workers) and increased β, or increased probability of symptoms (e.g., the elderly) and increased p_S. We assume that these risk groups can be tested at a rate (i.e., d_S and d_A) different from the rate in the general population. The overall TPP will depend on the proportion of tested individuals in these target groups. We use our previous formulas to assess TPP and TSP stratified by risk group and the potential for bias in the overall TPP caused by testing of these groups.

Assessment of TPP-incidence and TSP-incidence relationships using simulations

To assess the accuracy of equations (3) and (4), we simulated the SEIR model described above using an adaptive tau-leaping method (R package adaptivetau). From the simulations we calculated the weekly confirmed incidence, TPP, TSP, and supply/demand ratio (stratified by risk group where relevant), varying testing parameters d_A, d_S, and p_I. We then explored the impact of temporal variation in testing in 2 ways: by inducing linear changes in testing parameters and by drawing available tests T^* from a lognormal distribution to represent random fluctuations in capacity. Finally, to understand the effect of targeting testing at high-exposure and high-vulnerability populations, we varied the size of the risk groups, the relative hazard of infection and probability of symptoms, and relative rates of testing between the high- and low-risk groups. In simulations, we used parameters from existing literature where possible (5–12) (Web Table 1), with initial R₀ drawn from a uniform distribution between 3 and 5. We modeled a lockdown with a reduction in R₀ to 0.8–1 at 21–35 days after the start of the simulation.

Work with empirical data

In addition to our simulations, we used data from the COVID Tracking Project (13) to examine the relationship between TPP and confirmed cases across states, and within states across time, with population data for states from the US Census Bureau (14). We used the derived equations to plot the expected relationship between TPP and confirmed cases if all states had the same testing parameters, by fitting equation (3) to the observed confirmed cases using ordinary least squares regression. Similarly, we fitted equation (3) to observed confirmed cases for a time series within a single state to identify periods of time in which the trend in TPP is not expected given the trend in confirmed cases. In addition to national data, we used data from the Oregon Health Authority (15), which reports the proportion of coronavirus disease cases with various symptoms to illustrate trends in symptoms over time.

RESULTS

Relationship between TPP and confirmed incidence

In Figure 2, we vary each model parameter in turn, with the other parameters fixed at default values (Web Table 1), to explore its univariate relationship with TPP. As the rate of confirmed infections increases (Figure 2A), the TPP rises because there are more symptomatic, SARS-CoV-2–positive individuals in the population demanding tests. In addition, testing strategy can affect the TPP and TSP. Counterintuitively, if individuals with CLI are tested at a higher rate, the TPP will decrease (Figure 2B) if the rate of confirmed infections is held constant. In this case, higher rate of testing individuals with CLI means that individuals with non–SARS-CoV-2 CLI will also be tested at a higher rate, leading to lower TPP. If more asymptomatic individuals are tested (e.g., through expanding eligibility for testing), the TPP will decrease, albeit modestly (Figure 2C). If there is a shortfall of testing supply relative to demand (Figure 2D), TPP will increase because the confirmed cases represent a smaller proportion of infectious individuals, and more overall demand from infectious individuals leads to higher TPP. Finally, factors independent of policy decisions can affect the TPP. The test sensitivity has a negligible effect on the TPP (Figure 2E) when the confirmed incidence is held constant. On the other hand, if prevalence of non–SARS-CoV-2 CLI is higher (e.g., during an influenza outbreak), the TPP will decrease as more SARS-CoV-2–negative individuals will seek testing (Figure 2F).

Figure 2.

Test-positive proportion (solid line) and test-symptomatic proportion (dotted line) vary differentially along the gradients of testing strategies: rate of incident confirmed severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infections (A), rate of testing among symptomatic individuals (B), rate of testing among asymptomatic individuals (C), shortfall of test supply relative to demand (D), test sensitivity (E), and prevalence of non–SARS-CoV-2 coronavirus disease 2019–like illness (F). CLI, coronavirus disease 2019–like illness.

Figure 2 also shows the relationship between TSP and the model parameters. TSP increases modestly as infectious prevalence increases (Figure 2A) and decreases as the testing rate among asymptomatic individuals increases (Figure 2C). In contrast to the TPP, the TSP increases with higher testing rate of individuals with CLI (Figure 2B) and higher prevalence of non–SARS-CoV-2 CLI (Figure 2F), because both of these parameters lead to increased testing demand from symptomatic individuals. Testing supply shortfall (Figure 2D) and test sensitivity (Figure 2E) have negligible effects on the TSP.

Equation (3) provides intuition for how TPP and detected cases will change under different scenarios. During the exponential growth phase of an epidemic, TPP will rise as infectious prevalence rises rapidly (Figure 2A). Similarly, if the rate of new infections is declining and testing strategies remain constant, TPP will decrease over time. This observation provides a simple way to understand whether a fall in case numbers is due to a true decline in incidence or a shortage of test kits. If the infectious prevalence declines, the rate of confirmed cases and the TPP will both decline (Figure 2A), whereas if the supply of test kits is falling relative to demand but the infectious prevalence remains constant, the rate of confirmed cases will decrease (equation (2)) but the TPP will remain constant (equation (1); with constant infectious prevalence, TPP is independent of test kit supply). Similarly, concurrently rising TPP and confirmed incidence is an indication that infectious prevalence is truly increasing. If a rise in confirmed cases were due to increases in testing capacity alone, we would not expect TPP to increase.

Relationship between TPP and TSP

Figure 2 demonstrates how the TSP could provide further information to interpret changes in confirmed cases and TPP. The relationship between TSP and testing parameters is in some cases the inverse of the relationship between TPP and the parameters. For example, a decrease in TPP coupled with a rise in TSP over time provides evidence for an increase in prevalence of non–SARS-CoV-2 CLI, or in the rate of testing among individuals with CLI, over a change in testing rate among asymptomatic individuals. External data on changes in influenza-like illness over time can further narrow down the cause of dynamic changes in TPP and TSP.

The presence of subclinical infections does not substantially alter the relationship between TPP and confirmed incidence if the majority of confirmed cases and positive tests are from clinical cases. If confirmed incidence is fixed, infectious prevalence, and thus TPP, increase modestly as clinical fraction decreases.

Simulation results

We assessed the accuracy of the TPP and TSP formulas using data generated from an SEIR model. We fixed the testing parameters for all simulations and sampled before-lockdown R₀ uniformly from 3 to 5 (higher than observed values (6) to include parameter space in which supply of testing is limited), time of lockdown from 21 to 35 days, and after-lockdown R₀ from 0.8 to 1 in the absence of testing (5, 6). Figure 3 shows that if the supply of test kits is sufficient to cover the demand from symptomatic and asymptomatic individuals, the relationship between TPP and confirmed incidence is as in equation (3) (gray points) but that if there is limited supply of test kits the proportion of cases detected decreases while the TPP remains the same. Therefore, the TPP observed in the simulations is greater than predicted (black points). We observe a similar pattern for TSP. We found that the relationship between TPP and incidence was unaffected by the efficacy of quarantine (Web Appendix 2 and Web Figure 1).

Figure 3.

Confirmed incidence per 10,000 against test-positive proportion (A) and test-symptomatic proportion (B) at 4 weeks, from 10,000 susceptible-exposed-infectious-recovered simulations in which initial and lockdown R₀ values were randomly sampled from specified ranges (see text). The darkness of the points represents whether supply of testing was sufficient to cover demand at week 4, which is a function both of the availability of test kits and R₀. The solid line in each panel denotes the relationship in equations (3) and (4).

TPP was strongly correlated with the confirmed incidence rate (average Pearson correlation = 0.94 across all simulations in Figure 3). Correlation remained high in the presence of linear changes in testing parameters over time within a location but decreased when there was large variability in daily available test kits. As the variance of testing availability increased, the correlation between TPP and confirmed-incidence time series decreased (Web Appendix 2 and Web Figure 2), as decreases in confirmed cases due to testing shortage occurred without concurrent decreases in TPP.

Effect of differential testing within risk groups

High-exposure individuals are more likely to be infected, so infectious prevalence in this group will be higher than the general population. High-susceptibility individuals are more likely to have symptoms and thus more likely to be tested, so TPP will be higher in this group. However, the relationship between confirmed incidence rate and TPP remains the same unless the groups are tested at different rates.

We found that TPP at 35 days was positively correlated with relative testing rate in high-exposure groups among symptomatic individuals (Pearson correlation = 0.71) and negatively correlated with relative testing rate among asymptomatic individuals (correlation = −0.37). In contrast we found that TPP at 35 days was negatively correlated with relative testing rate in high-exposure groups among symptomatic individuals (Pearson correlation = −0.27) and asymptomatic individuals (correlation = −0.50). High rates of testing of symptomatic individuals in this group is highly effective at reducing transmission; the clinical fraction was 0.75. Therefore, higher testing is associated with lower infectious prevalence among the high-susceptibility group, and lower TPP.

If we had data on testing rates and cases stratified by risk group (e.g., job category, age), plotting confirmed incidence against TPP for each risk group could indicate whether there were testing differences between the groups. In simulations, the TPP-incidence relationship differed by risk group when the high-risk group was tested at a higher rate (Web Appendix 2 and Web Figures 3 and 4). See Web Appendix 2 for more details.

Comparison of TPP and confirmed incidence across US states

Figure 4 shows the relationship between confirmed incidence per 10,000 and TPP by state, at 4 different times relative to the start of the epidemic in each state. The parameter values that minimize the sum of squares are plotted as a regression line. If all states had the same testing parameters and sufficient supply of test kits, we would expect them to lie on the line as in Figure 3 (gray dots).

Figure 4.

Weekly average of observed confirmed incidence (x-axis) versus test-positive proportion (y-axis) on a log–log scale. Each point represents a state, with Colorado (CO; triangle), Massachusetts (MA; square), and Rhode Island (RI; filled circle) highlighted. The trend line is fitted using equation (3) for each time point separately. Each panel represents weekly averaged values at a set time point from the first day on which prevalence was estimated to have reached 0.01% in the state: A) 28 days; B) 42 days; C) 56 days; D) 70 days. Test data are from the Covid Tracking Project (13), and population data are from the US Census Bureau (14).

States that fall below the line have a lower TPP than expected given how many cases they have observed. Figure 2 shows that there could be several reasons for this: increased prevalence of non–SARS-CoV-2 CLI; increased testing of asymptomatic individuals; or increased testing of individuals with CLI and correspondingly higher proportion of infections detected. States that fall above the line have a higher TPP than expected given how many cases they have observed. The reasons for this are the inverse of those above. In addition, as in Figure 3, there could be a shortfall in supply of test kits relative to demand, leading to a lower rate of case detection than the average state. We observed a linear relationship between confirmed incidence and TPP, except in the first panel where there is some evidence of saturation of TPP at high incidence rates experienced early in the epidemic. Most states fall close to the average, while Massachusetts and Rhode Island appear to have consistently had lower-than-average TPP. On the other hand, Colorado appears to have consistently had higher-than-average TPP. Outliers are explained by changes in reporting of test data (for example, reporting serological and polymerase chain reaction tests separately that had previously been reported together) (13).

Figure 5 shows time series for individual states: 7-day rolling-average confirmed incidence (solid line) and “TPP-estimated” 7-day average confirmed incidence (dotted line), with parameters for each state estimated using ordinary least squares. Figure 5 demonstrates that often the TPP (and thus TPP-estimated incidence rate) correlates with observed incidence. Periods of time when this is not the case are indicative of dynamic changes in testing strategy leading to more toward positive or negative individuals being tested, a change in supply of test kits relative to demand, or a change in reporting of tests in the data that might not represent changes in the number of tests being administered.

Figure 5.

Seven-day rolling-average confirmed incidence (solid line) and “test-positive proportion-estimated” 7-day rolling average confirmed incidence (dotted line), using data from the Oregon Health Authority (15) and the COVID Tracking Project (13). Each panel represents a single state: A) New York; B) Michigan; C) Illinois; D) Oregon; E) Florida; F) Arizona. Parameters for estimated incidence are fitted to each time series separately. In (D), points represent the proportion of confirmed cases that were symptomatic, plotted on the same axis.

For example, in early April the TPP in all states, particularly New York (Figure 5A) and Michigan (Figure 5B), was higher than in May onwards, adjusting for the observed incidence rate, due to tests being given preferentially to sicker or higher-risk individuals. In Illinois (Figure 5C), the rise in confirmed cases in April was accompanied by a decrease in the TPP, suggesting an expansion of testing capacity and a systematic targeting of healthier individuals. In Oregon (Figure 5D), confirmed incidence rose in June and July while TPP rose less dramatically. In addition, the proportion of confirmed cases reporting symptoms is correlated with TPP and confirmed incidence. Although not the TSP as we have defined it, this metric gives an indication of how much testing is being conducted among asymptomatic individuals. In Florida and Arizona (bottom row), the rise and fall in cases in over the summer was accompanied by a rise and fall in TPP, and thus TPP-estimated incidence rate, as we would expect if infectious prevalence were increasing then decreasing.

DISCUSSION

We have presented a simple transmission model incorporating testing of SARS-CoV-2 to derive an expression for TPP as a function of well-defined parameters. We used this expression to understand how TPP changes with the confirmed incidence as well as other parameters related to testing and the presence of other pathogens in the population. In particular, our work can be used to build hypotheses for why a location or point in time has a higher or lower TPP than expected.

When comparing TPP between locations, it is important to compare the rate of incident confirmed cases at the same time. Within the United States, earlier in the epidemic, New York and New Jersey were pointed to as examples of states that had very high TPPs compared with the country average, but this analysis shows that they were in line with the average after adjusting for the observed incidence. We showed that high variability in testing availability reduces the correlation between TPP and incidence, underscoring the need to evaluate smoothed trends in TPP.

Policies related to testing of high-risk groups in the population can drive changes in TPP. We showed that increased testing in groups with higher prevalence of infection can increase the TPP, although high testing rates can efficiently control infection and thus reduce the TPP. We note that high TPP in small subsets of the population (e.g., incarcerated populations) is unlikely to cause large bias in the overall TPP unless the background incidence or testing rate is low or the population is small (e.g., at the county level). Publicly available data on testing in different high-risk populations is critical in understanding changes in TPP. To our knowledge, the Centers for Disease Control and Prevention website is the only source of such information stratified by age in the United States (16), and this information is available only for the whole country.

Other authors have attempted to infer the population prevalence in the United States using case data and TPP (17, 18). Our approach was not to make predictions or recommend absolute thresholds for TPP but to explore the effect of varying testing parameters on TPP and its relationship with incidence; many parameters affect TPP only through their effect on incidence. We expect the relationship to be nonlinear, but this nonlinearity occurs at higher infectious prevalence than is observed in the data we have used, meaning that the model is not identifiable as we have 5 parameters to fit a single linear gradient. We note that TPP and confirmed incidence alone do not provide enough information to infer the true prevalence or incidence, and we cannot assume that simple relationships proposed between these variables will hold in different settings (e.g., comparing March with June 2020 in the United States) (19).

The disease model presented here is a simplification of the true natural history. While we briefly presented an extended model, we could have considered further extensions. For example, heterogeneous mixing and superspreading, which have been observed for SARS-CoV-2 (20), and the details of nonpharmaceutical interventions (NPIs) (e.g., which population they are applied to, variation in adherence by risk group), likely affect the transmission dynamics. While these features could affect the change in TPP over time, the relationship between TPP and confirmed incidence would be similar. For example, heterogeneity of NPI efficacy by risk group would affect transmission dynamics, but we would expect that within risk groups there would be strong correlation between TPP and incidence over time.

We included random variation in testing availability to account for short-term, unpredictable changes in testing availability. Another source of variability in data is in the reporting of tests performed, for example, due to changes in guidelines or reporting delays, leading to variation in TPP that does not reflect true rates of testing. In addition, differences in which tests are included in the numerator and denominator affect the value of the TPP (e.g., the first test per person). We did not explicitly model repeated testing of individuals and thus did not examine the differences between available TPP metrics.

The assumption that allocation of test kits is proportional to demand implies that selection bias in the sample of individuals tested is independent of the number of test kits available. It might be that, in cases of extreme restriction in testing availability, more priority is given to sicker individuals seeking testing. Therefore, deviations from the expected TPP when testing availability is limited would be more extreme than observed in our simulations.

In conclusion, we have provided intuition for how TPP changes in response to the prevalence of infectious individuals in the population as well as with various testing parameters. Unless infectious prevalence is extremely high, we expect a linear relationship between detected incidence rate and TPP across locations, and deviations from this relationship can be interpreted using the model equations. In general, if a location has higher TPP than expected, this means testing is targeted more toward sicker individuals, or there is a shortage of test kits relative to demand. Increasing TPP should be interpreted as a warning sign that transmission is increasing. Bias in TPP caused by increased testing in certain risk groups can be alleviated by breaking down testing data by group (e.g., age and occupation) and separately reporting testing in high-risk institutional settings. Finally, the proportion of tested individuals showing symptoms would provide further information to explain changes in case numbers.