Bayesian group testing with dilution effects

Summary

A Bayesian framework for group testing under dilution effects has been developed, using lattice-based models. This work has particular relevance given the pressing public health need to enhance testing capacity for coronavirus disease 2019 and future pandemics, and the need for wide-scale and repeated testing for surveillance under constantly varying conditions. The proposed Bayesian approach allows for dilution effects in group testing and for general test response distributions beyond just binary outcomes. It is shown that even under strong dilution effects, an intuitive group testing selection rule that relies on the model order structure, referred to as the Bayesian halving algorithm, has attractive optimal convergence properties. Analogous look-ahead rules that can reduce the number of stages in classification by selecting several pooled tests at a time are proposed and evaluated as well. Group testing is demonstrated to provide great savings over individual testing in the number of tests needed, even for moderately high prevalence levels. However, there is a trade-off with higher number of testing stages, and increased variability. A web-based calculator is introduced to assist in weighing these factors and to guide decisions on when and how to pool under various conditions. High-performance distributed computing methods have also been implemented for considering larger pool sizes, when savings from group testing can be even more dramatic.

1. Introduction

The group testing formulation originated by Dorfman (1943) has found use in a diverse array of applications, including coronavirus disease 2019 (COVID-19) testing. The motivating idea of group testing for a disease like COVID-19 is that, say if biomarker samples from subjects are pooled, and if the prevalence is low, most likely the test result for the pooled sample will be negative, indicating that all subjects in the pool are negative, using only one test. On the other hand, if the result indicates a positive test, and hence there is at least one positive sample present among the pool, then further testing can be conducted to identify the positive subjects. Dorfman suggested that all samples which comprise a positively tested pool should subsequently be tested individually. This approach has been adopted broadly. Nonetheless, in the presence of testing error, this approach can be problematic, in that classification error thresholds may not be reached. An important potential source of testing error is through dilution, which may occur when pooled samples contain few positives relative to a larger number of negatives. Another important aspect of COVID-19 is the constantly changing prevalence and individual risk levels, such as through seasonality, uneven vaccination rates, and the emergence of new variants. A Bayesian approach is well-suited for acknowledging and incorporating this heterogeneity into the classification process. Described here is a Bayesian framework for systematically addressing these important issues in pooled testing for COVID-19.

Recently, the Food and Drug Administration has released guidelines for allowing group testing of patients suspected of having COVID-19. There is thus a renewed flurry of interest in group testing for COVID-19 (Lohse and others, 2020; Majid and others, 2020; Hogan and others, 2020; Shental and others, 2020; Donoho and others, 2020), as it can play a fundamental role in efficient disease surveillance. As there has been a large percentage of asymptomatic cases, widespread testing is important for understanding and controlling spread. There also is a need for monitoring large swaths of the population through repeated testing, such as for front line and essential workers, and those at high risk. A potential drawback to such type of surveillance is the reliance on imprecise testing. Polymerase Chain Reaction (PCR)-based assays based on samples taken with nasal swabs can be highly accurate, although issues with sample collection or mispriming can lead to errors. These error levels also arise with pooling, and increase with pool size. We refer to this phenomenon as a dilution effect. For instance, false negative results from pooling with PCR-based assays that include one positive COVID-19 sample have respectively been estimated to range from 0.07, 0.09, and 0.19 for pools of size 5, 10, and 50 (Bateman and others, 2021). Fast, less expensive, and nonintrusive approaches for testing, such as those based on saliva, can broaden the scope of testing, but may be even less accurate. Emergence of variants also may affect testing error in the future.

A basis for the proposed group testing methods are models known as lattices, in which the classification states that represent the profiles of positivity among test subjects follow a partial ordering. This order structure is used to guide next stage pool selection, and importantly, it provides fundamental insight into the convergence properties of Bayesian group testing with dilution effects. The proposed Bayesian group testing approach explicitly acknowledges testing error and heterogeneity in individual risk levels through prior distribution specification, and can be applied with general response distributions, that is, either quantitative or categorical test responses. For COVID-19, these outcomes could be viral load level or type of variants detected. Importantly, individual-level classification error can be systematically be reduced to any error threshold, as our proposed approach has attractive optimality properties even in light of dilution effects in which correct classification can efficiently be made for all test subjects. A framework for sequential Bayesian classification on lattices has been described earlier (Tatsuoka and Ferguson, 2003). This current work differs from the previous formulation as we now consider test response distributions that depend on pool size and the number of positives in the pool. In terms of theoretical work involving partially ordered classification models, rules that rely on Kullback–Leibler information of response distributions that are specific to the experiment, such as in cognitive and educational adaptive testing, has been studied (Tatsuoka, 2014). Theoretical properties of group testing problems using partially ordered classification models also have been described (Ferguson and Tatsuoka, 2004; Tatsuoka, 2005).

Group testing has a rich statistical literature that has expanded upon Dorfman’s seminal work. An important result in group testing by Ungar (1960) gives a demarcation for when it is optimal to group test when responses are binary and there is no testing error. A Bayesian approach was first described in Sobel and Groll (1966), where binary responses without testing error are assumed. In addition to screening for disease, group testing has been used in genetics testing (Gastwirth, 2000; Sham and others, 2002), and to identify promising drug compounds (Xie and others, 2001; Remlinger and others, 2006). The phenomenon of testing error in group testing has been widely recognized (Graff and Roeloffs, 1972; Hwang, 1976; Litvak and others, 1994; Thierry-Mieg, 2006; Kim and others, 2007), including dilution effects with specific functional forms (Hwang, 1976; Zenios and Wein, 1998; Hung and Swallow, 1999). The possibility that subjects have different probabilities of being positive but with no testing error has been considered (Hwang, 1975; Black and others, 2012). Beyond binary, responses have been assumed to be normally distributed (Zenios and Wein, 1998), and to reflect viral load levels (Ghosh and others, 2020). Group testing has been studied in the context of multiplex and high throughput testing (Bilder and others, 2019; Donoho and others, 2020. In Bilder and others (2019), it is shown that taking into account heterogeneous prior probabilities can improve efficiency. Recently, methods for nonadaptive single-stage COVID-19 group testing have been suggested which are primarily based on the idea of compressed sensing (Shental and others, 2020; Ghosh and others, 2020). These methods have overlapping pools and can potentially identify the positive samples with a single stage of testing. However, they can only work well when prevalence rate is very low (1.3%), and are not necessarily convergent, as the confounding of positive cases is still possible. There also was no prior information included on samples and no dilution effect. Estimation of proportion such as for disease prevalence is an important application in group testing. Previous work has been based on binary outcomes (Sobel and Groll, 1966; Gastwirth and Hammick, 1989; Gastwirth and Johnson, 1994; Hughes-Oliver and Swallow, 1994; Brookmeyer, 1999; Tebbs, 2003).

Below, we will first introduce the proposed method and related notions, with an example. We then summarize theoretical properties for optimal designs in Bayesian group testing with general response distributions, and for an intuitive pooled test selection rule. -test look ahead analogs are presented as well, which select tests at a time, and can reduce the back and forth in preparation and testing. Finally, we introduce a web-based tool that implements the proposed algorithms and graphically represents results. Example scenarios are presented, which demonstrate the efficiency gains of group testing, and the trade-offs among the selection rules. As well, a brief note in the Appendix of the Supplementary material available at Biostatistics online is given on how Bayesian estimation of prevalence can be conducted within the proposed framework through mixtures of posteriors. This is relevant to surveillance of COVID-19. Proofs of theorems and a more thorough technical discussion are also given in the Appendix of the Supplementary material available at Biostatistics online.

2. Methods

2.1 Lattice models for Bayesian group testing

We illustrate how lattice-based classification models can be applied in Bayesian group testing with dilution effects.

Example 1

Consider all the possible subsets generated by subjects and , as in Figure 1a. Suppose each subset represents a profile of the subjects that are negative for COVID-19. Hence, one of these subsets represents the true state. The collection of these subsets thus form the classification model, and we henceforth refer to them as states: State denotes that both and are negative, state denotes that only is negative, and hence is positive. Similarly, state denotes that only is negative. State represents that both subjects are positive.

Denote the collection of these states as . In the above example, . These states (i.e., all the possible subsets of subjects that are negative) actually have a specific order structure in that they can be ordered by inclusion, and form what is known as a powerset lattice. When ordering by inclusion, note for instance that state is greater than states , , and because the set comprised of both and contains alone, alone, and , respectively. On the other hand, states and are incomparable, as there is no inclusion relationship between the respective subsets. In this example, there are two subjects, and four possible profiles of them being positive or negative.

In general, for subjects, there are such possibilities, corresponding to the powerset of the subjects. Note in Figure 1 are lattice models for and 5 subjects. Further, note that there is essentially a one-to-one correspondence between states in lattice model and potential pooled tests. There are possible way to pool tests, since the empty set, represented as , is not a possible test. So, in this example, for state , there is also a pooled test with samples from both and ; for state , there is a test consisting of a sample only from subject , etc. We denote a pooled test associated with a state as .

As the size of classification model and experiment pool grow exponentially in , significant computational challenges arise. We have implemented distributed computing and numerical algorithms that allow models with (over 33 million classification states and experiments) to feasibly be analyzed when comparing performance across algorithms. This number is further increasing with algorithmic and hardware improvements. An overview of computing approaches are given at the end of the Appendix of the Supplementary material available at Biostatistics online.

We adopt a Bayesian approach. An advantage is the allowance for individual-level prior information about positivity into the classification process, as well as systematic characterization of uncertainty from potential testing error. In a Bayesian framework, the posterior probability values embody the combined empirical and prior evidence as to which state in the classification model is the true one. Larger posterior probability values for the true state are clearly desirable, with a probability value of 1 indicating certainty in correctly identifying the true status of positivity for all the subjects. A stage of testing is an iteration of preparing a collection of (pooled) samples for testing and observing the results of the test(s). The posterior distribution after stages of testing on the collection of classification states will be denoted as , and for a state , the posterior probability value that state is the true state is . Corresponding prior distribution values are denoted as and .

For instance, prior to testing for COVID-19, each subject could be assigned a prior probability of positivity based on generalized linear regression modeling, with explanatory variables such as age, gender, location of residence, travel history, status of household contacts, community infection rates in relation to specific variants, etc. For a given profile of the subjects in terms of their positivity status, to obtain its prior probability, we take the product of the respective individual prior probabilities for their respective status in the profile. These values form a prior probability distribution across the possible states. After a (pooled) test result is observed, using Bayes rule, prior probabilities are updated to posterior probabilities. We illustrate these computations in the example below.

Example 1, continued. Suppose that for a subject the prior probability that he/she is a positive (e.g., has COVID-19) is 0.05, and independently for a higher at-risk subject it is 0.10. The prior probabilities for state membership follows: for state , , for state it is , , and . Possible tests include pooling and , and testing each subject individually. Suppose also that a test has two possible outcomes, indicating that either at least one positive is present in the pool, or that only negatives are in the pool. Assume that the specificity, the probability of observing a negative outcome given that the pool does in fact consist only of negative samples, is 0.99. Also, assume that the sensitivity, the probability of observing a positive outcome given that the pool contains at least one positive sample, also is 0.99 when all samples are positive, and reduces slightly to 0.98 when only one of two samples is positive.

Let the first test be comprised of pooled samples from and . Then, if a negative outcome is observed, it can be seen by Bayes rule that . Hence, combined with the prior information, this test result would lead to strong indication that both of the samples are negative. If instead a positive test outcome is seen, then . Suppose next that samples from and are tested individually in the next stage as in Dorfman, and the outcome for is negative, and for is positive. If we update the posterior probabilities for the two observed outcomes simultaneously, then, . In particular, the posterior probability that is positive is the sum of respective state membership probabilities that indicate that is positive: . This is clear evidence that indeed is positive. Similarly, note that the probability that is positive is , which indicates that is clearly negative.□

Making “optimal” pooled test selections becomes more interesting and complex as the number of subjects being considered for pooling increases, the individual risks of positivity vary, and as testing error increases through dilution effects. We consider the following criterion as a basis for optimality in pool selection, from a theoretical perspective. First, let be the true state (i.e., representing the correct identification of negatives and positives). Typically, converges exponentially to 0 at the order for some , and is called the rate of convergence. The mathematical definition of the rate of convergence is taken to be

We establish group testing designs and selection rules that attain the optimal rates of convergence (largest possible alpha) with probability one under general dilution effects. Note that rates are in terms of number of tests administered, with one test per stage. We show that they are also characterized in terms of Kullback–Leibler information values, which are discrepancy measures for probability distributions.

Fig. 1.

Lattice models

2.2 Dilution Effects and Optimal Designs

Note that as pool size increases, it is possible that testing outcomes can become less reliable. For instance, for a given number of positive samples, it may be increasingly harder to detect their presence as pool size increases. Another dilution effect can occur in relation to the number of positive samples in a pool. For instance, for a given pool size, it may be increasingly easier to detect the presence of positive subjects as their number increases. Such dilution effects and their effect on the optimality of pooled test selection in terms of attaining the optimal rate of convergence is characterized by Theorem A.1 in the Appendix of the Supplementary material available at Biostatistics online.

We assume the following conditions that characterize dilution effects, which are formally represented in terms of Kullback–Leibler information values (see Section A.2 in the Appendix of the Supplementary material available at Biostatistics online). We assume that (i) for a given number of positive samples, as pool size increases, pooled tests become less discriminatory in identifying the presence of positive samples; (ii) as the difference in number of positive samples between two pools increases for a given pool size, it is relatively easier to discriminate the respective test response distributions; (iii) for a given pool size and a fixed difference in the number of positive samples present, as the total number of positive samples increases, the respective test response distributions become harder to discriminate. Finally, we assume a nonrestrictive technical condition, (A4) in the Appendix of the Supplementary material available at Biostatistics online. Theoretical demarcations when it is preferable to group test depend on which state is true. We illustrate examples of demarcations in Section A.3 of the Appendix of the Supplementary material available at Biostatistics online, which show that group testing is attractive asymptotically even under strong dilution effects.

Under these conditions, optimal designs can be considered for three case: (i) all subjects are positive (the bottom state in the lattice), (ii) all subjects being tested are negative (the top state in the lattice), and (iii) there is a mix of negative and positive subjects (a state in the lattice “in-between” the top and bottom states). In Theorem A.1, we see that optimal strategies for attaining the fastest possible rates of convergence are: (i) test subjects individually and (ii) to group test samples from all subjects. For the third case, optimal strategies are more complex. The covers and anticovers of the true state (states directly above and below it) must be considered, as these states pose the most difficulty in discriminating the true state, and hence have posterior probability values that converge to zero the slowest. Under general conditions, the optimal strategy is to group test the negatives and individually test the positives. Optimal rates depend on the statistical discriminatory efficiency of the pooled tests, as measured by Kullback–Leibler information. They also depend on the complexity of the lattice around the true state, as measured by the number of states directly surrounding the true state. Respective tests are selected proportionally so that the rate at which the slowest of the nontrue state posterior probability value converges to zero is maximized. In other words, it is desirable that the posterior probabilities of the covers and anticovers converge to zero at the same rate. More technical details are discussed in Section A.3 of the Appendix of the Supplementary material available at Biostatistics online. In practice, the true state identity is not known. Next, we propose a testing selection rule that will eventually identify the true state and adopt the corresponding optimal strategy, hence attaining the optimal rate of convergence, no matter the true state.

2.3 Bayesian halving algorithm and analogous -test look ahead rules

A key to obtaining systematic efficiency gains is to base the sequential selection of pooled tests on the Bayesian information provided by the posterior probability values for the states. In Theorem A.2 of the Appendix of the Supplementary material available at Biostatistics online, we will establish that under general conditions for dilution effects, an intuitive selection rule for pooling, referred to as the Bayesian halving algorithm (BHA), attains the optimal rate for the true state posterior probability to converge to the value 1. While this is an asymptotic property, given that the rates are exponential, attaining optimal rates is practically attractive for finite testing horizons as well, and insures that any error thresholds will be attained eventually with sufficient testing, regardless of prior specification.

A key concept from order theory is the idea of an up-set for a state in the lattice. This concept is useful for motivating the proposed test selection rules. For a state , denote its up-set as . This is defined as the subset of states within the lattice that are at least as great as (i.e., contain) the state in question. In Figure 1a, ; for , for , and for it is all four of the states. Practical interpretations of these up-sets are as follows: the up-set of is all the states for which a pool of and would contain only negative subjects, the up-set of is all the states such that is negative, etc. Conversely, the complement of the up-set of is defined as , and these are the states for which a pooled test would contain at least one positive sample. Similarly, is comprised of the states for which an individual test of a sample from subject would be positive, etc. Hence, the up-set of a state and its complement generate a partition of the classification states according to whether the corresponding pooled test would contain all negatives or not.

Now let us consider rules for pooled test selection based on . It is desirable to have a rule that adopts optimal strategies and attains optimal rates of convergence of the true state posterior probability value to 1, no matter which state is true, and under dilution effects, which are likely to arise especially for large pool sizes. The following intuitive pooled test selection rule possesses these properties, as we establish with Theorem A.2 in Section A.4 of the Appendix of the Supplementary material available at Biostatistics online.

Consider the BHA, which selects the pool that minimizes

(1)

The BHA systematically partitions the lattice model, so that regardless of the actual observed outcome, through Bayes rule, one of the two partitions of states will get increased posterior probability values, while the states in the other partition will get decreased values. This systematically encourages posterior mass to eventually accumulate to one state. This property supports that purposeful and systematic pool choices will be made as test results are observed that will lead to correct classification. From Example 1, we compute . Hence, it would be attractive to first select , and to group test.

We also extend the halving algorithm to an analogous -test look ahead rule, , so that pools are selected for simultaneous testing, as opposed to the one at a time testing implicit with the proposed halving algorithm. Selecting pooled samples at a time may be more practical, as this can reduce the back and forth involved in preparing pooled samples for testing based on observed results. We define a stage of testing to be the simultaneous submission of one or more pooled samples for testing. Given at a stage , the idea behind our proposed -test version of the halving algorithm is to partition the posterior mass on the lattice model into partitions according to sets of the following form. We choose pooled samples that minimize

For example, when , this criterion involves selecting the pair and that splits the posterior mass on the lattice model into parts as equally as possible, minimizing

(2)

We can approximate the minimization in (2) by sequentially selecting the experiments one at a time. First, given current posterior probability , we choose with the halving algorithm, that minimizes (2) given and , etc. This approximation is computationally much faster.

A halving algorithm that uses prior probabilities is previously described in Black and others (2012), where samples are successively split in half for positive pooled tests while also accounting for heterogeneity in risk. This approach is related to what we propose here, but our approach is fully Bayesian, with posterior probability updating serving as the basis for selection and stopping. Litvak and others (1994) present a set of several rules that are related to the halving algorithm. However, the user has to select a rule from this set. These rules differ in the amount of re-testing of pools that is conducted. In contrast, the proposed BHA is an automated rule. It also can be used with general test response distributions.

3. Results

3.1 Simulations

For the BHA and its -test look ahead counterparts, , we have created a web-based calculator and code for high performance computing (HPC) distributed computing that generates important performance statistics relating to expected values and standard deviations of number of tests and stages, as well as classification rates, and false positive and false negative rates. The website address is https://bayesgrouptest.case.edu, and the GitHub location for the HPC implementation code is given below. These performance statistics can be used as the basis for comparison and selection of group testing strategies depending on a specific set of conditions. Being able to flexibility assess the trade-offs between different group testing strategies is important for disease surveillance, as prevalence levels and individual risks, and testing approaches and lab capacities can greatly vary over time and location. Overall performance is derived by averaging state-level expected performance with corresponding state prior probability values for lattice classification. Required inputs are a vector of prior probabilities of positivity for each subject, a matrix with the probabilities indicating presence of a positive that depend on pool size and the number of positives in the pool, and target posterior probability error thresholds for positive and negative individual-level classification. Pooled test sequences are also generated. Note that this tool currently only employs binary outcomes for pooled tests (“positive is present”, or not), and henceforth only that case is considered.

Example 2

We next consider examples with as an illustration, with varying individual prior probability risk levels: (i) the prior probability value of positivity for each subject is , (ii) all such values are 0.20, (iii) -mix denotes that -subjects have the prior probability value 0.20, while the rest have the value 0.02. We also assume a dilution effect model modified from Hung and Swallow (1999), which depends on constants and , the number of positives in a pool , denoted as , and pool size . Suppose responses are binary, with specificity , and one minus the sensitivity value

For our examples, we assume , and . As an illustration, when pool size is 12, sensitivity ranges from 0.9384 when only one subject is positive, to 0.99 when all of the 12 subjects in the pool are positive. For one positive subject in a pool of size 5, 10, and 50, sensitivity is respectively 0.9759, 0.9529, and 0.7942. These dilution effects are comparable for those found with COVID-19 PCR-based pooling Bateman and others (2021). The webtool can generate matrices of dilution effects for varying and . The dilution effect conditions required in Theorem A.2, which relate to the optimality of the BHA, are satisfied with this example. In these settings, we consider performance of individual testing, -test look ahead rules, and the BHA (also denoted as = 1).

Finally, stopping of testing is assessed at the individual level. Note that the posterior probability that a subject is negative (positive) is equal to the sum of the posterior probability values of all the states for which the subject is represented as negative (positive). In our example, an individual subject is classified and no longer considered for further testing once this posterior classification error of being negative or positive is at or below at the respective given error threshold. The negative threshold can be stricter to reduce the false negative rate. After subject-level classification, the lattice model is reduced accordingly, mainly to ease computational burden associated with the larger lattice size. Posterior probabilities are renormalized among remaining states. For instance, in Figure 1a, if is removed, then states and are merged, and are merged, etc., and posterior probabilities are correspondingly aggregated.

Simulations are based on exhaustive analyses of possible test response sequences, up to a number stages that varies by algorithm. This variation was mainly due to numerical challenges as increases, and computational time limitations. Hence, the reported averages and error levels are approximations. Note for = 1 to 4, we construct simulations up to a maximum of 24, 16, 12, and 11 stages, respectively, and up to 5 stages for individual testing simulations. The overall classification rates by the respective upper stage limits through various prior probability settings (low risks through relatively high risks, nonheterogeneous through heterogeneous) are high, for instance ranging from 99.41% up to 99.99% when the error threshold for stopping is 0.001. For test response sequences that remain unclassified after a maximum stage has been reached, for the computation of performance statistics, it is assumed that the respective test and stage numbers are one more than those last observed. Also, unless stated otherwise, it is assumed that prior probabilities of positivity correctly reflect the risk of the individuals. Lastly, for the individual testing of subjects, respective two-state lattices (negative or positive status) were employed, using the same prior risk and response probabilities, and same error thresholds as with the corresponding group testing lattice model of size . For a set of individually-tested subjects, the total number of tests are summed, and the maximum number of tests administered to any one individual is used to represent the number of stages needed in testing the subjects.

Figure 2a shows, as expected, that in terms of average number of tests needed to classify all subjects, group testing with the BHA ( = 1) has the smallest average number of tests and individual testing the largest, across all the scenarios and selection rules considered here. Indeed, the reduction achieved through group testing can be dramatic. Testing efficiency is particularly striking when the prior probability values of positivity for subjects are small. This gain in testing efficiency reduces as increases among -test look ahead rules. For the same range of priors and , other performance statistics related to expected number of stages, standard deviations of number of tests and stages, and false positive and negatives rates are graphically displayed in Figure A.1 of the Supplementary material available at Biostatistics online. In terms of the number of stages, individual testing generally has the smallest average while the case has the largest. However, the -test look ahead rule can at times have even smaller expected number of stages. The -test look ahead rules serve as a compromise between the flexibility of single test at-a-time ( = 1) group testing versus the relative logistical ease of individual testing. We only consider as this still allows for adaptability with respect to test responses, as well as numerical feasibility. Larger values of require greater computational resources in generating and analyzing possible binary testing sequences. The total number of branches grows exponentially, with base and exponent equal to the stage number. As for the standard deviations for the number of tests and stages, no clear differences arise, although for the latter, BHA is slightly elevated. For false negative rates, they are slightly higher for individual testing given stricter thresholds and -mix cases, and BHA consistently has slightly smaller error rates. Overall, false negative rates appear to increase as -values increase.

In Figure 3, classification error thresholds are set to 0.01. Note that as increases, the ratio of expected number of tests for individual testing relative to the proposed group testing rules also increases. For instance, for the case when , as increases from 6 to 16, note that for the BHA, this ratio grows from 4.33 to 6.38. These ratios also increase for , although their magnitudes lessen as increases. As the prior risk increases, these ratios and the advantage of group testing lessens, as noted in the mixed and scenarios. For instance, for the BHA, the ratios range from 1.89 to 1.96, and for , 1.21 to 1.75. Conversely, in Figure 4, the ratio of the expected number of stages with group testing versus individual testing also increases in . For instance, for the case when , as increases from 6 to 16, note that for the BHA, this ratio increases from 1.1 to 1.51. Interestingly, the rate of decrease appears slower as increases, as the ratio when is equal to 1.05 for . As prior risk increases, these ratios are larger, and the relative “cost” of group testing increases. Note when , for the BHA, the ratios range from 1.64 to 3.16. For , the ratios of expected stages are smaller. For , ratio values range from 0.64 to 0.88, which actually indicates that expected number of stages may be even less than for individual testing at lower prevalence levels.

Variability in the number of tests and stages per rule also is an important consideration. See Figure A.2 of the Supplementary material available at Biostatistics online. Note that the standard deviation in the number of stages is approximately the standard deviation for the number of tests divided by , the number of tests conducted per stage. Hence, as increases, the variability in number of stages can be even smaller than individual testing. Individual testing has lower variability in number of tests. The variability in number of tests and stages per algorithm appears to slightly increase in . Overall, for , unless the number of stages is relative inexpensive and easy to process, individual testing becomes more attractive. Still, the 4-test look ahead rule also should be considered, given that it has better expected testing efficiency and lower expected number of stages, but with higher variability. We stress that a final decision depends on “local” capacity and costs.

Fig. 2.

Comparison of expected number of tests using different classification error thresholds for stopping (0.01, 0.005, and 0.001), and under different prior probability scenarios. = 12. The first row of prior probability scenarios are for = 0.02, 0.05, 0.1, 0.15, and 0.20; the second row is for -mix, -mix, -mix, and -mix.

Fig. 3.

Ratio of expected number of tests, individual testing versus = 1 to 4. N = 6 to 16 over three prior probability scenarios. Error threshold is 0.01.

Fig. 4.

Ratio of expected number of stages, to 4. versus individual testing. to 16 over three prior probability scenarios. Error threshold is 0.01.

Example 3

In Table 1, we consider robustness with respect to the misspecification of the dilution effect and the misspecification of prior probabilities in classification. We also consider the Dorfman procedure, and the scenario when there are no dilution effects, to provide insight into the relative loss in efficiency in testing due to dilution effects more generally. Except for the Dorfman procedure results, we employ the BHA ( = 1). Note that for , the number of tests and stages are the same. Three prior probability scenarios are considered ( = 0.02, = 0.20, and -mix, which denotes that all prior probabilities are equal to 0.02 expect one with = 0.20), as well as two different stopping thresholds (0.001 and 0.01 respectively for negative and positive posterior classification error). In the examples, . Computationally, the maximum number of stages was set to 24, and the dilution effect is the same as in Example 2. “Classification ” indicates the probability that the error thresholds have been met, through 24 stages. The first column, “Dilution effect correct,” shows performance with correct specification of the dilution effect; “No dilution effect present” shows performance when there is no dilution effect, either in Bayesian updating or in the pooled test responses; “Misspecification of dilution effect” reflects when no dilution is assumed in Bayesian updating when actually the pooled test data does have a dilution effect; “Misspecification of Prior” reflects that the prior probabilities do not match the actual prevalence or risk of positivity, so that individual risk is specified as 0.2 when the actual value is 0.02, and it is specified as 0.02 when the individual risk is actually 0.20; the “Dorfman” column illustrates performance with this procedure.

First, we note that in comparing when the dilution effect and prior probability values are modeled correctly, versus when there is no dilution effect and probability values are modeled correctly, there can be longer test sequences under dilution effects. Interestingly, when one “ignores” the dilution effect, and assumes constant sensitivity (0.99 here), we can see higher false negative error rates, with test sequences in some cases appearing to be relatively shorter than correct modeling and having similar length to the actual nondilution case. Another important scenario is when the prior probabilities are misspecified. Asymptotically, theoretical convergence results as described in Theorem A.2 assure that regardless of prior specification, with sufficient testing, the correct classifications can be made. However, less stringent error thresholds for stopping can lead to short sequences when risk is underspecified, and relatively higher false negative rates. If one is unsure about prior specification, then stringent thresholds are even more important for maintaining low error rates. Moreover, note that when prior probabilities are correctly specified, testing is reduced, which reflects how the Bayesian approach can improve efficiency. We have also conducted analyses of the Dorfman procedure in the dilution effect setting, and they are presented in Table 1. Note that the Dorfman procedure either performs relatively poorly in terms of false negative rates, or does not even reach error thresholds within the maximum stage limit, so that classification rates are low. Under dilution effects, the Dorfman procedure is not generally appropriate.

Computation of performance statistics presents combinatorial challenges as , and the number of stages increases. To address these challenges and systematically evaluate simulations under a wider range of scenarios that are beyond the practical scope of the webtool in terms of computational requirements, we utilize state-of-the-art Big-Data technologies. We have designed and developed a simulation framework based on Apache Spark Zaharia and others (2010). Currently, we have conducted simulations for as large as 25, which involves 33,554,432 states in the lattice model. Using BHA, when and the error threshold for stopping is 0.001, the total classification rate with a maximum number of 30 stages is 94.72%, the false positive rate is 0.0103%, the false negative rate is 0.0212%, and the expected number of stage/tests is 5.13. The ratios of expected number of tests and stages for this scenario are approximately 5.16 (individual testing versus BHA) and 2.50 (BHA versus individual testing). For , and error threshold of 0.01, these ratios are 7.48 and 1.735. In comparing these values with those in Figure 3, where the same threshold of 0.01 was applied, note that the efficiency gains from group testing continue to increase as becomes larger.

Table 1.

Robustness analysis of dilution effect test response distribution specification and prior specification, and Dorfman procedure performance, under 3 different prior risk scenarios, with BHA.

	Dilution effect correct	No dilution effect present	Misspecification of dilution effect	Misspecification of prior probability	Dorfman
(a) 0.02
Expected tests	3.188	2.218	2.17	5.81	3.315
Expected stages	3.188	2.218	2.17	5.81	1.21
Classification %	99.9	99.99	99.99	99.99	94.97
False positive %	0.008	0.038	0.065	0.035	0
False negative %	0.01	0.025	0.134	0.0003	0.0034
Test/Stage Std Dev	0.685	0.621	0.753	0.88	1.878
(b) 1-mix
Expected tests	4.476	3.54	3.502	6.657	4.798
Expected stages	4.476	3.54	3.502	6.657	1.345
Classification %	99.96	99.98	99.97	99.99	89.08
False positive %	0.008	0.031	0.055	0.11	0
False negative %	0.009	0.026	0.129	0.04	0.0028
Test/Stage Std Dev	0.649	0.622	0.757	0.928	1.924
(c) 0.2
Expected tests	11.08	11.05	11.06	12.83	10.9
Expected stages	11.08	11.05	11.06	12.83	1.9
Classification %	99.88	99.91	99.89	92.48	23.67
False positive %	0.156	0.171	0.17	0.052	0
False negative %	0.0009	0	0.006	0.168	0
Test/Stage Std Dev	1.239	1.08	1.272	1.481	2.125

4. Conclusion

In a Bayesian group testing framework, dilution effects are considered that depend on pool size and the number of positives in a pool. It is seen that even when dilution effects are quite significant, asymptotically optimal strategies rely on group testing. Moreover, an intuitive BHA utilizes the natural order structure in the group testing problem that becomes explicit when viewed as a lattice model. Importantly, it is shown to attain optimal rates of convergence. -test look ahead analogs of this halving algorithm also are proposed, which still can take advantage of pooling and adaptability, while reducing the number of stages in testing at the expense of testing efficiency. Computational tools have been developed that allow for assessment of the proposed group testing approaches, given inputted prior probabilities of positivity per subject, dilution effects and classification error level thresholds. These tools include the use of distributed computing to address the Big Data lattice representations for larger , and they establish the feasibility and practicality of simulated group testing designs that involve performance assessments up to and beyond. Future work will focus on extending the computational feasibility for even larger . The respective expected number of tests and stages (and standard deviations), classification error, and comparisons to individual testing are generated. Balancing the expected number of tests versus stages, as well as the respective variances and error rates, can be weighed across different -test look ahead rules, and compared against individual testing. Deciding when to group test depends on the relative expense of tests versus stages, as well as the other performance statistics, and preferences may vary across testing sites.

Detailed discussion on the estimation of dilution effects is deferred here, as approaches will depend on the test response distribution model. Note that this framework can seamlessly incorporate more complex outcomes. In the future, it will be of interest to see if such outcomes can provide formalized insight into the precise number of positive subjects within a pool, as opposed to just applying binary thresholds that indicate presence only. It also could be possible to consider estimating response probabilities with uncertain disease status (reflected by posterior probabilities in classification) using Bayesian latent class estimation approaches that can include nonparametric density estimation (Tatsuoka, 2002; Tatsuoka and others, 2013). Varied and well-characterized pooled test data should become more accessible as group testing becomes more widespread through population-level surveillance. Finally, we conjecture that it will be possible to embed BHA-based testing sequences in high-volume, automated workflows, which will reduce the practical burden of requiring higher number of stages.

5. Software

Source code available at https://github.com/ben072292/bayesgrouptestbackendhttps://github.com/ben072292/bayesgrouptestfrontend The web tool referenced in Section 3 is available at https://bayesgrouptest.case.edu.

Supplementary material

Supplementary material is available at http://biostatistics.oxfordjournals.org.

Funding

NSF DRL-1561716, R01 MH65538, NSF CCF-2132049, and UL1TR002548.