Designing and Interpreting Limiting Dilution Assays: General Principles and Applications to the Latent Reservoir for Human Immunodeficiency Virus-1

Limiting dilution assays are widely used in infectious disease research. These assays are crucial for current HIV-1 cure research in particular. Here we offer new tools to help investigators design and analyze dilution assays based on their specific research needs.

Abstract

Limiting dilution assays are widely used in infectious disease research. These assays are crucial for current human immunodeficiency virus (HIV)-1 cure research in particular. In this study, we offer new tools to help investigators design and analyze dilution assays based on their specific research needs. Limiting dilution assays are commonly used to measure the extent of infection, and in the context of HIV they represent an essential tool for studying latency and potential curative strategies. Yet standard assay designs may not discern whether an intervention reduces an already miniscule latent infection. This review addresses challenges arising in this setting and in the general use of dilution assays. We illustrate the major statistical method for estimating frequency of infectious units from assay results, and we offer an online tool for computing this estimate. We recommend a procedure for customizing assay design to achieve desired sensitivity and precision goals, subject to experimental constraints. We consider experiments in which no viral outgrowth is observed and explain how using alternatives to viral outgrowth may make measurement of HIV latency more efficient. Finally, we discuss how biological complications, such as probabilistic growth of small infections, alter interpretations of experimental results.

Human immunodeficiency virus (HIV)-1 infection persists in patients despite decades of effective antiretroviral therapy (ART), necessitating lifelong treatment. Limiting dilution viral outgrowth assays provide an essential tool for studying this residual infection. These assays detect latently infected resting CD4⁺ T cells by probing their ability to produce virus that infects other cells in culture following cellular activation [1, 2]. This latent reservoir of infected cells harbors stably integrated HIV-1 DNA coding for replication-competent virus and, as such, is believed to be the major barrier to curing HIV infection. Because replication-competent proviruses cannot be distinguished by polymerase chain reaction methods from the much more common defective proviruses (coding for virus that fails to replicate) [3–5], viral outgrowth remains the gold-standard marker of latent infection. Dilution assay statistics are used to estimate the frequency of latently infected cells from binary results of outgrowth experiments [6]. Likelihood-based dilution assay statistics have been part of the microbiologist's basic toolkit for nearly a century [7], and today they are indispensable to HIV-1 cure research.

Recent reports of interventions that may reduce the latent reservoir [8–14] have renewed optimism for achieving HIV-1 cure or long-term antiretroviral-free remission. Potential curative strategies are now being evaluated by how much they reduce reservoir size. Therefore, it is essential to improve understanding of statistics used in outgrowth assays and to establish a consistent approach for reporting the precision of the resulting measurements. Towards this goal, we present principles for the design and interpretation of dilution assays, discuss these principles in the context of HIV-1 latency, and offer a simple computational tool for estimating the frequency of latently infected cells in a sample (“infectious units per million” or “IUPM”) from assay results. The same statistical approach—and same program—can be used for a broad range of dilution assays designed to quantify viral reservoirs.

In a dilution assay, wells of a tissue culture plate are seeded with varying amounts of material from a presumed-infectious source, together with infectable reporter cells. In the viral outgrowth assay for latent HIV-1 [1, 2, 6], the infectious material is a known quantity of purified resting CD4⁺ T cells obtained from an HIV-1-infected donor. A very small fraction of these cells carries a latent HIV-1 genome that can cause replication-competent virus after cellular activation. A larger fraction of the cells carry defective viral genomes and are ignored in this analysis. Thus, throughout this paper, we refer to latently infected cells that are stimulated to release replication-competent virus in the assay simply as “infected cells.”

The input cell number in each well is typically varied in geometric series of successive dilutions. Sufficient dilutions are carried out so that some wells contain no infected cells. Uninfected lymphocytes (either from a MOLT-4/CCR5 T-cell line or phytohemagglutinin-stimulated, CD8-depleted lymphoblasts from uninfected donors [15]) are added to each well to serve as the reporter system, so that virus released from a single cell can spread through the culture and produce a detectable level of infection. The frequency of infected cells in the original population of resting CD4⁺ T cells, as well as a confidence interval (CI) around this frequency, are then estimated based on the pattern of wells in which viral outgrowth was observed. If desired, this fraction can be scaled by an appropriate number (approximately 10¹¹ to 10¹² total resting CD4⁺ T cells in a human body [16]) to estimate total latent reservoir size.

Before discussing details of assay design and statistical methods to estimate infected cell frequency, we note two issues inherent in this type of assay. First, the sensitivity—or lower limit of detection (LLD)—of the assay depends on the total number of cells sampled. If C resting CD4⁺ T cells are distributed among all the wells, then it is impossible to distinguish between infected cell frequencies near or less than 1/C. Precisely at this frequency, assuming perfect experimental conditions, at least one well will show outgrowth approximately 63% of the time, whereas the remaining 37% of the time no wells will turn positive. Therefore, to reliably distinguish a frequency of 0.1 IUPM from a nonexistent infection, one would need to assay a minimum of 10 million cells, and ideally several times that number. The section, Dealing with all-negative data, further discusses lower limits of detection.

Second, the distribution of cells into wells determines the precision of the assay, that is, its ability to distinguish between different infected cell frequencies. Precision can be described by CIs: more precise assays have smaller intervals, allowing for finer-grained comparisons between experimental conditions. Figure 1 illustrates how increasing the number of wells in an assay can improve its precision. Confidence intervals are discussed more generally in the next section.

Figure 1.

Precision of dilution assays increases with the number of wells into which a sample is divided. Consider two samples of the same size (total number of cells) collected from different donors. Sample A contains 2 latently infected cells, whereas Sample B contains 8 (ovals at left). Five possible assay setups are shown, distributing each sample into 1 to 5 equal wells (columns). Each assay results in a different pattern of possible outcomes for the samples, with the probability of each outcome shown in rows. The final row shows the probability that each assay yields the correct trend, ie, that more wells turn positive in Sample B than in Sample A. Positive wells (viral outgrowth) are shown in dark red. If all cells from a sample are deposited into a single well (first column), then it is impossible to distinguish between the two samples using a binary readout: the extra infected cells in Sample B are “redundant,” and each sample will show outgrowth. If the samples are divided among more wells (second through fifth columns), then the infected cells are less likely to appear redundantly in the same well, meaning that the two samples are more likely to show different outgrowth patterns. In this particular case, each sample must be divided into at least 4 equal wells to obtain the correct trend at least 95% of the time.

Sensitivity requires many cells, and precision requires distribution into multiple wells. Meeting both goals presents experimental challenges. Finally, we present likelihood statistics to estimate infected cell frequency, and we recommend an approach for optimizing experimental design to achieve any desired set of sensitivity and precision goals. Although our discussion focuses on HIV-1 infection, the same approach applies to any setting where frequency of infected cells is measured.

LIKELIHOOD STATISTICS PROVIDE A FLEXIBLE APPROACH FOR ESTIMATING INFECTED CELL FREQUENCIES

The most common method for analyzing limiting dilution assays involves maximum likelihood estimation of infected cell frequencies. This general method can be used for any experiment in which the observed outcome depends on one or more unknown parameters in a probabilistic way. Here, the only unknown parameter is the infected cell frequency, and the maximum likelihood method provides both a central estimate and CI for this frequency. If the infected cell frequency lies within the assay's limits of detection, then the maximum likelihood estimate is expected to be quantitatively meaningful, falling between but not equal to the extreme values 0 IUPM and 1 000 000 IUPM. Furthermore, this estimate will be precise —with CIs smaller than a specified size — for frequencies within the assay's (smaller or coinciding) limits of quantification. These concepts are further explained in Figure 2. In general, the size of the CI varies even for frequencies between these limits, and so CIs must be reported explicitly to allow comparison of infected cell frequency over time or among patients.

Figure 2.

Schematic for describing quality of maximum-likelihood estimates of infected cell frequency. The thick blue line plots the typical value (median or geometric mean) of the measured infected cell frequency $\hat{\theta }$ (y-axis) vs true frequency θ (x-axis), for an idealized experimental setup; this line is dashed outside the limits of detection for the assay. These limits are defined for a significance level α (eg, 0.05). Below the lower limit of detection (LLD), at least α of experiments result in all wells being negative. Above the upper limit of detection (ULD), at least α of experiments result in all wells being positive. In an accurate assay, the estimate tracks the diagonal line $\hat{\theta }=\theta$ (light gray dashed line) between the limits of detection. The blue shaded region plots the confidence interval (CI), within which the middle (1 – α) of estimates fall. Between the lower and upper limits of quantification (LLQ, ULQ), the CI is smaller than a specified size, and a fraction less than α of experiments result in all-positive or all-negative wells. Note that it is possible for LLD = LLQ and/or ULD = ULQ.

The basic principles of maximum likelihood analysis for dilution assays have been well understood since the work of Fisher [7] (see also [17]). These principles have also been extended to a Bayesian framework, which explicitly accounts for how prior knowledge alters interpretation of experimental results [18]. A study by Meyers et al [6] presented a complete analysis of maximum likelihood estimates and CIs resulting from all possible experimental outcomes of a particular 5-fold limiting dilution assay design with two replicates per dilution. This limiting dilution scheme was used in early studies of HIV-1 latency [1] for a particular reason. The low frequency of latently infected cells necessitates sampling a large number of cells, but blood volume concerns limit the number of replicate wells that can be done with higher input cell numbers. This assay design is widely used in standard assays of the latent reservoir [19], but it may not be optimal for all experimental situations. In particular, infected cell frequencies may lie outside this assay's limits of detection or quantification, and CIs may be too large to address particular research questions. Fortunately, custom assays can be designed to satisfy any desired limits and CI sizes; in Box 1, we present a simple procedure for doing so. Table 1 shows several recommended designs resulting from this procedure, and Box 2 gives the step-by-step calculation for deriving one of these designs.

Box 1. Procedure for designing a limiting dilution assay.

In designing an assay, the experimenter must choose a range of input cell numbers and the number of replicate wells to set up at each input cell number. This choice determines the lower and upper limits of detection (LLD, ULD), lower and upper limits of quantification (LLQ, ULQ), and confidence interval (CI) size ϵ (expressed as a number of base-10 logs). The limits of detection determine the range in which the assay rarely produces all-positive or all-negative results. Between the limits of quantification, the assay furthermore can distinguish infected cell frequencies up to a range of ϵ logs. Figure 2 further explains these concepts. Below, z_α refers to the critical normal distribution value corresponding to significance level α (eg, $z_{0.05}=1.96$, $z_{0.01}=2.58$).

The following procedure is guaranteed to produce an assay design with specified limits $LLD\le LLQ<ULQ\le ULD$ and maximum CI size ϵ, for a significance level α, aiming to use the fewest wells necessary (derivation in Supplementary Information). Box 2 provides a step-by-step application of this procedure to obtain one of the assay designs described in Table 1.

1. Select desired limits and maximum CI size. Experimental requirements may constrain feasible values:

   1. If C cells are available to distribute across all wells, then the lower limit of detection is constrained: LLD ≥ ln(1/α)/C.
   2. If the experimenter wishes to perform at most n replicate wells per input cell number, then the CI size is constrained: $ϵ\ge 1.08z_{\alpha }/\sqrt{n}$.
   3. If C cells are available and the experimenter wishes to perform at most n replicate wells per input cell number, then in addition to the above constraints on LLD and $ϵ$, we have $LLQ\ge 1.99n/C$.
   4. If C cells are available and the experimenter wishes to guarantee a CI size of at most ε, then in addition to the above constraint on LLD, we have $LLQ\ge (2.31z_{\alpha }^2)/(C{ϵ}^2)$.
   5. If the experimenter wishes to use only d distinct input cell numbers, that constrains the width of the region between the limits of quantification: $ULQ/\text{LLQ}\le 5^{d-1}$.
2. To guarantee that the CI is at most ϵ logs for infected cell frequencies within the limits of quantification, choose a limiting dilution series in which

   1. Maximum input cell number c_max is at least $1.59/LLQ$ cells,
   2. Minimum input cell number c_min is at most $1.59/ULQ$ cells,
   3. Each input cell number differs by no more than a factor of 5 from the previous one, and
   4. Each input cell number has at least $1.16\times (z_{\alpha }/ϵ{)}^2$ replicates.
3. To guarantee the desired limits of detection, further require

   1. The total number of cells across all wells is at least $\text{ln(}1/\alpha )/LLD$, and
   2. The smallest input cell number is at most $\ln (1/(1-{\alpha }^{1/n}))/ULD$, where n is the number of replicate wells of that cell number.

Often, the dilution series provided by Step 2 will also satisfy the conditions in Step 3 and may provide limits of detection even farther apart than required. In this case, nothing needs to be changed. However, if the dilution series does not meet condition 3A, wells of any input cell number can be added to make up the difference in total cells. To meet condition 3B, either the number of replicates for the smallest well may be increased or wells of a smaller input cell number may be added. It is possible for this condition to require a few additional wells beyond that specified in the experimental constraints in Step 1.

This procedure may yield input cell numbers that are impractically large for loading into wells. From the standpoint of ensuring the required limits and CI sizes, it is always acceptable to replace a well containing many input cells by multiple wells containing fewer input cells each, keeping the same total number of cells. For instance, replacing 2 wells of 10 million cells apiece by 10 wells of 2 million cells apiece can only serve to increase precision of the maximum likelihood estimate. If it is experimentally convenient to have all wells with the same input cell number, then this principle can likewise be used to do so.

Table 1.

Recommended Dilution Assay Designs Satisfying a Range of Experimental Requirements as Provided in Box 1^a

Experimental Requirements				Resulting Assay Design
Limits of Detection, Quantification (IUPM)			CI Size (Log10)	No. of Different Cell Input Numbers (Obtained by 5-Fold Dilution)	Largest Well (Input Cells)	Smallest Well (Input Cells)	No. of Replicates (Largest to Smallest)	Total Cells (Millions)
LLD	LLQ	ULQ = ULD
1.2	1.6	800	<1.5	6	1 × 106	320	2 each	2.5
0.48	1.6	800	<1.0	6	1 × 106	320	5 each	6.2
0.14	1.6	800	<0.5	5	1 × 106	1600	18 each	22
0.21	0.50	12.5	<1.5	4	3.2 × 106	2.56 × 104	(4, 2, 2, 3)	14
0.15	0.50	12.5	<1.0	4	3.2 × 106	2.56 × 104	5 each	20
0.04	0.50	12.5	<0.5	3	3.2 × 106	1.28 × 105	18 each	71
0.04	0.10	12.5	<1.5	5	1.6 × 107	2.56 × 104	(4, 2, 2, 2, 3)	72
0.03	0.10	12.5	<1.0	5	1.6 × 107	2.56 × 104	5 each	100
0.008	0.10	12.5	<0.5	4	1.6 × 107	1.28 × 105	18 each	360
0.008	0.02	12.5	<1.5	6	8 × 107	2.56 × 104	(4, 2, 2, 2, 2, 3)	360
0.006	0.02	12.5	<1.0	6	8 × 107	2.56 × 104	5 each	500
0.002	0.02	12.5	<0.5	5	8 × 107	1.28 × 105	18 each	1800

Abbreviations: CI, confidence interval; IUPM, infectious units per million; LLD, lower limit of detection; LLQ, lower limit of quantification; ULD, upper limit of detection; ULQ, upper limit of quantification.

^a Assay designs use a significance level α = 0.05, corresponding to 95% CIs. The dilution series commonly used in HIV-1 latency studies [1, 6, 19] is shown on the first row. Simulated performance of the two rows in bold is shown in Figure 3. Because decreasing the LLD and CI size each involves using more cells according to the procedure in Box 1, these two requirements are decreased together in this table.

Box 2. Designing a limiting dilution assay: case study.

Here, we show how the procedure in Box 1 can be used to create the assay design on line 11 of Table 1. Suppose that 500 million purified resting CD4⁺ T cells are available from each HIV-infected individual, we wish to distinguish 10-fold differences in infected cell frequency with 95% confidence, and we expect never to encounter infected cell frequencies above 12.5 infectious units per million (IUPM) in the experiment. We follow the three steps in Box 1:

1. Select desired limits and maximum confidence interval (CI) size. The cell constraint is C = 500 million cells. To distinguish 10-fold differences, the desired CI size is $ϵ={\text{log}}_{10}10=1$. The significance level is α = 0.05 (95% confidence), and $z_{\alpha }=1.96$ is the associated critical normal distribution value. These values imply the following:

   • Constraint 1A applies: the minimum feasible lower limit of detection (LLD) is ln(1/α)/C = ln(1/0.05)/(5 × 10⁸) = 6 × 10⁻⁹ per cell, or 0.006 IUPM,
   • Constraint 1D applies: the minimum feasible lower limit of quantification (LLQ) is $(2.31z_{\alpha }^2)/(C{ϵ}^2)=(2.31\times {1.96}^2)/(5\times {10}^8\times 1^2)=2\times {10}^{-8}$ per cell, or 0.02 IUPM.

To use the available cells fully, we set the lower limits to their minimum feasible values. Based on our belief in the maximum possible infected cell frequency, we set both upper limits to $1.25\times {10}^{-5}$ per cell, or 12.5 IUPM.

1. To guarantee the limits of quantification:

   1. Maximum input cell number c_max is at least $1.59/LLQ=1.59/(2\times {10}^{-8})=8\times {10}^7$ cells,
   2. Minimum input cell number c_min is at most $1.59/ULQ=1.59/(1.25\times {10}^{-5})=1.3\times {10}^5$ cells,
   3. At least four 5-fold dilutions are needed to span the input cell numbers (the ratio c_max/c_min is at least 625, or 5⁴),
   4. Each input cell number has at least $1.16\times (z_{\alpha }/ϵ){\hspace{0.17em}}^2=1.16\times (1.96/1{)}^2=4.5$ replicates.

These requirements can generally be satisfied by setting c_max equal to its minimum required value and performing the minimum required number of dilutions. Doing so yields a provisional assay design with input cell numbers $[8\times {10}^7,1.6\times {10}^7,3.2\times {10}^6,6.4\times {10}^5,1.28\times {10}^5]$ and n = 5 replicates for each input cell number.

1. To guarantee the desired limits of detection:

   1. The total number of cells across all wells is $\text{ln(}1/\alpha )/LLD=\text{ln(}1/0.05)/(6\times {10}^{-9})=500$ million cells, which is satisfied by the provisional assay design,
   2. The smallest input cell number is at most $\text{ln(}1/(1-{\alpha }^{1/n}))/ULD=\text{ln(}1/(1-{0.05}^{1/5}))/(1.25\times {10}^{-5})=6.4\times {10}^4$ cells, which is not satisfied by the provisional assay design.

To meet the requirement in Step 3B, we must perform one additional 5-fold dilution beyond that specified in the provisional design. The final assay design has input cell numbers $[8\times {10}^7,1.6\times {10}^7,3.2\times {10}^6,6.4\times {10}^5,1.28\times {10}^5,2.56\times {10}^4]$, with 5 replicates for each input cell number. Because the final dilution results in an input cell number smaller than the required $6.4\times {10}^4$, the assay has somewhat more power at the upper end than originally required (ULQ = ULD = 31 IUPM, instead of 12.5 IUPM). In this case, one could reduce the number of replicates for the smallest input cell number to 3 and still meet the original requirement.

Formally, maximum likelihood estimation of infected cell frequency relies on four assumptions justifying the use of Poisson distributions to approximate the number of infected cells in each well. First, cells in the sample are drawn from a much larger total population (eg, 50 million cells sampled from approximately 10¹¹ to 10¹² total resting CD4⁺ T cells in the body). Second, the infected cell frequency is small (eg, frequency less than 1 in 100 ensures approximation error less than 0.5%). Third, infected cells are distributed randomly among all replicate wells. Fourth, any well containing one or more infected cells is guaranteed to test positive.

Together, these assumptions constitute the “single-hit Poisson model.” In this model, the probability that a well with c cells tests positive, given an infected cell frequency θ, is $1-e^{-c\theta }$. The number of positive wells of a given input cell number c follows the binomial distribution specified in Figure 3. When wells of different input cell numbers $c_1,c_2,c_3,\dots$ are used, the results from each are independent, and so the probability of an observed experimental result is simply the product of individual binomial probabilities. The program IUPMStats (http://silicianolab.johnshopkins.edu) uses this calculation to estimate the infected cell frequency θ, as well as 95% CIs, from user-specified assay results. Figure 3 shows the performance of this method for two of the assay designs proposed in Table 1. Additional software tools using related methods are discussed by Hu and Smyth [20].

Figure 3.

Performance of the maximum likelihood estimator in simulations, using the two assay designs highlighted in Table 1. Left column: assay on row 4 (confidence interval [CI] < 1.5 logs); right column: assay on row 9 (CI < .5 logs). (A) Maximum likelihood estimate $\hat{\theta }$ and 95% CI plotted in blue; the diagonal line shows the case of a perfect unbiased estimator. Asymptotic CIs are reported, using likelihood of log-transformed θ. (B) Bias $\hat{\theta }/\theta$ plotted in blue; the horizontal line at 1 shows the case of a perfect unbiased estimator. (C) Size of the estimated 95% CI plotted in black (note different y-axis scales). (D) Binomial probability expression used to estimate infection frequency, assuming n replicate wells of c cells apiece. According to this expression, the probability that all wells are negative equals e^−Cθ, where C is the total number of cells across all replicate wells. A–C plot the actual infection frequency $\theta$ used in simulations on the x-axis. Each point on the curves is the geometric mean (A and B) or arithmetic mean (C) of 20 000 replicate simulations using the same $\theta$ (step size 0.025 logs). Curves are solid where <5% of simulated assays yield all-negative or all-positive results, dashed at 5%–50%, and not shown at >50%. Blue (A and B) or gray (C) shaded regions show the middle 95% of simulations; jaggedness results from the discrete nature of the dilution assay. Left to right in each panel, the thin vertical lines show the LLD, the LLQ, and the ULQ = ULD. Note that the assay shown at right is more sensitive (20-fold lower LLD), more precise (narrower shaded regions in A and B, smaller CI in C), and more accurate (curve in A better tracks the diagonal; curve in B better tracks the horizontal).

DEALING WITH ALL-NEGATIVE DATA: ABSENCE OF EVIDENCE IS NOT EVIDENCE OF ABSENCE

If all wells in an assay test negative for viral growth, then the maximum likelihood estimate of infected cell frequency is zero. Yet this is not a useful estimate; it is possible for some of a patient's cells to be infected $(\theta >0)$, but for none of these infected cells to be sampled. The probability of this all-negative outcome follows from the equation in Figure 3D.

To avoid this outcome, the procedure in Box 1 prescribes an assay design based on a specified LLD. The total number of cells C is chosen so that the probability of finding an all-negative result is small (eg, α = 0.05) if θ = LLD. As θ falls below the LLD, the probability of an all-negative result increases, making it difficult to distinguish between infection frequencies below this limit. For significance level α, the LLD equals $\text{ln(}1/\alpha )/C$. For example, if we want to be nearly certain (>95%) of detecting infection at frequencies as low as 0.01 IUPM, then 300 million cells are needed.

When an all-negative result does occur, however, some estimate for infected cell frequency is needed. Under conservative assumptions (uniform Bayesian prior on infected cell frequency between 0 and 1 000 000 IUPM), and for sufficiently many total cells C (>500), this result implies, with probability $(1-\alpha )$, that the infected cell frequency falls below the LLD. The LLD, then, is a conservative upper bound for the estimated frequency. If a more central estimate is needed, then under the same assumptions, there is a 50% chance that infected cell frequency is less than $\text{ln(2)}/C$; this value is called the “posterior median” estimate. For infected cell frequencies near this value, longitudinal samples may switch between testing all-negative and having some wells positive, even if the true frequency remains constant over time.

It is important not to overinterpret these statements: the LLD is not a “hard cutoff” below which the assay is useless. Furthermore, it is possible for the maximum likelihood estimate $\hat{\theta }$ to be below the LLD even if some wells are positive. One should therefore take care (1) when drawing conclusions from negative results or (2) when comparing negative results to other experiments where $\hat{\theta }$ is near the LLD. In general, formal hypothesis testing is required when comparing estimates near limits of detection. The online tool, ELDA, provides a method for comparing estimates between two or more assays [20].

It is likewise important to stress that the absence of evidence is not evidence of absence: the most sobering recent demonstration of this principle is the case of the “Mississippi Child,” who underwent combination ART for the first 18 months of life after being infected with HIV-1 in utero. Despite subsequent interruption of therapy, viral rebound did not occur immediately and repeated outgrowth assays found no replication-competent virus, raising tentative hopes that early aggressive treatment succeeded in preventing establishment of the latent reservoir [11]. These hopes were dashed when the virus rebounded after 28 months off therapy [14]. Four separate viral outgrowth assays were done throughout the interruption, with a total of 66 million resting CD4⁺ T cells analyzed. Because the latent reservoir decays very slowly [2, 21], it is reasonable to combine negative assay results and estimate that, with 95% probability, there were fewer than $\text{ln(}1/0.05)/66=0.045$ infectious units per million cells [14]. Consistent with the observed viral rebound, mathematical modeling of infection dynamics suggests that even this low frequency is insufficient to make cure likely [22].

Further complicating interpretation of all-negative results, not every resting CD4⁺ T cell that harbors replication-competent virus necessarily produces outgrowth in an assay. By one estimate based on study of adults undergoing long-term ART, for each latently infected cell detected via outgrowth, up to 60 more cells may go undetected [3]. It is not yet known whether this “invisible reservoir” generally exists in early-treated children, and the analysis in the preceding paragraph suggests that it is not necessary to invoke this additional infection source to explain the Mississippi Child's rebound viremia. This complication of viral outgrowth assays implies that an IUPM estimate may not translate directly to a literal number of infected cells but rather serves as statistical proxy for this quantity. We take up this issue further in the section, When the assumptions fail.

NEW DILUTION ASSAYS TO MEASURE VIRAL RESERVOIR: BETTER SENSITIVITY FOR EQUAL CELL INPUT?

Given the time, labor, and large cell input required for sensitive measurement of latent HIV infection, there is great interest in the development of assays to quantify HIV reservoirs more efficiently. One recent innovation is the Tat/Rev Induced Limiting Dilution Assay (TILDA), which measures cell-associated multiply-spliced HIV-1 RNA (msRNA) transcription in serial dilutions of maximally stimulated CD4⁺ T cells [23]. As with outgrowth assays, TILDA uses dilutions of a known number of cells distributed across replicate wells and gives a binary readout. Consequently, the same maximum likelihood approach — and the IUPMStats program — can be used. Unlike outgrowth assays, however, the use of msRNA measurement in TILDA eliminates the need for infectable reporter cells and permits a rapid turnaround time (days, versus weeks for outgrowth assays). Because TILDA measures an upstream process that is necessary [24–26], but not sufficient for assembly of replication-competent virus, it is not surprising that the number of msRNA-producing units per million (“msRUPM”) exceeds outgrowth-based IUPM in individuals receiving ART [23]. As a result, TILDA (and other assays of upstream processes) will appear more sensitive than outgrowth, in the sense that fewer total input cells are needed to detect at least one positive well. Yet it remains to be shown whether msRUPM or related measurements are clinically useful proxies for the size of the replication-competent latent reservoir, given that this assay may detect mRNA production from cells with defective proviruses [3].

WHEN THE ASSUMPTIONS FAIL

As described in the above section on maximum likelihood statistics, the statistical framework for estimating infected cell frequency relies on several assumptions. One way in which these assumptions may fail is for each infected cell to cause a positive result with probability less than one. There are several ways that a latently infected cell, capable of releasing replication-competent virus upon stimulation, may nonetheless fail to trigger outgrowth. For instance, variation in host gene expression may stall initiation of HIV transcription [3, 27]. Furthermore, even if transcription starts, noisy transactivation feedback circuits may fail to cause elongation of viral transcripts [3, 28–30]. Some stimuli increase HIV-1 gene expression but do not result in packaging and export of virus particles [13, 31]. Finally, the virions produced may fail to establish an exponentially growing infection [22]. However, these types of failure do not change the basic statistical framework of the assay, only its interpretation. In each scenario described, functionally intact provirus escapes detection in the outgrowth assay. In this light, the quantity being estimated in the assay is a weighted sum of outgrowth probabilities, considering all cells carrying potentially inducible replication-competent proviruses. For example, a measured frequency $\hat{\theta }=2$ IUPM is consistent with a population where 1 per million cells have 100% chance of outgrowth, while an additional 10 per million have 10% chance of outgrowth. Therefore, the “infectious unit” of IUPM refers to a statistical quantity causing viral outgrowth under laboratory conditions that may consist of many infected cells. This interpretation of IUPM follows common usage in the study of infectious diseases, which treats an infectious unit as the quantity of pathogen causing a single infection in expectation [32], in contrast to the less frequent usage treating it as the minimum quantity capable of causing infection [33]. Sensitivity of IUPM measurements to assay design further attests to the chance nature of outgrowth. Although rarely discussed, this sensitivity is tacitly documented in the literature by experimental protocols that recommend longer periods of monitoring for outgrowth if surprisingly few wells turn positive [19].

A second type of failure arises if infected cells are not distributed randomly among wells, which may occur, for instance, if tissue from an infected donor is not fully dissociated. In this case, infected cells may cluster in the same well, causing underestimates in infected cell frequency. Poor mixing may also lead to counterintuitive assay results—eg, positive results at small input cell numbers but not at large numbers. Tests for goodness of fit to the single-hit Poisson model can flag such outcomes [6]; an approximate χ² test is included for this purpose in IUPMStats. Standard experimental procedures are designed to ensure random mixing of cells and avoid these outcomes.

Third and most troubling is the possibility that outgrowth depends on the number of infected cells in a nonindependent way. There are several mechanisms by which multiple infected cells occupying the same well may interact synergistically to increase the probability of detectable outgrowth. Human immunodeficiency virus, like both simian and murine retroviruses, can saturate host restriction factors at high viral concentrations, increasing the probability that a cell is successfully infected [34]. In addition, sustained viral production depends on intracellular concentration of the viral protein Tat [28, 29], which may be able to cross cell membranes and reach critical concentrations more readily when multiple infected cells are present [35]. If these concentration-dependent mechanisms occur under experimental conditions, then estimated infected frequency $\hat{\theta }$ may increase if input cells are plated at higher concentrations. In this case, a more complex “multi-hit” model would be needed to estimate the size of the latent reservoir.

Accurate measurement of the latent infection in HIV continues to be a major challenge. As the field continues to develop therapies to reduce latency, principled design and interpretation of dilution assays will become increasingly important in prioritizing these therapies for clinical trial. We hope that our online tool (http://silicianolab.johnshopkins.edu) can assist the field in this task.

Definitions

All-positive (all-negative) results occur if all wells in an assay test positive (negative). These results are the least informative for estimating the infected cell frequency. Assays generally should be designed to avoid these results.

Limits of detection are defined for a particular assay design and a desired significance level α (typically 0.05). If the true infected cell frequency lies between the limits of detection, then with probability at least (1–α), the assay produces neither an all-positive nor an all-negative result.

The confidence interval (CI) for a given significance level α is a range that is computed from the results of an assay. If the assays were repeated many times on samples with the same true infected cell frequency, then the CI would include the true value (1–α) of the time.

Limits of quantification are defined for a particular assay design, a desired significance level α, and a desired CI size (expressed as a number of base-10 logs). If the true infected cell frequency lies between the limits of quantification, then with probability at least (1–α), the assay produces neither an all-positive nor an all-negative result, and the CI is smaller than the desired size.

Sensitivity is an assay's ability to detect very low infected cell frequencies. It is measured by the lower limit of detection.

Precision is an assay's ability to distinguish between similar infected cell frequencies. It is measured by the CI size. Outside the limits of quantification, an assay tends to be less precise (larger CI size).

Bias is the tendency of an assay to report estimated infected cell frequencies higher or lower than the true value. A perfectly unbiased (or accurate) assay produces estimates that are symmetrically distributed around the true value. Figure 3B measures bias as the estimated frequency divided by the true frequency; a value near 1 indicates lack of bias.

Supplementary Material

Supplementary material is available online at Open Forum Infectious Diseases (http://OpenForumInfectiousDiseases.oxfordjournals.org/).