Randomized gates eliminate bias in sort‐seq assays

Abstract

Sort‐seq assays are a staple of the biological engineering toolkit, allowing researchers to profile many groups of cells based on any characteristic that can be tied to fluorescence. However, current approaches, which segregate cells into bins deterministically based on their measured fluorescence, introduce systematic bias. We describe a surprising result: one can obtain unbiased estimates by incorporating randomness into sorting. We validate this approach in simulation and experimentally, and describe extensions for both estimating group level variances and for using multi‐bin sorters.

1. INTRODUCTION

Quantitative, multiplexed assays relying on fluorescence activated cell sorting (FACS) followed by high‐throughput sequencing are critical to modern biology and molecular engineering because they enable construction of large scale datasets connecting sequence to function. For example, these “sort‐seq” assays are widely used to profile the strength of protein–protein binding interactions via yeast display. ¹ , ² , ³ , ⁴ In particular one (i) synthesizes a library of 10⁴ to 10⁵ DNA sequences encoding proteins that may bind to a target of interest; (ii) transforms the library into yeast such that each putative binder is expressed on the surface of a population of cells; (iii) incubates cells with fluorescently labeled target protein; (iv) physically separates 10⁶ to 10⁸ cells based on binding affinity by FACS; and finally, (v) quantifies the prevalence, and thereby binding affinity, of each library member by high throughput sequencing. Due to biological and technical variability, there is a distribution over (log) fluorescence for each library sequence, and the challenge is to estimate the means of each of these distributions (Figure 1a,b). For example, for binding interactions, this mean fluorescence relates directly to biophysical quantities of interest including dissociation constants and binding energies. ⁴ , ⁵ , ⁶

FIGURE 1.

Schematic overview of randomized gates. (a) Distributions of log fluorescence for different cell populations and (b) their hypothetical true and estimated means. (c) An example of histogram approach with deterministic collection into four bins and (d) an example of randomized collection approach with two bins. (e) Estimated means of the randomized gating scheme are more accurate than the histogram approach as the number of collected cells increases

In previous work, cells are deterministically segregated into one or more collection tubes (referred to as “bins”) based on their measured fluorescences, and the mean fluorescence of each population is estimated from the histogram of observed sequence counts in each bin (Figure 1c). Peterman and Levine ⁷ compare the error associated with different strategies for collecting and analyzing such data, and they show that average squared error is the sum of contributions from bias and variance (e.g., Hastie et al., ⁸ Chapter 7.3). The variance arises from experimental noise and variability across cells, and it can be reduced by increasing the number of cells screened. The bias arises from the discretization of the space of log fluorescence into bins (Figure 1b,c); for example, narrow distributions can be sorted all into the same bin but have means as different as the bin width. Moreover, this bias poses reproducibility challenges; the direction and magnitude of bias depends on how the bins are chosen, but this choice is subjective and commonly depends on variable experimental conditions. Because even the most sophisticated FACS machines can sort cells into at most six bins, resolution is limited. This low resolution limits the value of sort‐seq data in quantitative analyses, for instance, by prohibiting computation of precise binding energies. This challenge has spurred much work on how to effectively reduce histogram bias. ⁵ , ⁶ , ⁷ , ⁹ One common approach seeks to overcome the resolution limits of histograms by assuming fluorescence is log‐normally distributed for each population and using maximum likelihood estimation to estimate moments. ² , ⁹ , ¹⁰ , ¹¹ However, on real data, this assumption is violated and the resulting estimates can have greater bias than the naive approach (Figure S1).

2. RESULTS AND DISCUSSION

In this work, we show that the bias generated using histograms can be eliminated altogether by incorporating randomness into FACS collection strategies with as few as two bins (Figure 1d), thereby obtaining arbitrarily accurate estimates with many cells (Figure 1e). To do this, we take a statistical approach. We consider a population of cells that pass through a 2‐bin sorter, each with log fluorescence F independently and identically distributed according to a density function p _F . Our target of interest is the mean log fluorescence, ${\mathbf{\mu }}_F=\int f{p}_F\left(f\right)\mathrm{d}f.$ Let B denote the bin (either 1 or 2) into which a cell is collected, and let Y ₁ and Y ₂ be the counts of cells in Bins 1 and 2 after sorting, respectively. In multiplexed sort‐seq assays, we obtain Y ₁ and Y ₂ for thousands of populations, and our goal is to accurately estimate the mean of each population simultaneously.

For standard binning, a gate is chosen for each bin that defines the range of values F for which cells are collected into that bin; so, the bin B is deterministic once F is measured (e.g., as in Figure 1c). We instead consider randomized gates which define for each bin the probability of collecting a cell at each fluorescence (as in Figure 1d) and rely on pseudo‐random numbers to determine the bin. For estimating population means, when the fluorescence measurements fall between lower and upper bounds L and U, one first sorts using randomized gates such that for any f on the interval [L,U],

$$\mathrm{\mathbb{P}}\left(B=1|F=f\right)=1-\frac{f-L}{U-L}\text{and}\mathrm{\mathbb{P}}\left(B=2|F=f\right)=\frac{f-L}{U-L}.$$ (1)

The counts are then combined into an empirical estimate of ${\mathbf{\mu }}_F$ as $\widehat{\mathbf{\mu }}=\left(U-L\right)\cdot Y_2/\left(Y_1+Y_2\right)+L$.

While one might expect introducing randomness to decrease precision by introducing additional noise, $\widehat{\mathbf{\mu }}$ is directly informative to the mean fluorescence. In particular, $\widehat{\mathbf{\mu }}$ is an unbiased estimate of the true population mean in the sense that the average value we would expect for $\widehat{\mathbf{\mu }}$ if we repeated the sort‐seq experiment many times is equal to ${\mathbf{\mu }}_F$ (see Theorem 1 in Section 4).

This unbiasedness theorem guarantees that, in contrast to the histogram approach, we can get arbitrarily accurate estimates by screening a larger numbers of cells (Figures 1e and S2). More precisely, recalling that the mean squared error (MSE) is the sum of the bias squared and the variance, ⁸ unbiasedness implies that the error of $\widehat{\mathbf{\mu }}$ is dictated solely by its variance. Moreover, $\widehat{\mathbf{\mu }}$ allows a transparent trade‐off between the number of cells sorted per population and the precision of the estimates; notably, with as few as 400 cells, a 95% confidence interval for ${\mathbf{\mu }}_F$ will cover at most 10% of the range from L to U (Section 4).

2.1. Randomized gates provide superior accuracy to histograms in simulation

We used a simulation study to explore the implications of unbiasedness on estimation accuracy with the randomized gate approach relative to the standard histogram approach. In this study, we simulated fluorescence of 250 cells from log‐normal distributions with different means and variances (Figure 2a). We then simulated sorting these cells based on their fluorescence either with four deterministic gates of equal width or with two randomized gates as dictated by Equation (1). For the deterministic gates, we constructed histograms and computed estimates of the mean fluorescence as the average of the bin centers weighted by the fraction of cells they contained; and for the randomized gates, we estimated the mean as $\widehat{\mathbf{\mu }}$. Figure 2b,c report the performance of these estimates in terms of MSE, along with their bias and variance components. As expected, the randomized gates approach has negligible bias except for broad distributions violating the conditions of our theorem (Section 4).

FIGURE 2.

Simulation study reveals improved estimation properties obtained with randomized gates as compared to histograms. (a) Fluorescence values of cells are drawn independently from log‐normal distributions with different scales and with varied means, where the black arrows represent simulated changes of the means. (b) The relative performances of estimates from histograms and randomized gates across a range of mean log fluorescences in terms of mean squared error (ratios greater than 1 reflect lower error with randomized gates and ratios below 1 reflect lower error with histograms). (c) The mean squared error (left) decomposed into bias (center) and variance (right) for both estimates. All points are the average across 200 replicates, each with N = 250 cells

With even as few as 250 cells per population, the MSE of the histogram approach is dominated by bias. Accordingly, the unbiased randomized approach typically provides more accurate estimates. Notably, 250 cells is fewer than is the typical in sort‐seq assays; with larger samples, more pronounced improvements are obtained (Figure S2). Because the histogram estimates are systematically biased toward bin centers, they can however be more accurate for narrow distributions with means near bin centers (Figure 2b).

2.2. Experimental implementation via shifting gate thresholds

We next tested our approach experimentally. Current FACS software does not support randomized gate programming, so we devised an experimental approximation in which we manually changed the gating threshold 20 times during sorting at regular intervals (Section 4). We tested this procedure in the context of a binding assay using yeast display. ¹² We synthesized DNA encoding four mini‐protein binders to the SARS‐COV‐2 receptor binding domain (RBD) with a range of binding affinities. ¹ While the value of this approach is greatest for highly multiplexed assays with many thousands of sequences, we chose this small number so that we could also test each binder easily in serial. We separately transformed and expressed each design in yeast and then incubated the populations with RBD. Both the target and binders were fluorescently labeled, and we considered the log ratio of target to binder fluorescence as an expression normalized proxy for binder strength. ⁶ We measured each sample on a Sony SH800 cell sorter separately, recording the binding signal for each binder (Figure 3a). We then pooled the samples together and sorted 1,000,000 cells, collecting 50,000 cells at each of the 20 thresholds (Section 4).

FIGURE 3.

Agreement of binding signal of de novo designed binding proteins measured via yeast display in multiplex with ground truth values obtained in clonal yeast. (a) Distributions of samples measured clonally by flow cytometry, and distributions of pooled samples during sorting with a shifting gate boundary. Black triangles represent 6 of the 20 stopping points for the shifted gate. (b) Agreement of clonal and multiplexed binding signal. The x‐axis is measured by flow cytometry while the y‐axis is a multiplexed measurement by next‐generation sequencing. Error bars represent size of the steps used when shifting the threshold

The multiplexed measurements largely recapitulate the ground‐truth clonal measurements (Figure 3b), with the exception of design candidate 2018, for which the multiplexed estimate is below the clonal one. We suspect this is due to dissociation of some of the target protein in the time between the clonal and multiplexed measurements; kinetics experiments suggest dissociation occurs rapidly for this design. ¹

3. CONCLUSION

In the supplementary note, we additionally describe two extensions of this idea. First, because the differences in the variability of fluorescence across each population is often of interest (in addition to mean fluorescence), we show how to extend the approach to estimate the variance for each population and validate this approach in simulation (Figure S4). Second, we describe how to effectively take advantage of sorters that sort into more than two bins simultaneously to obtain more accurate estimates. We view these contributions as a starting point for future work of using randomness to obtain precise, multiplexed estimates.

We have shown how to obtain precise, multiplexed estimates in sort‐seq experiments with a simple strategy that incorporates randomness. With as few as two randomized gates, this mathematical technique allows one to collect more accurate data than one could previously obtain with four or six bin sorters. Moreover, this greater accuracy is attained with less sophisticated hardware and less downstream experimental effort. While we have emphasized studies of binding affinity, we believe our strategy is applicable to a wider range of applications of sort‐seq assays including studying transcriptional regulation ¹⁰ , ¹³ and protein stability, ¹¹ and building datasets for protein design. ¹⁴ Widespread implementation of randomized gates in FACS and community adoption of this strategy, will greatly simplify and improve sort‐seq assays by eliminating a common bias in this ubiquitous assay. We believe this will allow FACS to play a more central role in screening settings, for construction of reliable datasets for machine learning models in bio‐design applications, and for building datasets for quantitative models in biology more generally.

4. MATERIALS AND METHODS

4.1. Unbiasedness of estimates from randomized gates

The advantage of the randomized gates presented in Equation (1) is that the resulting counts in each bin (Y ₁ and Y ₂) may be combined as $\widehat{\mathbf{\mu }}=\left(U-L\right)\cdot Y_2/\left(Y_1+Y_2\right)+L$ to estimate ${\mathbf{\mu }}_F$ without bias. We make this statement precise and present a theorem that guarantees when this is the case.

For an estimator $\widehat{\theta }$ of a fixed estimand θ, the estimator's bias is the expected value of its error $\mathbb{E}\left[\widehat{\theta }-\theta \right|\theta ]$ conditioned on that particular value of θ. An estimator is called unbiased if, regardless of the value of the estimand, the bias is equal to zero—that is, if $\mathbb{E}\left[\widehat{\theta }\right|\theta ]=\theta$ for every θ. Theorem 1 states that this property holds for $\widehat{\mathbf{\mu }}.$

Theorem 1

(Unbiasedness with randomized gates). If the support of p_F is bounded between L and U, then $\widehat{\mathbf{\mu }}$ is an unbiased estimator of mean fluorescence. That is, $\mathbb{E}\left[\widehat{\mathbf{\mu }}\right]={\mathbf{\mu }}_F$.

We begin by rewriting the probability that a cell is collected into Bin 2 to expose the connection between this quantity and ${\mathbf{\mu }}_F$:

$$\begin{array}{l}\mathrm{\mathbb{P}}\left(B=2\right)={\int }_L^U{p}_F\left(f\right)\mathrm{\mathbb{P}}\left(B=2,|,F=f\right)\mathit{df}\\ //\mathrm{via}\mathrm{law}\text{of total probability}\text{support assumption}\\ ={\int }_L^U{p}_F\left(f\right)\left(f-L\right)/\left(U-L\right)\mathit{df}\\ //\mathrm{by}\text{Equation}\left(1\right)\\ =\left({\mathbf{\mu }}_F-L\right)/\left(U-L\right).\end{array}$$

If N = Y ₁ + Y ₂ total cells are collected, then the count in the second bin is distributed as $Y_2\mid N~\text{Binomial}\left(\left({\mathbf{\mu }}_F-L\right)/\left(U-L\right),N\right)$ and has mean $\mathbb{E}\left[Y_2\right]=N\cdot \left({\mathbf{\mu }}_F-L\right)/\left(U-L\right)$. Accordingly, for any N total number of cells, $\mathbb{E}\left[\widehat{\mathbf{\mu }}|Y_1+Y_2=N\right]=N\cdot \left({\mathbf{\mu }}_F-L\right)/\left(Y_1+Y_2\right)+L={\mathbf{\mu }}_F$. When N is random as well, then by the law of iterated expectation, $\mathbb{E}\left[\widehat{\mathbf{\mu }}\right]=\mathbb{E}\left[\mathbb{E}\left[\widehat{\mathbf{\mu }}|Y_1+Y_2=N\right]\right]={\mathbf{\mu }}_F$ as desired.

Notably, this theorem holds for any distribution p _F satisfying the support condition and does not require any parametric assumptions such as log‐normality.

4.2. Trade‐off between number of cells sorted and precision of estimates

The relative simplicity of the estimate $\widehat{\mathbf{\mu }}$ leads to a transparent trade‐off between the precision and scale of the experiment. Recalling that $Y_2\mid N~\text{Binomial}\left(\left({\mathbf{\mu }}_F-L\right)/\left(U-L\right),N\right),$ the variance of $\widehat{\mathbf{\mu }}$ is

$$\mathrm{Var}\left[\widehat{\mathbf{\mu }}|N\right]=\frac{{\left(U-L\right)}^2}{N^2}\mathrm{Var}\left[Y_2\right]=\frac{{\left(U-L\right)}^2}{N}\mathrm{\mathbb{P}}\left(B=1\right)\mathrm{\mathbb{P}}\left(B=2\right).$$

To construct a confidence interval for ${\mathbf{\mu }}_F,$ we can therefore first approximate the standard error of $\widehat{\mathbf{\mu }}$ by $\frac{U-L}{\sqrt{N}}\frac{\sqrt{Y_1{Y}_2}}{N},$ and appeal to approximate normality of the Binomial distribution for moderate to large N to report ${\mathbf{\mu }}_F=\widehat{\mathbf{\mu }}\pm 2\frac{U-L}{\sqrt{N}}\frac{\sqrt{Y_1{Y}_2}}{N}$ with 95% confidence. Because $\sqrt{Y_1{Y}_2}/N$ can be at most 1/2 (if Y ₁ = Y ₂), the size of this interval is at most $2\left(U-L\right)/\sqrt{N}.$ Therefore, to estimate ${\mathbf{\mu }}_F$ to within one tenth of the range with high confidence, at most N = 400 cells are needed, since in this case $2\left(U-L\right)/\sqrt{N}=\left(U-L\right)/10$.

For scale, commercial machines sort on the order of 10,000 cells per second, and typical assays sort tens of millions of cells divided amongst many populations. Thus, a library of 100,000 populations could be screened to high precision with on the order of 1 hr of sorting time.

4.3. Simulation details

In the simulations depicted in Figure 2, we compare against the standard approach of using a histogram to estimate ${\mathbf{\mu }}_F.$ Consider a K bin histogram. For each bin k, if the range of fluorescences collected is from lower bound l _k to upper bound u _k , then $\mathrm{\mathbb{P}}\left(B=k|F=f\right)=\mathbf{1}\left[l_k\le f<u_k\right].$ The histogram estimate then corresponds to combining the resulting counts as

$${\widehat{\mathbf{\mu }}}_{\text{Hist}}=\sum _{k=1}^K\frac{Y_k}{N}\left(\frac{u_k+l_k}{2}\right).$$

In order to use the unbiased estimator, both in simulation and in practice, we must slightly extend the randomized gate definition proposed in Equation (1). In particular, Theorem 1 assumes that the support of the fluorescence density p _F is bounded between L and U (i.e., that for $F~p_F,\mathrm{\mathbb{P}}\left[L\le F\le U\right]=1$). In practice, this may not be the case. But, as previously stated, Equation (1) returns negative “probabilities” outside of this range. Therefore, we propose to “clip” the collection probabilities at the boundaries, and instead define

$$\begin{array}{l}\mathrm{\mathbb{P}}\left(B=1\right|F=f)={\left(1-\frac{f-L}{U-L}\right)}_{†}\text{and}\\ \mathrm{\mathbb{P}}\left(B=2\right|F=f)={\left(\frac{f-L}{U-L}\right)}_{†}\end{array}$$

where † denotes clipping between zero and one such that, for a scalar $x,{\left(x\right)}_{†}=\max \left(\min \left(x,1\right),0\right).$ This ensures that $\widehat{\mathbf{\mu }}$ is well‐defined, but gives up unbiasedness in situations where the support assumption of Theorem 1 is violated. This bias is apparent, for example, at the right and left sides of the left panel of Figure 2c.

4.4. Experimental approximation of randomized gates with shifting thresholds

Because current FACS software does not support randomized gate programming, we devised an experimental approximation in which we manually changed the gating threshold 20 times during sorting at regular intervals. Specifically, we use a gate that collects all cells with fluorescence above a threshold into Bin 2 and those below the threshold into Bin 1, and we shift that threshold over the course of the collection from the lower limit L to the upper limit U. In theory, this approach exactly recovers Equation (1) in the limit that the threshold is shifted continuously from L to U at a constant rate. This is because for a cell with fluorescence f between L and U, the probability that it is collected into Bin 2 is the fraction of the experimental time during which the threshold is below f, which is (f − L)/(U − L). This approximation does not, however, account for possible changes in the distribution, p _F over time. Such changes occur in binding assays, for example, when nontrivial labeled target protein dissociates over time. This challenge is a disadvantage of the approximation relative to randomized gates that could in theory be implemented into sorters.

4.5. Yeast display and deep sequencing

EBY100 yeast cells expressing each of the four mini‐protein binders were grown in C‐Trp‐Ura media. Binder protein expression was induced by replacing the growing buffer with SGCAA and incubating at 30°C for 24 hr. ¹⁵ The induced cells were labelled with 250 nM biotinylated receptor binding domain target protein, washed twice with PBSF (PBS + 1% BSA), then labelled again with anti‐c‐Myc fluorescein isothiocyanate (FITC) and streptavidin‐phycoerythrin (SAPE). The experiments were performed on a Sony SH800 cell sorter. Sixty thousand cells were recorded for each binder to reflect the individual distribution of baseline PE signal intensity. In the shifting gate experiment, a square area (AreaTotal) with side length (L) was pre‐determined at the SH800 collection panel. The area was divided into 2 separate collection gates, Gate1 and Gate2 (corresponding to Bin 1 and Bin 2 in Equation (1)). Gate2 was in an isosceles right triangle and started with a small area in the right‐bottom corner of AreaTotal and Gate1 took up the remaining. The yeast cells were run through the SH800 and each cell went into either the Gate2 or Gate1 collection tube if its log PE/FITC signal was in the range of AreaTotal. All other cells were discarded. After collecting 50,000 cells, the cell flow was paused, Gate2 was shifted both leftwards and upwards for L/10 and cell flow continued. Because the proprietary software for operating the sorter allowed setting gate positions only through a point and click graphical user interface (rather than numerically), we measured out gate increments by pixel distance on the display using a ruler. The above shifting process repeated 19 times for a total of 20 collections. The cells collected in Gate1 and Gate2 were then grown, and 1 × 10⁷ cells from each gate were barcoded and the sequences for each cell were determined by Illumina next‐generation sequencing. ¹¹ The number of cells collected by each gate for each population was estimated from the proportion of sequencing reads attributed to each population and the number of cells collected into the gates.

Because the number of cells collected by each gate was not made directly available through the proprietary software, we estimated this from the raw exported data. In particular, we imported the data using the FlowCal python package ¹⁶ and computationally implemented the gates and filters (including for forward and backward scatter).

4.6. Sensitivity of maximum likelihood inference to non‐normality of real data

Likelihood‐based inference is a common strategy used with the intent to circumvent the resolution limitation of the histogram approach. ² , ⁹ , ¹⁰ , ¹¹ However, this approach can fail on real data. In particular, existing likelihood methods rely on the assumption that for each of the cell populations the fluorescence values are log normal distributed, $\log F~\mathcal{N}\left(\mu ,{\sigma }^2\right)$ where the mean log fluorescence $\mu ={\mathbf{\mu }}_F$ is the target of inference and σ ² is the typically unknown variance of the population.

We evaluate performance of maximum likelihood inference in this situation with simulations using data sub‐sampled from a flow cytometry dataset of binding signal of a computationally designed mini‐protein binder to ActRII. Data were collected using yeast display as previously described except with the addition of a supplemental binding protein, protein A, the binding signal $\log \left(\text{FITC}/\mathrm{PE}\right)$ was recorded for approximately 10,00,000 cells. The distribution of this signal is highly non‐Gaussian (Figure S1A).

We first compared the performance of the maximum likelihood approach (described in greater detail below) to the randomized approach on downsampled datasets with N = 250 cells with the same set‐up described in Figure 2. As in the earlier simulations, the randomized approach provides improved MSE across most simulation conditions (Figure S1B). This improvement is again explained by estimation bias, which is mitigated by the randomized approach (Figure S1C). Though one might expect the benefit of maximum likelihood would appear for larger sample sizes (e.g., due to the asymptotic efficiency of maximum likelihood estimation in theory), this is not the case. In fact, due to the bias of maximum likelihood, the relative improvement of the randomized approach is larger at N = 1,000 cells (Figure S1D). Moreover, Figure S1E demonstrates that the maximum likelihood approach does not empirically provide more accurate estimates even under correct specification (with fluorescences sampled as in Figure 2a).

4.7. Maximum likelihood estimation

To estimate ${\mathbf{\mu }}_F,$ likelihood‐based approaches consider the counts in each of K bins (Y ₁, Y ₂, …, Y _K ), since the measured fluorescence values cannot be disambiguated when multiple populations are sorted in multiplex. These counts follow a multinomial distribution as

$$Y_1,Y_2,\dots ,Y_K~\text{Mult}\left(\mathbf{\pi }\left(\mu ,{\sigma }^2\right),N\right),$$

where $N={\sum }_{k=1}^K{Y}_k$ is the total number of cells sorted into any bin and $\mathbf{\pi }\left(\mu ,{\sigma }^2\right)=\left({\pi }_1,{\pi }_2,\dots ,{\pi }_K\right)$ are the normalized bin probabilities. In particular, if for each bin k the range of fluorescences collected is from lower bound l _k to upper bound u _k , then

$${\pi }_k=\frac{\mathrm{\Phi }\left(\frac{u_k-\mu }{\sigma }\right)-\mathrm{\Phi }\left(\frac{l_k-\mu }{\sigma }\right)}{{\sum }_{k\prime =1}^K\mathrm{\Phi }\left(\frac{u_{k\prime }-\mu }{\sigma }\right)-\mathrm{\Phi }\left(\frac{l_{k\prime }-\mu }{\sigma }\right)},$$

where Φ(·) is the cumulative density function of the standard normal. The log likelihood function is then

$$\log p\left(Y_1,\dots ,Y_K;\mu ,{\sigma }^2\right)=\log N!-\sum _{k=1}^K\log Y_k!+\sum _{k=1}^K{Y}_k\log {\pi }_k,$$

where the dependence of each π _k on μ and σ ² is left implicit. The maximum likelihood approach is to return μ that maximizes this expression,

$${\widehat{\mathbf{\mu }}}_{\mathrm{MLE}}=\underset{\mu }{\arg \max }\left[\underset{{\sigma }^2>0}{\max }\log p\left(Y_1,\dots ,Y_K;\mu ,{\sigma }^2\right)\right].$$

This optimization problem is not analytically tractable, and its constraints and non‐convexity pose challenges for local, gradient‐based optimizers. So we instead solve the optimization approximately with a grid search.

AUTHOR CONTRIBUTIONS

Brian L. Trippe: Conceptualization (equal); formal analysis (equal); funding acquisition (equal); investigation (equal); methodology (equal); software (equal); validation (equal); visualization (equal); writing – original draft (equal); writing – review and editing (equal). Buwei Huang: Data curation (equal); formal analysis (equal); investigation (equal); validation (equal); writing – original draft (equal); writing – review and editing (equal). Erika A. DeBenedictis: Data curation (equal); formal analysis (equal); investigation (equal); validation (equal); writing – original draft (equal); writing – review and editing (equal). Brian Coventry: Data curation (equal); formal analysis (equal); investigation (equal); validation (equal); writing – original draft (equal); writing – review and editing (equal). Nicholas Bhattacharya: Formal analysis (equal); investigation (equal); methodology (equal); writing – original draft (equal); writing – review and editing (equal). Kevin K. Yang: Conceptualization (equal); formal analysis (equal); investigation (equal); methodology (equal); project administration (equal); software (equal); supervision (equal); validation (equal); writing – original draft (equal); writing – review and editing (equal). David Baker: Data curation (equal); investigation (equal); project administration (equal); resources (equal); supervision (equal); validation (equal); writing – original draft (equal); writing – review and editing (equal). Lorin Crawford: Conceptualization (equal); funding acquisition (equal); investigation (equal); methodology (equal); project administration (equal); supervision (lead); visualization (equal); writing – original draft (equal); writing – review and editing (equal).

Supporting information

Appendix S1 Supporting Information

Figure S1. Maximum likelihood estimation of mean fluorescence assuming log normality is non‐robust on real data. (A) Fluorescence values are drawn independently from empirical distributions of binding signal of mini‐protein binders to ActRII obtained by flow cytometry and rescaled to have an range of different standard deviations (0.01 in red, 0.05 in green, and 0.20 in blue, respectively). (B) The ratio of the MSE of estimates from the maximum likelihood approach on histogram counts to unbiased estimates from randomized gates. (C) The estimation bias for the maximum likelihood estimates with the histogram counts (left) versus the randomized gates (right). All points in panels (B) and (C) are the average across 200 replicates, each with N = 250 cells. (D) The ratio of the MSE of the estimates is not substantially different even with a greater number (N = 1,000) of cells or (E) under correct specification with log normal distributed observations.

Figure S2. The error of estimates with randomized gates decreases with increasing sample size. Here, we consider N = 100, 250, and 1,000 total cells, respectively. The bottom row shows the relative performances of estimates from histograms and randomized gates across a range of mean log fluorescences in terms of mean squared error (ratios greater than 1 reflect lower error with randomized gates and ratios below 1 reflect lower error with histograms).

Figure S3. Randomized gates for (A) estimating population variances as defined by Equation (S1) and (B) optimally estimating population means with 4‐way sorting.

Figure S4. Average squared error of standard deviation estimates of various p _F at different sample sizes. Here, we consider (A) N = 1,000, (B) N = 5,000, and (C) N = 25,000 total cells. Depicted are the performances of (left) histogram based estimates and (right) the randomized estimate $\sqrt{{\widehat{\mathbf{\sigma }}}^2}$. In panel (D), we show the distributions from which we simulated the data.

Trippe BL, Huang B, DeBenedictis EA, Coventry B, Bhattacharya N, Yang KK, et al. Randomized gates eliminate bias in sort‐seq assays. Protein Science. 2022;31(9):e4401. 10.1002/pro.4401