Authors Response to Correspondence About an Improved Compensation Method

Recently we proposed an optimization based method for compensating flow cytometry data that is able to use any combination of single or multiple stained controls (1). We demonstrate that our method is superior to the standard approach to compensation supported by commercial flow cytometry software which uses only single stained control samples. Two investigators have published critiques (2, 3) of our manuscript to which we now respond.

Roederer questions one aspect of our paper, our empirical finding that using multiple stained and single stained controls together give better overall compensation results (2). In his critique, he ignores our principal result, which consists of an optimization method for compensation that allows the use of additional samples beyond the single stained samples that are used for compensation in the commercial software developed by Roederer.

In our study, we defined a parameter, |Δ|, that measures the deviation from perfect compensation. Using this parameter, we discovered that optimization-based compensation (using both single- and multiple stained controls) performed better than conventional compensation (using only single stained controls). Roederer’s commentary does not address the theory of multi-control compensation and compensation accuracy presented in our paper. We respond to the specific issues he raises in the order they are presented by him.

Roederer states that he does not understand the meaning of our statement that multi stained samples are closer to the experimental N-stained samples than to single stained controls.“…it is unclear what the authors mean by “closer to the N-stained sample”. What does “closer” mean in a mathematically defined way?” Roederer asks us to explain mathematically what closer means when comparing integers, such as a 9-stained experiment, an 8-stained control and a 1-stained control. 9 is considered closer to 8 than to 1 because after subtraction, the remainder is smaller. Mathematically,

$$\mathrm{N}-(\mathrm{N}-1)<\mathrm{N}-1\text{all}\mathrm{N}>2.$$

While Roederer only asks for a mathematical relationship, the N-labeled experiment is also closer (e.g. more similar) biophysically in experimental space to the N-1 labeled experiment. With increasing number of reagents, there may be perturbation of membrane, proteins or other cellular components that alter the fluorescent behavior of dyes. Because of these perturbations, changes may not be linear when estimating spillover, leading to errors when linearly extrapolating to N-labeled samples using only single-labeled controls.

The non-linearity of the fluorescent spillover effects is raised in the critique by Rajwa (3) in which he states: “…the authors did not attempt to introduce a true nonlinear model…In fact, they seem to suggest that M obtained through Eq.(6) can be used somehow in a standard linear mixture model in order to estimate α. But this is a logical contradiction! The matrix M (and consequently S) obtained from an invalid linear model cannot be used in a nonlinear mixture model and certainly cannot improve the original linear model, which has been demonstrated to be untrue in the specific case studied by the authors.”

We agree with Rajwa about the nonlinearity of the system and the problems with the linear extrapolation from single stained samples favored by Roederer. There are two ways in which the nonlinearity can be addressed. We could have correctly modeled the nonlinear physics of the system and increased the number of model parameters considerably. Unfortunately this approach would require an impractical number of samples for adequate parameter estimation and would lead to a satisfying theoretical result that would not be useful for experimentalists. We instead opted to base our approach on using flow cytometry controls that are typically already done by experimentalists, such as single-labeled compensation samples and (N-1) “leave one out” controls used for gating. We therefore chose to take the first order approximation of the nonlinear model and keep the original number of model parameters. This approximation is formally similar to the linear model used by conventional compensation but the elements of the transformation matrix ( $\overline{\overline{S}}$ in ref. 1) will be different from the spillover coefficients (elements of $\overline{\overline{S^0}}$ matrix in ref. 1).

Roeder states that he is: “ unaware of any reference claiming that single stained controls are not rigorous for compensation and strongly dispute their claim that this was “long recognized””. Roederer’s own publications contradict his critique: “compensation problems increase when more than four colors are used, but these problems can be minimized if staining combinations are chosen carefully” (4). If single stained compensation worked well as the number of dyes were increased, there would be no compensation problem and one could just rely on the compensation to correct for spectral overlap. Compensation problems increase as the number of dyes increase because the linear extrapolation from single stained to multi stained samples is imperfect. Experimentalists are well aware of the distortions introduced by conventional compensation. Indeed, one of the motivations for developing mass cytometry has been the limitations and inaccuracies introduced by conventional compensation [for discussion see (5)].

Roederer comments that “The authors are wrong in stating that the “compensation method described in this article requires homogeneously stained controls…Having mixed staining of different cells would be less efficient in delivering dye per cell than uniform staining, but, if adequate data are taken, the averages will be valid” We disagree. If there is significant deviation from homogeneously stained cells the averages are not adequate for compensation. In our original article (1), homogeneously stained controls have been defined as follows: “For example in a triple-stained control every cell is stained by 3 fluorochromes and the number of double-stained, single-stained and unstained cells is zero or negligibly small”. Deviation from uniform (homogenous) staining will not alter the estimate of compensation parameters as long as the averages are not significantly shifted. In the case of strong deviation from uniform staining one has to select the cluster of fully (homogenously) labeled cells and use that reduced set of data as the control.

Roederer claims: “…A priori, we predict that simply doubling the number of identical controls would increase the precision on compensation estimates by 40%....Indeed, simply adding additional single stained compensation controls should increase the compensation precision more so than multiple stained controls. Or, equivalently and far more practical, more events for each sample could be collected, thus increasing the precision of the measurement of each control. To justify their premise, the authors must demonstrate that additional multiple stained controls perform better than additional single stained controls…” Roederer’s claim has been tested experimentally and is not valid. We did conventional compensation by using single stained controls where the sizes of the original single stained controls are reduced by 50%. The results are shown in Figure 1a–c. Figure 1a is equivalent to the figure in our original article (Figure 1b in ref. 1). It shows the goodness of conventional compensations of full size single stained controls and multiple stained controls, i.e. the compensations of single stained controls are perfect and not so good for multiple stained controls. Similarly Figure 1b and 1c show the goodness of the conventional compensations when one half and the other half of the original single stained controls are utilized, respectively. The overall deviation from perfect compensations is characterized by the |Δ| parameter.

Figure 1. Compensations of the FCM data of 5-stained dentritic cells and U937 cells.

Subfigures a–c) refer to the dentritic cell data where conventional compensations are performed at three different data sets of single stained controls. Subfigures d–f) refer to the steps of the optimization based compensation of the U937 cell data.

a) Result of conventional compensation of dentritic cell data(complete data sets of single stained controls were utilized). This figure is equivalent with Figure 1b in ref. 1. 〈 $C_j^{(v)}(\overline{\overline{S^0}})$〉 is the compensated average intensity of the j-th stain that is detected at the j-th detector in control sample v. In conventional compensation the spillover matrix, $\overline{\overline{S^0}}$ is determined from Eq. 3 in ref. 1 by using data from single stained controls. At the abscissa each column is marked by a (v)_ j symbol, where vector v and subscript j specify the control experiment and the detector where the intensity was measured, respectively. Only those averages are plotted that belong to missing dyes, i.e. the j-th component of the v vector is zero. After correct compensation all these averages should approach zero. At the abscissa symbols referring to single stained controls are framed by red. b) Result of conventional compensation (one half of the data sets of each single stained control were utilized). c) Result of conventional compensation (the other half of the data sets of each single stained control were utilized). d) The column plot shows 29 averages, 〈 $D_j^{(v)}$〉 of fluorescence intensities detected in 10 control samples of the U937 cell data (see the v vectors of the controls in the text). At the abscissa each column is marked by a (v)_ j symbol (see legends to subfigure a). e) Result of conventional compensation. f) Result of optimization based compensation. 〈 $C_j^{(v)}(\overline{\overline{S}})$〉 is the compensated average intensity of the j-th stain that is detected at the j-th detector in control sample v. For the optimization based compensation matrix $\overline{\overline{S}}$ was determined by minimizing the objective function in Eq. 9 in ref. 1.

In every subfigure F is the sum of the squares of the plotted averages. In subfigures a–c) $\mid \mathrm{\Delta }\mid =\sqrt{\frac{F}{32}}$ is the root of the mean of the squares of the plotted averages, while in subfigures e–f) $\mid \mathrm{\Delta }\mid =\sqrt{\frac{F}{29}}$.

When doubling the sizes of the single stained controls (i.e. going from Figures 1b or Figure 1c to Figure 1a) the overall deviation from perfect compensation does not decrease significantly. In one case |Δ| changes from 45.9 to 42.0 a mere 9.2% decrease, while in the other case |Δ| increases by 3.8%. In spite of Roederer’s expectation in this example we could not improve the compensation of the multi stained controls by simply increasing the sizes of the single stained control data. On the other hand our optimization based compensation method decreased |Δ| from 42.0 to 23.9, a significant 43% decrease (see Figure 1b, c in our original article, ref. 1).

Roederer states: “The authors provided no information about how many cells were collected for each sample, what the fluorescence intensity was in each channel, what the actual spillover values were, nor what the noise contribution (autofluorescence) was in each channel.” At the end of our original article (1) we stated: “An executable file containing the compensation program and the experimental data files utilized in this work are available at the authors’ website (www.tsb.mssm.edu/Primeportal).” At the time of the original article’s publication, these data files contained, and still contain, all the requested information except the spillover coefficients. As a Supplementary Material we have now provided two additional output files containing: 1) the spillover matrix for conventional and for optimization based compensation, 2) the data needed for creating figures like Figure 1 in the original article (1).

Roederer states: “…there were no examples as to how this might be applied in a practical way. For example, in the authors’ own 5-color experiment, there are a total of 30 possible single or multiple stained controls: which of these 30 should be used?…” We show good results using all single stained and all N-1 (leave one out) stained controls.

This is our initial recommendation in this paper and these controls are routinely performed in many flow cytometry studies, including those by Roederer. Therefore, if N-1 controls are already being run, the use of the improved algorithm allows improved compensation without additional experimental work. However, our algorithm is general and researchers can use our approach to assess the improvement in compensation obtained and relative practicality of using any combination of control samples.

Finally Roederer comments: “…any scientific presentation must provide evidence of reproducibility, variability, extensibility, and robustness. This cannot be achieved through the presentation of data from a single experiment…”

We present the result of another experiment performed on U937 cells. The utilized five fluorochromes are: (1) FITC (CD45-FITC Beckman Coulter, Cat# PN IM0782U), (2) PE (anti-human CD45-PE BioLegend, Cat# 304008), (3) Pacific Blue anti-human CD45-Pacific Blue BioLegend, Cat#304022), (4) PE-Cy7 (anti-human CD45-PE-Cy7 BioLegend, Cat#304016), (5) APC (CD45-APC Beckman Coulter, Cat# PN IM2473U), while the characteristics of the respective five band pass filters (center wavelength/band width in nm) are: (1) 530/30, (2) 575/30, (3) 450/50, (4) 780/60, (5) 660/20.

Besides the five single stained controls we utilized five multiple stained controls too. The v vectors characterizing the multiple stained controls are: (0,0,1,1,1) (1,0,0,1,1) (1,0,1,0,1) (1,0,1,1,0) (1,0,1,1,1), where v=(0,0,1,1,1) for example specifies a control sample where the U937 cells were stained only by the 3^rd, 4^th and 5^th dyes.

Figure 1d–f are equivalent to the first three subfigures of Figure 1 in our original article (1), however these subfigures refer to our experiment with U937 cells. In the case of this extra set of experiments the comparison of conventional and optimization based compensation results in the same conclusions. Figure 1e shows that conventional compensation of single stained controls are perfect, however the compensation of multi stained controls are not so good. On the other hand for optimization based compensation (see Figure 1f) the deviation from perfect compensation is homogeneously distributed along the controls and the root of mean of the squares of the deviations, |Δ| = 49.2, is considerably smaller than for conventional compensation, |Δ| =83.2. Also, by using this data set one can test the validity of Roederer’s hypotheses for conventional compensation, i.e. doubling the size of the data sets of single stained controls a 40% reduction is expected in the value of |Δ|. As we did at Figure 1a–c we used half of every data set of the single stained controls to perform conventional compensation with these reduced controls. In spite of Roderer’s expectation, there was no significant change in the |Δ| value: for the first half of the single stained controls |Δ| = 83.6, while for the second half |Δ| = 82.8.

The data files of our experiment with U937 cells are available at the Supplementary Material.

In conclusion, in both the conventional and optimization based approach, compensation is an affine transformation of the data of an N-stained sample. In conventional compensation the elements of the transformation matrix are determined from N single stained controls and an unstained sample. These matrix elements, the so called spillover coefficients, guarantee the perfect compensation of single stained controls but in our two examples do not result in correct compensation of the multiple stained controls. In spite of Roederer’s expectation (2) we could demonstrate that the quality of the compensation of multi stained controls does not practically change by changing the size of the data sets of single stained controls.

However, the optimization based compensation utilizing both single and multiple stained controls successfully adjusts the values of the elements of the transformation matrix improving the overall compensation of the controls and specifically the compensation of the multi stained controls.

We believe that multi stained controls are biophysically closer to the N-stained sample than the single stained controls. Thus compensation using single- and N-1 stained controls provides the best overall compensation for different cell populations.