Comparison of highly sensitive, multiplex immunoassay platforms for streamlined clinical cytokine quantification

ABSTRACT

Introduction

Selecting the optimal platforms to quantitate cytokines is challenging due to varying performance and the plethora of options available.

Aims

To compare performance of three highly sensitive, multiplex assays on three different platforms – MSD S-plex, Olink Target 48, and Quanterix SP-X – to MSD V-plex which is widely used for quantitative cytokine assay.

Methods

Serum and stimulated plasma samples were analyzed across each platform. The proportion of quantifiable samples was compared for each analyte and correlation analyses were performed to relate the data. For MSD S-plex, parallelism and antibody pair knockdown experiments gauged specificity of the kit.

Results

MSD S-plex was the most sensitive multiplex platform followed by Olink Target 48, Quanterix SP-X, and MSD V-plex. Concentrations across platforms differed greatly for some cytokines, but all platforms showed strong correlation. Results for MSD S-plex were confirmed by parallelism and knockdown.

Conclusion

MSD S-plex should be a priority platform for ultra-sensitive assay. Olink Target 48 offers an enticing combination of sensitivity and multiplex capability that warrants consideration when many cytokines require quantitation. MSD V-plex, MSD S-plex and Olink quantitative assays offer high utility across drug development programs, but fit-for-purpose performance should be assessed on a per-analyte basis.

1. Introduction

Cytokines are small, soluble pleiotropic proteins that control the development and responses mediated by immune cells [1]. Dysregulation of cytokines is associated with many diseases including cancer and various autoimmune disorders and may also result from unintended systemic inflammatory responses triggered by some therapeutics [2,3]. In drug development, it is therefore important to quantitate cytokines to assess the impact of drugs that influence immune response, to aid in characterizing disease biology, and to monitor for patient safety [4]. However, selecting the optimal platforms and assays to quantitate cytokines is challenging due to varying performance and the plethora of options available.

While proteomics-based approaches are often utilized in early clinical studies as a screening tool to identify clinically relevant cytokines, there is an increased need for robust, highly sensitive, and quantitative cytokine measurements as a drug progresses through development. This is especially challenging given that homeostatic concentrations of cytokines in blood are low [5]. In addition, prior to identification of priority cytokines as biomarkers for efficacy and/or safety, many potentially important cytokines may need assessment in an exploratory setting. Multiplex platforms are useful in this regard as they allow quantification of many cytokines at once, require less sample volume per-analyte, and may be more cost-effective if data from the entire plex is utilized. Platforms historically used for such analysis include traditional ELISA, Meso Scale Discovery (MSD) V-plex (Meso Scale Diagnostics; Rockville, MD), Luminex (Bio-Techne; Minneapolis, MN), Quanterix SIMOA (Quanterix; Billerica, MA), and more recently ProteinSimple ELLA (Bio-Techne; Minneapolis, MN). While the accuracy and reliability of these platforms to measure a given analyte have been validated extensively, none of these individual platforms provides both the sensitivity required to measure low-abundance markers and the ability to measure many analytes at once. MSD V-plex, for example, is a versatile plate-based platform that allows simultaneous measurement of up to 10 markers but provides only a moderate improvement in sensitivity compared to ELISA [6–11]. Quanterix SIMOA, on the other hand, is a bead-based platform that is capable of accurate readouts in the fg/mL range [6,7,12–15], but currently only offers up to 4-plex assay kits. Furthermore, although Luminex and ELLA offer intriguing flexibility in designing bespoke multiplex assays, Luminex shows consistently poor sensitivity compared to ELISA and MSD V-plex [6,7,9,16,17] whereas ELLA offers at best a modest boost in sensitivity compared to traditional immunoassay platforms [7,18,19].

In recent years, several new off-the-shelf quantitative immunoassay technologies have touted improved sensitivity in a highly multiplexed format. These platforms include MSD S-plex, Quanterix SP-X, and Olink Target 48 (Olink; Uppsala, Sweden). MSD S-plex has a similar multiplex capability compared to V-plex, and data provided by the vendor suggests accurate quantification in the low fg/mL range [20–25]. It has been shown that MSD S-plex is approximately as sensitive as Quanterix SIMOA [26,27] confirming that it is an ultra-sensitive platform. SP-X is a plate-based platform that offers up to 10-plex assay kits and is touted by Quanterix as being ultra-sensitive, although it does not rely upon the traditional single-molecule counting technology associated with the company. One study found that Quanterix SP-X was significantly more sensitive than both MSD and Quanterix SIMOA for custom assay development [28], but we could not find any other platform comparisons that included SP-X. Finally, Olink Target 48 is an off-the-shelf assay technology that offers a 45-plex assay where concentration can be reported on a quantitative or non-quantitative basis. Non-quantitative Olink assays appear to correlate well with established platforms such as MSD [16,29]. Data from the vendor also confirms this for the quantitative readout [30], however it is not clear the level of sensitivity the quantitative option provides.

The overall goal of this study is to evaluate the performance of MSD S-plex, Quanterix SP-X, and Olink Target 48 using MSD V-plex as a reference since it is widely used in industry. To our knowledge, the performance of these platforms has not been directly compared. Specifically, we aim to understand the level of sensitivity and precision offered by each platform, validate the relative accuracy of each platform through correlation, and confirm the specificity of individual samples for each analyte. We aim to confirm specificity through parallelism and antibody pair ‘knockdown’ experiments. While parallelism is a well-known aspect of biomarker assay validation that is critical to assessing the relative accuracy of an assay [31], it is not feasible to perform when endogenous analyte concentrations are low. We therefore designed a bespoke experiment which we call antibody pair knockdown where verified antibody pairs for each cytokine are spiked into individual samples to block the cytokines of interest from capture/detection in the assay. This knockdown strategy serves to gauge specificity when endogenous analyte concentrations are low and to confirm parallelism results when endogenous analyte concentrations are sufficiently high.

2. Materials and methods

2.1. Serum samples

Healthy and non-small cell lung cancer (NSCLC) human serum samples were selected at random from an inventory of samples obtained from BioIVT (Westbury, NY). Descriptive statistics for the human patient samples are displayed in Table 1. In addition, 3 healthy plasma samples subject to whole blood stimulation were acquired from an internal blood donation unit at GSK (Collegeville, PA). All samples were stored in a freezer set to maintain −70°C for no more than 2 years at the time of analysis. All samples were sourced ethically, and their research use was in accord with the terms of the informed consents under approved IRB protocols (WCG 20161665 and Advarra CR00303097).

Table 1.

Normal and NSCLC patient characteristics.

Characteristic	Normal	NSCLC
n	30	10
Sex
Male	15	8
Female	15	2
Mean Age ± SD (years)	42.6 ± 13.4	60.2 ± 8.0
Race
Caucasian	2	9
Black	8	0
Hispanic	20	0
Native Hawaiian or other pacific islander	0	1
Not Given	0	0

2.2. Reagents and chemicals

Meso Scale Discovery V-plex Proinflammatory Panel (PIP) 1 human Kit and S-plex Proinflammatory Panel 1 human Kit were purchased from Meso Scale Diagnostics. Quanterix SP-X CorPlex Human Cytokine Panel 1 was purchased from Quanterix.

For whole blood stimulation experiments, Phorbol-12-myristate-13-acetate (PMA) was purchased from Millipore Sigma (Burlington, MA). Lipopolysaccharide (LPS), IFN-γ, and Ionomycin were purchased from ThermoFisher (Waltham, MA). Whole blood was collected with BD Vacutainer™ Plastic Blood Collection Tubes with K2 EDTA: Hemogard™ Closure purchased from Fisher Scientific (Hampton, NH).

For knockdown experiments, all antibodies were purchased from R&D Systems (Minneapolis, MN) according to vendor recommendations for ELISA matched antibody pairs: catalog numbers MAB610 and AF-210-NA (TNF-α), MAB611 and AF-219-NA (IL-12), MAB2172 and AF-217-NA (IL-10), MAB206 and AF-206-NA (IL-6), MAB604 and AF-204-NA (IL-4), MAB202 and AF-204-NA (IL-2), MAB601 and AF-201-NA (IL-1β), MAB2852 and AF-285-NA (IFN- γ), and MAB317 and AF-317-NA (IL-17A).

2.3. Whole blood stimulations

Three samples from GSK blood donation unit were subject to whole blood stimulation with PMA (25 ng/mL) and Ionomycin (1 µg/mL), LPS (10 ng/mL), LPS and IFN-γ (25 ng/mL), and PBS (unstimulated control). Stimulants were spiked into 15 mL whole blood and incubated at 37°C and 5% CO₂ for approximately 20 hours. The tubes were then centrifuged at 1500 × g for 10 mins. Plasma was aspirated from the tube, pipetted into 250 uL aliquots, and stored at −70°C for later use.

2.4. Sample analysis

Samples for MSD V-plex and Quanterix SP-X assays were analyzed in duplicate and samples for MSD S-plex were analyzed in singlet at GSK (Collegeville, PA) according to the manufacturer’s instructions. For each analyte the stated kit analytical LLOQ was verified by actual performance of the corresponding calibration curve; the lowest calibrator point with ≤ 25% coefficient of variation (%CV) and ≤ 25% relative error (%RE) at or above the stated LLOQ was used as the analytical assay LLOQ [32]. For ease of comparison, all LLOQs mentioned in this study refer to dilution corrected LLOQs. The dilution corrected LLOQ represents the analytical LLOQ multiplied by any additional dilution factor applied to samples. Sample results were excluded from analysis if the %CV was > 20% [32] or if ‘split’ replicates were observed (i.e., one replicate was within the quantifiable range and one replicate was not).

Samples for Olink Target 48 (T48) Cytokine assay were analyzed by Olink (Uppsala, Sweden). The samples were analyzed in singlet in accordance with standard kit procedure.

For MSD S-plex, the same samples analyzed in the full 9-plex version of the assay in singlet were also analyzed in duplicate using a 6-plex version of the assay where detection antibodies for IL-1β, IFN-γ, and IL-17A were omitted from the procedure.

2.5. Correlation analyses

For correlation analyses, sample results were converted to log scale as the concentration data spanned several orders of magnitude. Deming regression was performed in GraphPad PRISM for analytes where an adequate number of quantifiable sample results were obtained across platforms (IL-6, IL-10, TNF-α, and IFN-γ). D’Agostino-Pearson tests were used to confirm that the majority of datasets were normally distributed; accordingly, Pearson correlation coefficient was also reported for each platform combination.

2.6. Knockdown sample preparation

For MSD S-plex, we performed a ‘knockdown’ experiment to assess the specificity of the kit. Verified antibody pairs for each cytokine in the kit were spiked into individual samples to block the cytokines of interest from capture/detection in the assay. For each sample, antibodies for each cytokine were spiked at 5 µg/mL yielding a total of 90 ug/mL per sample (10 µg/mL per antibody pair across 9 cytokines). 95% matrix percentage was maintained for each sample. Samples were incubated with shaking for approximately 30 minutes, aliquoted, and frozen at −70°C for subsequent analysis in the assay.

2.7. Parallelism assessment

Parallelism was performed in select samples in the MSD S-plex assay. Samples that were selected for parallelism experiments included those that had high endogenous cytokine concentrations, those that showed nonspecific interference via knockdown experiments, as well as one sample stimulated with PMA + Ionomycin. Samples were analyzed in duplicate at the kit minimum required dilution (MRD) and diluted 2-fold in the kit-provided assay buffer. While there is no consensus on biomarker assay parallelism acceptance criteria [33,34], we focused on accuracy of each dilution-corrected concentration compared to the undiluted concentration where conservatively ≤ 30% RE was considered acceptable. We did not include %CV in our acceptance criteria as equal precision across platforms is not assumed. The %RE was calculated for the dilution-corrected values and was acceptable if ≤ 30%. Only samples that could be carried out through a total of 2 additional dilutions beyond the recommended assay MRD within the quantifiable range of the curve were reported.

3. Results

3.1. Sample concentration results

Normal and NSCLC serum concentration results for pro-inflammatory cytokines available across at least three of the four platforms are shown in Figure 1. A summary of the proportion of samples that were above the assay LLOQ can also be found in Table 2. MSD S-plex was the most sensitive platform across all analytes and was the only platform that was able to quantify IL-2, IL-4, IL-12p70, and to some degree IL-1β reliably across the sample set (although Quanterix SP-X data was not available for IL-2 and Olink T48 data was not available for IL-12p70). Olink quantified samples at an overall higher rate compared to MSD V-plex for most cytokines including IL-6, IL-10, TNF-α, IFN-γ, and IL-1β. However, underperformance was observed for select analytes. For example, Olink was not able to quantify IL-2 or IL-4 in any samples despite LLOQs of 119 fg/mL and 59.6 fg/mL respectively. Quanterix SP-X quantified samples at a better rate than MSD V-plex for IL-10 and TNF-α, however it should be noted that most samples in the SP-X data set were NSCLC samples that appear to have higher endogenous IL-10 and TNF-α concentrations compared to normal. Excluding IL-10 and TNF-α, SP-X was marginally more sensitive compared to MSD V-plex. Interestingly, Quanterix SP-X was not able to quantify IL-1β in any samples despite having an LLOQ of 96 fg/mL, which is the lowest stated LLOQ for this analyte across the platforms. MSD V-plex was the least sensitive platform, as < 10% of samples were above the assay LLOQ for IL-12p70, IL-4, IL-2, TNF- α, and IL-1β.

Figure 1.

Performance of MSD V-plex (n = 40), MSD S-plex (n = 40), olink target 48 (n = 24), and Quanterix SP-X (n = 12) multiplex assays for pro-inflammatory cytokines in healthy and NSCLC serum samples. The dotted line in each dataset represents the assay LLOQ. For informational purposes, samples that were <lloq were plotted at the assay LLOQ. Platforms are denoted by color; healthy serum and NSCLC serum are denoted by shape (solid circle and hollow diamond for healthy and NSCLC respectively).

Table 2.

Summary of quantifiable normal and NSCLC serum sample results for common analytes across platforms.

		IL-10	IL-12p70	IL-4	IL-6	IL-2	TNF-α	IFN-γ	IL-1β
MSD S-plex	# of Reportable Samples	40	40	40	40	40	40	40	40
	# of LLOQ samples	0	0	12	0	1	0	0	21
	% Above LLOQ	100%	100%	70%	100%	98%	100%	100%	48%
Olink T48	# of Reportable Samples	24	n/a	24	24	24	24	24	24
	# of LLOQ samples	0	n/a	24	0	24	9	4	21
	% Above LLOQ	100%	n/a	0%	100%	0%	63%	83%	13%
Quanterix SP-X	# of Reportable Samples	11	12	9	12	n/a	12	12	12
	# of LLOQ samples	0	11	5	5	n/a	0	8	11
	% Above LLOQ	100%	8%	44%	58%	n/a	100%	33%	8%
MSD V-plex	# of Reportable Samples	39	40	40	39	40	39	35	40
	# of LLOQ samples	29	39	40	19	38	36	10	40
	% Above LLOQ	26%	3%	0%	51%	5%	8%	71%	0%

n/a indicates analyte was not included in assay kit.

Complete sample concentration results for all samples across the platforms can be referenced in Supplementary Table S1.

3.2. Precision results

Precision was evaluated by assessing the %CV of samples analyzed in duplicate. A summary of the within-run precision data for normal and NSCLC serum samples can be found in Table 3. A summary of the within-run precision data for the stimulated plasma sample set can be found in Table 4.

Table 3.

Within-run precision of normal and NSCLC serum samples.

Parameter	MSD V-plex	Quanterix SP-X	MSD S-plex (6-plex)	MSD S-plex	Olink T48
Sample set	Normal and NSCLC serum samples	Normal and NSCLC serum samples	Normal and NSCLC serum samples	Select parallelism samples analyzed at the MRD	Precision not evaluated; samples analyzed in singlet
Total results (samples x analytes)	400	120	240	90
Number of “Split” results	8	4	7	6
Number of <LLOQ results	291	48	24	6
Number of >ULOQ results	0	0	0	0
Number of results with both replicates within the quantifiable assay range	101	68	209	78
Number of results with acceptable % CV (≤20%)	101	67	200	69
Percentage of acceptable %CV	100%	98.5%	95.7%	88.5%

Table 4.

Within-run precision of stimulated plasma samples.

Parameter	MSD V-plex	Quanterix SP-X	MSD S-plex	Olink T48
Sample set	Stimulated plasma sample set	Stimulated plasma sample set	Stimulated plasma sample set	Precision not evaluated; samples analyzed in singlet
Total results (samples x analytes)	117	78	105
Number of “Split” results	8	3	0
Number of <LLOQ results	52	9	3
Number of >ULOQ results	0	7	18
Number of results with both replicates within the quantifiable assay range	57	59	84
Number of results with acceptable % CV (≤20%)	39	53	83
Percentage of acceptable %CV	68.4%	89.8%	98.8%

Percent CV was reported for normal and NSCLC serum samples as well as the stimulated plasma sample set analyzed in duplicate in the MSD V-plex assay. Out of a total of 400 results for the normal and NSCLC samples (40 samples x 10 analytes), 8 results were not reportable due to split results and 291 were <LLOQ. Out of the remaining 101 results, 101 (100%) were reported with an acceptable %CV (≤20%). For the stimulated plasma set, out of a total of 117 results, 8 results were not reportable due to split results and 52 results were <LLOQ. Out of the remaining 57 results, 39 (68.4%) were reported with an acceptable %CV.

Percent CV was also reported for normal and NSCLC serum samples as well as the stimulated plasma sample set analyzed in duplicate in the Quanterix SP-X assay. Out of a total of 120 results for the normal and NSCLC samples (12 samples x 10 analytes), 4 results were not reportable due to split results and 48 samples were <LLOQ. Out of the remaining 68 results, 67 (98.5%) were reported with an acceptable %CV (≤20%). For the stimulated plasma set, out of a total of 78 results, 3 results were not reportable due to split results, 9 results were <LLOQ, and 7 results were >ULOQ. Out of the remaining 59 results, 53 (89.8%) were reported with an acceptable %CV.

For MSD S-plex, while the normal and NSCLC sample set were analyzed in singlet, we evaluated precision using the full 9-plex assay through select parallelism samples that were analyzed in duplicate at the assay MRD (full parallelism results presented in section 3.5). Out of a total of 90 results (10 samples x 9 analytes), 6 results were not reportable due to split results and 6 samples were <LLOQ. Out of the remaining 78 results, 69 (88.5%) were reported with an acceptable %CV (≤20%). Failed %CVs were observed for IL-10, IL-17A, IL-4 (one instance each) as well as IL-12p70, TNF-α, and IL-2 (two instances each). For the stimulated plasma set, out of a total of 105 results, 3 results were <LLOQ and 18 were >ULOQ. Out of the remaining 84 results, 83 (98.8%) were reported with an acceptable %CV.

In addition, we evaluated a 6-plex version of the MSD S-plex PIP kit. The same normal and NSCLC serum set was analyzed in duplicate according to manufacturer’s instructions but detection antibodies for IL-1β, IFN-γ, and IL-17A were omitted from the procedure. Out of a total of 240 results (40 samples x 6 analytes), 7 samples were not reportable due to split results (all instances for IL-4) and 24 samples were <LLOQ (23 instances for IL-4). Out of the remaining 209 samples, 200 samples (95.7%) were reported with an acceptable %CV (≤20%). Failed %CVs were observed for IL-10 (two instances), IL-4 (one instance), and TNF-α (five instances).

Precision was not evaluated for Olink Target 48 Cytokine which is run in singlet per manufacturer’s instructions.

Complete sample concentration results can be referenced in Supplementary Table S1.

3.3. Platform correlation results

Platform correlation matrices were compiled for healthy and NSCLS serum samples as well as stimulated plasma samples for IL-6 (Figure 2), IL-10 (Figure 3), TNF-α, (Figure 4), and IFN-γ (Supplemental Figure S1) as these cytokines had an adequate number of quantifiable samples across all four platforms. It should be noted that success of the stimulations varied by stimulation condition, analyte, and platform; while many stimulated sample results were within the quantifiable assay range at each assay MRD, some stimulation results were <LLOQ and others were >ULOQ and thus excluded from correlation analyses. In addition, we included both serum and plasma results together in this analysis because they generally correlated to the same degree for the various cytokine and platform combinations. However, this may not be expected for other cytokines or assays that quantitate cytokines differently in serum versus plasma.

Figure 2.

Correlation matrix of quantifiable IL-6 concentration results for MSD V-plex and MSD S-plex (n = 21), MSD V-plex and olink T48 (n = 17), MSD V-plex and quanterix SP-X (n = 6), MSD S-plex and olink T48 (n = 29), MSD S-plex and quanterix SP-X (n = 7), and olink T48 and quanterix SP-X (n = 10). Sample results are reported in fg/mL and converted to log scale as the concentration data spanned several orders of magnitude. The solid line in each graph shows Deming regression and the dotted line is the line of identity. Pearson’s correlation is also shown for each platform combination.

Figure 3.

Correlation matrix of quantifiable IL-10 concentration results for MSD V-plex and MSD S-plex (n = 13), MSD V-plex and olink T48 (n = 12), MSD V-plex and quanterix SP-X (n = 11), MSD S-plex and olink T48 (n = 35), MSD S-plex and quanterix SP-X (n = 17), and olink T48 and quanterix SP-X (n = 15). Sample results are reported in fg/mL and converted to log scale as the concentration data spanned several orders of magnitude. The solid line in each graph shows Deming regression and the dotted line is the line of identity. Pearson’s correlation is also shown for each platform combination.

Figure 4.

Correlation matrix of quantifiable tnf-α concentration results for MSD V-plex and MSD S-plex (n = 8), MSD V-plex and olink T48 (n = 8), MSD V-plex and quanterix SP-X (n = 6), MSD S-plex and olink T48 (n = 24), MSD S-plex and quanterix SP-X (n = 17), and olink T48 and quanterix SP-X (n = 16). Sample results are reported in fg/mL and converted to log scale as the concentration data spanned several orders of magnitude. The solid line in each graph shows Deming regression and the dotted line is the line of identity. Pearson’s correlation is also shown for each platform combination.

Correlation across the platforms for IL-6 ranged from r = 0.998 to r = 0.958. Using MSD V-plex as a reference, Pearson correlation coefficients for MSD S-plex, Olink, and Quanterix SP-X were 0.958, 0.998, and 0.998 respectively.

Correlation across the platforms for IL-10 ranged from r = 0.966 to r = 0.695. Lower correlation coefficients were observed for platform combinations involving Olink and may be driven by plasma samples stimulated with LPS + IFN-γ that did not appear to correlate in the same manner as other sample types. Notably, all LPS + IFN-γ stimulated samples analyzed in the Olink assay were marked with a “warning” tag, indicating that these samples did not meet QC criteria applied to each individual sample in the run and that the results from these samples should be interpreted with caution. While this sample set did not appear problematic across all Olink analytes, results for LPS + IFN-γ stimulated samples likely contributed to lower correlation coefficients observed for Olink platform comparisons where this sample set was quantifiable. Using MSD V-plex as a reference, Pearson correlation coefficients for MSD S-plex, Olink, and Quanterix SP-X were 0.966, 0.960, and 0.958 respectively.

Correlation across the platforms for TNF-α ranged from r = 0.997 to r = 0.894. Using MSD V-plex as a reference, Pearson correlation coefficients for MSD S-plex, Olink, and Quanterix SP-X were 0.951, 0.916, and 0.997 respectively.

Correlation across the platforms for IFN-γ ranged from r = 0.920 to r = 0.808 (plasma samples stimulated with LPS + IFN-γ were omitted from the dataset as they contained recombinant IFN-γ). Using MSD V-plex as a reference, Pearson correlation coefficients for MSD S-plex, Olink, and Quanterix SP-X were 0.920, 0.824, and 0.912 respectively.

3.4. MSD S-plex ‘knockdown’ experiments

MSD S-plex was the most sensitive platform in our study and could quantify several cytokines at baseline that were not quantifiable using other platforms, such as IL-2, IL-4, IL-12p70, and IL-1β. Although S-plex could quantify these cytokines in many samples, in many instances endogenous concentrations were still too low to gauge nonspecific interference via parallelism. Instead, we performed knockdown experiments where verified antibody pairs were spiked into individual samples. If the kit is specific for a given cytokine, the antibody pair spiked result should be <LLOQ. The spiked values can be compared to the unspiked values to determine what percentage of the sample response is authentic. For this experiment, samples below the quantitation range were reported at the LLOQ and those above the quantitation range at the ULOQ. To calculate the percentage of the unspiked sample response that was authentic, the following formula was used:

$$\frac{\left(\mathit{Unspiked}\mathit{result}-\mathit{knockdown}\mathit{result}\right)}{\left(\mathit{Unspiked}\mathit{result}-\mathit{LLOQ}\right)}\mathit{x}100$$

Results for normal and NSCLC serum samples are shown in Figure 5. Nearly all knockdown samples were <LLOQ for each cytokine, indicating that the responses from these samples are 100% authentic. However, there were occasional knockdown samples that had quantifiable results, suggesting that a portion of the response in these samples arises from nonspecific interference. We considered any measurement that was ≥ 95% authentic to have negligible levels of nonspecific interference. Overall, there were 8 results out of 360 measurements (40 samples x 9 cytokines) considered to have significant nonspecific interference, including two instances for IL-10, one instance for IL-12p70, one instance for IL-4, two instances for TNF-α, one instance for IL-2, and one instance for IL-17A. The results suggest that despite occasional instances of nonspecific interference, the MSD S-plex pro-inflammatory kit is highly specific even in the low fg/mL range, as 352/360 (97.8%) of sample responses were authentic.

Figure 5.

MSD S-plex antibody pair knockdown data. Endogenous (unspiked) sample results are depicted by red bars. The result for the same sample spiked with a cocktail of antibody pairs (knockdown sample) for each cytokine is depicted in blue. The dotted line in each dataset represents the assay LLOQ. For informational purposes, samples that were <lloq were plotted at the assay LLOQ. To calculate the percentage of the unspiked sample response that was authentic, the difference between the unspiked and knockdown result was divided by the difference between the unspiked result and the assay LLOQ. Nearly all knockdown samples measured <lloq for each cytokine, indicating that the responses from these samples are 100% authentic for a given cytokine. Samples with an authentic response < 95% are denoted with black arrows, indicating that a portion of the response in these samples arises from nonspecific interference. Overall, the results show that the MSD S-plex pro-inflammatory kit is highly specific for pro-inflammatory cytokines even in the low fg/mL range.

Stimulated plasma samples were also subject to knockdown. All samples in the set demonstrated an authentic response, confirming that increases in measured concentrations observed in the stimulated samples were indeed attributed to increases in the relevant cytokine concentrations. These results can be found in Supplementary Table S2 which contains the full knockdown dataset.

3.5. MSD S-plex parallelism results

For parallelism experiments, we selected some samples that had high endogenous cytokine concentrations (Normal 6, 8, 16, and 24), all samples which showed significant nonspecific interference via knockdown experiments (NSCLC 7, Normal 3, 14, 17, and 22) as well as one sample stimulated with PMA + Ionomycin. Since we intentionally picked samples which exhibited nonspecific interference, this dataset likely exhibits a higher rate of failure for parallelism compared to a randomly selected sample set. Samples that could not be carried out through at least 2 additional dilutions beyond the recommended assay MRD within the quantifiable range of the curve were excluded from analysis.

Generally, samples free from nonspecific interference (demonstrated via knockdown experiments) showed acceptable parallelism, whereas samples with significant nonspecific interference failed parallelism. A specific example for the 3 highest responding IL-10 samples is presented in Supplementary Figure S2. The sample with nonspecific interference fails parallelism. The 2 samples with 100% authentic responses showed acceptable parallelism.

Parallelism results across all analytes are shown in Table 5. There were 4 samples with instances of significant nonspecific interference (2 for IL-10 and 2 for TNF-α) remaining the dataset after <LLOQ results were removed. Three out of these four samples failed parallelism, while the passing sample showed increasing %RE across dilutions. Out of the other 46 instances where a ≥ 95% authentic response was observed, 41 samples had acceptable %RE for the majority of dilutions that were within the quantifiable range (89.1% across all analytes, failures observed for IL-6, IL-10, IL-4, TNF-α, and IL-1β). Across the entire dataset, parallelism was acceptable in 42/50 cases (84%). We did not consider samples with acceptable %RE but unacceptable %CVs as failures in this analysis.

Table 5.

MSD S-plex parallelism results.

Red indicates %RE > 30%.

*Indicates %CV > 30%.

% Auth. = % Authentic.

4. Discussion

Comparing performance of immunoassays that quantitate cytokines and other biomarkers is challenging as varying calibrators, capture/detection antibodies, assay buffers, incubation times, incubation temperatures, as well as capture/detection methods influence the resulting sample concentrations. Such assays thus should not be expected to yield accurate “absolute” concentrations; they are relative assays. In addition, their sensitivities should be judged by the proportion of samples that are quantifiable rather than the stated LLOQ.

While immunoassays offer relative quantification, two assays that measure the same analyte should correlate. We found that IL-6, IL-10, TNF-α, and IFN-γ showed good correlation across each platform combination and thus yield accurate relative quantitation. As expected, although these platforms correlated, measured concentrations in some cases differed greatly. For example, concentrations of MSD V-plex IFN-γ were on average more than 10-fold higher compared to MSD S-plex IFN-γ and over 20-fold higher compared to Olink T48 IFN-γ despite correlation coefficients of 0.920 and 0.824 respectively. These disparities in concentration might reflect a core difference in the way each assay captures/detects IFN-γ, perhaps due to different antibodies utilized or varying performance of the recombinant protein selected as the calibrator in each assay. Despite different measured concentrations, each assay can still be fit-for-purpose.

In this study, we found that MSD S-plex was by far the most sensitive multiplex platform we evaluated. The most difficult cytokines to detect were IL-1β and IL-4 which were quantifiable in 48% and 70% of samples analyzed on S-plex respectively. While we did not evaluate Quanterix SIMOA in this study due to its limited ability to multiplex, other studies have shown that MSD S-plex is approximately as sensitive [26,27]. Olink Target 48 Cytokine was more sensitive than MSD V-plex for most analytes but not as sensitive as MSD S-plex. Despite LLOQs of 119.2 fg/mL for IL-2 and 59.6 fg/mL for IL-4, Olink was not able to quantify these cytokines in any samples that we analyzed. It is unclear whether the IL-2 and IL-4 components of the assay require further optimization or if these cytokines cannot be quantified due to sensitivity limitations with the Olink platform. Lastly, we found that Quanterix SP-X was marginally more sensitive compared to MSD V-plex.

We also found that MSD V-plex showed the best precision as judged by proportion of acceptable %CVs for the normal and NSCLC serum samples analyzed in duplicate, as every sample result had an acceptable %CV when both replicates were quantifiable. Quanterix SP-X also had very good precision in this regard (98.5%). Although MSD S-plex had a slightly lower proportion of samples with acceptable %CVs in this sample set (95.7% in the 6-plex assay and 88.5% for parallelism samples prepared at the assay MRD), we considered this level of precision to be adequate given that the majority of sample results were in the fg/mL range. The MSD S-plex assay also outperformed its V-plex counterpart in the stimulated plasma sample set (98.8% for S-plex and 68.4% for V-plex). It appears that this result is mostly driven by one sample that consistently yielded poor %CVs across analytes when subject to various stimulation conditions in V-plex but not S-plex (Supplementary Table S1). We considered the possibility of a dilution error for this set of stimulated samples in MSD V-plex but did not find any trends consistent with this across other samples analyzed in the same columns/rows. These results underscore the fact that platform, assay, user, and sample-related factors all contribute to a method’s ability to reliably quantify a given analyte. Overall, we find that MSD S-plex and Quanterix SP-X yield consistent responses within a given run that are comparable with MSD V-plex, however full assay validations of these kits should be performed to ensure consistent performance from run-to-run.

We performed follow-up analysis on the MSD S-plex assay to confirm specificity of kit which, given the strong correlation data demonstrated in this study, implies specificity of the other platforms evaluated. Since parallelism is difficult to perform when endogenous concentrations are low, we used a knockdown strategy where verified antibody pairs are spiked into each sample to block the cytokine of interest from assay capture/detection. We found that the MSD S-plex PIP is highly specific for each cytokine in the 9-plex, with some occasional instances of nonspecific interference. Samples that exhibited nonspecific response via knockdown experiments failed parallelism in most instances when the sample could be reliably diluted beyond the kit MRD. In contrast, most samples with an authentic response showed acceptable parallelism, as judged by %RE, across all analytes. These results suggest that in lieu of parallelism, which can only be performed when endogenous concentration is sufficiently high, antibody pair knockdown is a useful tool to gauge nonspecific interference in immunoassay. This strategy can assess specificity in a high number of samples at once, reports on the percentage of response in a given sample that arises from nonspecific interference, and only requires a few µg antibodies per analyte. While this experiment should not replace parallelism, which helps to ensure the appropriate assay calibrator has been selected and that the appropriate MRD has been applied, antibody pair knockdown can be used to supplement parallelism findings or can utilized when parallelism cannot be assessed. Ideally, at least one antibody in the pair should be polyclonal in order to block all epitopes on the cytokine of interest. It should also be investigated further whether a single polyclonal antibody is sufficient to block all such epitopes.

5. Conclusion

We conclude that MSD V-plex remains a robust, reliable platform already embedded in many labs employing biomarker technologies that is a practical option for quantitating cytokines and other biomarkers with sufficient endogenous concentrations. However, we find that MSD S-plex offers several advantages over other platforms currently in use. It has approximately the same level of sensitivity as Quanterix SIMOA, and this sensitivity is required to measure many key pro-inflammatory cytokines and other biomarkers at baseline. In addition, it has greater multiplex capability than Quanterix SIMOA, runs on the same instrument as MSD V-plex which is widely available, has a low sample volume requirement, and delivers reliable concentration data in a plate-based format which may be preferable to bead and/or fluidics-based assays. Overall, while performance should be more rigorously assessed on a per-analyte basis, our view is that MSD S-plex should be a priority platform for ultra-sensitive assay needs. One limitation of MSD S-plex remains that a user-developed assay option is currently not available.

In addition, we find that Olink Target 48 offers an enticing combination of sensitivity and multiplex capability that warrants consideration when many cytokines require quantitation. Although it should not be considered an ultra-sensitive platform, Olink Target 48 offers quantitation of 45 high-utility cytokines in one assay and requires as little as 2 µL sample volume which can be useful for rare matrices. While our study demonstrates that Olink Target 48, and as an extension other quantitative Olink panels such as Flex and Focus, can offer accurate and sensitive relative quantitation when compared to other established immunoassay platforms, such high-dimensional panels should be more rigorously assessed on a per-analyte basis.

Ultimately, it is our view that MSD V-plex, MSD S-plex, and quantitative Olink assays offer high utility across drug development programs given the combined capabilities of these platforms in regard to sensitivity, relative accuracy, multiplex capability, off-the-shelf availability of assays for a wide variety of analytes, and availability of these platforms at labs employing biomarker technologies throughout industry. However, for particularly important cytokines that may be biomarkers for drug efficacy and/or safety, single-plex assays should be considered. Looking forward, a single platform that is not limited by plex or sensitivity remains elusive. Emerging immunoassay technologies such as NULISA, which boasts superior sensitivity to Quanterix SIMOA and the capability to simultaneously quantify of hundreds of soluble proteins at once, should be evaluated and considered for future drug development programs [35].

Funding Statement

Funding for this analysis was provided by GSK. The authors were employed by GSK when the work was completed and are shareholders of GSK.

Article highlights

• Selecting the optimal platforms and assays to quantitate cytokines is challenging due to varying performance and the plethora of options available.

• We aimed to compare performance of three highly sensitive, multiplex assays on three different platforms – MSD S-plex, Olink Target 48, and Quanterix SP-X – to MSD V-plex which is widely used for quantitative cytokine assay.

• Healthy serum, cancer serum, and stimulated plasma samples were analyzed across each platform.

• The number of samples above the LLOQ was compared for each analyte.

• Deming regression and correlation analyses were performed to relate the data.

• For MSD S-plex, parallelism and antibody pair knockdown experiments, where verified antibody pairs for each cytokine were spiked into individual samples to block the cytokines of interest from capture/detection in the assay, were used to gauge specificity.

• MSD S-plex was the most sensitive multiplex platform followed by Olink Target 48, Quanterix SP-X, and MSD V-plex.

• Measured concentrations across platforms differed greatly for some cytokines, but all platforms showed strong correlation.

• Within-run precision of each platform was adequate, excluding Olink which was analyzed in singlet.

• Concentration results for MSD S-plex were confirmed by parallelism and knockdown.

• Samples that exhibited nonspecific response via knockdown experiments failed parallelism in most instances.

• Quantitative immunoassays should not be expected to yield accurate “absolute” concentrations; they are relative assays since varying reagents and methodology influence the resulting sample concentrations.

• Assay sensitivities should be judged by the proportion of samples that are quantifiable rather than the stated LLOQ.

• Strong correlation results across each platform combination further suggests that each assay yields accurate relative quantitation.

• In lieu of parallelism, which can only be performed when endogenous concentration is sufficiently high, antibody pair knockdown is a useful tool to gauge specificity in immunoassay.

• Acceptable parallelism and successful knockdown experiments in the MSD S-plex assay imply accuracy and specificity across all platforms when considered together with correlation results.

• MSD V-plex remains a robust, reliable platform that is a practical option for quantitating cytokines and other biomarkers with sufficient endogenous concentrations.

• MSD S-plex offers several advantages to traditional ultra-sensitive assay platforms and should be a priority platform for ultra-sensitive assay.

• Olink Target 48 offers an enticing combination of sensitivity and multiplex capability that warrants consideration when many cytokines require quantitation.

• MSD V-plex, MSD S-plex and Olink quantitative assays offer high utility across drug development programs, but fit-for-purpose performance should be assessed on a per-analyte basis.

• Platforms such as NULISA that further expand the limits of sensitivity and plex should be investigated.

Disclosure statement

The authors have no relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript. This includes employment, consultancies, honoraria, stock ownership or options, expert testimony, grants or patents received or pending, or royalties.

Writing disclosure

No writing assistance was utilized in the production of this manuscript.

Supplemental data

Supplemental data for this article can be accessed online at https://doi.org/10.1080/17576180.2024.2442190