Abstract
Background:
Multiplexed protein analysis platforms are a novel and efficient way to characterize biomarkers in a variety of biological samples. Few studies have compared protein quantitation and reproducibility of results across platforms. We utilize a novel nasosorption technique to collect nasal epithelial lining fluid (NELF) from healthy subjects, and compare the detection of proteins in NELF across three commonly used platforms.
Methods:
NELF was collected from both nares of twenty healthy subjects using an absorbent fibrous matrix and analyzed using three different protein analysis platforms: Luminex, Meso Scale Discovery (MSD), and Olink. Twenty-three protein analytes were shared across two or more platforms, and correlations across platforms were assessed using Spearman correlations.
Results:
Among the twelve proteins represented on all three platforms, IL1α and IL6 were very highly correlated (Spearman correlation coefficient [r] ≥ 0.9); CCL3, CCL4, and MCP1 were highly correlated (r ≥ 0.7); and IFNɣ, IL8, and TNFα were moderately correlated (r ≥ 0.5). Four proteins (IL2, IL4, IL10, IL13) were poorly correlated across at least two platform comparisons (r < 0.5); for two of these proteins (IL10 and IL13), the majority of observations were below the limits of detection for Olink and Luminex.
Discussion:
Multiplexed protein analysis platforms are a promising method for analyzing nasal samples for biomarkers of interest in respiratory health research. For most proteins evaluated, there was good correlation across platforms, although results were less consistent for low abundance proteins. Of the three platforms tested, MSD had the highest sensitivity for analyte detection.
Introduction
There is growing interest in the use of multiplexed protein analysis platforms to analyze biomarkers in a wide variety of biological samples. Multiplexed assays offer the advantage of analyzing of several biomarkers of interest simultaneously while utilizing small quantities of sample1, are cost-effective, and can be used to characterize localized inflammatory mediators and cytokines in sites of interest to respiratory and environmental health researchers2–4.
Of particular interest in respiratory health is the utility of non-invasive sample collection methods for evaluation of disease and environmental exposure biomarkers, as circulating biomarkers have been shown to be poorly correlated with biomarkers in respiratory samples5. One potential sampling method to fill this gap is the collection of nasal epithelial lining fluid (NELF) using nasosorption, a method to noninvasively sample the upper respiratory epithelium6. Assessment of biomarkers in NELF can provide insights into inflammatory processes and immune mediators in the upper respiratory tract. Cytokines in NELF have been show to correlate well cytokines in the lower airway5. However, methods for measuring respiratory immune mediators across the field vary; thus, interpreting reproducibility of results across studies is a major challenge.
Several assay platforms exist using varied methodologies for protein analysis, such as bead-, ELISA-, mass spectrometry-, and protein extension-based assays. Studies analyzing circulating mediators from serum and plasma comparing use of these platforms have demonstrated variable results with respect to the sensitivity, accuracy, and inter-platform correlations of proteomic assays using commonly available biological specimens, like serum or plasma7–10. However, the performance of multiplexed assays to evaluate respiratory samples have not yet been investigated. Understanding how well results correlate across protein analysis platforms, and which platforms may be most reliably used for the analysis of respiratory samples, is vital to advancing the field of respiratory biomarker research.
To fill this gap and evaluate performance of commonly utilized protein analysis platforms, our study compares three protein analysis platforms – Luminex, Meso Scale Discovery (MSD), and Olink– using NELF samples from healthy subjects. We utilize paired NELF samples to assess the sensitivities of each assay and observe correlations in protein quantitation across assay platforms.
Methods
Sample Collection
Healthy adult subjects between 18 and 75 years of age were recruited for study participation. Exclusion criteria included cigarette smoking, a history of bleeding or other blood-related disorders, recent nasal surgery, and current signs of active viral infection. Twenty healthy adults were enrolled and basic demographic information was collected. Nasal epithelial lining fluid was collected from both nares of each subject using a synthetic absorbent matrix (Nasosorption FX·i, Mucosal Diagnostics, Midhurst, England). Following a procedure that has been previously described6,11, nasal strips were inserted into the anterior portion of the inferior nasal turbinate of each nostril following pre-moistening of nares with normal saline. Following insertion, nostrils were clamped shut for two minutes with a padded nose clip. Strips were then stored at −20°C in microfuge tubes until elution. Samples from both nares were eluted a and eluate from both nares for each subject was combined and subdivided into three aliquots which were then frozen at −20°C until analysis. This protocol was approved by the University of North Carolina at Chapel Hill Biomedical Institutional Review Board (Study #21–0022) and all methods were performed in accordance with relevant guidelines and regulations including obtaining written informed consent.
Protein Analysis Assays
Each of the three sample aliquots was analyzed by one of three different protein analysis platforms: Human Immunotherapy Luminex Performance Assay 24-plex Fixed Panel (R&D Systems, Minneapolis, MN); V-Plex Human Cytokine 30-plex Kit (Meso Scale Discovery [MSD], Rockville, MD); Olink Target 96 Inflammation (Olink Biosciences, Uppsala, Sweden). Samples were analyzed once using the Luminex and MSD platforms, and were analyzed in triplicate using the Olink platform. The proportion of samples detectable within the dynamic range for each assay (as defined by the manufacturer) was documented. For Olink, limit of detection (LOD) is defined as 3 standard deviations above background and reported in pg/mL. In our analysis, an Olink sample was categorized as below limit of detection (LOD) if one or more of the three measurements were below LOD. For MSD, the lower limit of detection (LLOD) corresponded to the average signal 2.5 standard deviations above the background. For Luminex, the lower limit of quantitation (LLOQ) was utilized as defined by the platform software.
Statistical Analysis
Log 2 normalized signal intensity was utilized across platforms for data analysis. For Olink, the three data points for each participant were averaged. All observations, including those outside of the dynamic range for each assay, were included in the statistical analyses. Twenty-three protein analytes were shared across two or more platforms. Coefficients of variation (CV) were calculated for the three Olink measurements, and mean CVs were calculated for each protein analyte. Correlations across platforms were assessed using Spearman correlations given protein concentrations were non-normally distributed. All analyses were performed in RStudio (version 2022.07.1+554).
Results
Twelve of the proteins evaluated were analyzed on all three platforms (Table 1). There were also six proteins available only on the chosen Olink and MSD assays (IL5, IL7, MCP4, TNFβ, CCL11, VEGF), two proteins available only on the chosen Olink and Luminex assays (IL17a, CXCL10), and three proteins available only on the chosen Luminex and MSD assays (IL15, IL12p70, GMCSF).
Table 1.
Proportion of samples outside of range of detection/quantitation for proteins analyzed by Olink, MSD, and Luminex
| Olink | MSD | Luminex | ||
|---|---|---|---|---|
| Protein | #(%) <LOD | #(%) <LLOD | #(%) <LLOQ | #(%) >ULOQ |
| CCL3 | 2 (10%) | |||
| CCL4 | 13 (65%) | |||
| IFNγ | 2 (10%) | 4 (20%) | ||
| IL1α | 1 (5%) | |||
| IL2 | 20 (100%) | 1 (5%) | ||
| IL4 | 19 (95%) | 17 (85%) | ||
| IL6 | 6 (30%) | |||
| IL8 | 13 (65%) | |||
| IL10 | 19 (95%) | 3 (15%) | ||
| IL13 | 20 (100%) | 18 (90%) | ||
| MCP1 | ||||
| TNFα | 4 (20%) | 19 (95%) | ||
| IL5 | 19 (95%) | 8 (40%) | ||
| IL7 | ||||
| MCP4 | 4 (20%) | |||
| TNFβ | 3 (15%) | 10 (50%) | ||
| CCL11 | ||||
| VEGF | ||||
| IL17a | 2 (10%) | 19 (95%) | ||
| CXCL10 | ||||
| IL15 | 1 (5%) | 11 (55%) | ||
| IL12p70 | 19 (95%) | |||
| GMCSF | 6 (30%) | |||
The proportion of samples within the range of quantitation for each assay varied widely. For MSD, 5% of samples were below the LLOD. For MSD, TNFβ had the largest proportion of samples below LLOD (n=10 [50%]) (see Table 1). Luminex had a significantly larger proportion of samples below the lower limit of quantitation (LLOQ) with several proteins (CCL4, IL4, IL13, TNFα, IL17a, IL15, IL20p70) below LLOQ for the assay in greater than 50% of measurements. Luminex also had one protein, IL8, above the upper limit of quantitation (n=13 [65%]). For Olink, five proteins (L2, IL4, IL5, IL10, and IL13) were below LOD in ≥95% of samples.
Among the twelve proteins tested on all three platforms, there was overall a moderate-to-high degree of correlation across platforms, with some exceptions (Table 2). Two proteins (IL1α, IL6) were very highly correlated (Spearman correlation coefficient [r] ≥ 0.9), three proteins (CCL3, CCL4, MCP1) were highly correlated (r ≥ 0.7), and three proteins (IFNɣ, IL8, TNFα) were moderately correlated (r ≥ 0.5) across all three platforms (Table 2, Supplemental Figure A1). Four of the six proteins analyzed only by Olink and MSD platforms (IL7, MCP4, CCL11, VEGF) were very highly correlated (r ≥ 0.9). Finally, four proteins (IL2, IL4, IL10, IL13) were poorly correlated across at least two platform comparisons (r < 0.5). The majority of measurements for each of the poorly correlated proteins were below the LOD for Olink and/or Luminex as described above.
Table 2.
Spearman correlation coefficients (r) for proteins analyzed by Olink, Luminex, and MSD platforms
| Olink + MSD | Olink + Luminex | Luminex + MSD | ||||
|---|---|---|---|---|---|---|
| Protein | r | p | r | p | r | p |
| CCL3 | 0.90 | <0.01 | 0.78 | <0.01 | 0.74 | <0.01 |
| CCL4 | 0.97 | <0.01 | 0.82 | <0.01 | 0.75 | <0.01 |
| IFNγ | 0.81 | <0.01 | 0.66 | 0.00 | 0.72 | <0.01 |
| IL1α | 0.97 | <0.01 | 0.95 | <0.01 | 0.95 | <0.01 |
| IL2 | 0.02 | 0.92 | 0.11 | 0.63 | 0.82 | <0.01 |
| IL4 | 0.25 | 0.29 | −0.24 | 0.31 | 0.13 | 0.59 |
| IL6 | 0.96 | <0.01 | 0.98 | <0.01 | 0.94 | <0.01 |
| IL8 | 0.56 | 0.01 | 0.67 | 0.00 | 0.78 | <0.01 |
| IL10 | −0.07 | 0.77 | 0.28 | 0.23 | 0.66 | 0.00 |
| IL13 | 0.28 | 0.24 | 0.24 | 0.31 | 0.39 | 0.09 |
| MCP1 | 0.96 | <0.01 | 0.89 | <0.01 | 0.82 | <0.01 |
| TNFα | 0.62 | 0.00 | 0.60 | 0.01 | 0.72 | <0.01 |
| IL5 | −0.31 | 0.18 | ||||
| IL7 | 0.99 | <0.01 | ||||
| MCP4 | 0.93 | <0.01 | ||||
| TNFβ | 0.70 | <0.01 | ||||
| CCL11 | 0.94 | <0.01 | ||||
| VEGF | 0.95 | <0.01 | ||||
| IL17a | 0.74 | <0.01 | ||||
| CXCL10 | 0.97 | <0.01 | ||||
| IL15 | 0.94 | <0.01 | ||||
| IL12p70 | 0.22 | 0.36 | ||||
| GMCSF | 0.49 | 0.03 | ||||
r ≥ 0.9
0.7 ≤ r < 0.9
0.5 ≤ r < 0.7
r < 0.5
Among the 20 proteins tested by Olink that were shared across one or more other platforms in this analysis, mean CV of the three measurements for each protein ranged from 0.28% (for IL8) to 101.39% (for IL4) (Supplemental Table A1). A majority of all 92 of the proteins tested by Olink (n=67, 73%) had a mean CV <10% for the triplicate runs (Supplemental Table A2).
Discussion:
We found that protein analysis platforms are comparable in their measurement of protein biomarkers in NELF for proteins quantified within the dynamic range for each assay. A majority of proteins evaluated by two or more platforms in our study were moderately to highly correlated, suggesting consistency of results among the platforms tested. However, sensitivities of the platforms varied, and influenced the comparability of results across platforms. Our study indicates that utilizing a variety of multiplexed protein analysis platforms for NELF analysis is feasible and results across platforms are consistent, particularly for higher abundance proteins, and can likely provide valuable insights into the complex cytokine milieu of the upper respiratory tract.
Despite the high correlations observed for the majority of proteins in this analysis, there were several proteins with poor correlations across all platforms (IL2, IL4, IL10, IL13). The majority of measurements for each of these proteins were below the LOD for at least one of the platforms tested. Other studies measuring protein biomarkers in the nasal fluid of adults (by MSD or individual ELISA) have also found relatively low levels of IL4, IL10, and IL135,12. Our findings suggest that for lower abundance proteins such as these, the assays chosen here performed less reliably and reproducibility across platforms was compromised. This limitation may be mediated by selecting assay platforms with higher sensitivity for lower abundance mediators.
The varying proportion of samples within the range of quantitation for each assay may be based on the different analytic methods employed by each platform. Olink is a proximity extension assay requiring binding of antibody pairs to target proteins followed by extension of oligonucleotides linked to these antibodies once brought into close proximity, a method to increase specificity and reduce cross-reactivity of the assay13. Olink does not directly quantify protein target concentration, but quantifies DNA sequences generated by linkage and extension of the oligonucleotides on a normalized scale in proportion to measured background signal, while MSD and Luminex both provide quantitative measures of analyte directly proportional to the intensity of light emitted by the sample7. While comparison of these results using Spearman correlations allows for these differences in quantitative methods, it is important to note that we are not directly comparing protein concentrations across analysis platforms. Additionally, despite the platform similarities between MSD and Luminex, a higher proportion of samples were below the LOD for Luminex, which is not unexpected given MSD’s larger dynamic range14,15.
The triplicate sample analyses performed on the Olink platform offers an opportunity to assess the precision of this assay and its performance analyzing both high- and low-abundance proteins. Mean CVs were calculated as an estimate of precision for each protein, and were <10% for the majority (n= 67, 70.5%) of proteins. However, in each instance where at least one of the measurements were below the LOD for the Olink assay, CVs for the triplicates were >10% and ranged as high as >100%. For example, 95% of samples were below the LOD for at least one of the three Olink measurements for IL4, IL5, and IL13, and mean CVs were 62.43%, 101.39%, and 56.04% for each of these proteins, respectively. This level of variability suggests that observations outside of the dynamic range of each assay are, not unexpectedly, less precise and reliable. For cytokines that are of particular interest to respiratory health researchers that appear to be of low abundance in the nasal fluid – e.g. IL4, IL13 – analysis with a more sensitive assay (e.g. MSD) or individual ELISAs may be preferable. Repeated analysis of samples using the MSD and Luminex platforms would provide useful information about the precision of these assays and should be addressed in future studies.
To our knowledge, this is the first study comparing the performance of protein analysis platforms for the analysis of nasal epithelial lining fluid. Other studies comparing multiplexed protein analysis platforms have largely focused on the analysis of blood components and include other popular assay technologies, such as SOMAscan and Myriad Rules Based Medicine (RBM)7–10. A recent large-scale comparison of proteomics platforms for the analysis of serum and plasma samples from multiple respiratory health cohorts showed significant variation in correlation among platforms tested (including comparisons of SOMAscan with Myriad RBM, Olink, MSD, and individual ELISAs)8. Median Spearman correlation coefficients for the nine cytokines tested by both SOMAscan and MSD were poor and ranged from 0.28–0.38 in the two cohorts examined, while median r=0.36 for proteins tested by both SOMAscan and Olink in a separate cohort of patients; there were no direct comparisons of cytokine measurements by Olink and MSD. Another recent analysis examined correlations between the SOMAscan and Olink platforms in serum samples and found that the minority of 817 proteins tested by both platforms were well-correlated (14.7% r≥0.75)9. Analyses comparing an earlier version of the Luminex and MSD platforms demonstrated similar results with regard to the sensitivities of each assay and demonstrated good correlation for IL-8 quantitation, but showed significant variability with regard to the absolute quantity of analyte detected7. The variable results observed in these previous studies highlight the need for further testing and validation across platforms to determine the optimal assay for each sample type and protein of interest and provide assurance of reproducibility across studies.
Our study demonstrates that proteins of interest can be quantified with consistency in nasal fluid samples using widely available multiplexed protein analysis platforms. There is growing interest in the analysis of nasal epithelial lining fluid as a non-invasive means of characterizing immunologic responses at the level of the respiratory epithelium16. This methodology may be particularly useful for quantifying airway inflammation in asthma and allergy research and assessment of acute and chronic immunologic responses to environmental exposures in the upper airway17–19. Based on our results, noninvasive sampling of the nasal mucosa for proteomic analysis is a promising area for future large-scale studies. The optimal platform may vary based on the relative abundance of cytokines and proteins of interest in the nasal fluid when studying higher-risk individuals – i.e. those with respiratory or allergic disease, or with a history of exposures of interest. Overall, our findings confirm that analysis of nasal epithelial lining fluid is feasible, and that protein analysis platforms are comparable in their measurement of higher-abundance protein biomarkers in nasal fluid.
Supplementary Material
Sources of financial support
(National Institutes of Health Grant/Award Numbers): Zetlen: T32HL116275
Maecker: 2U19AI057229, 2U19AI104209
Nadeau: A gift from the Sean N. Parker Center for Asthma & Allergy Research at Stanford University
Rebuli: T32ES007126, K01ES032837, R21ES032928, CR 83578501 - Disclaimer: This research was funded in part by U.S. EPA Cooperative Agreement CR83578501, but has not been subjected to review and does not necessarily reflect EPA policy.
Rice: R01ES031252, P30ES000002, U01HL146408
Footnotes
The authors declare that they have no conflicts of interest with regard to the content of this report.
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
References
- 1.Kingsmore SF. Multiplexed protein measurement: Technologies and applications of protein and antibody arrays. Nat Rev Drug Discov. 2006;5(4):310–320. doi: 10.1038/nrd2006 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Chawes BLK, Edwards MJ, Shamji B, et al. A novel method for assessing unchallenged levels of mediators in nasal epithelial lining fluid. J Allergy Clin Immunol. 2010;125(6):1387–1389.e3. doi: 10.1016/j.jaci.2010.01.039 [DOI] [PubMed] [Google Scholar]
- 3.Majewski S, Zhou X, Miłkowska-Dymanowska J, Białas AJ, Piotrowski WJ, Malinovschi A. Proteomic profiling of peripheral blood and bronchoalveolar lavage fluid in interstitial lung diseases: An explorative study. ERJ Open Res. 2021;7(1). doi: 10.1183/23120541.00489-2020 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Takahashi K, Pavlidis S, Ng F, et al. Sputum proteomics and airway cell transcripts of current and ex-smokers with severe asthma in U-BIOPRED: an exploratory analysis. doi: 10.1183/13993003.02173-2017 [DOI] [PubMed] [Google Scholar]
- 5.Payton AD, Perryman AN, Hoffman JR, et al. Cytokine signature clusters as a tool to compare changes associated with tobacco product use in upper and lower airway samples. Published online 2022. doi: 10.1152/ajplung.00299.2021 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Rebuli ME, Speen AM, Clapp PW, Jaspers I. INNOVATIVE METHODOLOGY Novel applications for a noninvasive sampling method of the nasal mucosa. Am J Physiol Lung Cell Mol Physiol. 2017;312:288–296. doi: 10.1152/ajplung.00476.2016.-Reliable [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Chowdhury F, Williams A, Johnson P. Validation and comparison of two multiplex technologies, Luminex® and Mesoscale Discovery, for human cytokine profiling. J Immunol Methods. 2009;340(1):55–64. doi: 10.1016/j.jim.2008.10.002 [DOI] [PubMed] [Google Scholar]
- 8.Raffield LM, Dang H, Pratte KA, et al. Comparison of Proteomic Assessment Methods in Multiple Cohort Studies. Proteomics. 2020;20(12):e1900278. doi: 10.1002/PMIC.201900278 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Haslam DE, Li J, Dillon ST, et al. Stability and reproducibility of proteomic profiles in epidemiological studies: comparing the Olink and SOMAscan platforms. Proteomics. 2022;22(13–14):2100170. doi: 10.1002/PMIC.202100170 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Katz DH, Robbins JM, Deng S, et al. Proteomic profiling platforms head to head: Leveraging genetics and clinical traits to compare aptamer- And antibody-based methods. Sci Adv. 2022;8(33):5164. doi: 10.1126/SCIADV.ABM5164/SUPPL_FILE/SCIADV.ABM5164_TABLES_S1_TO_S6.ZIP [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Thwaites RS, Ito K, Chingono JMS, et al. Nasosorption as a Minimally Invasive Sampling Procedure: Mucosal Viral Load and Inflammation in Primary RSV Bronchiolitis. Br Rep J Infect Dis J Infect Dis ®. 2017;215:1240–1244. doi: 10.1093/infdis/jix150 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Lopez EM, Stepp WH, Ebert CS, et al. Site-specific detection and differential levels of immune mediators in the sinonasal mucosa. Int Forum Allergy Rhinol. Published online 2022. doi: 10.1002/ALR.23056 [DOI] [PubMed] [Google Scholar]
- 13.Assarsson E, Lundberg M, Holmquist G, et al. Homogenous 96-plex PEA immunoassay exhibiting high sensitivity, specificity, and excellent scalability. PLoS One. 2014;9(4). doi: 10.1371/journal.pone.0095192 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Günther A, Becker M, Göpfert J, Joos T, Schneiderhan-Marra N. Comparison of Bead-Based Fluorescence Versus Planar Electrochemiluminescence Multiplex Immunoassays for Measuring Cytokines in Human Plasma. Front Immunol. 2020;11:2486. doi: 10.3389/FIMMU.2020.572634/BIBTEX [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Zhang Y, Li X, Di YP. Fast and efficient measurement of clinical and biological samples using immunoassay-based multiplexing systems. Methods Mol Biol. 2020;2102:129–147. doi: 10.1007/978-1-0716-0223-2_6/FIGURES/1 [DOI] [PubMed] [Google Scholar]
- 16.Rebuli ME, Glista-Baker E, Hoffman JR, et al. Electronic-cigarette use alters nasal mucosal immune response to live-attenuated influenza virus: A clinical trial. Am J Respir Cell Mol Biol. 2021;64(1):126–137. doi: 10.1165/RCMB.2020-0164OC/SUPPL_FILE/DISCLOSURES.PDF [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Noah TL, Zhou H, Zhang H, et al. Diesel exhaust exposure and nasal response to attenuated influenza in normal and allergic volunteers. Am J Respir Crit Care Med. 2012;185(2):179–185. doi: 10.1164/rccm.201103-0465OC [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Hentschel J, Müller U, Doht F, et al. Influences of nasal lavage collection-, processing- and storage methods on inflammatory markers-Evaluation of a method for non-invasive sampling of epithelial lining fluid in cystic fibrosis and other respiratory diseases. J Immunol Methods. 2014;404:41–51. doi: 10.1016/j.jim.2013.12.003 [DOI] [PubMed] [Google Scholar]
- 19.Scarpa MC, Kulkarni N, Maestrelli P. The role of non-invasive biomarkers in detecting acute respiratory effects of traffic-related air pollution. Clin Exp Allergy. 2014;44(9):1100–1118. doi: 10.1111/cea.12373 [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.