Categorizing Concentration Confidence: A Framework for Reporting Concentration Measures from Mass Spectrometry-Based Assays

Abstract

Background:

Innovation in mass spectrometry-based methods to both quantify and perform discovery has blurred the lines between targeted and untargeted assays of biospecimens. Continuous data—concentrations or intensity values generated from both methods—can be used in statistical analysis to determine associations with health outcomes, but concentration values are needed to compare measurements from one study to another to inform policy making decisions and to develop clinically relevant thresholds. As a single solution for discovery and quantitation, new hybrid-type assays derive concentration values for chemicals or metabolites but with varying degrees of uncertainty that may be greater than traditional quantitative assays. There is no current single standard to guide reporting bioassay concentrations or their uncertainty in concentration values from hybrid assays. Even when measures are robust, obtained with high scientific rigor, and provide valuable data toward risk assessment, unknown uncertainty can lead to bias in interpretation of reported data or omission of reported data that does not meet the strict criteria for absolute quantitation.

Objective:

The objective of this commentary is to articulate a scheme that enables investigators across bioanalytical fields to easily report analyte measurement assurance on the same scale from quantitative, untargeted, or hybrid assays that include a range of concentration confidences.

Discussion:

We propose a simple scheme to report concentrations for targeted and untargeted analytes. Level 1 is a confirmed concentration following established tolerances in a fully quantitative assay while level 5 is a tentative intensity from a typical untargeted assay. This framework enables easy communication of uncertainty in concentration measurements to aid cross-validation, meta-analysis, and extrapolation across studies. It will facilitate interpretation while supporting analytical advancement and allow clear and concise measurement reporting across a broad range of confidences. https://doi.org/10.1289/EHP15465

Introduction

Uncertainty in measures affects the direction and size of estimates. Uncertainty can lead to epidemiological reports with conflicting results, even in well-designed studies designed to test causal hypotheses in humans. In addition, uncertainty and then extrapolation among studies can affect the accuracy of dose–response relationships in toxicology or drug efficacy. While reporting uncertainty in quantitative analysis uses established standards, developing new analytical methods requires new tools to report the uncertainty associated with those methods. For example, the Biomonitoring, Environmnetal Epidemiology, and Short-lived Chemicals (BEES-C) instrument evaluates the quality of biomonitoring data on short-lived chemicals and helps users better understand the limitations.¹ In addition, annotation confidence criteria help users understand the specificity of measurements in untargeted analysis for biological and exposure interpretation.² However, while new analytical advances combine untargeted and quantitative analysis using high-resolution mass spectrometry (HRMS) and low resolution mass spectrometry (QqQ) techniques, there is currently no tool to report the uncertainty of measures from these hybrid workflows easily.

Researchers combine the strengths and weaknesses of untargeted and targeted analyses to develop hybrid strategies to quantify exogenous chemicals and endogenous metabolites, while simultaneously measuring additional metabolites and chemicals with less precision and accuracy (Figure 1).^3–6 A traditional discovery workflow for an untargeted assay includes analysis of samples using HRMS. Data are collected on hundreds to thousands of analytes, and relative peak intensities across two or more groups (e.g., cases and controls, exposed and not exposed) are compared using advanced statistics. In contrast, quantitative assays with targeted analysis focus on the precise measurement of one to tens of a priori determined chemicals to obtain absolute concentrations using strict criteria to minimize uncertainty. These assays provide higher throughput and lower costs than an untargeted assay that employs HRMS techniques. However, there is increased interest in performing quantitation and discovery at the same time (i.e., a hybrid approach) because of limited sample availability, the cost and time to perform multiple assays, and the advancement of HRMS instrumentation and quantitative software.⁷ Newly developed workflows include simultaneous quantitation and discovery (SQUAD) with HRMS technology: It combines an a priori set of analytes with authentic chemical standards and calibration curves in a targeted workflow and then remines the data for new analytes in an untargeted workflow.⁸ There is a label-free hybrid approach combining HRMS tandem mass spectrometry (MS/MS) data collection across the metabolome coupled with sequential dilution and quantitation using a QqQ.⁹ There is a pseudo-targeted method that uses HRMS to generate a targeted list of multiple reaction monitoring (MRM) ion pairs, which are then measured in samples on a QqQ to increase sensitivity.^10,11 These advancements in quantitation using untargeted methodologies accelerates science with valuable data, but they also bring ambiguity in error associated with the reported concentrations that are not easy to convey clearly or concisely.

Figure 1.

Hybrid analysis combines the advantages of untargeted and targeted analysis for risk assessment.

Here, we propose a scheme to report confidence in concentration measures that do not meet the accuracy and uncertainty criteria expected from an absolute quantitative assay but are still robust measures valid to help address policy and risk. Our scheme enables investigators to compare across assays more easily by binning reporting measures. It is not intended to replace established protocols to report absolute quantitation or untargeted analysis. It can complement these existing protocols and fill gaps in reporting for a range of assays not classically described by either technique in a single inclusive reporting scheme. For example, it is common for quantitative and untargeted assays to use quality control (QC) samples to calculate interbatch and intrabatch variability.^6,12 Therefore, using these metrics as criteria for a congruent scale across the range of assays is straightforward. Through this framework, we hope to facilitate transparency, improve data sharing, and initiate discussions with epidemiologists and risk assessment researchers for more robust and versatile use of these chemical measurements.

Concentration Confidence Scheme

In Table 1, we propose a concentration confidence scheme to report confidence in analyte concentrations generated from nontraditional targeted and untargeted assays (e.g., hybrid assays) that is also applicable to traditional targeted and untargeted workflows. To determine the confidence level for reporting a given analyte, we suggest that a user begins at level 1 and work down to level 5 until the analyte meets all method criteria and metric thresholds in that level. This should be repeated for each analyte in the study because the measurement confidence varies across the different analytes, especially in hybrid assays. Reporting the concentration confidence as level 1, 2, 3, 4, or 5 for each analyte should be included in the manuscript Methods section or provided in the supplementary tables along with additional method validation metrics such as limit of detection (LOD), detection frequencies, coefficient of variation (CV), and etc. Further, we encourage reporting the analyte concentration confidence into shared data repositories, such as the Human Health Exposure Analysis Resource (HHEAR) Data Center (https://hheardatacenter.mssm.edu/) and Metabolomics Workbench (https://www.metabolomicsworkbench.org), to enable researchers to include metadata reporting with their individual analyte measures. This framework also accommodates variability in calibration practices, such as the use of surrogate vs. matrix-matched standards,¹³ and addresses quantification challenges across both xenobiotics and endogenous chemicals.¹⁴ It enhances harmonizability for targeted, untargeted, and hybrid analyses by providing guidance on selecting confidence levels applicable for all workflows.

Table 1.

Concentration confidence scheme. Proposed reportable quantification confidence levels for analytes based on thresholds of accuracy and uncertainty obtained from quantitative, hybrid, and untargeted assays.

Quantification confidence level	Method criteria			Threshold metrics			Quantification confidence indicator
	Native standards	Labeled internal standards	Calibration curve	Accuracya	Intrabatch variabilityb	Interbatch variabilityb
1	Analyte-specific	Analyte-specific	Multipoint ($\ge 6$ levels), internal	$\pm$ Up to 20% RSD	$\le 15\%$	$\le 15\%$	Confirmed
2	Analyte-specific	Analyte-specific, class-specific, or similar physicochemical properties	Multipoint ($\ge 6$ levels), internal or surrogate	$\pm$ Up to 30% RSD	$\le 15\%$	$\le 30\%$	Strong
3	Analyte-specific	Analyte-specific, class-specific, or similar physicochemical properties	Multipoint ($\ge 6$ levels), internal or surrogate	$\pm$ Up to 40% RSD	$\le 30\%$	$\le 50\%$	Good
4	Nonspecific	None	Single or multipoint with 1–5 levels, internal or surrogate	$\pm$ More than 40% RSD, or none	No threshold	No threshold	Fair
5	None	Not used for quantitation	None	None	Not applicablec	Not applicablec	Tentative

Note: RSD, relative standard deviation.

^a Accuracy for level 1 is calculated from standard reference materials (SRM), which can be National Institute of Standards and Technology (NIST) samples, proficiency testing materials, or well-characterized pools used by multiple labs. For levels 2–4, reference material can also be spiked in the sample matrix. Quality controls (QCs) are injected into every analytical run of a project. Analyst confidence in the SRM results can influence the level selection.

^b Interbatch and intrabatch variability (i.e., uncertainty) is calculated from any quality QC sample that is measured repeatedly throughout the analysis.

^c Intrabatch variability $<30\%$ is typical but not required. After normalization and batch correction, interbatch variability is assessed and reported for selected metabolites or chemicals.

Level 1, confirmed.

The criteria for level 1 is the highest confidence reporting and is based on extensive literature regarding absolute quantitation.¹⁵ To achieve confidence level 1, every batch uses isotope dilution with a labeled version of the analytes of interest (i.e., internal standard calibration curve) and a matrix-matched (recommended) or non–matrix-matched calibration curve (performing within 5% deviation of matrix-matched slope calibration curve of at least six nonzero levels). Calibration curves must have $r^2>0.95$ and cover $>95\%$ of the range of study samples (100% is ideal). While an analyte-specific sample preparation protocol is suggested, it is not required as long as thresholds for reporting concentration accuracy of reference standards are met [$\pm 20\%$ relative standard deviation (RSD)], and there is repeatability (intrabatch variability $\le 15\%$) and reproducibility (interbatch variability $\le 15\%$) of QC results. If any of the method criteria or metric thresholds are not met, the analyte does not meet the requirements to report a level 1 and a less confident level must be reported. Confirmed quantification applies when authentic reference standards are analyzed concurrently with samples, matching exact mass, isotope pattern, retention time, and MS/MS spectrum.¹⁶

Level 2, strong.

Level 2 confidence includes method criteria such as isotope dilution using a labeled analog from the chemical class of the analyte of interest (i.e., surrogate calibration curve) and/or a non–matrix-matched calibration curve with six or more levels. While an analyte-specific sample preparation protocol is suggested, it is not required as long as the metric thresholds for reporting concentration accuracy ($\pm 30\%$ RSD) and intrabatch variability ($\le 15\%$) of the reference standards are met for every batch, and interbatch variability is $\le 30\%$. This level includes analytes for which no externally validated reference standards are available. Therefore, even when metric thresholds meet those for level 1 [i.e., accuracy is $\pm 20\%$ RSD, uncertainty ($\le 15\%$)], this is the highest confidence level possible when analyte-specific labeled standards are not used.

Level 3, good.

Analysis is performed using the method criteria for level 1 or level 2 (analyte-specific standards or surrogate, matrix-matched curve or non–matrix matched with six or more levels) but does not achieve the metric threshold criteria for uncertainty for level 2. Accuracy should be $\pm 40\%$ RSD, and intrabatch and interbatch variability should be $\le 30\%$ and $\le 50\%$, respectively. If any of these metric thresholds are not met, a less confident level must be reported.

Level 4, fair.

Analysis is performed using an external calibration curve without isotope dilution and a limited number of calibration levels ($n=1–5$). Reference standards are not required, although uncertainty of the analyte should be reported even if there is no threshold. There is no threshold for intrabatch or interbatch variability at this level. Therefore, this is the most confident level that can be reported for an analyte that uses only one to five calibration curve levels (analyte-specific standards or surrogate, matrix matched curve or non–matrix matched) in the method. This is also the confidence level to report when at least one of the metric thresholds is not met for level 3 (i.e., accuracy $>\pm 40\%$ RSD and intrabatch and interbatch variability $>30\%$ and $>50\%$, respectively).

Level 5, tentative.

Criteria for the lowest confidence level are reserved for analytic measures that do not meet any of the requirements for quantitation. Intrabatch and interbatch variability should be reported. Robust data analysis allows intensity measures (not concentrations) to be used as continuous measures in statistical analysis. Concentration units are not reported with these measures. This is the most confident level that can be reported for an analyte in the absence of any calibration curve or the absence of isotope labeled standards (analyte-specific or surrogate) used in the method.

We recognize that an analyte may be classified at a lower level on the scheme due to analytical restrictions but may be practically equivalent to a higher level. For example, labeled standards may not be commercially available, or an externally validated reference sample for assessing accuracy may not be available to reach level 1. It is common practice in complex lipidomics experiments that seek to quantify hundreds of lipid species in the same run¹⁷ to use nonphysiologically occurring lipid species¹⁸ or to use an external standard curve (constructed from a common lipid) to quantify all individual lipid species in a particular class.¹⁹ Additionally, nonconventional matrices such as hair, teeth, dried blood spots, or nails may not have proper QC materials, so surrogates must be used.²⁰ While these criteria may require reporting a “lower” level of quantitation confidence according to the current scheme and expert experience, concentrations measured in these biological samples are still valuable and valid.

Selected Examples

Here, we provide examples of typical and novel analysis workflows used in biomedical and environmental analysis and the respective concentration confidence levels that will be reported using the proposed level scheme (Table 1).

Case 1.

Urinary metabolites of phthalates²¹ and phenols²² were measured according to CDC-published methods.^23,24 This involved sample-specific preparation using enzyme deconjugation and sample cleanup after solid-phase extraction. Labeled standards for each analyte of interest were utilized for isotope dilution with a seven-level calibration curve. Analytes were quantified using a QqQ mass spectrometer with MRM. Batch-wise RSDs of QCs during analysis of the study specimens were $<20\%$ for target analytes present in reference or fortified materials above the LOD. Intrabatch precision for QC analytes above the LOD had a CV of $<10\%$, while interbatch precision remained below 15%. Recovery rates for batch QCs ranged from 80% to 120%. Quality assurance (QA) and QC protocols include those described by the Mount Sinai HHEAR Targeted Lab,^25,26 and these assays meet the concentration confidence Level 1 criteria for all analytes.

Case 2.

A dilute-and-shoot assay was developed to measure six carotenoids by QqQ mass spectrometry (lutein, $\mathrm{\beta }$-cryptoxanthin, zeaxanthin, $\mathrm{\alpha }$-carotene, $\mathrm{\beta }$-carotene, and lycopene).²⁷ To reduce labor time and prevent potential metabolite degradation, plasma samples were deproteinized with methanol and centrifuged, followed by injection. Seven level calibration curves for each analyte were generated with a single internal standard, $\mathrm{\alpha }$-tocopheryl acetate. Among the six carotenoids, lutein and zeaxanthin coeluted and were measured as a mixture. Furthermore, lycopene stereoisomers were summed to obtain the final concentrations. Linearity for all carotenoids was $>0.99$. Intrabatch and interbatch precision values were evaluated with National Institute of Standards and Technology (NIST) samples (ranging from 6.2% for $\mathrm{\beta }$-carotene to 12.4% for $\mathrm{\beta }$-cryptoxanthin for interbatch CV). A matrix effect from 105% to 115% was observed. Accuracy at low concentrations of NIST samples was $<20\%$ for all metabolites, except for $\mathrm{\alpha }$-carotene, which was 30%. Accuracy at high concentrations was 35.9% for lycopene and 108% for $\mathrm{\beta }$-cryptoxanthin. $\mathrm{\alpha }$-Carotene and $\mathrm{\beta }$-carotene meet the criteria for confidence reporting level 2, lycopene meets the criteria for confidence reporting level 3, whereas $\mathrm{\beta }$-cryptoxanthin and a mix of lutein and zeaxanthin meet the criteria for level 4.

Case 3.

Using an untargeted metabolomics workflow with HRMS, a hybrid assay was developed to quantify per- and polyfluoroalkyl substances (PFAS) including perfluorooctanesulfonic acid (PFOS) and perfluorooctanoic acid (PFOA) in plasma (example based on Goodrich et al.²⁸). A six-level matrix-matched calibration curve using isotope dilution and labeled PFOS and PFOA was used. All endogenous metabolites were measured in the absence of labeled standards or calibration curves. Plasma metabolites, PFOS, and PFOA in the study samples and NIST SRM1957 and 1958 were extracted using a simple solvent protein crash. PFOS and PFOA were measured using the MS1 ion within a m/z of $5\text{\hspace{0.17em}ppm}$ from the standards. Accuracy and uncertainty for PFOS and PFOA were within 10% and $<15\%$, respectively. All other endogenous metabolites quality control was unreported. PFOS and PFOA meet the criteria for confidence reporting level 1, while all other metabolites and chemicals measured in the assay meet the criteria for confidence reporting level 5.

Case 4.

A hybrid assay named globally optimized targeted mass spectrometry (GOT-MS) was developed to expand and optimize detection capabilities and improve quantification, notably amino acids.²⁹ A global search of precursors and products ions by liquid chromatography-triple quadrupole (LC-QqQ) mass spectrometry was followed by selecting MRM transitions for identification and quantitative analysis. Five samples were spiked with labeled ${\mathrm{U}\text{-}}^{13}{\mathrm{C}\text{-}}^{15}\mathrm{N}$-amino acids and diluted to generate five-level calibration curves for amino acids without using native standards. Isoleucine and Leucine coeluted and were measured as a mixture. Intrabatch variability was $<3.0\pm 2.2\%$ without amino acid labeled standard normalization and $<4.1\pm 3.4\%$ with amino acid labeled normalization. Interbatch variability of amino acids decreased from $8.3\pm 3.4\%$ to $4.8\pm 4.0\%$ after normalization with labeled amino acids. The linearity ($R^2$) of amino acids was $0.82\pm 0.26\%$ for GOT-MS. Amino acids meet the criteria for confidence reporting level 4, whereas all other metabolites and chemicals measured meet the criteria for confidence reporting level 5.

Discussion

The availability of analytic methods that can increase the biomarker information obtained from nonrenewable biological samples using HRMS is an important advancement in environmental health research. However, the authors’ experience is that there is skepticism about the comparability of biomarker concentrations obtained through HRMS and the traditional targeted sample analyses, and this has been a roadblock to accepting study results that report HRMS-derived concentrations. In particular, establishing exposure–outcome associations requires results from multiple studies in varied populations where exposure measurements are considered comparable. The proposed concentration confidence levels describe the uncertainty around concentration measures from untargeted, hybrid, and quantitative analysis on the same scale. Therefore, it can provide information for the evaluation of whether the results of a study that uses HRMS-derived concentrations can be used in combination with other studies to establish exposure–outcome associations. For example, an analyte reported from different studies with measurement confidence level 1 can easily be combined for meta-analysis and cross-study evaluation. Within the HHEAR program, measurements that reach levels 1 and 2 are considered harmonizable based on the published or publicly available proficiency testing materials’ reference values and tolerance ranges that indicate an accuracy of $\ge 70\%$ recoveries and a precision of $\le 30\%$ RSD as an acceptable benchmark (CHEAR/HHEAR Targeted Lab Analysis and Quality Assurance Working Group, personal communication). While it is likely that additional levels can also be considered, when and how studies from multiple confidence levels can be aggregated requires further evaluation, which will influence the development and modification of future schemes. For example, an analyte reported from different studies with measurement confidence level 1 can be easily combined for meta-analysis and cross-study evaluation.

In addition to reporting concentration confidence for each analyte according to Table 1, we propose a way to report assay performance using accuracy criteria as well as intrabatch and interbatch variability of the analytes. Therefore, this scheme can provide a reporting tool for cross-validation of untargeted and hybrid assays in the absence of proficiency testing and round-robin validation schemes, which are common to quantitative analysis. By using these confidence levels as targets when optimizing new assays, this framework can influence study design and analytical method development.

There are particular challenges to quantitation of some exogenous and endogenous chemicals in complex matrices like plasma and urine. As a result, greater confidence level in the concentration confidence scheme is achieved through approaches that account for preanalytical variability and matrix effects. For example, a matrix-matched calibration curve is used as the same biological matrix as the samples to account for potential degradation during sample preparation and analysis.³⁰ This approach ensures that calibration standards and samples are affected similarly by matrix components.³¹ To manage ionization variability and matrix effects, especially at low concentrations typical of exogenous chemicals, usage of isotopically labeled internal standards is highly effective³² because these standards have identical chemical and physical properties to the target analyte and experience the same ionization effects, allowing for accurate quantification even in complex matrices.³³ When analytical standards are limited, which is particularly relevant for emerging exogenous chemicals, surrogate analyte or surrogate matrix approaches can be utilized.³⁴ The surrogate analyte method uses a stable isotope–labeled standard as a surrogate for calibration in the actual biological matrix, while the surrogate matrix method employs an alternative matrix devoid of the target analyte.¹⁴ Both approaches require demonstration of parallelism between calibration standards and the target analyte in the biological matrix to ensure accurate quantification.³⁴ Where analyte-free matrices are not available, typical of endogenous chemicals, techniques such as standard addition or background subtraction can be employed.¹⁴ By combining these strategies as needed for selected exogenous and endogenous chemicals, analysts can develop robust methods that achieve high confidence in concentration measures across different types of analytes, contributing to a comprehensive framework for reporting concentration measures from mass spectrometry-based assays.

While valuable as providing a framework to compare across methods, we recognize some limitations to this scheme, which can lead to subjectivity in reporting. In particular, there may be higher or lower confidence in a given sample measurement due to measurements below a limit of detection (LOD), the use of blank correction or recovery correction, calibration curve extrapolation outside calibration levels, or other analytical conditions that could apply during data acquisition and curation. The scheme does not address the range of annotation in untargeted data, which can affect analyte specificity.² Further, the choice of a surrogate for quantitation can also impact the reliability of the measures, especially for complex biological samples.³⁵ As a result, regulatory clinical or environmental risk assays, in particular, will likely require significant adaptation to the proposed scheme to meet standards or fit-for-purpose.³⁶ However, as an initial framework, we capture most bioanalytical scenarios while maintaining a simple reporting process.

The variability in analytical capacity across institutions and limited access to high-quality standards are indeed significant barriers to achieving level 1 confidence in the proposed framework similar to challenges discussed by Clarke et al.³⁷ For those laboratories with reliability concerns for quantification confidence, we propose adopting a tiered approach similar to that suggested by Sarmad et al.³⁸ This “fit-for-purpose” strategy can accommodate various research settings and analytical capabilities, allowing laboratories to implement the highest level of quality assurance feasible within their constraints.³⁸ For laboratories with limited resources, we recommend prioritizing the principle of utility in analyte selection, focusing on practical value rather than quantity.³⁹ Additionally, emphasizing methodological innovation for developing high-throughput equipment could contribute to increased automation and accessibility.³⁹ To address the challenge of limited access to high-quality standards, we suggest focusing on the development of reference methods and materials across laboratories.³⁹ This can help ensure reliability and reproducibility across different laboratories, even those with resource constraints. By incorporating these considerations and providing flexible solutions, we believe the framework can be used across a wide range of research settings, ultimately improving the quality of reporting and comparability of mass spectrometry-based assays in various fields.^2,39

Our confidence scheme focused on molecular mass spectrometry methods for organic chemicals. However, there are similarly stringent QC thresholds in atomic mass spectrometry for quantifying trace elements. Atomic mass spectrometry is based on argon plasma, which dramatically reduces matrix components compared to molecular mass spectrometer applications.⁴⁰ In addition, the main factors that determine the relative sensitivity of different elements are fixed by the atomic first ionization potential, atomic mass, and isotopic abundance.⁴⁰ Therefore, in some applications, external calibration methods or using single standards can generate quantitative measures that may or may not reach the thresholds considered for absolute quantification. Therefore, the development of a similar scheme may be helpful to the trace elements community for discussion and use.

Technology advancements in HRMS now enable increasingly reliable measurements. These methods approach reliability provided by quantitative assays that were once limited to QqQ MS. As novel bioanalytic workflows continue to strive toward broader analyte coverage, lower limits of detection, and better precision, established criteria for reporting only targeted or untargeted measures—at opposite ends of the spectrum—are no longer sufficient. Therefore, we propose a scheme for reporting concentration confidence measures from MS-based bioassays. The scheme maximizes the utility of the available measurement techniques while still communicating the associated error. We anticipate that this scheme will help clarify the muddled middle—to facilitate clear and concise reporting of measures on the same scale across a broad range of confidences, including targeted and semiquantitative measures. The current scheme is simple and provides flexibility for application in emerging technologies to report measurements or as target metrics for method development. Through individual use and dissemination of the concept at symposia, conferences, and consortia with the metabolomics, exposomics, and biomonitoring communities, we aspire that the community will adopt and refine this framework. As analytical methods evolve and adapt, we encourage discussion across the bioanalytical communities on concentration confidence schemes and best reporting practices.

Conclusions and opinions are those of the individual authors and do not necessarily reflect the policies or views of EHP Publishing or the National Institute of Environmental Health Sciences.