The Time Has Come for Quantitative Protein Mass Spectrometry Tests That Target Unmet Clinical Needs

Abstract

Protein mass spectrometry (MS) is an enabling technology that is ideally suited for precision diagnostics. In contrast to immunoassays with indirect readouts, MS quantifications are multiplexed and include identification of proteoforms in a direct manner. Although widely used for routine measurements of drugs and metabolites, the number of clinical MS-based protein applications is limited. In this paper, we share our experience and aim to take away the concerns that have kept laboratory medicine from implementing quantitative protein MS. To ensure added value of new medical tests and guarantee accurate test results, five key elements of test evaluation have been established by a working group within the European Federation for Clinical Chemistry and Laboratory Medicine. Moreover, it is emphasized to identify clinical gaps in the contemporary clinical pathways before test development is started. We demonstrate that quantitative protein MS tests that provide an additional layer of clinical information have robust performance and meet long-term desirable analytical performance specifications as exemplified by our own experience. Yet, the adoption of quantitative protein MS tests into medical laboratories is seriously hampered due to its complexity, lack of robotization and high initial investment costs. Successful and widespread implementation in medical laboratories requires uptake and automation of this next generation protein technology by the In-Vitro Diagnostics industry. Also, training curricula of lab workers and lab specialists should include education on enabling technologies for transitioning to precision medicine by quantitative protein MS tests.

Introduction

Proteins have been measured in clinical laboratories as markers for several disease states since the early examples of urinary albumin as indicator of kidney disease in 1827 and the first tumor marker (Bence Jones protein) in 1845.¹ Nowadays, proteins are routinely quantified using automated immunoassays, but it is widely acknowledged that this technique has several major drawbacks.² In the past decades, mass spectrometry (MS) has demonstrated feasibility for quantitation of protein biomarkers with clear advantages compared to immunoassays.^1,2 Such MS protein tests follow a bottom-up strategy, where peptide quantities are used to determine concentrations of protein targets and isoforms for specific clinical indications.

Hitherto, the adoption of MS-based protein tests in medical laboratories is rather limited, likely because the workflow of multiplexed (proteoform) testing is relatively complex, quality requirements with regard to allowable measurement uncertainty are stringent, and automation is not in place yet. It is of key importance to follow quality requirements that are common practice for all tests within medical laboratories. The earliest guidance for evaluation of liquid chromatography (LC) MS-based methods started from experience with small molecules^3,4 and was based on recommendations of the US Food and Drug Administration (FDA) and the European Medicines Agency (EMA). More specific LC–MS requirements have been available since the publication of guideline C50-A from the Clinical and Laboratory Standard Institute (CLSI) in 2007,⁵ which was later revised into a more systematic approach for development and validation of LC–MS methods in the CLSI C62-A guideline.^6,7 At the same time, the MS-based proteomics community proposed a three tier system using a fit-for-purpose approach for the discovery of protein biomarkers and anticipated translation into a medical test.⁸ This latter guidance document provides a good starting point for analytical validation but it does not include clinical validation. For the development of tier 1 assays that are suitable for implementation as a clinical test, it does not address their adoption in medical laboratories. A CLSI document C64 dedicated to quantitation of proteins by MS techniques is currently being drafted.⁹ As of May 2022, in Europe, the development of MS-based tests will also have to be compliant with the new regulations for In-Vitro Diagnostics (IVDR) 2017/746.¹⁰ A working group within the European Federation for Clinical Chemistry and Laboratory Medicine (EFLM) has established a cyclical framework for test evaluation of both CE-IVDs and laboratory-developed tests (LDTs) (Figure 1).^11,12 Importantly, development of an LC–MS method for quantitation of (multiple) biomarkers should only be commenced if there is an unmet clinical need in the clinical care pathway.¹¹⁻¹³ Recently, a checklist was reported to aid in the discussions with clinicians to identify and verify current clinical needs.¹⁴ Once the clinical gaps are identified, and the decision is made to invest in solving the gap, a test can be developed. This process consists of a preparation phase, followed by the method development phase and subsequent analytical and clinical validation of the anticipated test (Table 1), which should be in compliance with the upcoming IVDR 2017/746.^10,15 The total test process including implementation in a routine setting should be considered at an early stage in the development. This ensures high quality tests that meet predefined analytical and clinical performance specifications. From a technical point of view, proteotypic peptides are commonly measured using a triple quadrupole (QQQ)-instrument.^8,16,17 The peptide quantities reflect protein concentrations, provided that digestion conditions allow equimolar conversions.^18,19 Veracity of the protein quantity is inferred via agreement between multiple proteotypic peptides.²⁰ Furthermore, it is assumed that the protein of interest is present in the intact form in the matrix and that digestion is not hampered by potential post-translational or chemical modifications.²¹ Ideally, these assumptions should be checked and corrected for during rigorous analytical method validation.

Figure 1.

Protein MS as an alternative technology for immunoassay-based testing, potentially providing an additional layer of clinical information. The interplay between the key elements of the test evaluation process is summarized in a cyclical framework. The wheels dynamically link the elements that are driven by the clinical needs and intended use of the envisioned test in a clinical pathway aiming for improved patient outcome. In this paper, analytical and clinical performance of MS-based protein testing is discussed. Evaluation of the other three elements of the cyclical framework requires more data that will come available once MS-based protein testing is adopted in medical laboratories and used for longer time to evaluate thousands of specimens.

Table 1. Quality Criteria during Preparation and Implementation Phases of a Quantitative Protein MS Method to Be Used in Clinical Chemistry Routine Practice and Ahead of Clinical Evaluations.

no.	phase	action	key ref
0	identifying and verifying an unmet clinical need		(11, 12)
1	preparation phase including preanalytics	defining the measurand(s)	(22, 23)
		setting clinical and analytical performance specifications; conceptualize total testing process	(30−33)
		preanalysis; prevention of diagnostic errors at the stage of specimen collection	(36, 38, 39)
2	method development and analytical performance evaluation	selection of proteotypic peptides; quantitation and confirmation	(19, 46, 47)
		sample preparation; digestion conditions and use of proteases	(50, 51, 54)
		internal standard strategies; from SIL peptide to SIL protein and cost effectiveness	(20, 71)
		peptide prepurification; sample clean up and solid phase extraction (SPE)	(76, 77)
		LC–MRM–MS/MS analysis; data evaluation, separation of peptides, (dynamic) MRM detection, ion ratio selectivity	(7, 49, 78)
		calibration and traceability to certified reference materials	(10, 79)
		analytical validation	(6, 15, 81)
		maintaining long-term accuracy; documented in quality management system (QMS)	(81)
3	implementation	education and training; theoretical and practical	(83, 84)
		achieving consistent instrument performance	(81)
		maintenance plans for LC–MS equipment
		system suitability testing	(49,85)
		application checks
		internal quality control	(81, 89, 90)
		external quality assessment	(93−95)
		total testing checks
		troubleshooting: manual/instrument mistakes

In this paper, we focus on the potential of MS-based protein testing as an alternative technology for immunoassay-based testing. Stringent analytical performance and robustness have to be demonstrated before this enabling technology can be adopted into clinical practice. Different phases of test development will be highlighted as well as the rational selection of analytical performance specifications needed for ensuring the intended use of the test. The latter is based on the Milan hierarchy and goes beyond the specific LC–MS criteria for clinical adoption that we reported earlier.⁴

We conclude with presenting longitudinal, four-year analytical performance data of a multiplexed test for the quantitation of serum apolipoproteins.

Achieving Long-Term Robustness: The Preparation Phase

Test development should start with identification and verification of unmet clinical needs within the current clinical pathway (Table 1). Next, the candidate markers that address this need are determined^11,12 followed by a preparation phase that includes defining the measurand(s), predefining analytical and clinical performance specifications, and the purpose and mode of the test in the clinical pathway. Preferably a business-case is also developed. Short-sightedness during the preparation phase will undoubtedly lead to difficulties during method development and possibly result in suboptimal test performance and/or research waste.

Defining the Measurand(s)

A measurand is defined as “a quantity intended to be measured”. Although it may seem trivial to identify the protein measurand, it is emphasized that products from a single gene can be presented as a plethora of so-called proteoforms.²² These proteoforms arise from genetic variations or alternative splicing, but importantly, they can also be the result of post-translational modifications (PTMs). Potential PTMs have been reviewed earlier²³ and include, but are not limited to phosphorylation, methylation, acetylation and glycosylation. For these reasons a protein measurand should be defined at the molecular level. Depending on the clinical relevance of the proteoforms, the measurand could be defined as the sum of all proteoforms originating from a single gene²⁴ (e.g., cardiac troponin²⁵), a subgroup thereof (e.g., specific isoform such as IgG4²⁶), or even a single proteoform (e.g., carbohydrate deficient transferrin).^27,28 The clear definition of the measurand is thus needed to ensure the intended analyte is indeed targeted and quantified. Besides the molecular definition of the analyte the matrix from which the analyte is intended to be measured also requires careful consideration.

Setting Clinical and Analytical Performance Specifications

To ensure that a developed test is fit for clinical purpose, the predefined clinical performance specifications (CPS) should determine the required analytical performance specifications (APS) prior to the initiation of the method development. A hierarchy for APS has initially been developed in the (Stockholm) consensus agreement formulated in 1999 and has been revisited, named the Milan hierarchy, in 2014 by the European Federation of Clinical Chemistry and Laboratory Medicine (EFLM).²⁹ To ensure that the APS of new tests are in agreement with the required clinical performance, analytical performance criteria should ideally be set based on clinical outcome studies. However, as such data is often not available,³⁰ a second strategy is to use biological variation data from intraindividual and interindividual variations (CV_w and CV_g, respectively). In this approach, biological variation is the basis for establishing the minimal, desirable and optimal goals for imprecision (CV_a) and bias (B_a) and calculation of total allowable error for the biomarker test.^31,32 Recently a checklist was established to evaluate studies assessing biological variation³³ and a database with biological variation data may be found at https://biologicalvariation.eu. If biological variation data is not available, APS should be set based on the state-of-the-art performance. For LC–MS based protein quantitation, analytical CVs of <10% should be possible based on our own experience.^34,35 A final consideration in the preparation phase addresses the clinical pathway and role of the test in the pathway.

Preanalysis

The preanalytical phase occurs both outside and inside a medical laboratory. The former includes patient identification and specimen collection and transport, whereas the latter starts with specimen arrival and pre-examination, followed by possible centrifugation (of blood), decapping, and aliquoting. It should be recognized that the preanalysis is very critical for good biomarker research. Therefore, the provenance and quality of the clinical specimens should be documented and validated.³⁶ These challenges are often underestimated. Earlier, the HUPO plasma proteome project identified that the sample processing (time on the bench, centrifugation speed) all affect the quality of the specimen and subsequent analyte recovery.^37,38 Moreover, the efficiency of enzymatic digestion may be altered depending on the matrix,^37,39,40 and stability of proteins or proteoforms may vary in certain matrices. Therefore, the preanalytical phase and specimen collection tubes require careful consideration. In addition, the absolute concentration of the measurand determines the sample preparation strategy, as preconcentration might be necessary, e.g., using MSIA or SISCAPA (Mass Spectrometric ImmunoAssay or Stable Isotope Standards and Capture by Anti-Peptide Antibodies) approaches.^41,42 Historically, serum has been the matrix of choice for archiving patient materials in medical laboratories, but in general matrix choice is highly laboratory dependent.^37,39,40 However, it should be emphasized that development and validation of an MS-based protein test in a specific matrix certainly does not guarantee accurate results in a different matrix. Therefore, all of these parameters have to be thoroughly investigated.

Achieving Long-Term Robustness: Method Development and Analytical Performance Evaluation

Quantitation of proteins by MS via the bottom-up proteomics approach involves protein digestion with subsequent MS detection of the proteotypic peptides measured against an internal standard, thus reflecting the original concentration of the target protein.¹⁶ A three tier system⁸ has been proposed for the different levels of confidence and rigor of the technique used, with only tier 1 type tests suitable for implementation in the medical laboratory. In our previous work, various quality criteria were summarized including clinical chemistry principles for method development of quantitative bottom-up proteomics from the preanalysis to postanalysis.⁴ Furthermore, a recently proposed guideline structures the minimum information on a developed method that is required for publication of quantitative bottom-up proteomics tests in a clinical setting.⁴³ For absolute quantitation of target proteins, the establishment of a traceability chain according to ISO 17511:2020 is essential.

Peptide Selection

Quantitation of proteins via the bottom-up approach involves digestion of proteins by proteases (generally trypsin) into “signature” or proteotypic peptides that are detected using multiple reaction monitoring (MRM) in a triple quadrupole MS-system combined with LC. It is of great importance to benchmark genetic variations of a certain protein across populations, and to periodically keep track of reported mutations.⁴⁴ To further ensure accurate quantitation of proteins, it is recommended to measure at least two peptides per protein. It may be worthwhile to follow up multiple peptides in the early stage of method development^19,45 in order to have two remaining peptides per protein at the final stage. Some general rules apply for the best choice of peptides for optimal quantitation.^46,47 Preferably, proteotypic peptides should not contain missed cleavage variations or unstable amino acids (cysteine, methionine, tryptophan, asparagine, or glutamine) that could undergo chemical modifications^19,48 and they should not contain any PTMs or any known amino acid changes due to genetic variation. Moreover, transitions with interferences should be avoided.⁴⁹ Finally, when (two) optimal peptides are selected for a protein the best performing peptide may then serve as the quantifying peptide while the other is used for (qualitative) confirmation.³⁵

Proteolysis

The procedure of bottom-up sample preparation for protein quantitation should be tightly controlled and lead to equimolar conversion of proteins into quantifier peptides for the sake of accurate test results. Therefore, the experimental conditions should be optimized to ensure the stable production of peptides without missed cleavages.^50,51 Standard procedure for protein digestion involves denaturation and reduction of the proteins and alkylation of SH-groups. The fully unfolded proteins can then be split by proteolytic digestion cutting at specific residues (lysine and arginine for trypsin).^52,53 The efficiency of the tryptic digestion has been tested for various combinations of temperature, solvents, chaotropic agents and surfactants and optimal conditions can be demonstrated by time curves for the peptide generation.⁵⁴ However, the time curves for peptide generation nicely demonstrate that optimal conditions for different peptides are not unambiguous and some peptides are produced faster and/or with higher yield using different surfactant or solvents.⁵⁴ For quantitation of a large panel of protein biomarkers it will therefore be a major challenge to create uniform optimal conditions for absolute yields of all signature peptides. Next to optimal digestion conditions, activity and constant quality of the proteolytic enzyme for cleaving the protein should be guaranteed. Variability of proteolytic activity has been described for different types and sources of trypsin, which can influence digestion efficiency.^55,56 Important for trypsin stability is preventing autoproteolysis.⁵⁷⁻⁶⁰ Digestion performance can be tested in various ways, e.g. by monitoring missed cleavages or other peptide variants^19,61 or by using commercially available quality control samples.⁶²

Internal Calibration Strategies

With regard to internal standards (IS), several strategies can be used from addition of the stable isotope labeled (SIL) peptide only, to addition of a full length SIL protein.⁶³ The use of SIL peptide can correct for losses of the target peptide. It is expected that modifications to the endogenous peptide during digestion will also occur for the SIL peptide and the relative ratio between endogenous and SIL peptide thus remains constant.^19,64 Addition of SIL peptides with different labels at different time points before and after the digestion period enables the comparison of endogenous peptides produced relative to the labeled peptides present during the digestion and thus may be used to demonstrate changes of the (SIL) peptides during the sample preparation workflow.^61,65,66 Other solutions for internal standard strategies that have been explored are winged peptides (with sequences that still require tryptic cleavage).^67,68 Also SIL peptides can be obtained from so-called Protein Epitope Signature Tags (PrESTs),⁶⁹ containing a short and unique sequence of a protein of interest which can be produced in E. coli. Alternatively, artificial proteins can also be produced as concatamers of standard (Q) peptides (QCATs or QconCATs) which have the advantage of generating (SIL) peptides by endoproteolytic or chemical cleavage at once for multiple proteins.⁷⁰ The use of full length SIL proteins may be considered the “gold standard” for the accurate quantitation of proteotypic peptides by LC-MRM-MS/MS.⁷¹⁻⁷³ However, SIL versions of proteins are often not available, or at best as recombinant versions that are extremely costly. To this end, it is noted that digestion kinetics of a recombinant protein are not necessarily identical to those isolated from a native human matrix. The recombinant protein may differ with regard to secondary, tertiary or quaternary structure from the endogenous one, or differ in PTM content (or proteoform profile), thereby introducing a digestion bias. This difference between native and recombinant proteins should also be considered when selecting calibration materials.²⁰ Since cost effectiveness is also a key component in the cyclical framework during evidence-based test evaluation³⁰ the use of SIL proteins would make the test unaffordable, especially for highly multiplex protein assays.

Peptide Prepurification

For LC–MS analysis of crude serum samples after digestion cleanup of the sample reagent mixture is wanted in order to protect the analytical column and mass spectrometer from digest reagents and other serum or sample components. While it may be necessary to employ immunoaffinity based SISCAPA to enhance analytical sensitivity, peptide prepurification using either an in-line trapping column or cartridge or plate based solid phase extraction is often sufficient. A trapping column may be used online where peptides are bound and contaminating components are removed to waste before the start of the gradient elution. Next, the flow is switched in line with the analytical column for separation and elution of peptides and their subsequent MS measurement.^35,45 However, thorough cleanup of the sample digests by offline solid phase extraction (SPE), may be preferred.⁷⁴ Specific anion- or cation-exchange cartridges have been shown useful for peptide purification.⁷⁵ SPE with hydrophilic lipophilic balanced (HLB) cartridges has been applied in various studies.^76,77 In case that peptides elute directly from a trapping column onto the analytical column potential losses can become apparent from, for example blank runs between different samples. In case SPE is performed in an offline manner, clean up and recovery of the peptides will depend on both the pH and the percentage of organic solvent used for elution. Optimal conditions can be established experimentally for the set of peptides applicable.

The SPE cartridges efficiently remove matrix interferences, such as phospholipids and salts and thus potentially reduce ion suppression. The removal of matrix interferences by SPE is an important step in the long term protection of the LC column and stability of the MS triple quadrupoles as compared to the recurrent use of a trapping column. Furthermore, the 96-well format of SPE cartridges enables concentration of the sample in a (semi)-automated fashion.

LC–MRM–MS/MS Analysis

For the LC–MS measurement of multiple proteins a quantitation and confirmation peptide are measured for each protein. For accurate quantification of molecular proteoforms and/or specific proteins the metrological traceability concept has to be implemented according to ISO 17511:2020 calibration hierarchies. Key for accurate quantification is the unequivocal definition of the measurand. Clearly, when applying quantitative bottom up proteomics, the intact proteoform is not measured but rather a proteotypic peptide that is formed after equimolar digestion. It is emphasized that evaluation of peptide–protein equimolarity is crucial for each selected proteotypic peptide. For each peptide at least two, but preferably three MRM transitions are selected with one serving as the quantifying transition and others as qualifier transitions. The peptide MRM transitions should confirm ion ratios for both the endogenous and SIL-peptide. In order to minimize total analysis times, the high-throughput requirements determine limited LC-runtimes for routine applications in medical laboratories.

Data Evaluation

Peptide retention times have to remain stable between runs and from batch to batch in order to ensure proper integration of peak areas. In general multiple product ions of a peptide precursor ion will be detected and the combination of both with the most stable (and often highest) signal is used as quantifying transition as opposed to the sum of multiple transitions also used in research grade assays. For the other product ions a certain transition ratio will be found compared to the quantifying transition which can be used for confirmation of identity of the peptide. In case of interfering peptides with the same combination of precursor and product ion eluting at (almost) the same retention time the transition ratios will be influenced.^49,78 According to CLSI guidelines^5,6 deviation of transition ratios should be <20% from the standard ratio and such interferences should not influence the calculated concentrations of the peptide (protein) based on the quantifying transition⁷ which would be the case with summation of all transitions measured. An interference could be visible by changes in peak shape of a quantifying or qualifying transition. Outlier set up for peak width, symmetry and full width at half-maximum (fwhm) will detect double peaks or obvious peak broadening for one of the transitions thus identifying interferences of different precursor/product combination. In case interferences are observed a different choice of transitions could solve the problem. Alternatively, specific SPE conditions for cleanup of the sample could be altered or separation of the peptides could be adjusted by changing gradient conditions for elution.

Calibration and Metrological Traceability: Selection of Calibrators

In laboratory medicine the concept of metrological traceability is defined by the Joint Committee for Guides in Metrology as “the property of a measurement result whereby the result can be related to a reference through a documented unbroken chain of calibrations, each contributing to the measurement uncertainty”.⁷⁹ This concept is the basis for standardization and harmonization of medical tests; making test results comparable to allow universal exchange of patient laboratory data between institutions, nationally and at a global level.¹⁰ For quantitative proteomics tests aimed for clinical application a calibration strategy is required that takes into account metrological traceability of test results to the highest possible order if available. Therefore, calibrators should be used that guarantee traceability to secondary and finally primary reference materials and behave in a common way. An example of a traceability chain to SI is depicted in Figure 2 (source: ISO 17511:2020), recently discussed in more detail.⁸⁰

Figure 2.

Full metrological traceability toward International System Units (SI). Modified from ISO 17511:2020.

Analytical Validation

Thorough analytical validation is a prerequisite for ISO 15189⁸¹ accreditation (https://www.iso.org/standard/56115.html) and will also be required under the new EU regulatory framework IVDR 2017/746.⁸² A structured framework and detailed guideline for validation of new MS methods that may also be valid for protein quantitation has been developed within the CLSI guideline C62-A.⁶ Analytical performance recommendations are defined for bias, imprecision, sensitivity, specificity, matrix effects, carry over, interference, and stability. Various CLSI EP protocols are mentioned that will provide further information on the exact procedures to be applied. Acceptance criteria for e.g. imprecision, trueness and carry over are deduced from biological variation (if known) and should be selected in such a way that the test can meet its intended use.⁶ In case a test is foreseen on dedicated LC–MS instrument, the criteria for cleanliness and compatibility of the samples are much less stringent than when the test is performed on an instrument already in use for other applications. In the latter case, care should be given not to introduce additional test-to-test interferences.

Achieving Long-Term Robustness: Implementation

To achieve long-term accuracy of quantitative proteomics MS tests both the analytical system (all equipment needed for the test), and the application should consistently meet the predefined analytical performance specifications as outlined in the preparation phase. Adequate implementation of the complex MS technology within the routine medical laboratory setting is therefore important. In the laboratory quality management system (QMS), special attention should be given to the education of personnel and staff. Moreover, checks should be established to ensure consistent performance of the total testing process, including, the instrumentation as well as the application.

Education of Medical Laboratory Technicians and Staff

To ensure that medical tests provide accurate clinical information, ISO guidelines require in general proper training of the work force on the procedures needed for the application.⁸¹ In the case of mass spectrometry, most technicians and lab staff do not have a background in analytical chemistry and are less familiar with mass spectrometry.⁸³ Often, they have to be introduced to the complex analysis of quantitative bottom-up proteomics. MS-based quantitation of protein biomarkers is specialized work, which requires a thorough educational process that should be provided to employees at different levels from hands on technical staff to those dealing with management and directing biomarker quantitation for clinical utility.⁸⁴ The training program should at least consist of a theoretical section, that could be attended by both technicians and staff, and hands-on training, preferably specifically on the intended application, for the technicians. In our setting we entrust the development of MS-based methods to analytical chemists whereas the stable operation of LDTs is done by clinically trained lab technicians. For safeguarding the long-term robustness of the LDTs supervision of lab technicians on quality assurance aspects such as monitoring SST and QC results are a prerequisite. To make the MS-based test development sustainable, the head of the clinical chemistry department has established a multidisciplinary collaboration and alliance with analytical chemists (mass spectrometrists) for initial development but also throughout to provide an entry for troubleshooting and test improvement. Bridging technological knowledge on one side and clinical quality—according to ISO 15189:2012—on the other side required continuous exchange, collaboration, and communication.

Theoretical Training

The theoretical training program should ideally cover all phases of method development (see previous sections), to ensure that all individuals working on the LC–MS-based protein applications are aware of the strengths and potential flaws of the technique. Therefore, the theoretical section should include an introduction to QQQ-MS to understand the potential for analytical sensitivity and specificity, the basics of bottom-up proteomics (including the essentials of sample preparation, the use of internal standards and the design of calibration strategies) and, importantly, also data analysis, with an emphasis on the use of retention time, peak shape and area, ion ratios and concordance between peptides to monitor quality of the results. Such theoretical training programs focused on proteomics applications are generally not available and hence, the training should be tailor-made. Alternatively, the knowledge may be acquired through several summer schools, short courses and workshops. Several courses on the basics of mass spectrometry are available, including the North American Mass Spectrometry Summer School (https://www.ncqbcs.com/resources/training/summer-school/) or the summer school in Mass Spectrometry in Biotechnology and Medicine (http://msbm.org/). Similarly, summer schools dedicated to proteomics are also available (summer school advanced proteomics, http://www.proteomic-basics.eu). An alternative option could be short courses offered in concordance with scientific meeting such as Mass Spectrometry: Applications to the Clinical Laboratory (MSACL, https://www.msacl.org/) or the American Society of Mass Spectrometry (ASMS, https://www.asms.org). More recently, the American Association for Clinical Chemistry (https://www.aacc.org) now also offers online LC–MS education for clinical applications.

Practical Training

When starting with LC–MS in a clinical laboratory, the technicians that will be running the applications may not be familiar with the concept of quantitative proteomics and practical training is necessary. Courses on proteomics and MS should cover theoretical aspects as well as contain hands-on practical sessions. It is noted that the exact training needed is highly dependent on the application, and therefore, it is advised to do practical training on-site on the specific application and instrumentation. The results of an educational program should be documented in the QMS. Moreover, roles and responsibilities for employees should be based on their level of training and expertise and should be identified within the QMS of the lab.

Achieving Consistent Instrument Performance

Besides proper training of personnel, precautionary measures should be in place to ensure that the performance of both the instrumentation as well as the application are robust. To ensure that the instrumentation performs accurately, it should be maintained properly. However, just proper maintenance is no guarantee for accurate performance. Therefore, system suitability checks should also be performed. To ensure accurate test results, matrix-based internal and external quality control materials are used, similarly to other (non LC–MS based) test applications (Figure 3).

Figure 3.

Long-term accuracy requires meeting instrument performance indicators and test application performance criteria.

Instrument Maintenance

To ensure that the instruments used for quantitative proteomics by LC–MS are in good order, maintenance should be performed at three levels: regularly, annually and incidental in the case of malfunction. A laboratory technician should be trained to perform regular maintenance. This includes at least cleaning of the ion source in a weekly or biweekly interval, as well as cleaning of the inlet capillary every 3–6 months, depending on the occupancy rate of the instrument. Annual maintenance mostly pertains to cleaning of the LC, with an emphasis on the pump heads and changing of the pump oil of the MS rough pumps. Annual maintenance may be performed using a well-trained technical specialist, or could be outsourced to the instrument manufacturer or dedicated maintenance companies, for instance through service contracts. If a laboratory chooses not to use a service contract, it should ensure to have a well-trained LC–MS technical specialist who is capable of annual maintenance and incidental repairs.

System Suitability Testing

System suitability testing (SST) is a highly valuable procedure that is often overlooked during method development and implementation. Often, laboratories believe that the use of internal quality control (IQC) is sufficient to detect method errors and incorrect results. However, underlying instrument errors are sometimes masked by accurate IQC results.⁸⁵ Moreover, SST aids in detection of instrument errors and convincing of instrument manufacturers of malfunctioning instrumentation.

System suitability testing should consist of two phases: one is the accurate and consistent use of logbooks. It is important to monitor the vacuum, column temperature, flow rate, column pressure and precision of calibration, as well as any maintenance procedure.⁸⁶ Besides logbooks, system suitability testing should comprise repeated measurement of a well-defined sample, that reflects the biological analytes. For peptide-based quantitation, this can easily be accomplished using a mixture of synthetic peptides reflecting both endogenous and stable isotope labeled peptides. Briscoe et al.⁸⁵ described a design for an SST used in biopharmaceutical analysis. In the design, samples are tested at the start, middle, and end of a batch of samples. Using this strategy, the SST can detect changes in analytical sensitivity, analytical specificity, precision and carry over for relevant compounds of a test. Abbatiello et al.⁴⁹ described a system suitability test making use of a predigested sample of six proteins that was used for evaluation of different LC–MRM–MS/MS platforms in 11 laboratories. Here, several chromatographic and MS performance metrics were compared such as retention time drifts, MS detector responses, chromatographic resolution, peak capacity and tailing and carry over, demonstrating abnormalities in LC and MS performance among the participating laboratories. The aim of SST was to define the criteria to optimize replication of results within the LC–MRM–MS/MS platform. For comparison of SST at multiple sites longitudinal monitoring has been proposed based on Statistical Process Control (SPC) using the software package MSstatsQC in accordance with that for quality control samples using Levey-Jenning plots.^87,88 Automatic flagging for unwanted deviations can help to detect failures of the system regarding response, signal-to-noise, retention times and peak shape.⁸⁸ Only when the system fulfills the criteria of SST it is allowed to proceed with measurement of the actual samples, calibrators and IQCs.

Application Checks

Besides ensuring that the instrument is performing according to the specifications, it is also necessary to be convinced of the accurate performance of the full application. This may be done according to typical laboratory medicine procedures of IQC and external quality assessment (EQA). The procedures may be very similar to the ones already in use for batch processes, although in a multimarker quantitative protein MS test the selection of samples with appropriate levels for all proteins may be difficult.

Internal Quality Control (IQC)

Quality control samples for any diagnostic test are compulsory for ISO 15189⁸¹ accreditation, and often at least bilevel IQC samples are used. IQC should automatically detect batches in which errors occur, without rejecting accurate batches.⁴ Ideally, this is performed through a standard procedure, often based on Westgard rules,⁸⁹ and results are monitored in the laboratory information system (LIS). The selection of IQC samples should be done carefully. Ideally, IQC-samples are native, matrix-based materials just like the clinical samples to ensure similar behavior during sample preparation and analysis. Moreover, their stability should be assessed to ensure that the IQC materials are appropriate for longer-term monitoring of test performance. A general recommendation is to select IQC concentrations at or near medical decision points, but this is often not feasible for multiplexed tests. Behavior and trends of the IQC results around their respective targets can be followed in the so-called Levey-Jennings plots⁸⁷ and violation of the rules will be checked and should lead to corrective measures. For uniplex tests the IQC is typically performed at two levels, a physiological and pathological concentration, but it should be noted that this is often not feasible for multimarker panels. Here, a pragmatic choice may be taken by selecting available samples as long as there is spread between the samples for all important analytes. For both IQC levels concentrations are established with appropriate CLSI protocols with 1, 2, and 3 SD boundaries.⁸⁹ Importantly, these levels of imprecision should be in line with the allowable imprecision and bias according to the predefined APS.⁹⁰ In case of multimarker panels, each analyte should be monitored individually, and boundaries should be set individually. It has to be noted that correction for multiple comparisons may be necessary to avoid a high false rejection rate.⁹¹ In recent years, the six sigma model has been introduced for the evaluation of IQC samples,⁹² but so far, it has not yet been applied to LC–MS proteomics tests.

External Quality Assessment (EQA)

EQA or proficiency testing (PT) programs are meant to investigate comparability of test results between laboratories. For new LC–MS protein biomarkers EQA is often not available and in that case prolonged follow-up is required according to CLSI GP29.⁹³ EQA materials should be targeted either by a gold standard method or assigned by a peer group means. The importance of using commutable materials for correct value assignment has been clearly stated by Miller et al.⁹⁴ who described that the use of noncommutable materials will cause a matrix related bias, and inaccuracy of test results. Furthermore, when commutable samples have been used, agreement of results will reflect what would be seen for patient samples. As an example, in the Dutch EQA program the organizers make use of commutable, value-assigned materials and JCTLM listed reference measurement procedures (RMPs). This approach enables systematic monitoring of standardization efforts for years and demonstrates improvement of interlaboratory CVs for various analytes.⁹⁵

Total Testing Process Checks

Besides measures to ensure accurate analytical performance of a test, the embedding of the new test in the clinical care pathway should also be in place to enable future clinical validation and implementation of a novel MS-based quantitative protein test. Therefore, a concise implementation plan is required that contains multiple checks before final approval for clinical use may be obtained. This plan should at least contain information on the embedding of the test in the clinical care pathways, routing of samples in the laboratory, implementation of the test in the laboratory information management system (LIMS), decisions on turn-around times, reference intervals or decision limits, the confirmation and authorization of patient results, and communication to the relevant medical doctors.

Robust Performance of Quantitative Bottom up Proteomics: The Example of Serum Apolipoproteins

Within our laboratory, we have recently introduced quantitation of apolipoproteins for patient diagnostics. It is well-known that current cardiovascular disease (CVD) risk assessment based on serum lipid profiling overlooks a large group of CV risk patients (including women⁹⁶ and individuals with familiar dyslipidemias such as remnant disease and hyperlipoproteinemia (a)⁹⁷). Moreover, a reduction in low-density lipoprotein (LDL) cholesterol through therapeutic intervention is often not sufficient to avoid further cardiovascular complications.⁹⁸ There is now a substantial body of evidence for the quantitation of the functional proteins on lipoprotein particles as markers for CVD risk assessment,⁹⁹⁻¹⁰² and it is believed that apolipoprotein quantitation could therefore fulfill the currently unmet clinical need for cardiovascular patients.¹⁰³

An MS-based method was developed in accordance with CLSI guideline C62A and its intended use for precision diagnostics. First, the apolipoproteins of interest were selected: apoA-I, apoB, apoC-I, apoC-II, apoC-III, apoE, apo(a), apoA-II, and apoA-IV. For apoE the phenotype originating from its genetic polymorphism was also clinically relevant. Therefore, proteotypic peptides for each of these proteins, as well as the peptides specific for the apoE phenotypes were targeted in our LC–MS method. It was not possible to define analytical performance specifications based on outcome studies, therefore, biological variation derived analytical performance data were assessed. For apoA-I, apoB and apo(a) this resulted in TEa of 9.1%, 11.6%, and 24.1%, respectively. For the other apolipoproteins, a TEa of 20% was predefined based on the state of the art. The method development and provisional analytical validation has been described elsewhere.³⁵ Moreover, the apoE phenotyping was in good concordance with a genotyping method.¹⁰⁴ Analytical performance of the apolipoprotein multiplex test for precision diagnostics is optimized in such a way that the required (predefined) clinical performances for detecting and/or monitoring residual cardiovascular disease can be met. To prove the clinical value of the apo multiplex panel in addition to the conventional lipid panel, clinical outcome studies are underway that evaluate the predictive value of, e.g., the baseline apolipoprotein panel for predicting recurrent cardiovascular events.

A system suitability sample (SSS) was developed consisting of all synthetic versions of both the endogenous and stable isotope labeled peptides at a concentration of 0.15 μmol/L. Five replicate injections of (two μL of) this sample are followed by injection of a blank (the same solution without peptides) prior to a run, as well as immediately following a run.⁸⁵ To ensure sufficient analytical sensitivity, a threshold was specified for the area counts of all internal standard signals (all signals >20.000 counts). For the analytical specificity, the CVs of the ion ratios (ratio between the different transitions of a single peptide⁶) have to be within 15%; similarly, to ensure sufficient precision, the CVs over the relative responses (endogenous/SIL) had to be <10% within five replicates, and <15% difference between the SSS before and after the run. To ensure that the full peak is being detected, the optimum of the peak should deviate no more than 12 s from the retention time specified in the method, and carry-over is monitored for all peptides as the SIL peptide area (blank)/SIL peptide area (last SSS). The maximum allowable carry-over was set at 1%. If the system suitability test failed for any of the criteria, the instrument should be checked prior to the analysis of the clinical samples.

To ensure long-term stability of the method, an external calibration procedure was developed using native human serum based calibrators, that are traceable to WHO-IFCC international reference materials SP1-01 and SP3-07 for apoA-I and apoB, respectively. Traceability of apo(a) was secondary through immunoassay measurement to SRM 2B.¹⁰⁵ Currently, there are no reference materials available for apoC-I, C-II, C-III and E, but a working group of the International Federation for Clinical Chemistry (IFCC WG APO-MS, https://www.ifcc.org/ifcc-scientific-division/sd-working-groups/wg-apo-ms/) has developed a complete reference measurement system for global standardization of seven serum apolipoproteins.⁸⁰ Long-term stability in our laboratory is exemplified for three apolipoproteins in two QC samples (Figure 4, with Levey-Jennings plots of apoA-I; (1.32 g/L) ± 0.07 (1SD 5.3%), apoB; (1.02 g/L) ± 0.04 (1SD 4.2%) and apo(a); (102.0 nmol/L) ± 7.4 (1SD 7.3%)¹⁰⁶). We have been able to run the multiplexed apolipoprotein test stably for more than four years, even though a number of amendments were made (multiple instruments, introduction of SPE cleanup and adding additional apolipoproteins, several trypsin lots and several laboratory technicians performed the application). Given the experience with the multiplex apolipoprotein test we stress that stringent quality requirements during test development are essential for validation and implementation to achieve robust test performance.

Figure 4.

Levey–Jennings plots with concentrations of apolipoproteins A-I, B and (a) based on peptides VQPYLDDFQK, TEVIPPLIENR, and LFLEPTQADIALLK, respectively. Results from apoA-I and apoB were obtained from a single QC sample (n = 237) measured in 77 runs over four years (top and middle plot). Results from apo(a) were obtained from a second QC sample measured in 33 runs (n = 71) over three years (bottom plot). Recent measurements (2019-present) on a second Agilent 6495 QQQ-MS system in our laboratory have demonstrated similar long-term performance (data not shown).

Conclusions and Future Perspectives

In this era of steadily increasing needs for precision medicine, it is important to focus on test development and enabling technology that address clinical gaps in care pathways, in order to improve patient management and outcomes. Therefore, clinical needs should be properly identified and verified. Subsequent biomarker selection and test development and evaluation should be done in a precise manner, encompassing all key elements of the Test Evaluation cyclical framework established by the EFLM working group on Test Evaluation.¹¹ In this paper, we present our practice for developing quantitative MS-based protein tests. Once candidate markers of interest are selected, method development should be done after considering the measurand(s) and predefining analytical and clinical performance specifications for the intended use of the test. Actual method development and analytical performance evaluation experiments should be designed according to CLSI recommendations and in agreement with ISO15189:2012. It is imperative to have proper instrument maintenance, system suitability testing, internal/external quality control procedures in place, as well as training programs for developing respectively applicating personnel. With the precautions mentioned, we have demonstrated that LC–MS based protein quantitation can be performed in a robust manner in medical laboratories for years. Yet, for worldwide uptake of these next generation MS-based protein tests we call upon the In-Vitro Diagnostics industry for automating and integrating the workflow of both the sample preparation and LC–MS technology into a convenient and robotized MS-system. Moreover, a concerted effort is needed between IVD-industry, laboratory specialists, clinicians and researchers to implement high-quality MS-based protein tests with proven clinical performance and clinical utility at affordable costs.

The authors declare no competing financial interest.