Enhancing the accuracy of measurement of small molecule organic biomarkers

Abstract

Over two decades, the Organic Analysis Working Group (OAWG) of the Consultative Committee for Amount of Substance: Metrology in Chemistry and Biology (CCQM) has organized a number of comparisons for clinically relevant small molecule organic biomarkers. The aim of the OAWG community is to be part of the coordinated international movement towards accuracy and comparability of clinical measurements that will, in turn, minimize the wastage of repeat testing and unnecessary therapy to create a sustainable healthcare industry. International and regional directives/requirements on metrological traceability of calibrators and control materials are in place. Metrology institutes worldwide maintain infrastructure for the practical realization of metrological traceability and demonstrate the equivalence of their measurement capabilities through participation in key comparisons organized under the auspices of the CCQM. These institutes provide certified reference materials, as well as other dedicated value-assignment services benefiting the in-vitro diagnostic (IVD) industry, reference (calibration) laboratories and the clinical chemistry laboratories. The roles of these services in supporting national, regional, and international activities to ensure the metrological traceability of clinical chemistry measurements are described.

Introduction

A small molecule biomarker is a low molecular weight organic compound that is of use in healthcare for the prediction, diagnosis, and monitoring of diseases. As compared to the subjectivity of disease symptoms, biomarkers provide an unbiased and quantifiable means of characterizing diseases [1, 2]. Routinely, biomarkers are measured on fully automated in vitro diagnostic (IVD) devices that have the advantages of high throughput and ease of operation. To be truly valuable, the measurements made with these IVD devices should be both accurate and reproducible. While the accuracy of laboratory testing has progressed over past decades, for most laboratory tests, a result still needs to be interpreted based on the reference range for the method applied by the clinical laboratory.

Even in an era where patients are increasingly mobile as they seek treatment from different healthcare providers across hospitals or even countries the need for repeat testing [3] should be minimized. Evidence-based medicine for medical decision making is becoming more prominent in clinical practice [4]. Clinicians make use of cut-offs on clinical markers for diagnosis, for example, glucose [5] and/or haemoglobin (Hb) A_1c for diabetes mellitus [6], and creatinine for renal function [7]. Regardless of where, when and how the clinical results were provided, these results should be comparable. Together with accuracy control, one of the key approaches to achieving this is metrological traceability.

Globally, efforts on traceability in clinical measurements are on-going. In 2002, the Joint Committee for Traceability in Laboratory Medicine (JCTLM) was created through a Declaration of Cooperation between the International Bureau of Weights and Measures (BIPM), the International Federation for Clinical Chemistry and Laboratory Medicine (IFCC), and the International Laboratory Accreditation Cooperation (ILAC) [8, 9]. JCTLM identifies internationally accepted reference measurement procedures (RMPs), reference materials and reference laboratories to assist the IVD manufacturers in utilizing them to meet the requirements of the relevant ISO standards [10–14]. This has been particularly useful for manufacturers who needed to meet the requirements of the European Directive 98/79/EC [recently replaced by Regulation (EU) 2017/746] that necessitates establishing metrological traceability (these concepts have been previously described by others [15, 16]) for clinical measurements through available RMPs and/or available reference materials of a higher order [17, 18]. The IFCC also has a strong focus on traceability and has a dedicated Committee on Traceability in Laboratory Medicine [19]. Additionally, the IFCC’s Scientific Division has several working groups focused on establishing RMPs or secondary reference materials for priority analytes [20].

In the metrology arena, the International Committee for Weights and Measures (CIPM), which oversees the activities of the BIPM, created the Consultative Committee for Amount of Substance: Metrology in Chemistry and Biology (CCQM) in 1993 [21]. The Organic Analysis Working Group (OAWG) established in 1996 under the CCQM plays a part in supporting the international activities of the JCTLM. One of the main roles of the OAWG is to organize key comparisons to benchmark the capabilities of metrology institutes for the execution of “higher order” measurement procedures.

A brief history of OAWG clinical comparisons

In 1998–99, the National Institute of Standards and Technology (NIST, the national metrology institute of the United States of America) organized the first pilot study (denoted by “P” preceding the number in the study code) for international metrology institutes in the field of clinical chemistry. This involved the determination of total cholesterol in serum (CCQM-P6). Cholesterol was an important biomarker in the U.S. as cardiovascular disease was the country’s principle health problem and NIST was heavily involved in improving the nation’s quality control system for cholesterol measurement. By this time, NIST had published a definitive method [22] for total serum cholesterol by isotope dilution-mass spectrometry (IDMS) [23]. The pure cholesterol Standard Reference Material^® SRM^® 911b that was used for calibration and providing SI traceability of the measurement procedures applied in the study [Note: SRMs are certified reference materials (CRMs) provided by NIST] was also developed. Although the spread of results from the metrology institutes that participated was deemed acceptable for a pilot study, there was room for improvement.

The OAWG members discussed the results and concluded that incomplete hydrolysis of cholesterol esters may have contributed to biases in the CCQM-P6 results. Based on this knowledge, the follow-on key comparison (denoted by “K” preceding the number in the study code), being CCQM-K6 was coordinated and was successful in demonstrating international comparability [24]. Interlaboratory precision and the reported uncertainties were significantly improved even with two of the institutes using their own purity assessed cholesterol standard for calibration. The results from the comparison ensured international confidence in the cholesterol in serum CRMs being provided by the comparison participants.

The success of CCQM-K6 also spawned a series of complementary comparisons to test capabilities beyond cholesterol and tackle the growing list of biomarkers employed in routine clinical measurements. The OAWG recognized the impracticality of designing a single comparison for every biomarker, thus in 2001 they carefully selected two additional representative biomarkers with differing analytical challenges and mass fraction levels as analytes in subsequent studies: glucose in serum (CCQM-K11) and creatinine in serum (CCQM-K12) [25, 26]. Together with cholesterol, glucose and creatinine are the most frequently measured clinically-relevant substances in human biofluid samples and were selected to be representative of analytical challenges associated with well-defined and low molar mass organic substances with a range of polarity and mg/kg concentrations in blood. The design of these comparisons contributed to the current OAWG core comparison strategic approach, which selects model systems for key comparisons based on polarity and molecular weight .

After a nearly two decades of developing clinical measurement capability across the OAWG, the number of participating National Metrology Institutes (NMIs) and Designated Institutes (DIs) in current studies have doubled with a considerable spread of participating institutes from across the globe. Table 1 summarizes all the comparisons organized by the OAWG in the clinical area. Such comparisons provide the evidence for the equivalence of measurements made by member institutes and are used to underpin their services in the field of clinical chemistry .

Table 1.

OAWG Pilot studies and key comparisons for small molecule organic biomarkers

Measurand	Comparison1	Year	Coordinating Laboratory1	Participating NMIs and Dis1	Total Participating Metrology Institutes
Cholesterol in serum	CCQM-P6	1999	NIST	LGC, NIST, NMi, NMIJ, NRCCRM, PTB	6
Cholesterol in serum	CCQM-K6	2000	NIST	LGC, NARL, NIST, NMi, NMIJ,	7
Glucose in serum	CCQM-P8	2000	NIST	NRCCRM, PTB KRISS, NIST, PTB	3
Creatinine in serum	CCQM-P9	2000	NIST	KRISS, LGC, NIST, PTB	4
Cholesterol in serum	CCQM-K6.1	2001	NIST	NARL, NIST, VNIIM	3
Glucose in serum	CCQM-K11	2001	NIST	KRISS, NIST, PTB	3
Creatinine in serum	CCQM-K12	2001	NIST	IRMM, KRISS, LGC, NIST, PTB	5
Glucose in serum	CCQM-K11.1	2005	KRISS	CENAM, KRISS, NCCRM, NMIJ, VNIIM*	4
Creatinine in serum	CCQM-K12.1	2005	KRISS	CENAM, KRISS, NCCRM, NMIJ*	3
Cortisol in serum	CCQM-P77.a	2006	NIST	KRISS, PTB, NIST, NMIJ, NIM	5
Progesterone in serum	CCQM-P77.b	2006	NIST	PTB, NIST, NMIJ, NIM	4
Creatinine in serum	EURAMET.QM.K12	2010	LGC	EXHM, HSA, LGC, LNE, PTB	5
Non-peptide hormones in serum: cortisol	CCQM-K63.a	2008	NIST	KRISS, LGC, NIM, NIST, NMIJ, PTB	6
Non-peptide hormones in serum: progesterone	CCQM-K63.b	2008	NIST	CENAM, KRISS, LGC, NIM, NIST, NMIA, NMIJ, PTB	8
Comparison of value assignment of CRMs and PT materials: creatinine in serum	CCQM-K80	2010	NIST	CENAM, KRISS, LGC, NIM, NIST, PTB	6
Cholesterol in serum	CCQM-K6.2	2012	NIST	CENAM, HSA, INMETRO, INTI, KRISS,	10
Glucose in serum	CCQM-K11.2	2012	NIST	LNE, NIMT, NIST, UME, VNIIM CENAM, HSA, INMETRO, KRISS, LNE, NIMT, NIST, NMIJ, PTB, UME, VNIIM	12
Creatinine in serum	CCQM-K12.2	2012	NIST	CENAM, HSA, INMETRO, KRISS, LNE, NIM, NIMT, NIST, NMIJ, PTB, UME, VNIIM	11
Low-polarity analytes in a biological matrix: vitamin D metabolites in human serum	CCQM-K132/P169	2015	NIST	HSA, KRISS, NIM, NIMT, NIST, NMIA, UME	7
High polarity analytes in biological matrix: urea and uric acid in serum	CCQM-K109/P148	2016	HSA	CENAM, GLHK, HSA, INMETRO, KRISS, LGCa, LNE, NIM, NIMT, NIST, NMIA, NMIJ, PTB, UME, VNIIM	15c, 14d
Comparison of CRMs and value-assigned quality controls: urea and uric acid inhuman serum or plasma	CCQM-K142/P179	2016	HSA	CENAM, HSA, KRISS, NIMb, NIST	4c, 5d

^*Data not returned

^aNMI participated in urea only

^bNMI participated in uric acid only

^{c, d}Total participating institutes for urea and uric acid respectively

¹Abbreviations: CENAM: Centro Nacional de Metrología; EXHM: General Chemical State Laboratory; GL: Government Laboratory; HSA: Health Sciences Authority; INMETRO: Instituto Nacional de Metrologia, Qualidade e Tecnologia; INTI: Instituto Nacional de Tecnología Industrial; IRMM: Institute for Reference Materials and Measurements [later named Joint Research Centre (“JRC”)]; KRISS: Korea Research Institute of Standards and Science; LNE: Laboratoire National de Métrologie et d’essais; NARL: National Analytical Reference Laboratory [later became National Measurement Institute, Australia (“NMIA”)]; NIMT: National Institute of Metrology, Thailand; NIST: National Institute of Standards and Technology; NMi: NMi Van Swinden Laboratorium; NMIJ: National Metrology Institute of Japan; NRCCRM: National Research Center for Certified Reference Materials [later merged into the National Institute of Metrology (“NIM”)]; PTB: Physikalisch-Technische Bundesanstalt; UME: Ulusal Metroloji Enstitüsü; V VNIIM: D.I. Mendeleev All-Russian Institute for Metrology

Primary calibrators for traceability o the international system of units (SI)

Metrology institutes, IVD manufacturers, providers of target value-based external quality assessment schemes (EQAS), and laboratories involved in standardization programmes all need access to organic compounds with correctly assigned purity to deliver reliable SI-traceable measurements. This constitutes well-defined chemical entities such as drugs, nutritional markers, and endogenous metabolites and related substrates, with the list of desired compounds continuing to expand [27–30]. The chemical purity determination of organic calibrators to ensure SI traceability for clinical measurements is one of the core activities for the OAWG and has been recently described.

Since there are over a hundred of such organic clinical biomarkers, it is a challenge for any single metrology institute or even a single country to provide pure substance CRMs for all biomarkers. Thus, this is a truly international endeavour across many metrology institutes. Due to the importance of organic purity assessment, from 2008, the BIPM coordinated the CCQM-K55 series of comparisons dedicated to the purity assignment of a range of organic compounds. These comparisons on “primary calibration materials” were organized to enable member metrology institutes of the CIPM to uphold their Mutual Recognition Arrangement (MRA), by demonstrating the equivalence of their measurement procedures. Model compounds representing different sectors of the “molecular weight vs polarity” analytical space developed by the BIPM (see Fig. 1) were provided to establish the capabilities of metrology institutes that offer pure substance CRMs or purity assignment services [27]. The capabilities underpinning the purity assignment of routinely measured metabolites and substrates with low polarity and medium molecular weight such as cholesterol, cortisol, progesterone, and 25-hydroxyvitamin D₃ [25(OH)D3] were underpinned through successful participation in CCQM-K55.a and CCQM-K55.b, where the model compounds were 17β-estradiol (for low molecular weight and low polarity compounds) and aldrin (for medium molecular weight and low polarity compounds), respectively. For the same molecular weight range, the higher polarity metabolites and substrates such as amino acids, creatinine, glucose, and 5-tetrahydrofolic acid were underpinned through CCQM-K55.c and CCQM-K55.d, where the selected model compounds were L-valine (for low molecular weight and high polarity compounds) and folic acid (for medium molecular weight and high polarity compounds), respectively.

Fig. 1.

Plot showing the model compounds (red dots) in the CCQM-K55 series of organic purity key comparisons in different sectors of the “molecular weight vs polarity” analytical space developed by the BIPM. The common clinical markers (blue dots) are also mapped onto the same analytical spaces

In addition to the CIPM MRA’s list of CRMs as measurement services produced by metrology institutes, many higher order reference materials relevant to the health area, covering compounds such as steroids and their metabolites, are also listed on the JCTLM database. Both of these lists have evolved in conjunction with the CCQM-K55 purity assessment series [31–34]. The purity assignment of these types of compounds is made all the more challenging by the structural complexity of the analytes and the potential hygroscopicity of materials such as the steroid metabolites. The presence of low levels of structurally related impurities are inevitable in these types of materials and this requires the structural elucidation and assessment of relative response factors during purity assessment utilizing liquid chromatography with ultraviolet diode array detection (LC-UV/DAD). It included estimation of molecular masses for impurities to ensure effective transformation of molar response to mass fractions. The impact of pH when using LC-UV/DAD, such as in the case of amino acids, requires a good understanding of the physiochemical properties of not just the compounds but also the structurally related impurities.

Pure substance CRMs as primary calibrators for clinical chemistry are available from several institutes. These include (but are not limited to) analytes, such as glucose, creatinine, uric acid, folate vitamins, amino acids, hormones and steroids, or selected peptides and proteins from institutes such as the NIST, National Institute of Metrology (NIM, China), National Metrology Institute of Japan (NMIJ), National Measurement Institute, Australia (NMIA, Australia), LGC (United Kingdom), Korea Research Institute of Standards and Science (KRISS, South Korea), Centro Nacional de Metrologia (CENAM, Mexico), Health Sciences Authority (HSA, Singapore) and Government Laboratory (GL, Hong Kong SAR).

An IVD manufacturer’s technical consideration of a primary calibrator would include the measurement uncertainty of their purity values. For a pure substance CRM to be effectively used in the production of the manufacturer’s product calibrator, the uncertainty of the former would ideally be significantly smaller than the total uncertainty of the manufacturer’s standing procedure.

There is an increasing market preference for primary calibration solutions rather than neat primary materials. Provided that the materials are stable in solutions, such CRMs are often better alternatives, particularly when the pure compounds may be hygroscopic and require careful handling. Several NMIs are providing or are developing measurement services for the assignment of calibration solution, such as the vitamin D metabolite calibration solutions from NIST, and amino acids solutions from NIST and NIM, used as calibrators for peptides and proteins.

In 2015, BIPM coordinated the key comparison CCQM-K78.a on the assignment of mass fraction contents of multi-component amino acids in aqueous solution. Majority of the institutes applied isotope-dilution mass spectrometric (IDMS) methods where the native and isotopically labelled materials are in the same chemical form [35]. A smaller number of institutes also applied LC coupled with spectrophotometric methods. The level of agreement among the results displayed the capabilities related to the assignment of calibration solutions in the majority of the institutes.

Higher order reference materials as secondary calibrators and control materials

Matrix CRMs typically behave as secondary calibrators in the calibration traceability chain described in ISO 17511 since the value of the measurand is assigned by comparison to a primary standard. Provided that the uncertainties associated with the assigned values are “fit for purpose” or smaller than those expected for the working calibrators, these secondary calibrators are then used by IVD manufacturers to produce the working calibrators with the implementation of the manufacturer’s selected procedure. Matrix CRMs may also be used as trueness control materials for the assessment of bias of measurements. Such matrix CRMs available from metrology institutes include (but are not limited to) sterols, steroids and hormones (e.g. cholesterol, cortisol, progesterone, vitamin D) in serum, metabolites (e.g. creatinine, urea) in serum, trace metals (e.g. calcium, sodium) in serum, nutrients (e.g. beta-carotene) in serum, toxic components (e.g. oxychlordane, PCBs, mercury) in serum, steroid metabolites (e.g. testosterone glucuronide) in freeze-dried urine, drugs (e.g. cocaine, codeine) in urine and drugs (e.g. 6-monoacetylmorphine) in hair. As listed in the BIPM key comparison database, these are available from the Laboratoire national de métrologie et d’essais (LNE, France), D.I. Mendeleyev Institute for Metrology, Rosstandart (VNIIM, Russia), TÜBITAK Ulusal Metroloji Enstitüsü (UME, Turkey), NMIA, NIST, NIM, NMIJ, KRISS, CENAM, HSA, LGC and others. Many of these matrix CRMs for clinical chemistry are also listed in the JCTLM, such as (but not limited to) cholesterol, homocysteine, various carotenoids in serum and plasma, and nicotine and cotinine in urine from NIST; creatinine in serum from LNE, CENAM, HSA and LGC; glucose in serum from LNE and HSA.

The development of many of these CRMs has been facilitated through historical participation in OAWG comparisons (Table 1). In some cases, analogous matrix-based CRM measurement services have developed in tandem across several institutes. The OAWG recognized this as an opportunity to conduct a specific comparison to directly compare the individual CRMs in their capacity to deliver measurement services to their customers (i.e., the value of the measurand as stated in the product certificate). In 2010, NIST coordinated the first CRM comparison for clinical materials, CCQM-K80 involved a comparison of the values assigned to CRMs and proficiency testing (PT) materials for creatinine in serum [36, 37]. The demonstration of continued reliability of such materials is important, particularly when it impacts national or international programmes, an example being NIST’s participation in the US National Institute of Health (NIH) National Kidney Disease Education Program’s Laboratory Working Group to address stakeholder needs and the development of newer CRMs to include a lower level of creatinine to support the critical need for a paediatric-level material.

More recently, a second clinically-relevant comparison of CRMs CCQM-K142, was coordinated by HSA and NIST to verify the equivalence of available urea and uric acid in human serum/plasma CRMs provided by KRISS, NIM, CENAM, HSA, and NIST. These were the higher-order reference materials available at the time of the comparison for the determination of urea (used in the diagnosis of kidney disease or renal failure), as well as uric acid (used in diagnosis of gout). Both of these measurands are associated with other medical conditions including diabetes and the formation of kidney stones. The CRMs from participating institutes were shipped to HSA (the measurement laboratory) for independent measurement under repeatability conditions. 10 CRMs certified for urea and 12 CRMs certified for uric acid were tested. Except for one uric acid in frozen serum CRM exhibiting bias potentially due to the method applied for assignment of its reference value, all the CRMs demonstrated equivalence as higher-order reference materials for these clinically important measurands (see Fig. 2).

Fig. 2.

Plots showing the agreement (typically measured by relative degrees of equivalence, or DoE,%) in CCQM-K142 of the measured values for the urea (left) and uric acid (right) CRMs from metrology institutes with their assigned values. The green circle denotes the CENAM DMR-263b that was the only material to deviate from its expected value

In parallel with CCQM-K142, HSA also coordinated a traditional key comparison using urea and uric acid in frozen human serum as the model system. In this comparison, CCQM-K109, urea and uric acid were chosen as model compounds for clinical biomarkers with low molecular weight and high polarity in a biological matrix. After more than a decade of developing clinical measurement capability, the number of participating institutes in comparisons related to clinical chemistry has significantly increased. As participation in this comparison was “mandatory” for metrology institutes delivering service covered under the comparison’s scope, 15 NMIs/DIs participated in CCQM-K109 for urea and 14 for uric acid. The majority of these NMIs/DIs demonstrated measurement equivalence for their services. Despite the long experience of the OAWG in clinical comparisons there were still learnings from this comparison. The institutes who had reported the few values that deviated from the consensus discovered issues that included the calibrants, solubility, equilibration, instrumental bias, and improper storage of standard solutions. Successful demonstration of measurement equivalence in CCQM-K109, not only underpins the comparability of services for urea and uric acid in biological matrices such as blood, serum or urine, but also covers the entire scope of the key comparison, which is described as: low molecular weight (50–500 g/mol) and high polarity (pKow >2) analytes having high mass fraction values ranging from 10 mg/kg to 2000 mg/kg in biological matrix such as human serum, blood and urine.

The OAWG also aims to examine measurement areas of particular importance and the quality of 25-hydroxyvitamin D in serum measurements have been under considerable scrutiny over the last decade. NIST thus coordinated a key comparison, CCQM-K132, with specific measurement interest on 25(OH)D₃ and 25-hydroxyvitamin D₂ [25(OH)D₂] in human serum. In addition to assessing institutes’ capabilities for these priority measurands, the study extended the mass fraction range to 10⁵ to 10⁶ times lower than that previously demonstrated in the CCQM comparisons for cholesterol in serum, another similar nonpolar clinical analyte. Values for the analytes were in the 20 to 40 ng/g range for 25(OH)D₃ and 0.6 to 6 ng/g for 25(OH)D₂. While all the results were in good accord for 25(OH)D₃, two results were not in good accord for 25(OH)D₂ which was present at a more challenging lower analytical mass fraction range or < 1 ng/g.

CCQM-K132 successfully underpins NIST’s continued collaboration with the Vitamin D Standardization Program (VDSP) originally established by the US NIH in the execution of extensive comparability and commutability studies for serum 25(OH)D. This work now formally sits within the IFCC Scientific Division’s Vitamin D Working Group. In addition, this comparison also supports the comparability of reference measurement services such as for 25(OH)D₂ and 25(OH)D₃ in serum by UME, NIM, NIST and NMIA. The comparison may be used to underpin a wide range of services from the participating institutes for low molecular weight (100–500 g/mol) and low polarity (pKow < −2) analytes having low mass fraction values ranging from 1 ng/g to 500 ng/g, e.g. non-peptide hormones, 17β-estradiol, testosterone, progesterone and cortisol in serum.

The demonstration of measurement equivalence in international comparisons and traceability to the highest order reference materials ensures the continued robustness of the reference measurement system established to support the IVD industry, reference (calibration) laboratories and the clinical chemistry laboratories worldwide. The key comparisons organized under the auspices of the CCQM can support (in part) the listing of a reference measurement method or procedure in the JCTLM database. Examples include IDMS methods for the measurements of creatinine in serum by NIST, NIM and LGC; and glucose in serum by NIST and HSA. Companies such as IVD manufacturers can use the services of a reference measurement service provider listed in the database to assist with compliance with traceability regulations such as the EU IVD Regulations.

Another mechanism through which metrology institutes connect with the clinical chemistry community include their deep involvement in the coordination EQAS programmes, which allows clinical laboratories to assess not only the comparability of their results against their peers but also the accuracy of their methods, thus meeting the new demands of mobile patient management.

Accuracy-based external quality assessment schemes (EQAS)

EQAS programmes are important in ensuring the effectiveness of a clinical laboratory’s quality management system and serve as one of the main tools in assessing a laboratory’s performance during accreditation. Some metrology institutes are involved in the accreditation of EQA providers according to ISO/IEC 17043 [38], e.g. Instituto Nacional de Metrologia, Qualidade e Tecnologia (INMETRO, Brazil). Many EQAS providers are putting a greater focus on reference values in EQAS programmes, often termed “accuracy-based quality assurance programmes”.These programmes are vital in advancing the quality of laboratory medicine and hence, patient care since, unlike programmes with consensus target values, accuracy-based target values are not influenced by the composition and potential bias of IVDs involved in the programme. Accuracy-based programmes serve to reveal biases between peer group consensus values derived from different analytical platforms and encourage manufacturers to improve their methods to a point where clinicians can confidently make use of results from any platform for accurate diagnosis, treatment or dosage adjustments.

Ideally, metrology institutes that provide assigned values in EQAS programmes would have their measurement capabilities underpinned by participation in suitable key comparisons. They additionally need to be able to provide reference values with low enough uncertainties to allow effective performance evaluation. As general guidance, the standard uncertainty of the assigned value should be less than 0.3 times of the standard deviation for proficiency assessment [39].

Interlaboratory comparisons organized for clinical laboratories worldwide provide a good perspective of how they perform. One of the important early accuracy-based interlaboratory comparisons was organized in 2002. The International Measurement Evaluation Programme (IMEP)-17 programme which was organized by the Joint Research Centre (JRC) in collaboration with the IFCC and European Committee for External Quality Assurance Programmes in Laboratory Medicine (EQALM) enabled 1037 laboratories from 35 countries on all continents to compare their results against metrologically traceable values assigned by metrology institutes [40].

Several metrology institutes have provided traceable target values to EQAS programmes organized by themselves for their domestic clinical laboratories or for external providers. For instance, in Australia, NMIA develops reference measurement procedures for clinically important analytes in serum as prioritized by the Australasian EQAS provider, the Royal Australasian College of Pathologists Quality Assurance Programs (RCPAQAP). The RCPAQAP include target values for important endocrine hormones to assess clinical laboratory performance and improve cross-platform comparability. The use of accurate target values obtained from higher-order mass spectrometry techniques allowed for an improved assessment of bias that may originate from various immunoassay platforms. The NMIA RMPs involve two-dimensional liquid chromatography-tandem mass spectrometry approaches that utilize an initial two-dimensional liquid chromatography fraction clean-up to ensure the highest selectivity of the methods. The analytes currently involved are testosterone, 25(OH)D₃, cortisol and 17β-estradiol [41]. The levels in the RCPAQAP samples typically range from ≈ 0.15 nmol/L to 700 nmol/L. The demonstration of competence for this type of RMP has been via the specific CCQM-K132 comparison for Vitamin D in serum and via the CCQM-K109 comparison for urea and uric acid in serum.

Despite international standardization programmes within the clinical field, there is still considerable variability in results from routine clinical laboratories and many EQAS schemes still assess results against peer-group consensus values. An independent target value from a RMP allows this platform-specific evaluation to be overlaid with a metrologically traceable assigned value. Figure 3 shows an example of the NMIA reference value for a 25(OH)D₃ in serum RCPAQAP EQAS programme vs the overall median and mean. The scatter of participants’ results for analytes of this type is large.

Fig. 3.

Results from ≈ 80 laboratories for the 2015 Australasian RCPAQAP EQAS for 25(OH)D₃ versus the NMIA reference value. This sample is a mid-level sample at ≈ 130 nmol/L.

In Singapore, HSA has organized its annual EQA programmes since 2011 and has been accredited by the Singapore Accreditation Council (SAC) as a PT Provider in compliance with ISO/IEC 17043 [38] since 2013. By 2018, the programme had steadily expanded and comprised 17 clinical biomarkers, 8 of which are well-defined small molecule organic biomarkers namely: urea, uric acid, total cholesterol, total glycerides, high density lipoprotein-cholesterol, low density lipoprotein-cholesterol, glucose, and creatinine. The immediate focus of the programme is on clinical biomarkers associated with chronic diseases facing the Singapore population: diabetes mellitus and cardiovascular disease. The demonstration of competence for these measurands has been via the CCQM-K109 for urea and uric acid in serum, CCQM-K6.2 for cholesterol in serum, CCQM-K12.2/EURAMET.QM-K12 for creatinine in serum and CCQM-K11.2 for glucose in serum. The success of the HSA EQAS programme can be observed from the full participation by all public and private hospital laboratories and most private clinical laboratories in Singapore, the increase in the level of participation for the 17 clinical biomarkers, and tightening of the acceptable range of results.

In France, LNE has provided assignment of reference method target values to glucose, creatinine, total cholesterol, total glycerides, high density lipoprotein (HDL)-cholesterol and low density lipoprotein (LDL)-cholesterol in EQAS materials that were used to organize the first French accuracy-based programme in 2016 to which all 988 French clinical laboratories participated. The demonstration of competence by LNE for these measurands was achieved via successful participation in CCQM-K6.2 for cholesterol in serum, CCQM-K12.2 for creatinine in serum and CCQM-K11.2 for glucose in serum. An important development in this case was that for the first time in France, the mandatory EQAS organized by L’Agence nationale de sécurité du médicament et des produits de santé (ANSM) relied on CRMs of proven commutability as the EQAS materials. Previously, the mandatory French EQAS had relied on lyophilised sera of unknown commutability with target values being consensus values (i.e. mean of results obtained by all participants or laboratories that all use the same analyser). EQAS relying on samples with target values that have not been value assigned with a reference method do not make it possible to assess absolute bias of routine IVDs [42]. Indeed, consensus target values can be biased and can be greatly influenced depending on market share (e.g. if some IVDs are over-represented in the EQAS) [43]. For that reason, a French law published in 2016 [44] now makes it mandatory that target values associated with EQAS materials should periodically be determined with a reference method, if available .

In Thailand, the National Institute of Metrology, Thailand (NIMT) is also involved in domestic efforts on healthcare. In 2017, the Thailand Creatinine Standardization Working Group was established by the Department of Medical Sciences, Ministry of Public Health, in collaboration with experts on clinical chemistry and laboratory medicine in Thailand, including NIMT. The working group aimed to reduce interlaboratory variation in creatinine assays and to enable more accurate glomerular filtration rate (GFR) estimation. The working group has recently published guidelines for creatinine standardization in 2018. In addition, NIMT is collaborating with the Laboratory Quality Standards, Bureau of Laboratory Quality Standards in Department of Medical Science to implement reference values on EQAS for creatinine measurements. The demonstration of competence for creatinine measurement in NIMT has been via CCQM-K12.2 for creatinine in serum.

In Germany, PTB is involved in the accreditation of calibration laboratories according to ISO/IEC 17025 [45] and ISO 15195 [14] by assessment of their measurement capabilities. These laboratories are responsible for providing reference measurement values as target values for the evaluation of ring trials of clinical testing laboratories. Two organizations are appointed by the German Medical Association (GMA) for the provision of the EQAS in quantitative clinical measurements, namely the German Society for the Promotion of Quality Assurance in Medical Laboratories (INSTAND e.V.), a specialist scientific medical association, and the Reference Institute for Bioanalytics (RfB), part of a foundation founded by the German Society for Clinical Chemistry and Laboratory Medicine (DGKL). Their calibration laboratories provide over 30 target values using reference measurement procedures that are listed in the JCTLM database. To verify their competence as calibration laboratories working at a high metrological level, fulfilling the requirements of ISO/IEC 17025 and ISO 15195, the calibration laboratories regularly participate in collaborative surveys for calibration laboratories such as the RELA – IFCC EQAS for Reference Laboratories in Laboratory Medicine. The scheme covers a wide range of analytes (metabolites and substrates, enzymes, electrolytes, glycated haemoglobins, proteins, hormones, thyroid hormones, and therapeutic drugs). Participation in an EQAS programme such as RELA is also one of the requirements for listing of a reference measurement service by the JCTLM. The calibration laboratories are expected to review and critique their results and take remedial action where necessary. Beyond this, PTB in this context also provides reference measurement values for selected measurands on a needs and risk basis.

In China, NIM is involved in the accreditation of 10 national reference laboratories in accordance with the requirement of ISO/IEC 17025 [46] and ISO 15195 [14]. This number reflects the scale of the reference measurement system in China when compared to the current total number of 38 JCTLM’s stakeholder members internationally to date. NIM, being part of the China Joint Committee for Traceability in Laboratory Medicine formed between itself, the China National Accreditation Service for Conformity Assessment (CNAS) and the National Center for Clinical Laboratories (NCCL) of China, has organized EQAS programmes on glucose for the reference laboratories, and on homocysteine for both the reference and hospital laboratories in Shanghai.

In 2009, NIST administered the first accuracy-based programme for improving the comparability of vitamin D metabolite measurements with the NIST/National Institutes of Health (NIH) Vitamin D Metabolites Quality Assurance Program (VitDQAP) [47]. Individual studies conducted with the VitDQAP illustrated a significant bias between immunoassay and higher-order liquid-chromatography with mass spectrometric detection results for the determination of 25(OH)D_Total in a fortified serum material [48]. In this case, one of the blind samples in the Study 3 exercise of the VitDQAP was SRM 972 Vitamin D in Human Serum L3, which was a “normal” human serum that has been fortified (i.e., spiked) with 25(OH)D₂. The results for the immunoassay methods were significantly lower, suggesting that the augmented 25(OH)D₂ content in the SRM 972 L3 material was not sufficiently incorporated into the matrix and thus was not completely recovered via immunoassay techniques. Such results are invaluable in determining the suitability of candidate (or existing) CRMs for use across various analytical platforms. Arguably, such evaluations through quality assurance programmes could serve as a first pass in the assessment of candidate CRM commutability. This, and other “lessons learned” related to using non-human serum as an initial matrix [49], have been applied in the development of NIST’s replacement material, SRM 972a Vitamin D Metabolites in Frozen Human Serum.

NIST also supported the conversion of the consensus-based Vitamin D External Quality Assessment Scheme (DEQAS) to an accuracy-based programme by providing RMP-based values for the DEQAS samples between 2012 and 2019 [50, 51]. The PT scheme for vitamin D metabolites administered by the College of American Pathologists (CAP) has also converted to an accuracy-based programme in recent years. While NIST has not assigned target values for the CAP samples, a subset of CAP samples has been included in multiple rounds of commutability testing along with NIST SRMs and DEQAS samples [46, 52].

Emerging issues and conclusions

Within the clinical field, there is a range of evolving issues related to aspects such as new biomarkers and new approaches to the measurement of biomarkers. Neonatal testing (newborn screening) is an area where new approaches to testing are being examined. For example, the national healthcare programme in South Korea allows all newborns to have the opportunity for screening of inherited metabolic diseases by blood tests. Dried blood spot (DBS) is used as a sampling method for this newborn screening [48]. Thus, KRISS has been working to introduce DBS type CRMs, produced by loading whole blood. A batch of DBS type CRM has been developed for amino acid and acylcarnitine measurement with the goal of launching these in 2019. DBS screening is an area that will continue to grow, and metrology institutes will be required to expand their capabilities to provide services in this area.

The need for harmonization, standardization, and evidential traceability of the next generation of clinical measurements is evolving. The International Consortium for Harmonization of Clinical Laboratory Results (ICHCLR) has the role of reviewing priorities and maintaining a summary of measurand harmonization activities. The Consortium provides information regarding individual measurand’s ranking with respect to medical impact, status of current activities and any resources or links to other organizations which may be working on a given measurand. Many of these are proteins; however, several small organic measurands [e.g. bilirubin (conjugated), triiodothyronine (T₃), free T₃, digoxin, and cortisol in serum] have been identified in areas where there is an urgent need for harmonization to ensure that measurement results are suitable for medical decisions. The ICHCLR now sits within the IFCC framework and hence, aligning these activities will be a significant benefit. It should also be noted that several institutes [NIST, LNE, the National Institute for Biological Standards and Control (NIBSC) and the JRC] actively participate in IFCC working group on reference material commutability.

The current clinical measurement activities of the OAWG for “small” molecule organic biomarkers also have a significant intersection with the emerging bioanalytical challenges associated with peptide and protein analysis. Comparisons such as CCQM-K55.d for L-valine purity assessment and CCQM-K78.a for amino acids in acidic solution serve as part of the foundation to support the development of SI traceable measurements for larger peptide and protein biomarkers that are carried out under the CCQM Protein Analysis Working Group (PAWG). These activities contribute to the standardization of measurement methods for clinically relevant proteins, which also relies on effective collaboration between metrology institutes, IVD manufacturers, clinicians and EQAS providers. Examples of successful collaborative projects include activities on standardization of neurodegenerative disorders biomarkers (e.g. alpha-synuclein) and cardiovascular disease biomarkers such as Cardiac Troponin I, as well as procalcitonin, haemoglobin A2, growth hormone in serum, and albumin in urine.

The task of developing and eventually linking a SI-traceable measurement infrastructure to a broad array of emergent clinical applications will be daunting if undertaken by any single institute. The CCQM OAWG activities have supported metrology institutes in demonstrating measurement equivalence, and the institutes have in turn worked to embed their clinical services in national and international programmes, disseminating SI-traceability to clinical results.

Abbreviations

BIPM

International Bureau of Weights and Measures

CCQM

Consultative Committee for Amount of Substance: Metrology in Chemistry and Biology

CIPM

International Committee for Weights and Measures

CRM

certified reference material

DBS

Dried Blood Spot

DEQAS

Vitamin D External Quality Assessment Scheme

DI

Designated Institute

EQALM

External Quality Assurance Programmes in Laboratory Medicine

EURAMET

European Association of National Metrology Institutes

EQAS

External Quality Assessment Schemes

ICHCLR

International Consortium for Harmonization of Clinical Laboratory Results

IDMS

Isotope dilution-mass spectrometry

IMEP

International Measurement Evaluation Programme

IFCC

International Federation for Clinical Chemistry and Laboratory Medicine

ILAC

International Laboratory Accreditation Cooperation

ISO

International Organization for Standardization

IVD

In-vitro diagnostic

JCTLM

Joint Committee for Traceability in Laboratory Medicine

MRA

Mutual Recognition Arrangement

NMI

National Metrology Institute

OAWG

Organic Analysis Working Group

RMPs

Reference measurement procedures

SI

International System of Units

PT

Proficiency testing

RCPAQAP

Royal Australasian College of Pathologists Quality Assurance Programs

SRM

Standard Reference Material^®

VDSP

Vitamin D Standardization Program

VitDQAP

Vitamin D Metabolites Quality Assurance Program

Biographies

Katrice Lippa leads the Organic Chemical Measurement Sciences Group within the Chemical Sciences Division in the National Institute of Standards and Technology (NIST), which is the source of quality assurance programmes and hundreds of Standard Reference Materials for clinical and metabolomics, food and natural products, nutritional assessment and chemical manufacturing. She currently serves as Vice Chair of the Consultative Committee for Amount of Substance: Metrology in Chemistry and Biology (CCQM)’s Organic Analysis Working Group.

Lindsey Mackay is General Manager of Chemical and Biological Metrology in the National Measurement Institute, Australia (NMIA) and serves as Chair of the CCQM’s Organic Analysis Working. Her team in NMIA produces a wide range of pure organic and matrix chemical reference materials, gas standards and DNA reference materials, and also coordinates proficiency testing programmes covering food, environmental, illicit drug and gas measurements.

Tang Lin Teo is Acting Division Director and Laboratory Director of the Chemical Metrology Division in the Health Sciences Authority (HSA). She works closely with her teams in developing certified reference materials (CRMs) for clinical, food, water, cosmetics and pharmaceutical testings, and in organizing accuracy-based proficiency testing such as external quality assessment (EQA) programmes for Singapore’s clinical laboratories and medical clinics, with a focus on clinical markers of chronic diseases affecting the local population.

Sharon Yong is Analytical Scientist in the Organic (Clinical) Chemistry Section within the Chemical Metrology Laboratory in HSA. Her work covers the development of CRMs and reference measurement methods for biomarkers in biological fluids, peptides as calibrators, and coordination of the annual accuracy-based EQA Programme on haemoglobin (Hb) A_1c testing with the HSA team.

Qinde Liu is Consultant Scientist and Leader of the Organic (Clinical) Chemistry Section within the Chemical Metrology Laboratory in HSA and serves as one of the Vice Chairs of the Joint Committee for Traceability in Laboratory Medicine (JCTLM)’s Database Working Group. His work covers the development of CRMs and reference measurement methods for biomarkers in biological fluids, purity assessment of peptides and proteins, and coordination of HSA’s accuracy-based EQA Programmes.

Johanna E. Camara is Research Chemist at NIST and Technical Project Leader for a variety of clinically-relevant reference materials and has worked on project planning, liquid chromatography-mass spectrometry method development, measurements, and commutability studies of reference materials. She is also Quality Manager and represents NIST in several Standards Development Organizations.

Vincent Delatour coordinates R&D activities in bioanalysis in the Laboratoire National De Métrologie Et D’essais (LNE), the National Metrology Institute (NMI) in France, and serves as member of the JCTLM’s Working Group on Traceability, Education and Promotion. The goal of his work in LNE is to assess and improve comparability and accuracy of medical test results through the development of mass spectrometry-based reference methods and CRMs for clinically relevant biomarkers.

Tong Kooi Lee was Division Director of the Chemical Metrology Division in HSA since from its establishment in 2008 to May 2019. Since stepping down as Division Director, he has taken on an advisory role in the division.

Béatrice Lalere heads has headed the Biomedical and Organic Chemistry Department in LNE for several years. The activities aim to assess and improve comparability and accuracy of test results in the environmental and biomedical fields through the development of mass spectrometry-based reference methods and CRMs.

Gavin O’Connor is Head of Department for Biochemistry at Physikalisch-Technische Bundesanstalt (PT B) a n d Professor of Biochemical Metrology at the Technical University of Braunschweig. Having previously worked at LGC in the UK and the European Commission’s Joint Research Centre, his research has focused the provision of metrologically traceable results for the characterization of reference materials using IDMS. The focus of the Department of Biochemistry at PTB is on the development of high accuracy measurement procedures for clinically relevant targets.

André Henrion is responsible for bio-organic mass spectrometry at PTB. He has been engaged in the development of primary methods for quantification of small-molecular health status markers and protein biomarkers based on isotope dilution mass spectrometry. Beyond this, his present interest includes the use of cross-linking mass spectrometry (XL-MS) for measurement of protein structures and interactions.

Megumi Kato is Leader of the Bio-medical Standards Group in the National Metrology Institute of Japan (NMIJ), National Institute of Advanced Industrial Science and Technology (AIST). She has developed analytical techniques for protein, peptide, and amino acid measurements, and also some CRMs such as C-reactive protein solution and proteinogenic amino acids.

Masahiko Numata is Leader of Organic Primary Standards Group in NMIJ, AIST, and Visiting Professor at the Gunma University. He has developed analytical techniques for environmental analyses, and some CRMs such as polychlorinated biphenyls in sediments and mineral oils.

Ha-Jeong Kwon is Senior Research Scientist in the Center for Bioanalysis within the Division of Chemical and Medical Metrology in the Korea Research Institute of Standards and Science (KRISS). She has been working for several years on the development of analytical procedures and on the production of CRMs for clinical markers based on LC-MS/MS and ion chromatography.

Ji-Seon Jeong is Senior Research Scientist in the Center for Bioanalysis in KRISS and Associate Professor at the University of Science and Technology. She has been working for several years on the development of measurement standards in clinical markers based on the instrumental analysis (LC-MS, IC, HPLC, CE, and UV).

Bei Xu is Deputy Group Leader of the Organic Analysis Laboratory in NIM, China, and Leader of the JCTLM’s Metabolites and Substrates Review Team. In the past 22 years, she has worked on the development of reference methods and reference materials for small molecule organic bio-markers in laboratory medicine.

Dewei Song is Group Leader of the Clinical Chemistry Research Team within the Division of Chemical Metrology and Analytical Science in the National Institute of Metrology (NIM), China, and member of the JCTLM’s Proteins Review Team. He has been working for several years on the development of reference method and reference materials for laboratory medicine based on isotope labeling technology, chromatography and mass spectrometry.

Jintana Nammoonnoy is Metrologist in the Organic Analysis Group within the Chemical Metrology and Biometry Department in the National Institute of Metrology (Thailand). She has been working on the development of chromatographic methods for the determination of organic compounds in biological fluids and the production of CRMs for clinical measurements.

Wagner Wollinger is Deputy Head of the Organic Analysis Laboratory within the Chemical and Thermal Metrology Division in the Instituto Nacional deMetrologia, Qualidade e Tecnologia (INMETRO), the NMI in Brazil. He has been working for several years on the production of CRMs for chemical substances, tests of products for conformity assessment purposes and development of analytical methods based on nuclear magnetic resonance, chromatography and mass spectrometry.