Harmonization of Clinical Laboratory Test Results: The Role of the IVD Industry

Abstract

At the start of the 21st century, a dramatic change occurred in the clinical laboratory community. Concepts from Metrology, the science of measurement, began to be more carefully applied to the in vitro diagnostic (IVD) community, that is, manufacturers. A new appreciation of calibrator traceability evolved. Although metrological traceability always existed, it was less detailed and formal. The In Vitro Diagnostics Directive (IVDD) of 2003 required manufacturers to provide traceability information, proving assays were anchored to internationally accepted reference materials and/or reference methods. The intent is to ensure comparability of patient test results, regardless of the analytical system used to generate them. Results of equivalent quality allows for the practical use of electronic health records (EHRs) capture a patient’s complete laboratory test history and allow healthcare providers to diagnose and treat patients, confident the test results are suitable for correct interpretation, i.e., are “fit for purpose” and reflect a real change in a patient’s condition and not just “analytical noise.” The healthcare benefits are obvious but harmonization of test systems poses significant challenges to the IVD Industry. Manufacturers must learn the theory of metrological traceability and apply it in a practical manner to assay calibration schemes. It’s difficult to effect such a practical application because clinical laboratories do not test purified analytes using reference measurement procedures but instead deal with complex patient samples, e.g., whole blood, serum, plasma, urine, etc., using “field methods.” Harmonization in the clinical laboratory is worth the effort to achieve optimal patient care.

INTRODUCTION

The world is experiencing globalization and the clinical laboratory field is no exception. The goal is to provide optimal healthcare to the global population and clinical laboratory practice is inexorably moving towards harmonization. As stated by Greenberg, “An increasingly important objective in laboratory medicine is ensuring the equivalency of test results among different measurement procedures, different laboratories and health care systems, over time (1).” This requires harmonization and metrological traceability of assays to provide equivalence of results derived from different analytical systems (2). This has not been possible historically because assays provided by Industry have not been sufficiently comparable due to a lack of established reference materials and methods to “anchor” tests. As noted by Miller and Myers, “True and precise routine measurements of quantities of clinical interest are essential if results are to be optimally interpreted for patient care. Additionally, results produced by different measurement procedures for the same measurand must be comparable if common diagnostic decision values and clinical research values are to be broadly applied (3).”

A patient’s test history would be consistent if a single clinical lab performed all testing (i.e., same methodology, stable analytical performance, etc.) so a significant change in concentration (decrease or increase) would signal a meaningful clinical change. In reality, patients are increasingly mobile and two or more laboratories may test their samples. If the tests performed by different laboratories are sufficiently harmonized so as to produce essentially equivalent results (not necessarily quantitatively equal, but clinically equivalent), changes in concentration can be correctly interpreted by a healthcare provider. As explained by Gantzer and Miller “Clinical laboratory measurement results must be comparable among different measurement procedures, different locations and different times in order to be used appropriately for identifying and managing disease conditions (4).”

Harmonization is needed to use of electronic medical records/electronic health records (EMRs/EHRs) to capture all of a patient’s lab results in an electronic file available to patients and healthcare providers. Clinical laboratory results typically account for much of the information in EMRS but the benefit is negated if the cumulative values in EMR for the same analyte are not comparable. Perhaps not a problem for traceable analytes, e.g., electrolytes and glucose, but very much an issue for immunoassays such as thyroid and fertility hormones and cancer markers. Interpretation of sequential values using common reference intervals and medical decision levels (MDLs) is difficult, if not impossible. It’s been suggested laboratory data accounts for about 70% of clinical decisions. Hallworth has challenged that blanket statement but allows “The value of laboratory medicine in patient care is unquestioned (5). That value is greatly diminished without comparability of test results.

Cholesterol is a prime example of successful harmonization. Creating a reference measurement system (RMS) for this key lipid over about 30 years (1970 – 2000) coincided with a major reduction in mortality rates for coronary heart disease (CHD) in the US and also achieved a huge savings in healthcare dollars (1). The consequences of the lack of harmonization was demonstrated by an NIST report on calcium (Ca) that estimated the cost of a 0.1 mg/dL Ca bias can cost $8 - $31 for additional, but unnecessary, patient follow up testing (6). A bias of 0.5 mg/dL could results in an additional $34 - $89/patient. On an annual basis, a 0.1 mg/dL bias could translate into $60 - $199 million/year for about 3.55 million patients screened for Ca.

HARMONIZATION VS. STANDARDIZATION

In this paper “harmonization” is used interchangeably with “standardization,” though there is a distinction between the two (4). Standardization means results are traceable to higher metrological order reference materials and/or methods and ideally can be reported using SI units. Harmonization means results are traceable to some declared reference but accepted higher order reference materials and/or methods are not available and SI units are not applicable. Harmonization ensures comparability of results, enables application of clinical best practice guidelines and reference intervals, increases patient safety, and decreases medical care costs. Harmonization requires the cooperation of laboratories, academia, professional societies, metrological institutes, government agencies, EQA/PT providers, and industry. Two recent harmonization (actually, standardization) success stories mediated by Industry are creatinine and glycated hemoglobin (Hb A1c). Field assays for both of these analytes feature complete traceability chains and are firmly anchored by reference measurement systems. That said, ironically results for both assays are still typically reported in different units, creatinine in mg/dL (“conventional units”) and mmol/L (SI units), and Hb A1c in % Hb A1c (NGSP units) and mmol/mol (SI units).

METROLOGICAL TRACEABILITY

The In Vitro Diagnostics Directive (IVDD) of 2003 applies to Europe for the purposes of the CE mark, but has global implications. It requires manufacturers to establish the metrological traceability and uncertainty of kit calibrators. “Metrological traceability is defined in the VIM, clause 2.41 as the ‘property of a measurement result whereby the result can be related to a reference (a standard) through a documented unbroken chain of calibrations, each contributing to the measurement uncertainty. (1)” The IVDD doesn’t provide specifics but ISO 17511 (Metrological traceability of values assigned to calibrators and control materials) applies (7; see Fig 1.). It establishes a metrology infrastructure for assays. The IVDD requirements are incorporated in ISO 15189 (Medical laboratories- particular requirements for quality and competence) (8).

Figure 1.

General metrological traceability diagram from ISO 17511, in vitro diagnostic medical devices — Measurement of quantities in biological samples — Metrological traceability of samples assigned to calibrators and control materials, 2003

Abbreviations: ARML - Accredited reference measurement laboratory (such a laboratory may be an independent or manufacturer’s laboratory); BIMP - International Bureau of Weights and Measures; CGMP - General Conference on Weights and Measures; ML - Manufacturer’s laboratory; NMI - National Metrology Institute.

The symbol u_c(y) stands for combined standard uncertainty of measurement.

As White explains “Metrology, the science of measurement, provides laboratory medicine with a structured approach to the development and terminology of reference measurement systems which, when implemented, improve the accuracy and comparability of patients’ results (9).” Metrological principles are a relatively new in the clinical laboratory. For example, the Tietz Textbook of Clinical Chemistry (third edition, 1999) made no mention of “uncertainty” or “commutability” (10). The fourth edition (2006) mentioned uncertainty and commutability but only a definition of commutability was given (11). The fifth edition (2011) includes a discussion of uncertainty along with commutability (12). As noted by De Bievre, “Discussions with analytical chemists have revealed that basic concepts in metrology, including ‘traceability,’ are generally not an integral part of university or college curricula and are not treated in most text books of analytical chemistry” (13).

Metrology must be adapted to the clinical laboratory, but a practical approach is advisable due to differences between the disciplines. For example, Metrology is a “pure science” contrasting with the mixed science of clinical chemistry (combines several diverse sciences/technologies). National metrology institutes are “ivory towers” in comparison to clinical laboratories (“the trenches”). Metrology tests pure, well-defined analytes in simple matrices but clinical labs test complex, ill-defined analytes in challenging matrices (serum, plasma, urine, etc.). Metrology estimates expanded uncertainty (bias eliminated) while clinical labs focus on Total Error Allowable (TEa = bias + imprecision). Metrology seeks “absolute scientific truth” by reference method analysis but clinical labs deal in “relative truth” by field method analysis. Good metrology does not necessarily equal good clinical laboratory science but the clinical laboratory field needs to adapt Metrology concepts and “translate” them for practical application.

THE PILLARS OF HARMONIZATION

In anticipation of the IVDD, the Joint Committee for Traceability in Laboratory Medicine (JCTLM) was formed in 2002(1). It established three pillars of traceability: 1. reference measurement procedures (RMP), 2. reference materials (RM), and 3. a network of reference measurement laboratories. The JCTLM maintains a searchable database for all three on the International Bureau of Weights and Measures (BIPM) web site (14). The laboratory community has identified three other “pillars” in response to harmonization: 1. universal reference intervals and medical decision levels (MDLs), 2. accuracy based grading EQA/PT programs to ensure traceability of field assays is maintained and analytical bias is minimized or meets established criteria (e.g., CAP PT requirement of +/-6% of the NGSP target value for Hb A1c), and 3. harmonization of clinical laboratory practice and the total testing process (TTP), e.g., standardized nomenclature/terminology, reporting units, EBLM, etc.

The JCTLM goal is comparability of patient test results from different methods to ensure appropriate medical decision-making and optimal healthcare (15, 16). The components of a reference measurement system (RMS) are: 1. definition of the analyte, 2. RMP that specifically measures the analyte, 3. Primary and secondary reference materials, and 4. reference measurement laboratories. Analytes fall into two categories: 1. Type A (well defined; concentration in SI units; results not method dependent; full traceability chain), and 2. Type B (not well defined, heterogeneous, present in both bound and free state, not traceable to SI, rigorous traceability chain not available). The JCTLM provides a list of higher order RMs and RMPs and reference laboratories (17).

A requirement for harmonization is commutability. Commutability is defined as a property of a reference material, demonstrated by the closeness of agreement between the relation among the measurement results for a stated quantity in this material, obtained according to two given measurement procedures, and the relation obtained among the measurement results for other specified materials (4). In other words, fresh patient samples and materials such as calibrators need to provide an identical analytical response (see Fig. 2). Many secondary RMs are not commutable with native clinical samples and have failed to accomplish the intended goal of achieving harmonized results (4). Commutability is not a universal property of reference materials and must be proven with every field method. Well recognized by Metrology, commutability is not so widely appreciated in routine clinical laboratories. Historically, the commutability reference materials and calibrators prepared from them or traceable to them has not routinely been established. Noncommutability results in significant biases with field assays due to matrix effects, use of non-human forms of analyte, lack of antibody specificity, or other causes. The JCTLM now requires a commutability assessment of reference materials to be listed in its database. CLSI EP30 (Characterization and qualification of commutable reference materials for laboratory medicine) is a recent guideline (18). Metrology defines measurement uncertainty, or simply uncertainty, as a non-negative parameter characterizing the dispersion of the quantity values being attributed to a measurand, based on the information used (4). It is roughly equivalent to imprecision but ideally assay bias is eliminated prior to estimating uncertainty. CLSI EP29 (Expression of measurement uncertainty in laboratory medicine) is another recent guideline (19).

Figure 2.

Commutability is demonstrated if fresh patient samples and reference materials, e.g., calibrators, demonstrate an equivalent analytical response when tested by two methods

Commutable: same relationship for clinical samples and reference materials.

The fourth “pillar” of traceability- universal reference intervals- cannot be erected without the adoption of reference measurement systems and assay harmonization. Reference intervals for some analytes can be affected by various partitioning factors, e.g., age, gender, ethnicity, BMI (body mass index), and thus universal ranges may not be feasible. But such decisions can’t be made until harmonization has been achieved.

To meet the IVDD traceability requirement for result trueness and comparability requires the fifth “pillar:” validation of manufacturers’ metrological traceability by EQA/PT. EQA/PT programs using commutable samples with reference method target values allow accuracy based grading (20). Horowitz notes “Far too many laboratories consider proficiency testing just a necessary evil, little more than periodic pass–fail exercises we perform solely to meet regulatory requirements. Even for central-laboratory techniques, traditional PT suffers from ‘matrix effects,’ in that samples used for testing often react differently from native patient samples. Therefore, comparisons must be made only to peer groups, rather than to the ‘true value.’ What if the peer group as a whole is wrong? (20)” EQA/PT has typically been used to measure proficiency at performing a test and not the trueness of the test method or its performance relative to other method. For this reason, Miller concludes “Traditional PT materials are not suitable for field-based postmarketing assessments of a method’s trueness (21).” In one study, commutable serum-based material was assigned target values by reference methods for six enzymes (ALT, AST, CK, GGT, LD, and amylase) and was tested by 70 labs in Germany, Italy, and The Netherlands using six field methods (22). Results were graded on accuracy based on biological variability targets. For ALT, results were deemed acceptable for > 94% of the six commercial assays. Performance for the other five enzymes was variable and all methods demonstrated significant bias for CK. “Overall, it appears clear that method bias should be reduced by better calibration to the internationally accepted reference systems (22).”

The sixth harmonization “pillar” is the Total Testing Process (TTP). Plebani observed “Although the focus is mainly on the standardization of measurement procedures, the scope of harmonization goes beyond method and analytical results: it includes all other aspects of laboratory testing, including terminology and units, report formats, reference intervals and decision limits, as well as test profiles and criteria for the interpretation of results (23).” Harmonization of reporting units would seem easy to achieve but that’s not the case. “Even a change in the unit of hemoglobin (Hb) expression could potentially affect patient safety. Findings in a recent survey conducted in the UK revealed that 80% of laboratories were using g/dL, although g/L is the recommended unit … (23).” Harmonization of basic terminology and units is necessary but the international clinical laboratory community has yet to reach agreement. For examples of disharmony, see Table 1.

Table 1.

The necesity of reaching agreement over harmonization of basic terminology and units in the international clinical laboratory community: some examples of disharmony

Analyte	“Conventional units”	SI units*
ALT	U/L	mkat/L
Bilirubin	mg/dL	mmol/L
Cl	mEq/L	mmol/L
Glucose	mg/dL	mmol/L
Creatinine	mg/dL	mmol/L
Hb A1c	% Hb A1c	mmol/mol

* SI = International System of Units (Système International d’unités)

CHALLENGES FOR THE IVD INDUSTRY

Embracing metrological concepts and harmonization represents a paradigm shift for the in vitro diagnostics community. Manufacturers traditionally sought to differentiate themselves from competitors (e.g., by claiming a greater dynamic range, lower LoD, better precision, smaller sample size, etc.), and producing comparable patient results was not a priority. Lack of harmonization among field assays is evident from review of EQA/PT data, often of necessity reported by peer group (as opposed to accuracy based grading). In an era of harmonization, results from different systems should be comparable. Manufacturers are responding by: providing calibrator traceability/uncertainty information, restandardizing assays, testing commutability, etc., and they work with many professional organizations and each other to attain harmonization, but this is a new approach and challenge for the industry. Manufacturers have an integral role in educating customers about harmonization of assays, harmonization and clinical laboratory practice in general. Of course the age old question remains: “Where do manufacturers’ obligations end and the obligations of lab directors begin?” Manufacturers must provide “fit for purpose” tests, but labs must use the assays properly and effectively. When an assay “failure” occurs (and “failure” can apply to myriad issues and causes) does the fault lie with the manufacturer or with the lab and its use of the test?

A major challenge for manufacturers is to choose a total allowable error (TEa) goal from the many available options: CLIA requirements (U.S. specific); CAP; RCPA, RiliBÄK, or other EQA/PT provider specifications. A popular approach is to define TEa based on biological variability targets, but there are three targets from which to choose:

Minimum: TE_a < 1.65(0.75 CV_i)+0.375(CV_i² + CV_g²)^½ Desirable: TE_a <1.65(0.5 CV_i)+0.25(CV_i²+ CV_g²)^½ Optimum: TE_a < 1.65(0.25 CV_i)+0.125(CV_i² +CV_g²)^½ CV_i = individual biological variability; CV_g = group biological variability

An IFCC initiative is the Working Group on Allowable Error for Traceable Results (WG-AETR). This group concluded “Although manufacturers are compelled by the European IVD Directive, 98/79/EC, to have traceability of the values assigned to their calibrators if suitable higher order reference materials and/or procedures are available, there is still no equivalence of results for many measurands determined in clinical laboratories” (24). For some common analytes, such as sodium, current assays are too imprecise to meet TEa targets based on biological variation. The aim of harmonization is equivalent results but unfortunately, due to cost and limited resources, IVD manufacturers don’t always follow full traceability steps to value assign every new calibrator lot but rely on value transfer from an internally stored (“master”) calibrator material. In most cases, this procedure is probably valid, but a common complaint is calibrator lot to lot variability. The WG-AETR noted that when there are two traceability paths for a measurand, calibrators from different manufacturers may both be derived from valid traceability chains but produce non-equivalent results, as illustrated by Fig. 3. Equivalent results from two systems may be possible by using a correction factor determined by a correlation study.

Figure 3.

Manufacturers may prepare calibrators starting with traceability to the same reference material and/or reference method, but the calibrator manufacturing process may diverge at some point, resulting in significantly different results for the same measurand in the same patient sample if tested by the two field methods, despite metrologically acceptable traceability for each assay’s calibrators

The international clinical laboratory community has embraced harmonization. A prime example is the AACC’s ICHCLR (International Consortium for Harmonization of Clinical Laboratory Results) (2). The ICHCLR prioritizes analytes globally for harmonization and development of RMs and RMPs for listing in the JCTLM database, which will allow for comparable results irrespective of the laboratory, method, or the time when testing is performed. ICHCLR stakeholders include: clinical lab and medical professional societies, IVD manufacturers, metrology institutes, public health organizations, regulatory agencies, and standard-setting organizations. A similar initiative is Pathology Harmony in the UK (25). Pathology Harmony states: “as we move towards full electronic reporting of pathology results, we appreciate more fully that variations in things such as test names, reference intervals and units of measurement associated with our results is something that hinders progress.’’ In Australia, there is the RCPA (Royal College of Pathologists of Australasia) PITUS (Pathology Information Terminology and Units Standardisation Project) program that is dedicated to harmonization (26). PITUS in particular focuses on the interoperability of pathology test requesting and reporting. These initiatives and others are all supported by Industry.

MANUFACTURERS’ ROLE IN THE 21^st CENTURY

Industry support can be optimized when the harmonization initiatives are coordinated and prioritized. From the industry perspective there are limitations, costs and tradeoffs which need to be considered. Device manufacturers all have substantial product development priority lists and development schedules and personnel and financial resources are committed over long term periods to achieve strategic goals. The development process for a new product can be measured over years in our highly regulated environment. Further, the cost for each project can run into the millions of dollars. Reprioritization is possible and welcomed by industry when the results will provide benefit to the clinician, patient and healthcare system. Stellar examples such as creatinine, hemoglobin A1c and cholesterol have been pointed out in this manuscript.

The global drive for harmonization creates competing project priorities for companies. As manufacturers sign on to support harmonization projects, timelines that reflect development cycles (years) allow companies to reprioritize resources while maintaining projects that drive innovation, product health and portfolio development.

Harmonization may also require worldwide re-registration of products. Meeting the criteria of country specific regulatory agencies comes with additional considerations and complexities beyond the harmonization initiative. Registration timing is not equivalent in all countries and multiple products for a given measurand may need to be supported for an extended period of time. This impacts manufacturing resources and production costs.

It is imperative there be close coordination of industry, professional bodies and the global leaders of harmonization initiatives to ensure harmonization is successful. If companies could contribute to the prioritization of projects, design of experiment and contribute to the inputs we would be assured changes requiring product re-registration would be successful. This would also avoid unintentional competitive imbalances.

A significant consideration is the traceability of the reference assay. Device manufacturer’s typically register products using a predicate device to demonstrate product acceptance. In such cases proof of substantial equivalence is essential to demonstrate the assay is safe and effective. If a reference assay is a laboratory developed test the path to regulatory registration and the ability to commercialize the assay brings with it additional complications.

Lastly, a major consideration is whether the harmonization initiative provides benefit to the public. While accuracy is important, there are situations where existing assays may be relatively harmonized yet the reference method is very different from the commercialized assays. Under these special circumstances the cost of harmonization which includes physician education, patient safety and investment in product redevelopment must be carefully weighed to understand the benefit of harmonization.