Role and Responsibilities of Laboratory Medicine Specialists in the Verification OF Metrological Traceability of in vitro Medical Diagnostics

Summary

To be accurate and equivalent, laboratory results should be traceable to higher-order references. Furthermore, their quality should fulfill acceptable measurement uncertainty as defined to fit the intended clinical use. With this aim, in vitro diagnostics (IVD) manufacturers should define a calibration hierarchy to assign traceable values to their system calibrators and to fulfill during this process uncertainty limits for calibrators, which should represent a proportion of the uncertainty budget allowed for clinical laboratory results. It is therefore important that, on one hand, the laboratory profession clearly defines the clinically acceptable uncertainty for relevant tests and, on the other hand, end-users may know and verify how manufacturers have implemented the traceability of their calibrators and estimated the corresponding uncertainty. Important tools for IVD traceability surveillance are quality control programmes through the daily verification by clinical laboratories that control materials of analytical systems are in the manufacturer’s declared validation range [Internal Quality Control (IQC) component I] and the organization of External Quality Assessment Schemes meeting metrological criteria. In a separate way, clinical laboratories should also monitor the reliability of employed commercial systems through the IQC component II, devoted to estimation of the measurement uncertainty due to random effects, which includes analytical system imprecision together with individual laboratory performance in terms of variability.

Introduction

Clinical laboratories must provide useful information for patient care through the production of equivalent results across space and time. The achievement of this ideal framework in laboratory practice would have an important clinical, economical and ethical impact, significantly contributing to health care improvement by allowing the results of clinical studies undertaken at different locations or times to be universally applied.

There is an international agreement on the fact that, to become equivalent in the long run, results obtained by a calibrated routine procedure must be expressed in terms of the values obtained at the highest available level of the calibration hierarchy (1). In this regard, it is essential to build an unbroken metrological traceability chain that starts from the unequivocal definition of the measurand and ends, through a calibration hierarchy, at the level of the patient’s result (2). Only through a suitable metrological traceability chain the in vitro diagnostics (IVD) manufacturers can reliably transfer the measurement trueness from the highest level of the metrological hierarchy to the calibrators of commercial analytical systems used in clinical laboratories. The measurement bias along all the traceability steps, if any, should be appropriately eliminated to obtain unbiased results on clinical samples (3). The implementation of this metrology-based approach would allow to use common reference intervals and clinical decision limits and effectively apply evidence-based medicine (4). Moreover, the universal standardization of laboratory test results would allow to economize on the average aggregate cost of follow-up procedures and to achieve an important ethical dimension as it aims to affect the way diagnostic tests are used in order to guarantee optimal care for patients in a global world (5).

Considering that results produced by assays operating under unbiased conditions have in any case an associated uncertainty that derives both from uncertainties accumulated along the steps of the metrological chain and from random effects, an adequate estimation of this combined uncertainty should be performed (6, 7). These uncertainty values should then be compared with appropriate uncertainty limits in order to validate the clinical usefulness of the measurement (8).

An important point that should be highlighted in applying the previously illustrated approach is that it is no longer possible to consider separately the components of each commercial analytical system (i.e., platform, reagents, calibrators and control materials), which in terms of performance can only be guaranteed and certified by the manufacturer as a whole. Any change introduced by users or third parties (for instance, the use of reagents, calibrators or control materials from different suppliers) may indeed significantly alter the quality of the analytical system performance, removing any responsibility from the manufacturer and depriving the system (and, consequently, the produced results) of the certification originally provided (7). On the other hand, once IVD manufacturers have designed commercial systems that potentially meet the requirements of traceability and established quality for clinical application, it is the task of laboratory medicine specialists to verify if manufacturers have correctly implemented the traceability of their calibrators and if the performance of marketed systems is really appropriate for their clinical use. Here, we describe in more detail the role and responsibilities of laboratory professionals in doing this verification.

Role of the Laboratory Profession

The laboratory profession should clearly define the clinically acceptable uncertainty for results of relevant tests from which it is in turn possible to derive the components of uncertainty budget relative to various levels of the traceability chain (8). Furthermore, once the IVD product has been purchased by the end-users, laboratory professionals should verify that the process of system alignment to higher-order references has been correctly implemented and survey its analytical performance through appropriately structured quality control programmes.

Definition of the clinically acceptable uncertainty

Defining analytical performance specifications for each analyte is essential to make their determination clinically usable and to ensure that the measurement error does not prevail on the result (7, 9). These performance goals should be established by the laboratory profession according to recognized and widely accepted models. Particularly, the hierarchy of sources for deriving analytical goals of a laboratory measurement has been recently updated in a conference held in Milan. Although the essence of the hierarchy originally established in 1999 was supported (10), new perspectives have been forwarded prompting simplification and explanatory additions. Basically, the recommended approaches for defining analytical performance specifications should rely on the effect of analytical performance on clinical outcomes or on the biological variation of the measurand (Table I) (11). The attention is primarily directed towards the measurand and its biological and clinical characteristics, some models being therefore better suited for certain measurands than for others. For instance, the model based on biological variation is probably not appropriate for analytes showing high individuality: for those analytes, outcome-based data or, in their absence, the state of the art of measurement quality are the models to derive analytical specifications.

Table I.

Recommended models to be used for defining analytical performance specifications (Adapted from ref. 11).

Model 1: Based on the effect of analytical performance on clinical outcomes Done by direct outcome studies – investigating the impact of analytical performance of the test on clinical outcomes; Done by indirect outcome studies – investigating the impact of analytical performance of the test on clinical classifications or decisions and thereby on the probability of patient outcomes, e.g., by simulation or decision analysis.

Model 2: Based on components of biological variation of the measurand.

Model 3: Based on state of the art of the measurement (i.e., the highest level of analytical performance technically achievable).

Availability and quality of information about traceability and uncertainty of IVD systems

We recently recommended that IVD end-users should be able to access the following information on the employed analytical system and its calibration: a) which higher-order references (materials and/or procedures) have been used to assign traceable values to calibrators, b) which internal calibration hierarchy has been applied by the manufacturer, with a detailed description of each step, c) the value of expanded combined uncertainty of commercial calibrators, and d) which, if any, acceptability limits for uncertainty of calibrators were applied in the validation of the analytical system (7, 8). Taking the measurement of plasma glucose as an example, we showed that, in the best case, IVD manufacturers provide in their package inserts only the name of the reference material or procedure to which the assay calibration is traceable, without any description of the traceability implementation steps and their corresponding uncertainty (7). Organisations, such as the National Institute of Standards and Technology, Institute for Reference Materials and Measurements, IFCC, etc., are frequently mentioned and used as a »trusted brand«, often without any further explanation. Importantly, the declared uncertainty of commercial calibrators, when available, usually does not combine uncertainties associated with higher levels of the selected metrological traceability chain, appearing sometimes much lower than the uncertainty associated to the upper part of the traceability chain used to transfer the measurement trueness, which is, of course, impossible according to the traceability theory.

The task of laboratory professionals is therefore to contact IVD manufacturers and obtain the complete information, if not available in the assay or calibrator inserts. This verification of the implementation approach of IVD metrological traceability and of the uncertainty estimate is essential for laboratories in order to maintain and/or improve the quality of patient results. One should clearly be aware that the use of CE (»Communautés Européennes«) marking on IVD products by itself does not guarantee that the manufacturer has transferred trueness successfully, also because at present no normative verification of the manufacturer’s statements by a third party is expected. Moreover, even if the traceability of calibration is successfully implemented, this does not automatically ensure acceptable trueness and/or uncertainty for individual patient results. For instance, the selection of different types of traceability chains available for the same measurand may lead to different combined uncertainties at the level of commercial calibrators, not always permitting to fulfill the uncertainty goal at the level of patient results (7, 8). On the other hand, the uncertainty (including assay imprecision) associated with the available traceability chains may be too large, not permitting to meet specifications based on clinical needs (12, 13). Finally, the lack of analytical specificity of some assays for the measurand as the quantity subject to measurement may significantly influence the accuracy of an individual patient’s result even when a well-defined reference system is available. We previously used the example of serum creatinine to show how traceability, even when correctly implemented, does not correct for bias due to analytical non-specificity problems associated with alkaline picrate methods (8, 14, 15).

Daily surveillance of IVD analytical systems

Once the measurement system has been introduced into daily practice, the possible sources of degradation of its performance are numerous. It is therefore essential to put in place a continuous surveillance of the quality of performance of commercial assays. This surveillance basically relies on quality control programmes, which should, however, be redesigned to meet metrological criteria (6, 7). Particularly, the Internal Quality Control (IQC) has to be reorganised into two independent components: one devoted to checking the alignment of the analytical system and verification of the consistency of declared traceability during routine operations performed in accordance with the manufacturer’s instructions (IQC component I) and the other structured for estimating the measurement uncertainty due to random effects (IQC component II).

IQC component I: checking of system alignment

This programme checks whether in the course of an analytical run the system performance complies with the set goals, represented by the acceptable ranges of control material(s) provided by the manufacturer as a component of the analytical system (7). In this way, clinical laboratories may verify the consistency of declared traceability during routine operations, which should be strictly performed in accordance with the manufacturer’s instructions, with no clinically significant bias in the assumed traceable results. Any »out of control« signal must be considered to allow immediate corrective actions to bring the situation under control again (i.e., virtually »unbiased« results), before test reports related to the samples analyzed in the affected analytical run are issued (Figure 1).

Figure 1.

Internal Quality Control component I for checking the metrological alignment of the analytical system.

IQC component II: estimating the random uncertainty

Clinical laboratories should also monitor the reliability of the employed commercial system by estimating the uncertainty due to the random effects, which includes the analytical system imprecision together with the individual laboratory performance in terms of variability (6, 8). This IQC component II should provide, through mechanisms of retrospective evaluation, data useful for the knowledge of variability of the analytical system (e.g., the reagent lot-to-lot variation) and of its daily use by the individual laboratory (Figure 2). We previously emphasized how the IQC component II programme should be completely independent of the above described IQC component I addressed to check the alignment of the analytical system, using a different control material with well--defined characteristics (Table II) (6, 8).

Figure 2.

Internal Quality Control component II for estimating the random sources of measurement uncertainty.

Table II.

Main characteristics of a control material to be used for the Internal Quality Control (IQC) component II programme (Adapted from ref. 8).

Characteristic	Remarks
Material from a third-party independent source should be used	Material must be different from the control material used for checking the system alignment (IQC component I)
Material should closely resemble to authentic patient samples (fulfil commutability) (e.g., fresh-frozen pool)	Commercial non-commutable controls may provide a different impression of imprecision performance
Material concentration levels should be appropriate for the clinical application of the analyte measurement	When clinical decision limits are employed for a given analyte, materials around these concentrations should preferentially be selected

External Quality Assessment Schemes (EQAS) meeting metrological criteria

The participation of individual laboratories to EQAS that meet specific metrological criteria is mandatory to check the alignment of employed commercial systems to higher-order references. The requirements for the applicability of EQAS results in the performance evaluation of participating laboratories in terms of standardization and traceability of their measurements are reported in Table III (6, 7, 16). Examples in literature showing the effectiveness of this approach in noting traceability problems in commercially available assays for a specific measurand are available (15, 17–21). There are, however, few permanent EQAS covering these requirements, because some practical constraints, including technical (lack of certified materials, difficulties to prepare commutable samples, complicated logistics of distribution of frozen samples), psychological (lack of awareness of which quality factors make an EQAS important) and economic (higher costs) aspects, are still limiting their introduction (22). It is, however, clear that EQAS meeting metrological criteria have unique benefits that may permit to add substantial value to the practice of laboratory medicine (Table IV) (23).

Table III.

Requirements for the applicability of External Quality Assessment Scheme (EQAS) results in the evaluation of the performance of participating laboratories in terms of traceability of their measurements.

Characteristic	Aim
EQAS materials value-assigned with reference procedure by an accredited reference laboratory	To check traceability of a commercial system to the reference measurement system
Proved commutability of EQAS materials	To allow transferability of participating laboratory performance to the measurement of patient samples
Definition and use of the clinically allowable measurement error	To verify the suitability of laboratory measurements in clinical setting

Table IV.

Main benefits of External Quality Assessment Schemes meeting metrological criteria.

– Giving objective information about the quality of individual laboratory performance – Creating evidence about intrinsic standardization status/comparability of the examined assays – Serving as management tool for the clinical laboratory and diagnostic manufacturers, forcing them to investigate and eventually fix the identified problem – Helping the manufacturers that produce superior products and systems to demonstrate the superiority of those products – Identifying analytes that need improved harmonization and stimulating and sustaining standardization initiatives that are needed to support clinical practice guidelines

Conclusion

The laboratory profession plays a key role in checking the correct implementation of traceability of IVD systems and in ensuring that the uncertainty associated with patient results is clinically acceptable. To this aim, laboratory organizations, in addition to the definition of reference measurement systems useful for implementing traceability of patients’ results, should establish for each measurand the clinically acceptable uncertainty of its measurement that fits the purpose (6). This information should be available to reference material suppliers and IVD manufacturers to permit them to derive the proportion of the total uncertainty budget allowed for clinical laboratory results they may consume at the top and at intermediate steps of the calibration hierarchy (8). In addition, each clinical laboratory in daily work should verify the alignment of the employed IVD system, estimate the random sources of the measurement uncertainty and participate in appropriately structured EQAS in order to monitor over time its analytical performance and verify the suitability of performed measurements.