Limited comparability of creatinine assays in patients with liver cirrhosis and their impact on the MELD score

Abstract

Background & aim

Patients with end-stage liver disease require valid estimations of mortality for organ allocation and risk stratification. The model of end-stage liver disease (MELD) score is used for this purpose in most countries and incorporates bilirubin, International Normalized ratio, and creatinine. The aim of this study was to evaluate the comparability of creatinine results from different routine assays in the serum samples of patients with liver cirrhosis.

Methods

Residual material from 60 serum samples was available from patients in different stages of liver cirrhosis. Four centers participated; each center analyzed the samples with Jaffé-based and enzymatic routine assays in parallel. In addition, an accredited calibration laboratory certified the panel of samples by an internationally accepted reference measurement procedure (RMP) based on isotope dilution mass spectrometry (ID-MS). This method served as the independent reference.

Results

All routine methods displayed a high correlation to the RMP (r ≥0.937, p<0.001). Two enzymatic and two Jaffé-based methods provided results that were all within a ±20% range of the RMP. The other methods showed deviations >20% in up to 27% of the samples. The enzymatic methods were systematically lower, whereas the Jaffé-based methods were systematically higher (p<0.001). The resulting MELD scores differed from 0 to 4 points.

Conclusions

There are systematic deviations from the RMP. Jaffé-based assays gave higher results, whereas the enzymatic-based assays gave lower results compared to the results of the RMP. The comparability of results is limited and could be disadvantageous to patients listed for liver transplantation.

1. Introduction

Patients with advanced chronic liver disease have dramatically increased mortality. For many of these patients, liver transplantation is the only curative treatment option.

For suitable allocation of organs, objective parameters for assessment of short-time mortality are needed. The best evaluated parameter for short-time mortality in patients with end-stage liver disease is the lab-MELD (laboratory model of end-stage liver disease) score. It is calculated from the results of serum bilirubin, serum creatinine and the International Normalized Ratio (INR). The MELD score is used for liver allocation in many countries [1], [2].

Because of the limited organ availability, MELD diagnostics must be comparable between the transplantation centers to guarantee a fair organ-allocation process.

There is a comprehensive quality-assurance system for medical laboratories in Germany and many other countries [3].

To guarantee diagnostic confidence, analytical results are checked twice daily with two different internal controls. Four times a year, an external proficiency test for each parameter is performed according to the German quality-assurance system. The target values for the evaluation of these external proficiency tests are traceable to SI unit and assigned by a Reference Measurement Procedure (RMP) of higher metrological order based on IDMS analysis. The results in the proficiency tests for creatinine must meet the RMP target values ±20%. These proficiency tests are usually performed with artificial materials, which are not comparable to the complex matrix of samples of patients suffering from liver cirrhosis. There is already evidence that the results for creatinine in samples differ considerably [4]. However, until recently, only a limited number of assays had been studied and the results were not compared to the RMP.

The aim of this study was to evaluate the differences between different routine methods for creatinine measurement in comparison to the RMP and in context to the results of MELD scores in the samples of patients with end-stage liver disease.

1.1. Patients/methods

Sixty serum samples from 60 different patients with end-stage liver disease being evaluated for liver transplantation or listed for liver transplantation were available for this study.

These samples were residual materials from the clinical evaluations of the patients. The study was approved by the local ethics committee. For each sample, the initial results of the MELD parameters (bilirubin, creatinine by two different methods, and INR) were available. The MELD score was calculated using the following formula, according to the guidelines of the United Network for Organ Sharing [5]:

$$\begin{array}{c}10\times \{0\mathrm{0.957}\times Loge(creatinine\left[\mu mol/L\right]*A)\hfill \\ +0.378\times Loge(bilirubin[\mu mol/L]*B\hfill \\ +1.120\times Loge(INR)+0.643\}\hfill \\ \hfill \\ A=0\text{0.01131}=(\text{creatinine}[\text{mg}/\text{dL}])/(\text{creatinine}[\mu \text{mol}/L])\hfill \\ \text{B}=\text{0}\text{0.05848}=(\text{bilirubin}[\text{mg}/\text{dL}])/(\text{bilirubin}[\mu \text{mol}/L])\hfill \end{array}$$

The initial measurement was performed immediately after the sample arrived at the medical laboratory (center 1). Creatinine and bilirubin analyses were performed using Cobas^® 6000 and 8000 analyzers (Roche Diagnostics, Mannheim, Germany), according to the manufacturer's instructions. Creatinine was measured using the enzymatic assay Creatinine Plus Ver. 2 and the Creatinine Jaffé Gen. 2 (Roche Diagnostics, Mannheim, Germany) in parallel. Bilirubin was measured using the Bilirubin Total DPD Gen.2 kit (Roche Diagnostics, Mannheim, Germany). The prothrombin assay was performed using an ACL TOP 700 System (Instrumentation Laboratory, Bedford, MA USA) with the RecombiPlasTin 2G kit (Instrumentation Laboratory, Bedford, MA, USA), and the INR was calculated. The rest of the anonymized serum material was aliquoted and stored at −80 °C. One aliquot (250 µL) of each sample was then sent on dry ice to three medical laboratories at transplantation centers and one medical reference institute in Germany.

Participating medical laboratories were the University Hospital Leipzig (center 1), the Hannover Medical University (center 2), University Hospital Regensburg (center 3), Jena University Hospital (center 4) and the Reference Institute for Bioanalytics Bonn (center 5).

Table 1 lists the methods used for creatinine measurements in all participating centers.

Table 1.

Participating centers and methods for creatinine measurement. Abbreviations are composed of method type (*E = enzymatic test; J = assay based on the Jaffé principle); center number (1–4) and manufacturer (R= Roche Diagnostics, S= Siemens Healthcare Diagnostics. A= Abbott Diagnostics). REF = Reference Measurement Procedure (RMP).

Abbr.	Method*	Center	Manufacturer	Name of the test (Analyzer)
E1R	E	1	Roche Diagnostics, Mannheim, Germany	Creatinine Plus Ver. 2 (Cobas® 6000 and 8000 analyzer)
J1R	J	1	Roche Diagnostics, Mannheim, Germany	Creatinine Jaffé Gen. 2 (Cobas® 6000 and 8000 analyzers)
E2W	E	2	Wako Chemicals, Neuss, Germany	Creatinine M L-Type (Wako System Calibrator, Modular® analyzer)
E2R	E	2	Roche Diagnostics, Mannheim, Germany	Creatinine Plus Ver. 2 (Modular® analyzer)
J2R	J	2	Roche Diagnostics, Mannheim, Germany	Creatinine Jaffé Gen. 2 (Modular® analyzer)
E3S	E	3	Siemens Healthcare Diagnostics Inc.	ECREA (Dimension Vista® System)
			Newark, DE, USA
J3S	J	3	Siemens Healthcare Diagnostics Inc.	CREA (Dimension Vista® System)
			Newark, SE, USA
E4A	E	4	Abbott Laboratories	MULTIGENT Creatinine (Enzymatic) Assay (ARCHITECT® c System)
			Chicago, IL, USA
J4A	J	4	Abbott Laboratories	Creatinine Assay (ARCHITECT® c System)
			Chicago, IL, USA
REF	R	5	Calibration Laboratory I of the Reference Institute of Bioanalytics, Bonn, Germany	Mass Spectrometry Isotope Dilution Analysis

The RMP used in this study is an isotope dilution mass spectrometry method which is routinely in use for national creatinine proficiency tests and listed as reference method at the database of the Joint Committee for Traceability in Laboratory Medicine (JCTLM) [www.bipm.org/jctlm] [6]. The RMP was performed at least once for each sample at the ISO 17025 accredited calibration laboratory of the Reference Institute for Bioanalytics, Bonn, Germany. If there was sufficient material, two measurements were performed (50 of 60 samples), and the mean value was used for further analysis.

Centers 1–4 also performed total bilirubin analysis (centers 1&2: Bilirubin Total DPD Gen.2 kit [Roche Diagnostics, Mannheim, Germany]; center 3: TBIL REF K1167 (diazotized sulfanilic acid method, Dimension Vista® System, Siemens Healthcare Diagnostics Inc., Newark, PA, USA.); center 4: Total Bilirubin Assay, Methodology: Diazonium Salt [ARCHITECT® c System, Abbott Laboratories, Lake Bluff, IL, USA.]).

In addition, center 2 performed a measurement of direct bilirubin (Bilirubin Direct Gen. 2) [Roche Diagnostics, Mannheim, Germany].

1.2. Statistics

Data were analyzed using Excel 14.0 (Microsoft Corporation, Redmond, WA, USA) and SPSS 23.0 (IBM, Armonk, NY, USA). A p value <0.05 was considered statistically significant. Correlation analysis was performed according to Spearman's rank test. The Wilcoxon signed-rank test was used to assess whether the mean ranks of the methods differ from the RMP. Deviations between higher and lower bilirubin samples were analyzed by Mann-Whitney U test.

2. Results

The creatinine results ranged from 29.0 to 379.3 µmol/L. All methods for creatinine measurements displayed a high correlation with the RMP (r ≥0.937, p<0.001), but, except for two enzymatic methods (E1R (p=0.058) and E3S (p=0.457); see Table 1 for method details), all creatinine methods showed significant differences compared to the RMP. The results are summarized in Table 2.

Table 2.

Overview of the results from the different creatinine methods.

Method	E1R	E2R	E2W	E3S	E4A	J1R	J2R	J3S	J4A
Correlation-coefficient with RMP	0.984	0.988	0.994	0.994	0.980	0.975	0.987	0.937	0.957
Median deviation from reference (%)	−1.0%	−2.3%	−15.9%	0.0%	−11.4%	5.5%	−1.5%	8.6%	5.7%
Mean deviation from reference (%)	−1.6%	−2.1%	−16.2%	−0.3%	−12.5%	5.3%	−0.8%	10.6%	6.3%
Number of samples with Deviation >8.9% (a)	1	2	0	2	0	14	2	29	15
Number of samples with Deviation >20% (b)	0	0	0	0	0	0	0	7	3
Number of samples with Deviation <−8.9% (a)	4	3	55	2	47	0	2	0	1
Number of with Deviation <−20% (b)	1	0	16	0	5	0	0	0	0
Maximum positive deviation (%)	11.8%	14.0%	−2.3%	11.9%	−5.2%	18.1%	13.0%	57.8%	37.6%
Maximum negative deviation (%)	−21.1%	−15.5%	−30.6%	−13.8%	−31.3%	−8.4%	−16.6%	−2.6%	−10,7%
Diff. to RMP (***)	p = 0.058	p = <0.001	p = <0.001	p = 0.457	p = <0.001	p = <0.001	p = 0.023	p = <0.001	p = <0.001
Maximal tolerable concentrations according to the manufacturer's literature for total, direct and indirect bilirubin
- total bilirubin [µmol/L]	NA	NA	NA	NA	NA.	NA	NA.	NA	513
- direct (conjugated) bilirubin [µmol/L]	257	257	> 684	513	677.2	86	86	342	NA
- Indirect (unconjugated) bilirubin [µmol/L]	342	342	> 684	513	790.0	171	171	171	NA

^{NA = not available.}

^a8.9% corresponds to the allowable total error of creatinine based on biological variability published in reference 7.

^b20% is the maximum permissible deviation in proficiency tests for creatinine according to the German directive for quality assurance of medical laboratory investigations (German: RiLi-BÄK).

^***p-Value based on rank test for asymptotic significance (two-sided Wilcoxon-Test)

2.1. Creatinine results of identical methods after freezing-thawing cycle

The enzymatic methods E1R and E2R as well as the Jaffé-based methods J1R and J2R were identical, but analyses were performed at different times on different analyzers in different centers (Table 1). The results for the enzymatic tests showed no significant deviations between these two methods (Wilcoxon signed-rank test p=0.09), whereas the results of the Jaffé-based measurements differed significantly (p<0.001) between these assays. Different pre-analytical conditions may explain this finding. The methods E1R and J1R were performed immediately after the sample arrived at the laboratory. In contrast, samples were aliquoted and frozen before being transferred to laboratory 2 for analysis (E2R, J2R).

2.2. Comparison of the results from routine assays with the RMP

Only four of the nine routine methods provided results that were all within the ±20% range of the RMP results. Two of these were enzymatic and the other two were Jaffé-based methods. The other methods showed deviations >20% in different frequencies. One method showed deviations >20% from the reference target values in 16 of 60 (27%) results. 20% was fixed arbitrarily as it is the maximum permissible deviation in proficiency tests for creatinine according to the German directive for quality assurance of medical laboratory investigations (German: RiLi-BÄK) [3]. Instead of this arbitrary 20%, a more clinically meaningful criterion such as the biological variability-based total allowable error of 8.9% for desirable performance of creatinine assays could be used [7]. Using this cut off, all tests displayed at least two results with deviations >8.9%. Up to 55 of the 60 samples displayed deviations >8.9% for one method. Fig. 1 shows the results of the different assays for each sample in comparison with the results of the RMP.

Fig. 1.

Differences of different assays to RMP (y-axis) vs. absolute creatinine concentration (x-axis) [µmol/L]. A: enzymatic creatinine assays (A1: E1R, A2: E2W, A3: E3S, A4: E4A). B: Jaffé based assays: B1: J1R, B2: J2R, B3: J3S, B4: J4A). Dotted parallel lines: median ±1.96 SD; oblique grey lines: specific allowable total error for creatinine of 8.9% (from ref. 7). Assay E2R not displayed (see text for details).

The enzymatic methods were systematically lower, whereas the results of the Jaffé-based methods J1R, J3S and J4A were systematically higher. These deviations resulted in significant differences between the results of the enzymatic and the Jaffé-based methods (p<0.001).

2.3. Comparison of the results from the enzymatic and the Jaffé-based methods

The lowest creatinine values were always measured with enzymatic methods. Except for one sample, the highest values were always derived from the Jaffé-based methods.

The maximum differences between the lowest and highest results of the routine methods for each sample ranged from 12.2 µmol/L to 112.0 µmol/L (median 22.6 µmol/L or 28.1%). Maximum differences >20% between the lowest and highest results were seen in 48 of the 60 samples (80%). In 5 (8.3%) of the samples the maximum differences were >50%.

2.4. Influence of bilirubin concentrations on the results

We further analyzed the influence of bilirubin elevations on the observed differences. Total bilirubin values were available for all 60 samples. In 59 of these samples the results for direct and indirect bilirubin were also available.

Relevant interferences for the Jaffé-based assays are expected at lower bilirubin concentrations compared to the enzymatic assays, but the maximum tolerable concentration for total or indirect bilirubin to avoid relevant interferences differs substantially between the manufacturers (Table 2). For further analysis, we used the lowest cut off of 86 µmol/L for conjugated bilirubin or 171 µmol/L for indirect bilirubin supplied in the manufacturer's literature (method J1R and J2R). This cut off was the lowest value of bilirubin without a relevant interference, defined by the manufacturer as a recovery within ±10% of initial value at a creatinine concentration of 80 μmol/L.

Overall, 7 of 59 samples had direct bilirubin values of >86 µmol/L or indirect bilirubin values of >171 µmol/L. We analyzed these samples in comparison to the samples with lower bilirubin concentrations. Only two enzymatic methods (E3S (p=0.021) and E4A (p=0.026)) and none of the Jaffé-based methods showed significantly higher differences for creatinine in high bilirubin compared to low bilirubin samples. Interestingly, the method with the highest bilirubin tolerance (E2W) displayed the most pronounced deviations from the RMP.

2.5. Correlation analysis of creatinine differences

To characterize the influence of the severity of the liver disease on the systematic deviations between the RMP and the routine assays, a correlation analysis was performed. For three assays, the differences between the reference assay and the routine assays were significantly correlated with the MELD score (J3S (r =0.255, p=0.049), E3S (r =−0.332, p=0.010), E4A (r =−0.445, p<0.001)).

The two enzymatic assays were also significantly correlated with bilirubin (E3S (r=−0.289, p=0.025), E4A (r=−0.349, p=0.006)).

One of these and another assay were significantly correlated with creatinine concentrations (E4A (r=−0.481, p<0.001), J1R (r =0.298, p=0.021)).

2.6. Differences in creatinine and the impact on the MELD score

Influences of different methods on the resulting MELD score are summarized in Fig. 2. According to the requirements for the calculation of the MELD scores creatinine values <88.4 µmol/L (1 g/dL) are replaced by 88.4 µmol/L. Consequently, differences between values <88.4 µmol/L have no effect on the calculation of the MELD score. In our samples, 47% (28/60) had at least one creatinine result >88.4 µmol/L. In 3 of these 28 (10.7%) samples, there was no MELD difference. The other 25 of 28 samples (89.3%) showed differences. Seven samples (25%) showed a difference of 1 point, 15 (53.6%) samples showed a difference of 2 points, 2 samples (7.1%) showed a difference of 3 points, and 1 sample (3.6%) resulted in a maximum difference of 4 points in the resulting MELD scores.

Fig. 2.

Resulting differences between MELD scores with the different creatinine methods and the MELD score based on the RMP.

In higher MELD scores one additional MELD point corresponds to additional ~5% risk of 3-month mortality equivalent [8]. The sample with the difference of 4 points was taken from a patient with a very high MELD score. The lowest test results were from enzymatic tests giving MELD scores of 36 points (E4A) and 37 points (E1R), while the score using the reference method was 39 points. The highest score (40 points) was from a Jaffé-based method (J3S). The corresponding predicted 3-month mortality risk for a score of 36 would be ~85% compared to ~100% for a score of 40 [8]. The result from the RMP provided a MELD score of 39 points (~95% predicted three-month mortality). Compared to the RMP, the enzymatic assay (E4A) showed a 31.3% lower result (107 vs. 156 µmol/L). The concentration of bilirubin in this sample was very high (direct bilirubin was 269 µmol/L [15.7 mg/dL], and indirect bilirubin was 76.4 µmol/L [4.5 mg/dL]).

2.7. Comparability of bilirubin results

The bilirubin results were in good agreement between the four centers. The median results from each center were 35.1 µmol/L (Center 1), 33.1 µmol/L (Center 2), 33.3 µmol/L (Center 3), and 35.0 µmol/L (Center 4).

3. Discussion

The aim of this study was to evaluate the impact of different routine assays for creatinine on the organ-allocation process for patients listed for orthotopic liver transplantation. The acceptance of creatinine results by the transplantation organization is currently independent of the method and does not consider possible confounding factors. It has been shown that inter-laboratory differences have an impact on the results of the MELD score and consequently, the organ allocation process [9], [10], [11], [12]. In this context, Jaffé-based tests revealed higher creatinine results compared to enzymatic assays [4], [13], [14], [15], [16]. However, in these studies, the number of samples or the number of assays was limited and there was no systematic comparison with the RMP. To our knowledge, this is the first study analyzing creatinine results of the main Jaffé-based and enzymatic assays in parallel compared with the IDMS-based RMP in samples of patients with end-stage liver disease.

Obviously, the comparability of the results for different methods was limited in this study. This contrasts significantly with the results of routine proficiency tests in which the methods were usually in good agreement [17]. Elevated bilirubin results are known to interfere with these methods, but if the concentration of bilirubin exceeded the manufacturer's specifications, the results were still accepted by the transplantation organization for the calculation of the corresponding MELD score. The non-reporting of a result due to bilirubin concentrations above the manufacturer's specifications is not foreseen in the transplantation regulations and patients are listed and remain on the list only if regular laboratory results are provided. This raises the question whether laboratory reports should report values that were excessively influenced by elevated bilirubin concentrations and will lead to an incorrect MELD score.

From the analytical point of view, an enzymatic creatinine assay should be preferred for such samples because of the presumed better validity of the results in icteric samples compared to the results from Jaffé-based assays. However, this superior validity of enzymatic assays in icteric samples has not been proven in real samples from cirrhotic patients. Manufacturers usually validate their assays with samples from non-cirrhotic patients spiked with bilirubin. According to the manufacturer's specifications, all Jaffé assays were clearly more prone to interference in icteric samples compared to the enzymatic tests. Interestingly, we are not able to confirm this statement in general in our samples from cirrhotic patients. On the contrary, two enzymatic methods showed significantly higher deviations for the samples with high bilirubin concentrations.

Consequently, the differences between the assays could not be easily explained by the different bilirubin concentrations. We speculate that other interfering substances have a more relevant impact in this context. The material used for proficiency tests and for the assay evaluation by the manufacturer should ideally be comparable to routine samples, but because of limited availability and other reasons, there are substantial differences. In addition, perfect agreement of the results of routine methods with the RMP may not be transferable to samples from patients with advanced disease. The samples from patients with end-stage liver disease display a sophisticated matrix. There are different pseudochromogens that interfere with Jaffé-based methods. Metabolites and breakdown products of drugs accumulate, albumin concentrations are reduced, and bilirubin values are usually significantly elevated. All those factors could affect the results of the methods under routine conditions.

In summary, we conclude that neither the enzymatic nor the Jaffé-based method displayed a significantly better validity compared to the results of the RMP in this study. The results of this study prove that the chosen creatinine method is highly relevant to the organ allocation process. There were relevant differences in the resulting MELD scores in this cohort. Systematically lower values resulted from the use of enzymatic methods compared to the Jaffé-based methods.

There are several limitations to this study. In our sample cohort, most patients had presenting MELD scores ≤20 points (45 out of 60; 75%). Depending on organ availability, these patients usually will not receive an organ in the short term. For this reason, the differences in creatinine might not have a direct impact on organ allocation in these patients. Because there is an underrepresentation of patients with high MELD scores, the influence of different assays and methods may not be precisely estimated in our study. The differences of three assays compared to the reference assay showed significant correlations with the MELD score. The enzymatic assays showed negative deviations in contrast to the Jaffé-based assay, which showed a positive deviation with an elevated MELD score. According to the results of the correlation analysis for these three assays the deviations would become more pronounced for more advanced stages of liver cirrhosis.

One sample of a patient with a dramatically increased MELD score gave a score of 36 points with an enzymatic assay and 40 points with a Jaffé-based assay. In the organ-allocation process, these different results would probably have a direct influence on the patient because the corresponding 3-month mortality rate indicated by the enzymatic method would be approximately 85% (a 36 MELD score) compared to almost 100% (a 40 MELD score) for two Jaffé-based methods [8].

In 2016, the United Network for Organ Sharing Organ Procurement and Transplantation Network (OPTN/UNOS) introduced the calculation of the MELD-Na (incorporating serum sodium measurement) for organ allocation [18]. The MELD-Na has a higher predictive value compared to the MELD score. However, the MELD-Na has not yet been implemented by Eurotransplant and other organizations and sodium values were not available for this study. The lowest sodium concentration with the highest impact on the MELD-Na score used for calculation is defined as 125 µmol/L. If the patient with the MELD score between 36 and 40 points had a sodium concentration of ≤125 µmol/L the lowest MELD-Na score would be 38 and the highest still 40, resulting in a difference of 2 points compared to 4 points calculated using the MELD score. However, the MELD-Na score is based on the MELD score, and at sodium concentrations of ≥137 µmol/L the MELD-Na score is equal to the MELD score, also resulting in a 4 point MELD-Na score difference.

Because there were relevant differences between the different methods in samples of patients with end-stage liver disease, additional studies should be performed We recommend that the transplantation organizations should specify the methods suitable for MELD diagnostics.

However, we are convinced that this could serve only as a first step to improve the validity of the MELD score in respect of renal function because serum creatinine is known to have serious limitations as a biomarker. The most important limitation of creatinine in this context is the dependence on muscle mass. Physiologically, there are differences in muscle mass between women and men, but the MELD score does not differentiate between genders. Furthermore, studies have already revealed a relevant relationship between sarcopenia and mortality in patients with end-stage liver disease. In these patients, the MELD formula would underestimate mortality because of the reduced elevation of creatinine. Cystatin C is a promising alternative biomarker that is available for routine clinical use, but there are currently only limited data available concerning cystatin C concentrations in patients with end-stage liver disease and their use in the prediction of mortality.