Recalibration of Cystatin C using Standardized Material in Siemens Nephelometers

Abstract

Background

Cystatin C is a key GFR biomarker. Recently, Siemens recalibrated the assay based on certified reference material ERM-DA471/IFCC. The NIH funded longitudinal CKiD study has >3000 cystatin C measurements based on a pre-IFCC calibrator provided by Siemens. Since cystatin C values for CKiD are now standardized to IFCC certified reference material, it is important to relate the IFCC-calibrated results to the previous values so that there are no discontinuous results.

Methods

We diluted cystatin C ERM-DA471/IFCC (5.48 mg/L) into buffer and compared results with predicted ones. We then updated the cystatin C application on our BN II nephelometer to provide results based on pre-IFCC and IFCC calibrations of CKiD specimens simultaneously. We assayed 51 previously analyzed sera and 62 fresh additional specimens.

Results

The predicted concentrations from the IFCC standard were consistently 17% higher than the measured values using the pre-IFCC calibration (y=1.1686 x). Similarly, the re-run and fresh sample concentrations were 17% higher via the IFCC calibration than by the pre-IFCC calibration (y=1.168 x). There was very high reliability in the measurements using the previous calibration for re-run specimens (0.99) and for 33 pristine specimens using IFCC calibration (0.99).

Conclusons

We confirm the recalibration proposed by Siemens. To convert pre-IFCC results to IFCC-calibrated concentrations, the value is multiplied by 1.17. Conversely, one divides IFCC-calibrated results by 1.17 to estimate GFR via previously published pre-IFCC CKiD eGFR equations (Kidney Intl 82:445, 2012). For older adolescents cystatin C has already been standardized and can be directly applied to the CKD-EPI equations (NEJM 367:20, 2012).

Introduction

Cystatin C is a key biomarker for estimating kidney function; its reciprocal is highly correlated with glomerular filtration rate (GFR) in adults as well as in children[1–4]. Several years ago, a reference material for cystatin C was prepared so that the various assays could reach common agreement[5]. Calibration to such reference material has progressed in Europe but only recently has the FDA given approval to use such material in the United States. Based on preliminary studies from our laboratory we suspected that the cystatin C values would be increased compared to the current calibration[6]. In a longitudinal study such as the NIH funded Chronic Kidney Disease in Children (CKiD) study, a change in calibration would be expected to cause an abrupt shift in the yearly cystatin C values routinely obtained and accordingly the eGFR, and so a conversion between pre-IFCC and IFCC calibration would need to be developed.

In early 2018 Siemens announced that it was discontinuing the non-standardized N Latex Cystatin C Reagent Kit because the FDA had approved the ERM-DA471/IFCC reference material; re-standardized N Latex Cystatin C became available to users planning to measure IFCC-calibrated cystatin C concentrations. Siemens also communicated to us that the IFCC-calibrated cystatin C values would be 17.4% higher than the pre-IFCC values. The reference for this relationship was a letter which did not present original data.[7]. This correction is different from previous work by Inker et al[4] who published a correction of +11–12%. Accordingly, we found it necessary to perform our own IFCC calibration study.

Methods

Dilutions of ERM-DA471/IFCC

Four vials of ERM-DA471/IFCC were purchased and received in 2015 and 2016 from Sigma-Aldrich (St. Louis, MO). The IFCC-calibrated cystatin C concentration was 5.48 mg/L. We performed successive dilutions of each batch of this product into phosphate-buffered saline (PBS) so that the predicted concentrations ranged from 0.50 to 5.48 mg/L. Each group of 10–20 specimens was assayed on a Siemens BN II nephelometric analyzer via a 1:100 dilution in phosphate buffered saline (PBS) performed with a six-point calibration generated from multiple dilutions of a human cystatin C calibrator obtained from human urine, after confirming that the quality controls (1.06 mg/L low, and 1.93 mg/L high) were within 6% of the specific concentration[1]. Some of the assays were performed in duplicate and in those cases the average reading was compared with the concentration predicted from the dilutions.

Study specimens and measurements

The BN II analyzer software and cystatin C application was changed by the Siemens technical group in late 2018 so that the results of each sample could be simultaneously measured using the pre-IFCC and IFCC calibration respectively. Fifty one de-identified specimens from which two assessments of cystatin C using the pre-IFCC calibration were conducted with the first being performed in October 2017 and the second in December 2018 to February 2019 after the specimens had been stored at −80°C for slightly more than a year. These specimens were considered to have stable values of cystatin C according to the data of Rule et al[8] In addition, at the time of the second assessment using the pre-IFCC method we also utilized the IFCC method and obtained a third value of cystatin C for each specimen.

We also measured cystatin C using the pre-IFCC and IFCC methods on 62 pristine and freshly frozen specimens collected in 2018 and only subjected to one thawing. To directly determine the reliability of the IFCC method, we repeated the IFFC assay in 33 of the 62 specimens collected in 2018.

Lastly, repeated specimens from the first author (GJS) were assessed with both the pre-IFCC and the IFCC methods as an additional internal control. These data provided values of cystatin C lower than most of those from children with CKD but allowed determination of the validity and association between the pre-IFCC and the IFCC methods beyond those used in modeling the association.

Statistical methods

The development of the statistical model follows the paradigm of Carroll et al.[9] for constructing a general, errors-in-variables regression model. For this application, our goal is to model the values of the IFCC assay, the new method (Y), as a function of the true, unobserved values of the original assay; the old method (X), which is measured with error (ε), so that, instead of the true value (X) we observe the measured value (W = X + ε). This model will then be used to predict values of the new method from observed values of the old method. To accomplish this we need 1) an exposure model for the distribution of the unobserved, true values of the old method (X), 2) an error model for the observed values (W) and finally 3) the underlying outcome model for the observed values of the new method (Y). Thus we must model the observed values of both W and Y. The three components specify the distribution of the observed data. Because the true values of X are unobserved, however, the distribution of X must be integrated out to obtain the marginal distribution of the observed data. In our approach, the exposure and error models are fully parametric, and the required integration is performed numerically by the statistical software. The marginal likelihood then involves the parameters of all three component models, which are estimated by maximum likelihood.

For our application, we wish to model the values of the IFCC assay, the new method (Y), as a linear function of the true values of the original assay, the old method (X), which is measured with error (W). Using the approach of Carroll et al.[9] we must specify the three parametric model components. As is fairly standard we will assume that the unmeasured true values of the old assay (X) are normally distributed with unknown mean and variance. Second we assume that the measured value W is equal to X plus an independent, normally distributed measurement error (ε) with unknown variance. Finally the measured value of the new method, Y, is a linear function of X plus an additive error (ε₁) that is independently normally distributed with mean zero and unknown variance; this is the standard simple linear regression model. More formally the three model components are as follows. Let i index specimens (subjects), and j indicate replicate measurement of W.

$$\text{Exposure}\text{model}:X_i\sim N\left({\beta }_x,{\sigma }_x^2\right)$$ 1.

$$\text{Error}\text{model}:W_{ij}=X_i+{\epsilon }_{ij}{\epsilon }_{ij}\sim N\left(0,{\sigma }_w^2\right)$$ 2.

$$\text{Regression}\text{model}:Y_i=\alpha +\beta X_i+{\epsilon }_{1i}{\epsilon }_{1i}\sim N\left(0,{\sigma }_1^2\right)$$ 3.

The likelihood of the observed data in equations (2) and (3) is based on all three components, and has a total of six unknown parameters (three regression parameters and three variance components). Note that equation (2) is simply a one-way random effects model. The required data are paired measurements of the same specimens using both methods, as well as repeated measurements of the same specimens using the old method (the two sets of specimens need not be identical). Repeated measurements using the new method are useful but not necessary. One advantage of this approach is that it is straightforward to fit alternative regression models, such as a model with no intercept or a quadratic regression model.

Results

Figure 1 shows that serial dilutions into PBS of the certified reference material ERM-DA471/IFCC (5.48 mg/L) resulted in values that were consistently higher than those measured using the pre-IFCC calibration method (i.e. pre-IFCC = 0.856 x IFCC). The slope indicates that the predicted concentrations of the dilutions were 17% (1/0.856) higher than the pre-IFCC calibrated values over the entire range measured on the BN II instrument prior to the introduction of any software change.

Figure 1.

Pre-IFCC calibrated measurements plotted against dilutions of ERM-DA471/IFCC reference material in PBS. Most measurements were performed in duplicate and the average value utilized. Including the reference material used at full strength (5.48 mg/L), there are 20 predicted values over the physiologic range plotted for the relationship. The line pre-IFCC = 0.856 x IFCC reference) depicts the excellent fit of measurements via the pre-IFCC calibration method to the dilutions of the reference material.

Table 1 shows the descriptive statistics regarding the formal comparison of specimens via the pre-IFCC (old) and IFCC (new) calibrations. The primary findings show: 1) good agreement between repeated assessments with either the pre-IFCC (1.411 vs 1.426 mg/L) or the IFCC (1.562 vs 1.515 mg/L) methods; 2) the mean IFCC value was 1.170 times the mean of the pre-IFCC value among the 51 specimens measured after a second thaw following more than a year of storage; and the mean of the IFCC calibration was 1.164 times the mean of the pre-IFCC calibration among 62 pristine specimens after first thaw. The demographics of the subject samples for these two groups are seen in Table 2. Most of the subjects were male and white with eGFRs primarily in the 40 to 70 ml/min per 1.73 m² range, reflecting that most of their cystatin C values were in the 1–2 mg/L range.

Table 1:

Descriptive Statistics of the Cystatin C Assay

No. of specimens		Pre-IFCC 1st Thaw	Pre-IFCC 2nd Thaw	IFCC 2nd Thaw
51	Mean ± SD	1.411 ± 0.442	1.426 ± 0.475	1.669 ± 0.547
	Range	0.73 to 2.58	0.73 to 2.71	0.84 to 3.04
			Pre-IFCC 1st Thaw	IFCC 1st Thaw
62	Mean ± SD	NA	1.394 ± 0.795	1.622 ± 0.898
	Range	NA	0.60 to 3.82	0.76 to 4.69
				IFCC 1st Thaw	IFCC 2nd Thaw
33	Mean ± SD	NA	NA	1.562 ± 0.885	1.515 ± 0.903
	Range	NA	NA	0.77 to 4.32	0.72 to 4.23

Sera were assayed for cystatin C using a Siemens BN II nephelometer, which was calibrated in the usual way (pre-IFCC) and later according to standardized cystatin C reference material (ERM-DA471/IFCC).

Table 2:

Descriptive Statistics of the 107 Subjects who Contributed 113 Specimens in the IFCC Calibration Datasets

Characteristic	Median [IQR] or n (%)
	First set of specimens (n=50)	Second set of specimens (n=57)
Age, years	15.9 [12.6, 19.6]	15.5 [11.1, 19.3]
Male sex	36 (72%)	37 (65%)
White race	34 (68%)	42 (74%)
eGFR	52 [42, 66]	63 [45, 76]
SCr	1.26 [0.96, 1.84]	1.11 [0.82, 1.78]
Cystatin C (pre-IFCC)	1.36 [1.02, 1.69]	1.10 [0.90, 1.67]
Cystatin C (IFCC calibrated)	1.59 [1.19, 1.98]	1.30 [1.09, 1.90]

Table 3 shows the results from regression models with the aim of predicting IFCC calibrated values based on pre-IFCC values. The primary findings show: 1) the model allowing for an intercept revealed a non-significant and small intercept (= 0.028; 95% CI: −0.013 to 0.069). The correlation coefficient estimated using this model was very high, 0.9914 (0.0015) 2) the model without an intercept yielded a factor of 1.168 ± 0.006 as the one needed to multiply a pre-IFCC value to achieve an IFCC value, which is practically identical to and not significantly different from the 1.17 recommended by the manufacturer (p = 0.733); 3) the reliabilities of both the pre-IFCC and the IFCC methods were outstanding (~0.99); and 4) after excluding an outlying observation shown as the longest horizontal line at 1.01 mg/L in Figure 2, the results were robust.

Table 3:

Results of Regression Models Relating Pre-IFCC and IFCC calibrations

No. specimens	113	113	112a
No. observations	277	277	274a
Pre-IFCC	1.403 ± 0.062	1.406 ± 0.061	1.409 ± 0.062
IFCC
Intercept	0.028 ± 0.021	0	0
Coefficient of pre-IFCC	1.151 ± 0.014	1.168 ± 0.006	1.168 ± 0.006
Reliability of pre-IFCC	0.9895	0.9888	0.9912
Reliability of IFCC	0.9932	0.9937	0.9928

^aExcluding outlier shown in Figure as the longest horizontal line at 1.01.

Figure 2.

Comparison of serum specimens simultaneously measured using pre-IFCC and IFCC calibrated methods on a Siemens BN II. Horizontal lines indicate 51 specimens previously run slightly over a year previously with the line spanning the two values of the old (pre-IFCC) calibration. Diamonds identify 62 pristine specimens measured using pre-IFCC and IFCC calibrations. Open squares indicate values from GJS measured concomitantly with each run of 10–20 sera. The line describes the relationship between old and new calibrations (IFCC = 1.168 x pre-IFCC calibration); the intercept was not significant.

The reliability of the IFCC method was confirmed with direct calculation from the 33 specimens with repeated assessments using the IFCC calibration. Indeed, the reliability coefficient using these data was 0.9939, which is close to those reported based on regression models in Table 2.

Figure 2 depicts the data from the 51 specimens with two pre-IFCC assessments and one IFCC assessment with horizontal lines spanning from the two values of the pre-IFCC calibration and those of the 62 specimens with one pre-IFCC and one IFCC value with a diamond. Results obtained from repeated specimens from the first author (GJS) are depicted with open squares and were not used to derive the parameters of the regression models but were used as validation. The line corresponding to IFCC = 1.168 x pre-IFCC provides an excellent fit to the data, and the line with intercept (grey line using model I in Table 3) is practically undistinguishable. Furthermore, the data from the first author (GJS, open squares) fit well on the regression line.

Discussion

It is well known that serum levels of cystatin C have been much dependent on the methodology used to determine the concentration[10,5,11,12]. The variability in cystatin C calibrators and non-harmonized methods for determining cystatin C concentration has given rise to a large number of different cystatin C-based GFR prediction equations in children and adults[5,13,14,8,15–19]. Development of a certified reference material is desirable as it would eliminate most of the issues due to the variability in calibrators and assays; such an effort has been well underway in Europe and only recently has it become relevant in the United States[11].

While this is a reasonable endeavor, it would create confusion among the subjects in research initiatives such as the CKiD study. This observational study has been going on for over 15 years, and over 1000 children have been recruited in the United States and Canada. Many children have had multiple determinations of cystatin C, because it is an analyte that is regularly obtained at the annual visits as a means to estimate GFR. Converting to IFCC calibration would bring about an increase in the cystatin C concentration, which when applied to already published cystatin C eGFR equations would suddenly result in an underestimation of GFR and an apparent acute deterioration of kidney function, a concerning development for the patient and the study. Therefore, it is of great importance to reliably determine the relationship between cystatin C concentrations obtained via pre-IFCC calibration and those obtained using the newer IFCC calibration.

The approach to comparing the two calibration methods was resolved into two main efforts. The first was to obtain the reference standard ERM-DA471/IFCC and dilute it into PBS within the concentration range encountered clinically (see Figure 1). Such an approach was performed with 4 separate dilutions of vials of reference material. Due to the large dilution of the sample with PBS prior to analysis (100 fold), pre-dilution of the reference standard with PBS was not likely to greatly affect the commutability of the assay[5] The second step was to modify the software and cystatin C application so that both the pre-IFCC and IFCC calibrations could be obtained simultaneously on a given specimen (see Figure 2). Rather than simply use the new software that allows the IFCC calibration, we made sure the pre-IFCC calibration was maintained in our system and able to provide a result at the same time as the new IFCC calibration. The results of both steps were quite similar: data using the IFCC calibration were consistently 17% higher than the concentrations obtained via the pre-IFCC method. This was true over the whole range of cystatin C concentrations.

This simple relationship between pre-IFCC and IFCC calibrations on the BN II instrument allows us to continue to use the cystatin C-based and full CKiD GFR estimating equations in the CKiD study[1]. One caveat is that the comparative measurements were made between 2017 and early 2019. There has been a concern for drift of the Siemen-Dade-Behring assay over time[20,21]. Whereas this is concerning for correcting concentrations from early in the CKiD study, our personal experience using sera from healthy people as an additional internal control shows substantially less than 10% change over the past 10 years in our assay (data not shown). Another caveat is that the relationship between pre-IFCC and IFCC calibrations was established for cystatin C with specimens generally in the range of 1–2 mg/L, and therefore extrapolation to a level of 3.5 mg/L or to 0.6 mg/L may lead to erroneous conclusions. However, we did find (see Figures 1 and 2) that the pre-IFCC to IFCC relationship was maintained not only in the range of 1–2 mg/L but above 2 mg/L and below 1 mg/L. This result suggests that the relationship between pre-IFCC and IFCC calibrations is preserved: 1.17*pre-IFC=IFCC concentration in mg/L.

All cystatin C concentrations have been IFCC calibrated for CKiD since April 15, 2019. To use the GFR estimating equations one can simply divide the current IFCC-calibrated cystatin C value by 1.17 to obtain the pre-IFCC concentration and use that value to determine the eGFR. Obviously, the creatinine-based GFR estimating equations[22] do not require any modification, because creatinine is already referenced to IDMS standards[23–25]. For CKiD and other patients older than 18, one can divide the IFCC-calibrated value by 1.17 and the cystatin C-based eGFR calculated from the equations previously published[1]. Alternatively, one can use the IFCC-calibrated cystatin C concentration directly applied into the CKD-EPI equations[26,4].

In a 2011 letter to the Am J Kidney Disease, Inker et al[4] posited that IFCC calibration of cystatin C would result in values that were 11–12% higher than pre-IFCC measurements in a Siemens nephelometer. This study is in line with the trend we saw, however is ~5–6% lower than we observed in our study and that reported by Siemens[7]. Strength for utilizing the 17% correction for IFCC calibration comes from the agreement between Siemens[7] and our present data using totally different sera, and from the fact that we not only compared the calibrations of pre-IFCC and IFCC simultaneously in the same specimens, but when we diluted the ERM-DA471/IFCC reference material, the predicted results were 17% higher than those measured using the pre-IFCC calibration. A potential reasoning for this diffence between the current study finding and the previous is that the calibration was different in the 2011 than it is in our present study. Future alignment studys may be needed to address discrepancies should this be the case.

In summary, we have performed careful studies that consistently show the IFCC calibration for cystatin C to yield concentrations that are 17% higher than those obtained on the same nephelometer using the pre-IFCC calibration. This is evident from both simultaneous comparisons of the two calibrations as well as from preparing known dilutions of the reference material. We, in turn, recommend that the IFCC calibration be utilized for all cystatin C measurements in the United States. Currently, for pediatric specimens the results can be divided by 1.17 to estimate GFR using the previously published CKiD equations[1]. For adults and older adolescents using the CKD-EPI eGFRs, the IFCC calibrated results can be directly applied to the published equations.