Regional Implementation of Creatinine Measurement Standardization

Abstract

Because patients may receive care at multiple locations within a geographic area, serum creatinine measurements must be standardized across laboratories to enable comparisons of reported estimated glomerular filtration rate (eGFR). The results of a successful creatinine standardization program designed to minimize the contribution of laboratory error to the reporting of eGFR are reported; 107 laboratories, which tested creatinine on 124 analyzers from six different manufacturers, voluntarily participated. Each laboratory received a correction factor to apply to its creatinine measurements to standardize them to the isotope dilution mass spectrometry reference method. The adjusted values were then used to calculate eGFR using the Modification of Diet in Renal Disease (MDRD) equation. The standardization program reduced the average total error in the measurement of creatinine from 23.9 to 8.7% and the average analytical bias from 16.5 to 2.7%. Implementing this program on a larger scale could reduce the rate of incorrect classification of stage 3 chronic kidney disease by 84%.

Evidence-based clinical practice guidelines suggest that an estimate of GFR (eGFR) provides the best clinical tool to gauge kidney function.^1,2 To aid clinicians in their decision-making, laboratories are recommended to report eGFR routinely for adult patients by using creatinine-based equations. The four-variable Modification of Diet in Renal Disease (MDRD) equation³ is well suited for this purpose. In contrast to the Cockroft-Gault equation, the MDRD equation does not require the patient's weight to provide an accurate assessment of eGFR, and it has been reported as a valid indicator of kidney function.⁴ Various limitations to the MDRD formula are recognized, but, overall, it is a well-accepted estimate of kidney function.

Serum creatinine test results can vary significantly between clinical laboratories,^5,6 a fact that is often not well recognized by health care professionals. This variation is greater in the normal and near-normal range of creatinine measurements, and the difference may be of sufficient magnitude to change patient classification when an eGFR is calculated. Regional initiatives to implement eGFR reporting must first take steps to standardize the measurement of creatinine before it is used to calculate and report an eGFR.⁷ Creatinine can be standardized with an isotope dilution mass spectrometry (IDMS) reference method. An IDMS-traceable format of the MDRD equation has been developed.⁷

British Columbia introduced eGFR reporting in October 2003, and in March 2004, a provincial program was introduced to standardize the measurement of creatinine. This initiative involved 107 clinical laboratories throughout the province and was introduced as an interim strategy pending standardization of creatinine testing by instrument manufacturers, a process that is estimated to be completed by 2008. We report here the method in implementing this program and the ongoing monitoring of calibration and eGFR reporting errors by these laboratories. This may serve as a useful framework for other health regions considering the execution of a successful eGFR reporting program as per current best practice guidelines.

British Columbia is a province in Canada with a population of approximately 4.1 million people. The British Columbia Provincial Renal Agency, which is a branch society of the Provincial Health Services Agency, is an administrative structure providing a provincial framework to facilitate the implementation of renal initiatives.

The provincial creatinine standardization program is a voluntary project, which was carried out in partnership with the Canadian External Quality Assessment Laboratory. Canadian External Quality Assessment Laboratory and its collaborators design and implement external quality assessment programs that are used to monitor the analytical performance of clinical laboratories nationally and internationally. By using a commutable test sample (human serum), an external quality assessment program can be used to assess the accuracy of a laboratory's testing and to confirm the transfer of trueness (accuracy) from a reference method to field methods through a given calibration process.⁸

Results

The Creatinine Standardization Program is funded centrally and operates on a voluntary basis. The majority (94%) of clinical laboratories in British Columbia are participating. One region has opted out of the program, preferring to wait until the manufacturer of their instrumentation provides them with a creatinine method that has been standardized. When these laboratories are excluded from the data set, the participation rate for the remaining laboratories in the province is 99%. This rate of participation has remained unchanged since the inception of the program in 2005. From a clinical perspective, the eGFR reporting system has been implemented successfully and has been generally well accepted by most practitioners. Further details and the overall impact of the program form the basis of a separate publication.

Laboratory Standardization

The results of the baseline study are presented in Figure 1. The vast majority of analyzers in the province were found to be operating with a positive bias (calibration error) relative to the “true” value for creatinine as assigned by the reference method (Figure 2). The average calibration bias for the measurement of creatinine across the province was 16.5% (positive) when assessed relative to the true value. Calibration error will systematically skew all test results that are generated on a given analyzer. If the province of British Columbia were functioning as a single analyzer for the testing of creatinine in all adults in the province, then the reported creatinine test results would be 16.5% higher than they should be. After standardization, this bias was reduced to 2.7%. Figure 2B illustrates the marked shift downward in these test results after application of the assigned calibration correction equations for each laboratory.

Figure 1.

%TE for the measurement of creatinine (98.9 μmol/L) at baseline. The between-day precision (coefficient of variance) is also plotted (black bars).

Figure 2.

(A) Unadjusted creatinine performed on one test sample from each of the 107 laboratories in the province as compared with the reference sample. (B) Adjusted creatinine according to the assigned calibration correction equation for each laboratory. RV, reference value.

Figure 1 illustrates that there is a significant degree of variation in the measurement of creatinine province-wide. The percentage total error (%TE) ranged from 4 to 54% with an overall provincial average of 23.9%. Given the extremes, these data indicate that if a serum sample with a true creatinine concentration of 100 μmol/L were to be measured in the best laboratory in the province, then the reported result would fall between 96 and 104 μmol/L 95% of the time; if it were tested in the worst laboratory, then the reported result would be fall between 45 and 154 μmol/L 95% of the time.

The percentage of laboratories that were able to meet a 10% performance limit for creatinine and eGFR on this sample before and after application of the assigned correction equation for calibration bias is presented in Table 1. Before correction of calibration bias, 50% of the laboratories reported a test value that was within 10% of the assigned target value. After correction, this figure improved to 90%. This was reflected in the average %TE for the province, which decreased from 23.9 to 8.7% after implementation of the program.

Table 1.

Percentage of laboratories meeting a performance limit of reference value ±10% at baseline

Analytea	Pass (%)
Creatinine (uncorrected)	50.4
Creatinine (corrected)	90.3
eGFR (uncorrected creatinine)	58.7
eGFR (corrected creatinine)	86.6

^aReference value (RV) = 98.8 μ mol/L.

Performance data from a recent monitoring challenge are presented in Table 2, demonstrating that improvement performance has been maintained during the intervening 2-yr period. Although it is evident from this table that standardization does not perform as well at the lower concentrations of creatinine, the results are still better than before the correction of calibration bias.

Table 2.

Monitoring cycle (February 28, 2006)^a

Sample	A	B	C
Cr (μ mol/L; RV)	117.6	91.1	68.1
eGFR (ml/min per 1.73 m2; RV)	41	56	99
% of laboratories meeting a performance limit of RV ±10%
Cr (uncorrected)	50	38	26
Cr (corrected)	94	86	78
eGFR (Cr corrected)	90	80	74

^aCr, creatinine.

Program Cost

During year 1 of the program, the first and second components and one monitoring cycle of the third component were conducted. The cost of this “launch” phase was $335,000. This fee covered the costs associated with acquisition of the sample sets for the program, screening of these samples for viral pathogens, shipping of samples to Belgium for assignment of target values by IDMS, shipping of the samples to participating laboratories on dry ice, data acquisition, and performance reporting together with telephone and e-mail follow-up plus ongoing administrative costs. The third component of the program, composed of the ongoing monitoring and auditing of the participating laboratories three times per year, costs $135,000 per year. These figures include the cost of clerical, administrative, and professional support at both a clinical and a basic science level. Costs of errors as a result of lack of standardization are not the focus of this report; however, given the relatively low cost of this standardization program, in a province that performs >1 million serum creatinine measurements per year, the cost of changes in care, referral, and other issues as a result of lack of standardization is high.

Discussion

The program described here was designed to minimize the contribution of laboratory error to the provincial reporting of eGFR. This was accomplished by providing a postanalytical correction factor to each laboratory for standardizing its creatinine test results to the IDMS reference method before using the creatinine test result for calculating the patient's eGFR using the MDRD equation. The provincial creatinine standardization program reduced the average %TE in the province for the measurement of creatinine from 23.9 to 8.7% and the average analytical bias from 16.5 to 2.7%. Without correction of this calibration bias, the analytical systems in the province would on average be reporting creatinine test results that would be 16.5% higher than the true value (false positive).

Reporting of eGFR is now the standard of care in helping to identify, stage, and monitor patients with chronic kidney disease (CKD).^1,2 The measurement of creatinine in serum is the key determinant for the correct estimation of GFR. We provide a description of a large-scale provincial initiative to both initiate and maintain a successful eGFR reporting program that includes the standardization of creatinine testing in all provincial laboratories.

There is an increasing trend to use treatment guidelines that triage patients and invoke clinical management decisions on the basis of a test result. These guidelines often assume that laboratory testing has been standardized and that laboratory-to-laboratory variation in test results is minimal and can largely be ignored in the execution of the guideline. Unfortunately, this is not the case.

“Trueness” of a laboratory test result has impact on guideline-based medical decisions. If accuracy and consistency between laboratories is not achieved, then patients will be misclassified and incorrectly treated and may have significant attendant costs. One recent study in the United States estimated that laboratory calibration errors that affect the accuracy of calcium test results is costing the US health care system $66 to $199 million dollars per year⁹ as a result of the costs of the clinical decisions that were made in response to the inaccurate calcium test results.

The National Kidney Disease Education Program (NKDEP) Laboratory Working Group in collaboration with international professional organizations and manufacturers has developed a plan that when fully implemented will enable standardization and improved accuracy (trueness) of serum creatinine measurements in clinical laboratories worldwide. This plan is in the early stages of being implemented. On the basis of biologic variation, a %TE performance goal of 11.4% has been identified by this committee as being the minimum acceptable performance goal that clinical laboratories would have to meet for the provision of uniform and accurate eGFR.¹⁰ We chose to perform our initial standardization protocol on 27 different samples. We believed that this was more than enough to confirm precisely that %TE was within the acceptable range. Similar published studies of this kind have used as few as five survey specimens.⁶ The BC program was implemented as an interim strategy for the standardization of creatinine measurements in British Columbia until such time as the proposed NKDEP Standardization Program is fully implemented and deployed by the instrument manufacturers that subscribe to this program.

In this study, 90% of the participating laboratories were able to achieve a 10% %TE performance goal after correction of their calibration bias. This finding indicates that, in real terms, a %TE performance goal of 11.4% is certainly achievable once the manufacturers have revised their calibration processes to be traceable to the IDMS reference method. Accuracy-based human serum proficiency testing programs that incorporate clinical cases as are used in the BC program will play an important role in confirming that these revised calibrations have successfully transferred the trueness of creatinine test results to the field. That going forward, this transfer of trueness remains constant and the laboratories themselves have managed to implement successfully the routine reporting of eGFR on the basis of the IDMS-traceable format of the MDRD study equation.

An estimated 145,000 people in British Columbia are at increased risk for CKD. All of these people would receive a correct diagnosis on the basis of a decreased GFR (<60 ml/min) by a testing system that is operating with a 16.5% positive analytical bias for the measurement of creatinine, but their reported eGFR would be lower than actual. If all of the adults in British Columbia were to be tested using this analytical system, then 535,000 adult British Columbians (15 yr and older) would be added to an at-risk category for stage 3 renal disease when in fact they should not be (false positive; 2004 population statistics). The standardization program reduced the average provincial bias for the measurement of creatinine to 2.7%. Given this example, implementing this program would reduce the rate of false-positive results by 84% and keep 449,400 people from being incorrectly classified as being at risk.

It is difficult to estimate the costs associated with this magnitude of calibration error and the attendant rate of misclassification. At a minimum, one might assume that misclassified patients would require some form of follow-up to confirm the presence or absence of renal disease. Follow-up might consist of two office visits ($53.06) together with the a routine urinalysis ($4.98) and urinary microalbumin ($22.10) for a follow-up expenditure per patient of $80.14 (Medical Services Plan fees, 1998). Given this example, the potential savings to be realized from reducing the creatinine calibration bias from 16.5 to 2.7% would amount to $36 million. On the basis of this follow-up, the creatinine standardization program would pay for itself if it could reduce the rate of false-positive results by 0.3%.

The inappropriate labeling of patients as having stage 3 CKD (eGFR 30 to 60 ml/min) has implications far greater reaching than the cost of retesting. It results in a spike of newly identified prevalent patients with CKD in a given region, which affects health services planning. As important, patient-specific issues tied to disease labeling may have psychosocial implications. The impact on obtaining life and disability insurance for patients is not known.

Administrators initiating an eGFR reporting program must recognize the impact that laboratory error can have on the classification of patients at risk and the associated costs on the basis of this error. The first step is to minimize the contribution of laboratory error to the reporting of eGFR through a standardization program. Second, eGFR reporting must be unveiled along with an education strategy for clinicians, patients, and allied health professionals to understand the strengths and limitations of this as a diagnostic tool.

Regional laboratory standardization is a critical step in unveiling a large-scale program to report eGFR. Although significant up-front and maintenance costs are incurred in achieving this standardization, these costs are negligible when compared with the costs associated with the misclassification of patients.

Concise Methods

A total of 107 provincial laboratories, which tested creatinine on 124 analyzers from six different manufacturers, were invited to participate and elected voluntarily to take part in the program. The program consisted of three components: (1) Baseline assessment of all participating laboratories; (2) calibration of serum creatinine values in each laboratory and application individually to equations, tested by mathematical audit (Laboratory Standardization Procedure); and (3) ongoing audit and evaluation.

Phase 1: Baseline Assessment

Each laboratory received a common set of human serum samples for the testing of creatinine under stable testing conditions. The testing protocol program allowed within- and between-day imprecision, calibration bias, and %TE for the measurement of creatinine to be estimated for each laboratory. A total of 2604 human serum samples were tested during the baseline component of the program. The concentration of creatinine in these test samples ranged from 50 to 130 μmol/L. These concentrations were selected to cover the clinical range of interest for the determination of stage 3 renal disease (eGFR 30 to 60 ml/min per 1.73 m²) as defined by the National Kidney Foundation Kidney Disease Outcomes Quality Initiative (K/DOQI) guidelines.¹ This was done to reduce the impact that higher concentrations can have on least-squares regression models and to provide a better data set for demonstrating method bias over a range of concentrations that are clinically meaningful for the purpose of identifying early-stage renal disease. Subsets of these samples were also shipped on dry ice to a reference method laboratory in Belgium for the assignment of target values by the IDMS reference method for the measurement of creatinine. The International Federation of Clinical Chemistry Joint Committee on Traceability in Laboratory Medicine has approved this method for this purpose. Each sample was analyzed six times (duplicate analyses on three separate occasions), and the mean result was used to assign the target (true) value for creatinine to each sample.

Phase 2: Laboratory Standardization Procedure

An information package, which was written to describe the program and its underlying rationale, was distributed to each of the laboratories before the launch of the program. Subsequently, a comprehensive registration process was initiated to capture the contact and shipping information in addition to the details of the analytical systems and reference intervals that were in use in each of the laboratories.

Eighteen baseline and three check samples of human serum were shipped frozen on dry ice to the participating laboratories. In addition, each laboratory received three sets of frozen troubleshooting samples that were to be held for future use. Each laboratory performed a total of 30 creatinine assays during the baseline component of the program. Twenty-seven of these test results were used to calculate values for bias, absolute bias, variance, and %TE (%TE = bias + 1.96 × coefficient of variance) for the creatinine method as operated in each laboratory. These data were used to assign a laboratory/analyzer-specific linear regression equation for the correction of calibration bias. The remaining three results (check samples) were used to determine the impact of applying the assigned correction for calibration bias on the %TE for the measurement of creatinine in each laboratory. The calibration correction was applied when there was an improvement of >1% in the %TE as documented by the check samples. In addition, with the assignment of the calibration equation, the laboratories received a set of standardized reporting comments that were to be used with the reporting of eGFR.

Phase 3a: Mathematical Audit

Before going live with the reporting of eGFR in the province, the participating laboratories received a series of case studies for which they were asked to apply their assigned bias correction equation and to use the corrected creatinine test result in applying the modified four-variable IDMS-traceable format of the MDRD equation in calculating eGFR (GFR = 175 × standardized serum creatinine^−1.154× age^−0.203× 1.212 [if black] × 0.724 [if female]). This served as a mathematical audit and confirmed the laboratories' ability to apply their assigned correction equation for the postanalytical standardization of their creatinine test results and to apply this result correctly in calculating eGFR by the IDMS-traceable MDRD formula.

Phase 3b: Monitoring

The third component of the project involves an ongoing monitoring of all participating laboratories. Laboratories are sent three blind human serum samples three times a year. Each sample comes with a clinical case and has target values for creatinine assigned by the IDMS reference method. The laboratories are asked to provide their creatinine test results for each of these samples, the corrected (standardized) creatinine result that they would produce for each sample (if such correction is being applied), and the eGFR and comments that would be reported by their laboratory on the basis of the clinical information that came with the sample. After each monitoring challenge, a report is sent to the laboratory documenting its performance relative to the reference targets and the provincial performance criteria that have been established for the program. A report that summarizes the performance of the program provincially is also generated for distribution. The monitoring phase of the program is used to audit compliance and to confirm that the assigned calibration corrections are still appropriate for laboratories that are using them. This component of the program has also been used to audit other aspects of the testing process, including the consistency of reference intervals that are being used throughout the province for the reporting of serum creatinine test results in adults.

Disclosures

None.