New Decision Criteria for Selecting Delta Check Methods Based on the Ratio of the Delta Difference to the Width of the Reference Range Can Be Generally Applicable for Each Clinical Chemistry Test Item

Abstract

Background

Many laboratories use 4 delta check methods: delta difference, delta percent change, rate difference, and rate percent change. However, guidelines regarding decision criteria for selecting delta check methods have not yet been provided. We present new decision criteria for selecting delta check methods for each clinical chemistry test item.

Methods

We collected 811,920 and 669,750 paired (present and previous) test results for 27 clinical chemistry test items from inpatients and outpatients, respectively. We devised new decision criteria for the selection of delta check methods based on the ratio of the delta difference to the width of the reference range (DD/RR). Delta check methods based on these criteria were compared with those based on the CV% of the absolute delta difference (ADD) as well as those reported in 2 previous studies.

Results

The delta check methods suggested by new decision criteria based on the DD/RR ratio corresponded well with those based on the CV% of the ADD except for only 2 items each in inpatients and outpatients. Delta check methods based on the DD/RR ratio also corresponded with those suggested in the 2 previous studies, except for 1 and 7 items in inpatients and outpatients, respectively.

Conclusions

The DD/RR method appears to yield more feasible and intuitive selection criteria and can easily explain changes in the results by reflecting both the biological variation of the test item and the clinical characteristics of patients in each laboratory. We suggest this as a measure to determine delta check methods.

INTRODUCTION

Delta check is a quality control method that compares present and previous test results of patients and detects whether the difference between the two results exceeds pre-defined criteria. If the difference is smaller than the pre-defined criteria, the result is automatically reported; however, if the difference exceeds the pre-defined criteria, the result is transacted only after manual confirmation by laboratory personnel [1-4]. Delta check methods ensure the detection of pre-analytical errors, clerical errors, and random errors that cannot be detected using commonly used quality control methods, thereby improving the reliability of clinical tests [5-8]. Excessive or inappropriate use of delta check methods can, however, delay reporting times and increase workload owing to the need for additional manual validation of test results. The use of the most appropriate delta check methods in each laboratory can thus minimize workload and improve the rates at which errors are detected [7]. Most laboratories rely on one of the 4 standard delta check methods-delta difference, delta percent change, rate difference, and rate percent change-for each of the test items [8-10].

Time dependency and the extent of the variations in levels of each item are key primary considerations when choosing an appropriate delta check method. For instance, the delta method should be applied to electrolyte tests, as these always remain within a limited range owing to physiological homeostasis. By contrast, levels of uric acid fluctuate in a time-dependent manner, with fluctuations >2 mg/dL frequently encountered over the course of a month, albeit not over the course of a day. The rate method is therefore more appropriate for analyzing uric acid levels. In instances when the width of variation for the test results is quite large, the delta or rate percent change method should be applied, because the workload for additional manual validation is likely to increase if the delta or rate difference method is applied, due to greater changes in test results. If the width of variation for the test result is not large, the delta or rate difference method should be applied [8].

A major challenge in the selection of delta check methods that awaits resolution is that there are no established decision criteria for the selection of delta check methods suitable for newly introduced test items. Although the CV% of the absolute delta difference (ADD) [8] has been used as a decision criterion, a more reliable index that better reflects the variation in test results would be useful for the selection of delta check methods. Furthermore, the clinical characteristics of patients need to be considered when choosing an appropriate delta check method. The severe conditions of most inpatients usually account for substantial variations in their test results. This may warrant the use of a different delta check method than the methods used for outpatients, who are normally characterized by more stable conditions. Until now, however, no study has suggested separate delta check methods for inpatients and outpatients. In addition, different delta check methods can also be used for patients receiving special therapeutic procedures. For example, following transarterial chemoembolization (TACE), liver-specific enzymes such as AST and ALT may rapidly increase owing to the liver cell injury incurred during the treatment. In the case of hemodialysis, levels of creatinine and blood urea nitrogen (BUN) may rapidly decrease after the procedure. However, to our knowledge, no study has analyzed delta check methods to identify those approaches ideally suited for patients undergoing special therapeutic procedures.

Here we devise new decision criteria for selecting delta check methods that reflect the characteristics of the test items better. We suggest separate delta check methods for inpatients and outpatients and identify the most appropriate delta check method applicable to specific clinical conditions such as TACE or hemodialysis based on these criteria.

METHODS

1. Data collection

We collected the present results, reporting times, and sample types for 27 clinical chemistry tests reported by the Department of Laboratory Medicine, Asan Medical Center over a 5-week period (February 14-20 and April 4-May 1, 2011). We also collected test results reported over a 5-week period just before the present results, as previous results. Patients were classified as inpatient or outpatient; patients admitted to the emergency room of the institution were classified as inpatients, given that their clinical status was usually closer to that of inpatients than outpatients.

Overall, 811,920 and 669,750 paired test results were obtained from inpatients and outpatients, respectively. The overall research methodology of our study is represented schematically in Fig. 1.

Fig. 1.

Schematic diagram of the procedure for the determination of decision criteria for delta check method selection.

Abbreviations: ADD, absolute delta difference; DD, delta difference; RR, reference range; In, inpatients; Out, outpatients; TACE, transarterial chemoembolization; HD, hemodialysis.

2. Delta check methods based on the CV% of the ADD and time dependency

The delta difference, delta percent change, rate difference, and rate percent change were calculated using the collected test results for each item in the inpatient and outpatient data, respectively. We used these data to calculate the ADD. To analyze time dependency, the Pearson correlation coefficient (r) between the ADD and the time difference between the paired test results were calculated, and the time dependency of each item was assessed.

In addition, we calculated the CV% of the ADD for each item, and used this to assess the variation among the test results for each item. The delta check method for each item was determined using the CV% of the ADD and the Pearson correlation coefficient. The values that corresponded to 2.5% and 97.5% of the determined delta check method were presented as the lower and upper limits of delta check [8]. The cut-off values of the CV% of the ADD and the Pearson correlation coefficient were defined using the median values of the 27 items.

3. Delta check methods based on the ratio of the delta difference to the width of the reference range (DD/RR) and time dependency

We devised new decision criteria for selecting delta check methods based on the ratio of the delta difference to the width of reference range. We subsequently called this the DD/RR ratio. Our intent was to use this to distinguish the difference method from the percent change method. For the reference range of each item, the data used in the authors' hospital was used. For test items for which there was either an upper or a lower limit only, the width of the reference range was defined as the difference between 0 and the upper or lower limit. The cut-off value of the DD/RR ratio in the determination of the delta check methods was defined as the median of all of the items. To distinguish between the delta and rate methods, we used the Pearson correlation coefficient between the ADD and the time difference, as described in the preceding section.

4. Comparison of the 2 suggested delta check methods with previously reported delta check methods

We compared the 2 suggested delta check methods for each item in the inpatient and outpatient data, and further compared these 2 suggested delta check methods with those suggested in previous studies [8, 10].

5. Analysis of delta check methods for TACE or hemodialysis patients

We determined the delta check method based on the DD/RR and time dependency for data from 77 inpatients who underwent TACE and 72 patients who underwent hemodialysis at our institution between August 1 and 5, 2011. For the patients who underwent TACE, we analyzed the 5 items expected to be affected by the treatment: AST, ALT, alkaline phosphatase (ALP), total bilirubin, and direct bilirubin. The 5 items analyzed for patients who had undergone hemodialysis were creatinine, uric acid, total bilirubin, BUN, and direct bilirubin.

The cut-off values of the DD/RR and the Pearson correlation coefficient that were obtained from inpatients in the previous analysis were applied for evaluations of data obtained for inpatients who received either TACE or hemodialysis treatment. The result of the present analysis was compared with the result of the delta check method suggested for inpatients in the previous analysis.

6. Statistical analysis

All statistical analyses were conducted using the SPSS 13.0.1 software package (SPSS Inc., Chicago, IL, USA).

RESULTS

1. Former delta check method determined based on the CV% of the ADD and time dependency

To determine the most appropriate delta check method for inpatients, we used the median CV% of the ADD, and the median Pearson correlation coefficient between the ADD and the time interval between the paired test results. The median values for these parameters, obtained by analysis of all 27 items, were 121.63% and 0.049, respectively. Test items to which the delta difference method should be applied include electrolytes (e.g., calcium, phosphorus, sodium, potassium, chloride, and magnesium) and proteins (e.g., albumin). The items to which the delta percent change method should be applied include glucose, creatinine, total bilirubin, amylase, lipase, and creatine kinase (CK). The items to which the rate difference method should be applied include uric acid, direct bilirubin, total CO₂, iron, total iron binding capacity (TIBC), and LDL-cholesterol. The items to which the rate percent change method should be applied include enzymes, lipids, and lipoproteins. These include cholesterol, AST, ALT, ALP, BUN, triglyceride (TG), and HDL-cholesterol (Table 1).

Table 1.

Delta check methods for inpatients based on the CV of ADD and time dependency

^*The median CV% of the ADD (121.63%) and the median Pearson r between the ADD and the time difference between the paired test result (0.049) were used as the cut-off values for the determination of delta check methods for each test item; ^†The values that corresponded to 2.5% and 97.5% of the determined delta check method are presented as the lower and upper limits of delta check.

Abbreviations: ADD, absolute delta difference; ALP, alkaline phosphatase; T, total; BUN, blood urea nitrogen; D, direct; CK, creatine kinase; TIBC, total iron binding capacity; TG, triglyceride; Chol, cholesterol; DD, delta difference; DPC, delta percent change; RD, rate difference; RPC, rate percent change.

To determine the most appropriate delta check method for outpatients, we used the median CV% of the ADD, and the median Pearson correlation coefficient between the ADD and the time difference between the paired test results. The median values for these parameters, obtained by analysis of all 27 test items, were 105.06% and 0.052, respectively. The items to which the delta difference method should be applied include electrolytes such as calcium, sodium, potassium, chloride, and magnesium. The items to which the delta percent change method should be applied include glucose, creatinine, AST, ALT, total bilirubin, amylase, lipase, and CK. The items to which the rate difference method should be applied include uric acid, cholesterol, protein, albumin, phosphorus, total CO₂, iron, TIBC, and HDL-cholesterol. The items to which the rate percent change method should be applied include ALP, BUN, direct bilirubin, TG, and LDL-cholesterol (Table 2).

Table 2.

Delta check methods for outpatients based on the CV of ADD and time dependency

^*The median CV% of the ADD (105.06%) and the median Pearson r between the ADD and the time difference between the paired test result (0.052) were used as the cut-off values for the determination of delta check methods for each test item; ^†The values that corresponded to 2.5% and 97.5% of the determined delta check method are presented as the lower and upper limits of delta check.

Abbreviations: ADD, absolute delta difference; ALP, alkaline phosphatase; T, total; BUN, blood urea nitrogen; D, direct; CK, creatine kinase; TIBC, total iron binding capacity; TG, triglyceride; Chol, cholesterol; DD, delta difference; DPC, delta percent change; RD, rate difference; RPC, rate percent change.

2. New delta check method based on the DD/RR and time dependency

For inpatient data, the cut-off values of the DD/RR ratio were -0.2000 (the median value for the 27 items in cases in which the present result was smaller than the previous result), and 0.2000 (the median value for the 27 items in cases in which the present result was bigger than the previous result). These values were used to distinguish the difference method from the percent change method. The median value of the Pearson correlation coefficient between the ADD and the time difference was used to distinguish the delta method from the rate method. The value of this parameter was 0.049. The delta check methods that were determined as being applicable to the conditions in which the present result is smaller than the previous result, and to those in which the present result is greater than the previous result, were identical for the 27 items.

For outpatients, the cut-off values of the DD/RR ratio were -0.1714 (the median value for the 27 items in cases in which the present result was smaller than the previous result), and 0.1750 (the median value for the 27 items in cases in which the present result was bigger than the previous result). The median value of the Pearson correlation coefficient between the ADD and the time difference was used to distinguish the delta method from the rate method. The value of this parameter was 0.052. The delta check methods that were determined as being applicable to the conditions in which the present result is less than the previous result, and to those in which the present result is greater than the previous result, were identical for the 27 items (Table 3).

Table 3.

Delta check methods based on the ratio of delta difference to reference range (DD/RR) and time dependency

^*The cut-off values of the DD/RR ratio were -0.2000 (for value lower than previous one) and 0.2000 (for value higher than previous one); ^†The cut-off values of the DD/RR ratio were -0.1714 (for value lower than previous one) and 0.1750 (for value higher than previous one); ^‡The values that corresponded to 2.5% and 97.5% of the determined delta check method based on DD/RR are presented as the lower and upper limits of delta check.

Abbreviations: ALP, alkaline phosphatase; T, total; BUN, blood urea nitrogen; D, direct; CK, creatine kinase; TIBC, total iron binding capacity; TG, triglyceride; Chol, cholesterol; DPC, delta percent change; RD, rate difference; RPC, rate percent change; RR, reference range.

3. Comparison of the 2 suggested delta check methods with previously reported delta check methods

Comparison of our new delta check method based on the DD/RR ratio with the delta check method based on the CV% of the ADD showed that they were identical for 23 of the 25 items tested for inpatients, and for 23 of the 25 items tested for outpatients. The 2 items that gave different results for the 2 methods in inpatients were direct bilirubin and CK. The 2 items that gave different results for the 2 methods in outpatients were calcium and LDL-cholesterol.

For inpatient data, a comparison of the delta check method based on the CV% of the ADD with the delta check methods suggested for the same 18 items in previous studies [8, 10] showed that the results were identical to those of at least 1 of the 2 previous studies for all the items, and identical to those of both previous studies for 11 of the 18 items. For outpatient data, the results were identical to those of at least 1 of the 2 previous studies in 12 of the 18 items, and identical to those of both previous studies for 6 of the 18 items.

Comparison of our delta check method based on the DD/RR ratio with the delta check methods suggested in previous studies [8, 10] showed that our results obtained using inpatient data were identical to those from at least 1 of the 2 studies for 7 of 8 items. The only discrepancy was observed for assays of direct bilirubin. Analysis of the items for which variations were noted between the percent change method and the difference method indicated that the delta check methods for glucose, creatinine, total bilirubin, and direct bilirubin changed from the difference method in previous studies to the percent change method in this study, whereas that for uric acid changed from the percent change method in previous studies to the difference method in this study.

Using outpatient data, the results of our study were identical to those of at least 1 of the 2 previous studies for 6 items, but not for calcium, cholesterol, protein, albumin, ALT, phosphorus, and direct bilirubin. Analysis of the items for which variations were noted between the difference method and the percent change method indicated that the delta check methods for calcium, glucose, creatinine, total bilirubin, and direct bilirubin changed from the difference method in the previous studies to the percent change method in this study, whereas those for uric acid and cholesterol changed from the percent change method in the previous studies to the difference method in this study.

In summary, the delta check method that was suggested based on the DD/RR ratio could be clinically useful for 25 items (excluding direct bilirubin and CK) in inpatient samples, and for 19 items (excluding calcium, cholesterol, protein, albumin, ALT, phosphorus, direct bilirubin, and LDL-cholesterol) in outpatient samples (Table 4).

Table 4.

Comparison of the 2 suggested delta check methods with previously reported methods

Abbreviations: ALP, alkaline phosphatase; T, total; BUN, blood urea nitrogen; D, direct; CK, creatine kinase; TIBC, total iron binding capacity; TG, triglyceride; Chol, cholesterol; DD, delta difference; RR, reference range; DPC, delta percent change; RD, rate difference; RPC, rate percent change; ADD, absolute delta difference.

4. Identification of delta check methods for TACE or hemodialysis patients

We used the DD/RR ratio and time dependency to identify appropriate delta check methods for data from 77 inpatients who had undergone TACE. As a result, the rate percent change was suggested for the 5 items-AST, ALT, ALP, total bilirubin, and direct bilirubin-expected to be affected by TACE. The results were identical to those of the test that was conducted for inpatients for AST, ALT, ALP, and direct bilirubin, but not for total bilirubin.

In addition, the results of the same analysis that was conducted on 72 inpatients who had undergone hemodialysis were identical to those of the test that was conducted on all of the in-patients for 2 of the 5 items-creatinine, uric acid, total bilirubin, BUN, and direct bilirubin-that were expected to be affected by the hemodialysis. Rather than the rate method, however, the delta method was suggested for 3 of the 5 items: uric acid, BUN, and direct bilirubin. The comparison of the difference method and the percent change method showed that the results of the analysis are identical to those of the test that was conducted on all of the inpatients for creatinine, total bilirubin, BUN, and direct bilirubin, but not uric acid (Table 5).

Table 5.

Delta check methods for inpatients with TACE or hemodialysis based on DD/RR methods

^*The cut-off values of the DD/RR ratio were -0.2000 (for value lower than previous one) and 0.2000 (for value higher than previous one).

Abbreviations: TACE, transarterial chemoembolization; DD, delta difference; RR, reference range; ALP, alkaline phosphatase; T, total; BUN, blood urea nitrogen; D, direct; DPC, delta percent change; RPC, rate percent change.

DISCUSSION

For most of the delta check methods, it is generally considered clinically important if the measured value is higher than the previous value. In the case of electrolytes or HDL-cholesterol, however, it is also considered clinically important if the measured value is lower than the previous value. Thus, the selection of appropriate delta check methods to identify both these cases would enable a more efficient operation of laboratories by ensuring a more precise validation of measured values. The CV% of the ADD has been used as an index of the variation widths of each item in previous studies. However, this method cannot separately recognize those cases in which the measured value is either higher or lower than the previous value, because ADD converts the difference between the measured value and the previous value into an absolute value.

To overcome this limitation, we investigated whether the DD/RR ratio might offer a better index of the range in the variation of each item in this study. We hypothesized that this metric might be more suitable than the CV% of the ADD as an index of the variation range of each item, because it can reflect both the width of the reference range (an index of the biological variation range determined in each item) and the width of the delta difference (which represents the clinical characteristics of patients). There were no differences in terms of time-dependency between this study and previous studies. Moreover, there was no evidence of new variables, such as analyte half-life, in our study relative to the previous studies. This was because constant rates of production and elimination of items such as electrolytes and glucose preclude differences in the levels and half-lives of these items in clinical samples.

In this study, we used the DD/RR ratio both for cases in which the present result is higher than the previous result, and for those in which the present result is lower than the previous result. These were then compared using the delta check method based on the CV% of the ADD to validate the clinical reliability of the results. As a result, no difference was found between the delta check method that can be applied to cases where the present result is higher than the previous result, and the delta check method that can be applied to cases where the present result is lower than the previous result. The comparison of the newly suggested delta check method with the delta check method that was based on the CV% of the ADD showed that both were identical for all of the items tested in inpatients, except for direct bilirubin and CK, as well as for all of the items tested in outpatients, except for calcium and LDL-cholesterol.

Given that previous studies that proposed the use of delta check methods did not discriminate between inpatients and outpatients in their analyses, their results cannot be easily compared with the results of our study. However, the comparison of the DD/RR ratio with the delta check methods suggested in 2 previous studies for 18 items showed that they were identical to 1 of these 2 studies for 17 items (all except for direct bilirubin) for inpatient data, and to 1 of these 2 studies for 11 items (all except for calcium, cholesterol, protein, albumin, ALT, phosphorus, and direct bilirubin) for the outpatient data. These results show that when the DD/RR ratio is used as a new delta check method selection criterion, this method alone can be applied both to cases in which the present result is less than the previous result and to those in which the present result is greater than the previous result. They also show that the DD/RR ratio is clinically useful for the selection of the stratified delta check method as an index of the range of biological variation for each item. This can be attributed to the high level of consistency between the delta check method based on the DD/RR ratio and the delta check method based on the CV% of the ADD.

In this study, among the 9 items that showed a discrepancy between the delta check method based on the DD/RR ratio for inpatients and outpatients, we demonstrated a difference between the rate method and the delta method for 5 items. This is likely to be attributable to differences in the time-dependencies of the items due to the differences in the manner in which levels of different items were tracked in inpatients and outpatients. Whereas the percent change method is generally considered suitable for enzyme assays, the difference method is generally considered suitable for assays of electrolytes. In the cases of other classes of items, differences were dependent on the severity of the indications presented by patients and the characteristics of hospitals. If many patients with severe conditions are included, the range of variation for that item seems to be significant. Thus, the percent change method is expected to be more suitable than the difference method, and vice versa. In the cases of calcium, glucose, creatinine, total bilirubin, and direct bilirubin, the difference method was suggested in the previous studies, although the percent change method was suggested in this study. This is likely to be attributable to the fact that this study was conducted on a relatively large number of patients with diabetes and liver and kidney disorders. In addition, the previous studies were conducted on fewer than several thousands of patients, which is a smaller sample size than we used in this study.

For both the delta check methods that we investigated in this study and the critical value reporting used for validation systems in many laboratories, it has been reported that the establishment of stratified criteria for patient characteristics and clinical departments improves the efficiency with which laboratories operate, as it can reduce unnecessary validation procedures [11]. The authors' institution has also been running a customized reporting system that applies different criteria for each clinical department by reflecting patient characteristics that may differ between departments. Accordingly, in this study, based on the possibility that a particular treatment will affect test results depending on the test items, we identified the delta check methods that could be applied to patients who had undergone TACE or hemodialysis. For 4 of the 5 items that we expected to be affected, the results for the patients who had undergone TACE were identical to the results obtained for inpatients. Instead of the rate method, the delta method was selected for 3 items in patients who had undergone hemodialysis. This is likely to be attributable to the reduced effect of the correlation between the ADD and the time difference between the paired test results due to the conduct of the test within a regular time range of 0.98-1.01 day. On the other hand, in the selection of the difference method and the percent change method, the result was identical to the result obtained from inpatients for 4 items, albeit not for uric acid. This result shows that delta check methods that are generally applied to inpatients can also be applied to patients who have undergone TACE or hemodialysis.

In the case of mid-sized and small hospitals, the range of biological variation in test results and treatment methods may depend on the size of the hospital and the type of patient population. Thus, unlike in large hospitals, the data from a single hospital are unlikely to reflect the general characteristics of the subject group. In addition, the delta check method that was determined using the DD/RR ratio (the new criterion suggested in this study) can reflect the characteristics of each medical institution. Thus, the aforementioned selection criteria for delta check methods used in large hospitals can be applied to mid-sized and small hospitals. However, the selection criteria of mid-sized and small hospitals seem to be difficult to apply to large hospitals. Therefore, the results of the analysis that was conducted on the patients of a small hospital using the same criteria that were used in the authors' hospital were not compared with those obtained from the patients of the authors' hospital.

In conclusion, the new delta check method based on the DD/RR ratio is highly consistent with the delta check methods suggested in previous studies and the delta check method that was suggested based on the CV% of the ADD. This method is applicable to patients undergoing special therapeutic procedures such as TACE or hemodialysis, which can affect the results obtained for specific test items. Therefore, this method seems to be intuitive and generally applicable, while reflecting both the biological variation of test item and the clinical characteristics of patients in each laboratory. We suggest this as a measure to determine delta check methods for clinical chemistry test items.