Abstract
Background
Transcutaneous techniques to measure serum bilirubin have been validated in neonates but not in adult patients. We evaluated transcutaneous bilirubinometry (TcB) in adults at risk for or diagnosed with hepatic dysfunction to determine if this technology has clinical utility in quantifying the presence and magnitude of hyperbilirubinemia.
Design
Unblinded, consecutive hospitalized adult patients (n = 80) from the General Surgery, Trauma Surgery, and Liver Resection / Transplant services of a tertiary care, university affiliated medical center who were having serum bilirubin measurements performed underwent transcutaneous bilirubin measurement from the forehead, sternum, forearm, and deltoid. The transcutaneous bilirubin measurements were repeated each time serum bilirubin measurements were performed.
Results
Transcutaneous bilirubin measurements from the forehead correlated with serum bilirubin better (R=0.963) than measurements from the forearm (R = 0.792), deltoid (R = 0.922), or sternum (R = 0.928). Forehead TcB detected hepatic dysfunction (serum bilirubin ≥ 2 mg/dl) by receiver operator curves (Area Under the Curve, AUC = 0.971) as well as sternum (AUC = 0.970) and better than deltoid and forearm measurements (AUC = 0.935 and 0.893, respectively). A Bland-Altman plot, however, demonstrated that forehead measurements became less accurate as the magnitude of hyperbilirubinemia increased.
Conclusions
Forehead TcB correlated best with serum bilirubin levels but became less accurate at higher values. Refinements in the technology will be required before this technique, while promising, can be considered for routine clinical application in adults being evaluated for hyperbilirubinemia.
Keywords: Hepatic Function, Trauma, Transplantation, Spectroscopy
INTRODUCTION
The development of hepatic dysfunction impacts outcome for acutely ill, hospitalized adults even when the hepatic dysfunction is detected in association with diseases that do not primarily affect the liver such as end stage cardiac disease, acute respiratory distress syndrome (ARDS), sepsis, and multisystem trauma (1–8). However, the test of greatest utility for detecting and monitoring liver disease or hepatic dysfunction remains undefined (1). When evaluating the liver, most clinical laboratories perform a battery of analyses that assess hepatocellular injury (aspartate aminotransferase, alanine aminotransferase), bile production or excretion (bilirubin, alkaline phosphatase), and hepatic synthetic capacity (albumin, prealbumin, total protein). Serum bilirubin levels have been used to stratify the magnitude of acute hepatic functional derangement in critically ill adults (2–8) while hepatocellular enzymes are less useful in this regard (9, 10). Serum bilirubin levels are commonly utilized to quantify hepatic derangement in severity of illness scoring systems such as the Child-Pugh Score, Apache III, and multiple organ failure quantification scales that are used to monitor the status of critically ill patients (6, 11–15). In addition, bilirubin levels are frequently used either alone or with other parameters to monitor progression of disease or recovery in patients with biliary tract disease and hepatic failure as well as in patients undergoing biliary tract surgery or hepatic transplantation (16).
A rapid, accurate, and cost-effective method to quantify hyperbilirubinemia with noninvasive bedside testing offers an opportunity to decrease the time for clinical data to become available to clinicians, facilitate the detection of hepatic dysfunction through more widespread screening, avoid the need for frequent blood sampling and simplify monitoring. The use of transcutaneous techniques to measure serum bilirubin has been described in large populations of newborns being evaluated for pathological hyperbilirubinemia (17–19). Transcutaneous bilirubin (TcB) measurements have compared favorably to serum bilirubin measurements and have been projected to decrease the cost of neonatal screening for clinically significant jaundice (17). A systematic assessment of this technology in adults, however, has not been performed. We therefore evaluated the ability of transcutaneous bilirubinometry to assess serum bilirubin levels in a population of hospitalized adults.
MATERIALS AND METHODS
Patients were recruited from the General Surgery, Trauma Surgery, and Liver Resection/Transplant services at UPMC-Presbyterian from 9/2002 until 8/2003. Patients who were having total bilirubin measurements performed as part of their clinical care were identified and approached regarding participation in this study. The only inclusion criteria for this study was the performance of a serum bilirubin test ordered by the patient’s physician. There were no demographic, racial, ethnic, or medical exclusion criteria. Blood samples were obtained by phlebotomists and serum total bilirubin assessed in the clinical laboratory using a colorimetric reaction with the Vitros 700 System (Ortho-Clinical Diagnostics Inc., Rochester, NY). Dilutions were performed for samples that were out of the reportable range (0.1–27.0 mg/dl). After obtaining informed consent, the patients had TcB measurements performed using the BiliChek™ Noninvasive Bilirubin Analyzer (SpectRx, Norcross, GA). Measurements of TcB were performed and repeated during each subject’s hospitalization whenever serum total bilirubin measurements were performed as part of the subject’s ongoing clinical care. The time between blood sampling and TcB measurement was approximately 2–6 hours for each subject. All decisions regarding the frequency of blood sampling for bilirubin analysis were made by the subject’s physicians independent of this study protocol. This study was reviewed and approved by the University of Pittsburgh Medical Center (UPMC)/University of Pittsburgh Institutional Review Board and informed consent was obtained from all subjects or their legal representatives
Measurements of TcB were performed with the BiliChek™ which uses fiberoptic sensors and reflectance spectrophotometry to measure total bilirubin in the skin. The device performs a spectral analysis to incorporate intensity of reflected light, melanin content, and hemoglobin content (SpectRx) although melanin and hemoglobin were not specifically controlled for in this study. Measurements were performed according to the manufacturer’s instructions. Briefly, the fiberoptic tip measuring 0.5 cm is placed gently on the skin and a digital readout provided based on the average of three contacts. In order to account for variation in the force applied to the skin, triplicate readings were performed at each site and the mean value reported. Repeated readings at identical sites tended to vary by <2 mg/dl (data not shown). We began the study using a device with a measurable range of 0–20 mg/dl. Readings that were out of range (>20 mg/dl) were considered missing for statistical analysis and were not included in the correlation with plasma samples. After the initial 25 patients, the digital readout was altered by the manufacturer to increase the reportable range to >20 mg/dl. Since dermal thickness and melanin content of the skin can influence TcB readings (18, 19), we performed our initial analysis from the subject’s forehead, sternum, deltoid, and forearm to capture areas of skin with different degrees of thickness. Areas of skin involved with wounds, contusions, hematomas, surgical dressings or other conditions that might interfere with optical reflectance were avoided. Therefore all subjects did not have readings performed at each location. After the initial 52 patients were studied, an interim analysis (described in Results) revealed that the forehead was the most accurate location and further measurements were restricted to this site with duplicate readings performed at two different forehead locations.
Data are presented as the mean ± standard error. Linear regression analysis was used to evaluate the relation between TcB and serum bilirubin. The sensitivity and specificity of TcB to detect hepatic dysfunction defined as a bilirubin ≥ 2 mg/dl was assessed by Receiver-operating characteristic curves (ROC). Agreement between TcB and serum bilirubin measurement was assessed using methods described by Bland and Altman (20, 21) and using the Folded Empirical Cumulative Distribution Plot described by Krouwer (22). For the folded empirical cumulative distribution plot or mountain plot, serum bilirubin values were used as the reference standard and the percentiles for the differences were ranked and plotted against the percent difference (22). For Bland –Altman plots, forehead TcB measurements were used for analysis. The mean of the forehead TcB and serum bilirubin was taken as the “true” reference value since both methods can contain measurement error. We plotted the mean measurement difference between the two techniques against the mean of the two measurements to evaluate the relationship of measurement difference and magnitude of hyperbilirubinemia. We then performed linear regression of the difference between the methods against the mean of the two methods and determined the appropriateness of a linear model using standard regression diagnostics. We regressed the absolute value of the residuals on the mean of the two methods and obtained the 95% limits of agreement as described (21). In the absence of any significant relationship between the residual and the mean value, the estimated standard deviation is the standard deviation of equation 1. Statistical analysis was performed using Statview (Version 5.0, SAS Institute Inc, Cary NC), MedCalc (Version 7.6, MedCalc Software, Belgium) and Stata (Version 8SE, Stata Corp LP, College Station, TX).
RESULTS
There were 80 patients who were enrolled in this study. Demographic information is provided in Table 1. Forty-nine percent of the patients were men and the average age was 52.3 ± 2.0 years. The study group was predominately caucasian with a number of different illnesses as the primary admitting diagnosis (Table 1). There were a total of 440 paired serumtranscutaneous measurements performed with a mean of 5.6 ± 0.7 and a median of 3.5 measurements per patient (range 1–23).
Table 1.
Patient Demographics
n | 80 |
---|---|
Gender, M/F | 39/41 |
Age, y | 52.3 ± 17.6 (range 20–81) |
Race | |
Caucasian | 77 |
African American | 2 |
Asian | 1 |
Admitting diagnosis | |
Liver transplant | 24 |
Hepatic failure/cirrhosis/ESLD | 17 |
Biliary tract disease | 14 |
Trauma | 11 |
Malignancy | 9 |
Liver abcess/sepsis | 2 |
Coronary artery disease | 1 |
End stage renal disease | 1 |
Short gut syndrome | 1 |
Data represent mean ± SD.
There were 52 patients who had paired serum-transcutaneous measurements performed at multiple body locations. There were 208 total sets of measurements performed, which included 153 measurements from the forearm, 167 measurements from the sternum, 193 measurements from the deltoid, and 198 from the forehead. Not all sites were accessible in every patient due to the presence of wounds, splints, hematomas, and other visible abnormalities that could interfere with probe contact or spectroscopy measurements. Transcutaneous bilirubin measurements at each site significantly predicted serum bilirubin measurements (Figure 1, p<0.001 for all sites). Forehead TcB predicted serum bilirubin levels better then measurements taken from other sites (Figure 1).
Figure 1.
Correlation of transcutaneous bilirubin measurements with serum bilirubin. Subjects underwent paired serum-transcutaneous bilirubin measurements as described in Methods. Transcutaneous measurements were obtained from the A) forearm (serum = 0.65 + 1.33 × forearm, R = 0.792); B) deltoid (serum = −0.91 + 1.00 × deltoid, R = 0.922); C) sternum (serum = −0.66 + 0.90 × sternum, R = 0.928); D) forehead (serum = −1.55 + 1.18 × forehead, R = 0.963).
In scoring systems that quantify the magnitude of organ failure, hepatic dysfunction is frequently defined as a serum bilirubin ≥2.0 mg/dl (12–15). We therefore evaluated the sensitivity and specificity of transcutaneous techniques to detect serum bilirubin ≥2.0 mg/dl using ROC curves for the same set of 52 patients (Figure 2). The area under the ROC curve for measurements from the forehead was similar to that of the sternum (0.971 vs. 0.970, forehead vs. sternum, p = 0.95) while both sternum and forehead measurements identified hepatic dysfunction better than measurements from the forearm or deltoid (forearm AUC = 0.893; deltoid AUC = 0.935)(MedCalc).
Figure 2.
Receiver operator characteristic curve for transcutaneous bilirubin to detect hepatic dysfunction (serum bilirubin ≥ 2 mg/dl). Subjects underwent paired serumtranscutaneous bilirubin measurements as described and ROC curves were constructed for each location. Area under the curve (AUC) from forehead (0.971) and sternum (0.970) were statistically different compared to the AUC from forearm (0.893) and deltoid (0.935) (p< 0.03) (MedCalc).
Since forehead measurements provided readings that predicted serum values the best (Figure 1), we next evaluated whether this relationship could be improved by averaging multiple readings from the forehead. There were 47 subjects who had duplicate forehead measurements performed. As expected, the duplicate forehead readings correlated well with each other (R=0.983) but the mean of the duplicate measurements did not improve the relation with serum bilirubin compared to single forehead measurements (data not shown).
A folded empirical cumulative distribution or mountain plot was constructed to compare agreement of TcB measurements with serum bilirubin as the reference standard. As expected, forehead TcB measurements approximated serum bilirubin values better than measurements at other sites (data not shown). Forehead TcB values correlated with serum bilirubin values better than the mean of all TcB measurements (Figure 3) although the TcB measurements became less accurate at high bilirubin values. Approximately 95% of the forehead differences were between −5% and + 10% of the serum bilirubin percent difference.
Figure 3.
Mountain plot comparing transcutaneous measurements to serum bilirubin. The frequency of the differences between the mean of all transcutaneous measurements (dotted line) or forehead transcutaneous measurements (solid line) and serum bilirubin values were plotted against the difference. Data are taken from 52 patients with 198 individual forehead measurements performed.
Bland Altman plots were constructed to further assess the limits of agreement between transcutaneous and serum bilirubin measurements (Figure 4). The mean of forehead TcB measurements and serum bilirubin was the reference value as described in Methods. The relationship between the difference of TcB and serum bilirubin and the reference value became inverse at reference values of approximately 10 mg/dl. At values ≤ 10 mg/dl, the mean difference overestimated the reference bilirubin concentration and increased, as did the 95% limits of agreement, directly with the reference value (Figure 4B). At reference bilirubin values >10 mg/dl, the mean difference starts to reverse progressively towards negative to reflect slight underestimation with increasing reference bilirubin values and again, the 95% limits of agreement widened with larger values (Figure 4B).
Figure 4.
Bland-Altman Plot. The mean of forehead TcB and serum bilirubin was used as the reference standard and regression analysis performed according to the method of Bland and Altman (20). A) The values for individual comparisons are plotted and the fitted regression line is demonstrated. Agreement between the measurements would result in a regression line equal to one across all reference values. In B) the lower 95 % limit (diamond), average (triangle), and upper 95% limit of agreement (X) are demonstrated graphically. Data represent 431 individual forehead measurements from 80 subjects.
To evaluate the ability of TcB measurements to monitor trends in individual patients, we plotted TcB and serum bilirubin values over time for selected patients that had serial tests performed (Figure 5). While the accuracy of TcB measurements varied compared to serum bilirubin measurements, TcB frequently reflected the trends in serum bilirubin over time in individual patients.
Figure 5.
Serial bilirubin measurements. Sequential paired serum (filled) and transcutaneous (open) forehead bilirubin measurements were plotted over time from subjects 37, 38, 52, and 60.
DISCUSSION
The importance of hepatic dysfunction in the recovery from critical illness has been debated. This debate is often fueled by a fatalistic approach to liver failure generated in part by the lack of hepatic replacement therapy and the reliance on general supportive care. Nevertheless, hepatic dysfunction is associated with increased mortality in patients suffering ARDS, congestive heart failure, and multisystem trauma (2,3,8). While the optimal method to assess the hepatic dysfunction associated with critical illness has not been identified, hepatocellular enzyme and serum bilirubin levels are most commonly used for screening and monitoring. A number of quantitative measures of hepatic function have been developed but these techniques are frequently labor intensive, cumbersome, and have not been widely adopted (23–25). Serum bilirubin has been incorporated into several severity of illness scoring systems (12–15). The widespread use of bilirubin levels to monitor hepatic dysfunction may primarily be one of convenience since its measurement can be incorporated into automated panels of blood chemistry analyses. Nevertheless, bilirubin measurements are commonly assessed to monitor critically ill patients and evaluate their clinical course. Transcutaneous bilirubin measurements have been validated in children for the diagnosis of clinically significant neonatal jaundice (17–19). In this study, we evaluated whether transcutaneous bilirubinometry would correlate with serum bilirubin in adult patients.
Our data demonstrate that TcB correlated well with serum bilirubin measurements although the body site from which the measurement was taken had an effect on the correlation. Forehead TcB measurements correlated with serum bilirubin better than other sites and correlated better than the average value of all sites taken from the same individual. Forehead TcB measurement also performed better than other sites in detecting patients who had serum bilirubin values ≥ 2.0 mg/dl, a commonly accepted threshold for the diagnosis of clinically significant hyperbilirubinemia. While TcB measurements correlated with serum bilirubin, accuracy was less than desirable and was, in part, dependent on the magnitude of hyperbilirubinemia. TcB measurements overestimated the mean TcB-serum value at absolute values ≤10 and underestimated mean TcB-serum values at levels >10 (Figure 4). Despite this variability, however, TcB measurements approximated the general trend in serum bilirubin values when serially assessed in individual patients (Figure 5).
The importance of per-measurement variability and its potential impact on clinical decision making will not only be affected by the accuracy of the instrument but also by the clinical conditions in which the instrument is used. Two of the most frequent clinical indications for measurement of bilirubin in hospitalized adult patients are to detect hyperbilirubinemia associated with hepatic dysfunction or biliary tract disease and, for patients with known hyperbilirubinemia, to monitor progression or resolution of the disease process. In the former case, identification of a threshold value is important and so per-measurement accuracy, especially at lower serum bilirubin levels (≤ 2 mg/dl), will be essential. Our data demonstrate that TcB measurements perform reasonably well in this regard (Figure 3) but that room for improvement in precision exists. Figure 3 demonstrates that 95% of the data points from the forehead generated by TcB measurements had a percent difference compared to serum bilirubin values between −5% and + 10%. A 10% difference, however, may be important when evaluating whether a result exceeds a relatively low threshold value that would prompt further diagnostic evaluation. For monitoring patients with known hyperbilirubinemia, the absolute accuracy of the measurement may be of less overall importance than the ability to detect a proportionate change in bilirubin that may indicate clinical improvement or deterioration. As shown in Figure 5, TcB measurements were reasonably accurate in reflecting trends in serum bilirubin concentrations in individual patients.
There are several limitations that must be acknowledged and resolved before this technique can be considered for widespread clinical application. TcB has been primarily developed and evaluated in pediatric patients and incorporates estimates of skin thickness, melanin content, and hemoglobin content into the equations that transform reflectance measurements into bilirubin values. The device tested in this protocol was not modified for use in adults where skin thickness and melanin content may vary substantially from that of newborns. Our patient population consisted primarily of Caucasian patients and whether the device would perform differently in patients of other races and with different skin melanin content needs to be assessed further. In addition, hemoglobin content may vary among adult hospitalized patients depending on their clinical status and associated comorbid medical problems. Hemoglobin values were not measured or controlled for in this evaluation. Finally, blood samples for serum bilirubin were collected approximately 2–6 hours before TcB measurements were taken. Whether this time differential would affect the accuracy of TcB measurements would depend on the rate of bilirubin increase or decrease in a given clinical scenario. Figure 5 suggests that TcB measurements are able to detect changes that occur in serum bilirubin levels over the course of days but whether changes in serum bilirubin occur over shorter periods of time that would affect the accuracy of these results would need to be directly tested.
The optimal test for statistically comparing two methods of measurement of a clinical parameter that are changing over time and for which no true gold standard exists can be challenging to define. In this investigation, we have analyzed the performance of TcB in several ways that incorporate the manner in which clinicians commonly utilize this information. We did not assess variability between individuals or within repeated measures from the same individual. We acknowledge that accounting for multiple measurements from the same individual may modify the statistical interpretation of portions of our analysis (26). Further study controlling for many of the variables listed above will be required to precisely define if and how this device can be used in daily patient care. Whether this technology will prove useful in the clinical care of patients will require modification of the current technology for use in adult patients and further assessment of the accuracy of TcB measurements under a wide spectrum of clinical conditions and in a more heterogeneous population of subjects.
ACKNOWLEDGMENT
We would like to thank Scott E Kerr, Respironics, Murrysville, PA, for providing the BiliChek™ TcB analyzer and providing technical support.
This work was supported, in part, by NIH DK-55664 (BGH) and HD049109 (MRR).
Abbreviations
- TcB
Transcutaneous bilirubinometry
- ROC
Receiver Operator Characteristic curve
- AUC
area under curve
Footnotes
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Competing Interests Declared: None.
REFERENCES
- 1.McIntyre N. The limitations of conventional liver function tests. Sen Liver Dis. 1983;3:265–274. doi: 10.1055/s-2008-1040779. [DOI] [PubMed] [Google Scholar]
- 2.Reinhartz O, Farrar DJ, Hershon JH, et al. Importance of preoperative liver function as a predictor of survival in patients supported with thoracic ventricular assist devices as a bridge to transplantation. J Thorac Cardiovasc Surg. 1998;116:633–640. doi: 10.1016/S0022-5223(98)70171-0. [DOI] [PubMed] [Google Scholar]
- 3.Matuschak GM, Renaldo JE, Pinsky MR, et al. Effects of end-stage liver failure on the incidence and resolution of the adult respiratory distress syndrome. J Crit Care. 1987;2:162–173. [Google Scholar]
- 4.Matuschak GM, Renaldo JE. Organ interactions in the adult respiratory distress syndrome during sepsis: the role of the liver in host defense. Chest. 1988;94:400–406. doi: 10.1378/chest.94.2.400. [DOI] [PubMed] [Google Scholar]
- 5.Schwartz DB, Bone RC, Balk RA, Sziden JP. Hepatic dysfunction in the adult respiratory distress syndrome. Chest. 1989;95:811–875. doi: 10.1378/chest.95.4.871. [DOI] [PubMed] [Google Scholar]
- 6.Hebert PC, Drummond AJ, Singer J, et al. A simple multiple system organ failure scoring system predicts mortality of patients who have sepsis syndrome. Chest. 1993;104:230–235. doi: 10.1378/chest.104.1.230. [DOI] [PubMed] [Google Scholar]
- 7.Harbrecht BG, Doyle HA, Clancy KD, et al. The impact of liver dysfunction on outcome in patients with multiple injuries. Am Surg. 2001;67:122–126. [PubMed] [Google Scholar]
- 8.Harbrecht BG, Zenati MS, Doyle HR, et al. Hepatic dysfunction increases length of stay and risk of death after injury. J Trauma. 2002;53:517–523. doi: 10.1097/00005373-200209000-00020. [DOI] [PubMed] [Google Scholar]
- 9.Cassidy WM, Reynolds TB. Serum lactic dehydrogenase in the differential diagnosis of acute hepatocellular injury. J Clin Gastroenterol. 1994;19:118–121. doi: 10.1097/00004836-199409000-00008. [DOI] [PubMed] [Google Scholar]
- 10.Giannini E, Botta F, Fasoli A, et al. Progressive liver functional impairment is associated with an increase in AST/ALT ratio. Dig Dis Sci. 1999;44:1249–1253. doi: 10.1023/a:1026609231094. [DOI] [PubMed] [Google Scholar]
- 11.Pugh RNH, Murray-Lyon IM, Dawson JL, et al. Transection of the esophagus for bleeding esophageal varies. Br J Surg. 1973;60:646–649. doi: 10.1002/bjs.1800600817. [DOI] [PubMed] [Google Scholar]
- 12.Knaus WA, Wagner DP, Draper EA, et al. The Apache III prognostic system: risk production of hospital mortality for critically ill hospitalized adults. Chest. 1991;100:1619–1636. doi: 10.1378/chest.100.6.1619. [DOI] [PubMed] [Google Scholar]
- 13.Sauaia A, Moore FA, Moore EE, et al. Multiple organ failure can be predicted as early as 12 hours after injury. J Trauma. 1998;45:291–303. doi: 10.1097/00005373-199808000-00014. [DOI] [PubMed] [Google Scholar]
- 14.Marshall JC, Cook DJ, Christori NV, et al. Multiple organ dysfunction score: a reliable description of a complex clinical outcome. Crit Care Med. 1995;23:1638–1652. doi: 10.1097/00003246-199510000-00007. [DOI] [PubMed] [Google Scholar]
- 15.Fry DE, Pearlstein L, Fulton RL, Polk HC., Jr Multiple system organ failure: the role of uncontrolled infection. Arch Surg. 1980;115:136–140. doi: 10.1001/archsurg.1980.01380020006003. [DOI] [PubMed] [Google Scholar]
- 16.Dufour DR, Lott JA, Nolte FS, et al. Diagnosis and monitoring of hepatic injury. I. Performance characteristics of laboratory tests. Clin Chem. 2000;46:2027–2049. [PubMed] [Google Scholar]
- 17.Maisels MJ, Kring E. Transcutaneous bilirubinometry decreases the need for serum bilirubin measurements and saves money. Pediatrics. 1997;99:599–601. doi: 10.1542/peds.99.4.599. [DOI] [PubMed] [Google Scholar]
- 18.Yamanouchi I, Yamauchi Y, Igaraski I. Transcutaneous bilirubinometry: preliminary studies of noninvasive transcutaneous bilirubin in a multiracial predischarge newborn population to assess the risk of severe hyperbilirubinometry. Pediatrics. 2000;106:E17. doi: 10.1542/peds.106.2.e17. [DOI] [PubMed] [Google Scholar]
- 19.Onks D, Silverman L, Robertson A. Effect of melanin, oxyhemoglobin, and bilirubin on transcutaneous bilirubinometry. Acta Paediatri. 1993;82:19–21. doi: 10.1111/j.1651-2227.1993.tb12507.x. [DOI] [PubMed] [Google Scholar]
- 20.Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurements. Lancet. 1986;1:307–310. [PubMed] [Google Scholar]
- 21.Bland JM, Altman DG. Measuring agreement in method comparison studies. Stat Meth Med Res. 1999;8:135–160. doi: 10.1177/096228029900800204. [DOI] [PubMed] [Google Scholar]
- 22.Krouwer JS, Monti KL. A simple, graphical method to evaluate laboratory assays. Eur J Clin Chem Biochem. 1995;33:525–527. [PubMed] [Google Scholar]
- 23.Kholoussy AM, Pollack D, Matsumoto T. Prognostic significance of indocyanine green clearance in critically ill surgical patients. Crit Care Med. 1984;12:115–116. doi: 10.1097/00003246-198402000-00007. [DOI] [PubMed] [Google Scholar]
- 24.Lehmann U, Armstrong VW, Schutz E, et al. Monoethylglycinexylidide as an early predictor of post traumatic multiple organ failure. Ther Drug Monitor. 1995;17:125–132. doi: 10.1097/00007691-199504000-00004. [DOI] [PubMed] [Google Scholar]
- 25.Fabbri A, Bianchi G, Motta E, et al. The galactose elimination capacity test: a study of the technique based on the analysis of 868 measurements. Am J Gastro. 1996;91:991–996. [PubMed] [Google Scholar]
- 26.Bland JM, Altman DG. Agreement between methods of measurement with multiple observations per individual. J Biopharmaceut Statistics. 2007;17:571–582. doi: 10.1080/10543400701329422. [DOI] [PubMed] [Google Scholar]