Vitamin D Standardization Program (VDSP) Intralaboratory Study for the Assessment of 25-Hydroxyvitamin D Assay Variability and Bias

Abstract

An intralaboratory study assessing assay variability and bias for determination of serum total 25-hydroxyvitamin D [25(OH)D] was conducted by the Vitamin D Standardization Program (VDSP). Thirteen assays for serum total 25(OH)D were evaluated in a single laboratory including 11 unique immunoassays and one liquid chromatography – tandem mass spectrometry (LC-MS/MS) assay. Fifty single-donor serum samples, including eight samples with high concentrations of 25(OH)D₂ (> 30 nmol/L), were assigned target values for 25(OH)D₂ and 25(OH)D₃ using reference measurement procedures (RMP). Using four replicate measurements for each sample, the mean total percent coefficient of variation (%CV) and mean % bias from the target values were determined for each assay using the 50 single-donor samples and a 42-sample subset, which excluded 8 high 25(OH)D₂ concentration samples, and compared with VDSP performance criteria of ≤ 10% CV and ≤ ±5% mean bias. All 12 assays achieved the performance criterion for %CV, and 9 of the 12 assays were within ≤ ±5 % mean bias. The Fujirebio Inc. assay exhibited the lowest %CV and highest percentage of individual measurements within ≤ ±5% mean bias. Ten immunoassays exhibited changes in response due to the high 25(OH)D₂ samples with Abbott, Biomérieux, DiaSorin, DIAsource, and IDS-iSYS assays having the largest deviations. The Fujirebio Inc. and Beckman Coulter assays were only minimally affected by the presence of the high 25(OH)D₂ samples. Samples with high concentrations of 25(OH)D₂ provided a critical performance test for immunoassays indicating that some assays may not have equal response or recovery for 25(OH)D₂ and 25(OH)D₃.

Graphical Abstract

1. Introduction

Since 2010, the Vitamin D Standardization Program (VDSP), established by the U.S. National Institutes of Health, Office of Dietary Supplements (NIH-ODS), has coordinated activities to assist in the standardization of measurements of serum total 25-hydroxyvitamin D [25(OH)D], the primary marker of vitamin D status. Studies have demonstrated that assay results for the determination of serum total 25(OH)D, which is defined as the sum of 25-hydroxyvitamin D₂ [25(OH)D₂] and 25-hydroxyvitamin D₃ [25(OH)D₃], may vary depending on the assay used [1–5]. The analytical challenges and difficulties in assessing vitamin D status have been the subject of several recent reviews[4, 6–14]. The VDSP is a collaborative effort among the U.S. National Institutes of Health, Office of Dietary Supplements (NIH-ODS), the U.S. National Institute of Standards and Technology (NIST) [15], the U.S. Centers for Disease Control and Prevention (CDC), national survey laboratories in several countries, and vitamin D researchers worldwide [16]. Through the VDSP, a reference measurement system has been established consisting of reference measurement procedures (RMPs) at NIST [17], Ghent University [18], and CDC [19]; NIST Standard Reference Materials (SRMs) [20–22]; the CDC Vitamin D Standardization – Certification Program (VDSCP) [23]; and collaborations with two accuracy-based performance testing/external quality assessment (PT/EQA) programs, i.e., the U.S. College of American Pathologists (CAP) accuracy-based vitamin D (ABVD) program [24] and the U.K.-based Vitamin D External Quality Assessment Scheme (DEQAS) [25–27]. Importantly, the VDSP also established assay performance criteria for measurement variability and bias, i.e., Coefficient of Variation (CV) ≤ 10% and mean bias ≤ ±5% [28, 29].

One of the initial activities of the VDSP was the coordination of an interlaboratory comparison study in 2011 to benchmark measurement variability and bias for immunoassays and LC-MS/MS assays used to determine 25(OH)D [3] and to evaluate the commutability of SRMs and PT/EQA program study materials [30]. As a follow up to the first VDSP intercomparison/commutability studies, a second set of intercomparison/commutability studies was undertaken, denoted as Intercomparison Study 2 and Commutability Study 2. As part of Intercomparison 2, an intralaboratory study evaluated 12 immunoassays and one LC-MS/MS assay to assess variability and bias compared with VDSP criteria. Results of this single laboratory, multi-assay comparison study are reported in this paper. Results of the VDSP multi-laboratory Intercomparison Study 2 [31, 32] and Commutability Study 2 [33] are reported elsewhere.

2. Materials and Methods

2.1. Measurands

The measurand for the intralaboratory comparison study was human serum total 25(OH)D in concentration units of nanomoles per liter (nmol/L). Serum total 25(OH)D is defined as the sum of the concentrations of 25(OH)D₂ and 25(OH)D₃, without the inclusion of the concentration of 3-epi-25-hydroxyvitamin D₃ [3-epi-25(OH)D₃].

2.2. Intralaboratory Comparison Study - Coordination and Responsibilities

The intralaboratory study was coordinated by NIST and NIH-ODS, including acquisition of the single-donor serum samples and compilation and evaluation of the results. NIST was responsible for analyzing the 50 single-donor samples to assign values for the concentrations of 25(OH)D₂ and 25(OH)D₃. The University of Liège (Liège, BE) analyzed the 50 single-donor samples using the 12 different assays. Samples were distributed to University of Liège in November 2016 and the results were reported to NIST in January 2017. NIH-ODS and VDSP LLC were responsible for conducting the data analyses.

2.3. Single-donor Serum Samples

The single-donor serum samples used in the single-laboratory comparison study were the same sample set used in Intercomparison Study 2 [31] and Commutability Study 2 [33]. Single-donor serum samples from 50 human donors containing only endogenous vitamin D metabolites, which were prepared according to Clinical and Laboratory Standards Institute (CLSI) C37-A guidelines [34, 35], were obtained with a distribution of total 25(OH)D concentrations across the clinically-relevant range of 15 nmol/L to 150 nmol/L. However, eight samples were included with concentrations of 25(OH)D₂ > 30 nmol/L. A detailed description of the acquisition of the 50 single-donor samples is reported elsewhere [31]. The single-donor samples were stored at NIST at −80 °C until shipped to the University of Liege in November 2016 frozen on dry ice. The samples arrived frozen and were stored at −80 °C until the time of analysis.

2.4. Intralaboratory Comparison Study Design and Assays Evaluated

Analyses for the study were performed at the University of Liège during December 2016 for ten immunoassays and an LC-MS/MS assay; two additional assays were evaluated in June 2019 using the same 50 single-donor samples and protocol. The LC-MS/MS assay was based on the method reported by Fabregat-Cabello et al. [36], which has been certified by the Centers for Disease Control and Prevention (CDC) Vitamin D Standardization – Certification Program for the measurement of 25(OH)D [23]. The Diazyme assay was added as a new assay to the study. The Biomérieux assay was re-evaluated out of concern that there may have been a problem with the VIDAS instrument during the first evaluation; however, the results of the second evaluation using a new VIDAS instrument were not significantly different from the first evaluation, and therefore, both sets of results were included designated as Biomérieux I and Biomérieux II, respectively. The assay manufacturers, assay kits, and instrument models used in this study are summarized in Table 1, and additional details on the assay calibrators and reagents used are provided in Supplemental Table S1. The immunoassays included seven CLIA-based assays, and one each for ELFA-, ELISA-, EIA-, and ITA-based assays (see Table 1 for assay types).

Table 1.

Commercial Assays for Serum Total 25(OH)D used in VDSP Intralaboratory Study

Assay No.	Assay Manufacturer	Assay Kit/Instrument M odel	Assay Typea
1	Abbott	Architect 25-OH Vitamin D; Architect i1000	CLIA
2	Beckman Coulter	Access 25(OH) Vitamin D Total; Access-2	CLIA
3	Biomérieux I	VIDAS 25 OH Vitamin D Total; Vides	ELFA
4	Biomérieux IIb	VIDAS 25 OH Vitamin D Total; Vides	ELFA
5	DiaSorin	Liaison 25 OH Vitamin D Total; Liaison XL	CLIA
6	DIAsource	25OH Vitamin D Total ELISA; Thermo Fischer Multiskan FC w incubator	ELISA
7	Diazymeb	EZ Vitamin D Total	ITA
8	Fujirebio Inc.	25-OH Vitamin D; Lumipulse G1200	CLIA
9	IDS-EIA	25-Hydroxy Vitamin D EIA (IDS); Thermo Fischer Multiskan FC	EIA
10	IDS-iSYS	IDS 25 VitDS	CLIA
11	Roche	Vitamin D Total II; Cobas e411	CLIA
12	Siemens	Vitamin D Total (VitD); ADVIA Centaur XPT	CLIA
13	LC/MS/MS	ABSciex Q-Trap 6500/UPLC	LC-MS/MS

^aCLIA = Chemiluminescence Immunoassay

ELFA = Enzyme-Linked Fluorescence Assay

ELISA = Enzyme-Linked Immunosorbent Assay

EIA = Electrochemical immunoassay

ITA = Immunoturbidimetric assay

^bBiomérieux II and Diazyme assays were evaluated in June 2019; all other assays evaluated in December 2016

For each assay, duplicate measurements were performed on two separate days (n = 4) for each of the 50 single-donor serum samples (≈200 measurements per assay). Prior to analysis, the serum samples were removed from the −80 °C storage, thawed at room temperature, vortexed 30 s, and centrifuged at 3500 rpm for 10 min. The assays were performed using the routine laboratory operation procedures with normal internal QC criteria. Results were reported in nmol/L with three significant figures. The 11 unique immunoassays evaluated in this study included the most frequently represented immunoassays in recent DEQAS exercises [26], i.e., DiaSorin, Roche, Siemens, IDS-iSYS, and Abbott. To avoid repetition of the assay names, only the manufacturer’s name will be used since only one assay from a specific manufacturer was used.

2.5. Value Assignment of the Single-Donor Samples

The 50 single-donor serum samples were analyzed using NIST RMPs for determination of 25(OH)D₂ and 25(OH)D₃ [17]. The assignment of target values to the 50 single-donor samples is reported elsewhere [31] and the results are summarized in the Supplemental Table S2. The distribution of 25(OH)D concentrations in the 50 single-donor serum samples, arranged from low to high concentration, is shown in Supplemental Figure 1, with the contributions of 25(OH)D₂ and 25(OH)D₃ indicated.

2.6. Data Analysis

Mean bias (%) was determined using the following equation:

$$\mathit{\text{Mean Bias }}(\text{\%})=\frac{{\sum }_{j=1}^{j=50}{\sum }_{i=1}^{i=4}[(\text{ Test}\mathit{\text{Laboratory }}-\mathit{\text{NIST}})/\mathit{\text{NIST}}]\times 100}{N_{\mathit{\text{Total}}}}$$

where j = donor samples, i = sample replicates, NIST represents the assigned value, and N_Total = the total number of assays performing the measurements. The percent Coefficient of Variation (% CV) for each laboratory was calculated as the mean of the CVs for each of the 50 single-donor samples as calculated in the VDSCP. Ordinary least squares linear regression analysis was performed on both the 50- and 42-sample sets. The standardized residual plots from the linear regression analysis indicated heteroscedasticity in the regression model for the majority of the assays, particularly for the 50-sample set. We then performed a weighted least squares linear regression analysis using the 42-sample set, excluding the samples with high 25(OH)D₂ content, and observed that the heteroscedasticity was reduced, but still present in 10 of the 14 assays. For the 10 assays with significant heteroscedasticity, the maximum change in slope of the regression line when using the weighted regression analysis compared with the linear regression analysis was 13% with an average of 3.9% change across the 10 assays. As a result, we concluded that although weighting was recommended, it did not result in significant changes in the regression line slopes. The samples with high concentrations of 25(OH)D₂ introduce additional heteroscedasticity and when these samples are plotted with the linear regression line along with the 95% prediction intervals, they generally fall outside the prediction intervals. Therefore, we chose to use the linear (unweighted) linear regression analyses which allows the calculation of 95% prediction intervals for visual qualitative comparison of the differences in the performances of the various assays. All calculations, including CV, % bias, and linear regression with 95% prediction intervals, were performed using Stata software (College Station, TX) and some analyses were confirmed using Analyze-It, a statistical analysis add-in for Microsoft Excel (Analyze-It Software, Leeds, UK).

3. Results and Discussion

The focus of this comparison study was to assess the variability and bias in a single-laboratory setting of commonly used immunoassays and a commercial LC-MS/MS assay for the determination of serum total 25(OH)D. The study protocol of duplicate analyses on two days provided four replicate measurements for each single-donor sample for evaluation of each assay. Serum total 25(OH)D concentrations in the 50 single-donor samples ranged from 16 nmol/L to 148 nmol/L, with 25(OH)D₃ ranging from 10 nmol/L to 141 nmol/L and with most 25(OH)D₂ concentrations between 0.30 nmol/L to 3.0 nmol/L; however, 8 of the 50 samples had high concentrations of 25(OH)D₂ concentrations of > 30 nmol/L ranging from 32 nmol/L to 137 nmol/L, which provided a test of the assays’ capabilities and response to both 25(OH)D₂ and 25(OH)D₃. [37]. For the assay performance evaluations described below, the assay results are reported and compared for both the 50 single-donor sample set and for the 42-sample subset after removal of the samples with 25(OH)D₂ content > 30 nmol/L. The 42-sample subset may be considered as more representative set of clinical patient samples regarding the distribution of concentrations of 25(OH)D₂ [38, 39].

3.1. Descriptive Statistics

Descriptive statistics (N, mean, SD, and minimum and maximum values) for each assay are summarized in Table S3 (Supplementary Material) for the measurements of serum total 25(OH)D in both the 50-sample set and in the 42-sample subset and in Table S4 for the 8 high concentration 25(OH)D₂ samples (> 30 nmol/L). The mean values for serum total 25(OH)D in the 50 single-donor samples ranged from 69.0 nmol/L to 93.3 nmol/L for the different assays compared to the NIST mean value of 77.1 nmol/L as determined by using LC-MS/MS-based RMPs.

3.2. Measurement Variability

The results for the mean Coefficient of Variation (CV) as percent for each assay are summarized in Table 2 including the percent of individual sample with mean CVs less than 10% for both the 50- and 42-sample sets. A similar table with %CV and mean % bias for the 8-sample subset is provided in Table S5 (Supplemental Material). Box and Whisker plots of the %CV for the analysis of all 50 single-donor samples for each assay are shown in Figure 1A. The four immunoassays with the lowest %CVs for the 50 single-donor samples were Fujirebio Inc. (1.9 %), DiaSorin (2.6%), Abbott (2.7%), and Diazyme (2.9%) with SDs ranging from 1.0% to 1.6%. For comparison, the LC-MS/MS assay achieved a %CV of 4.5% for both the 50- and 42-sample sets with SDs of less than 1.8%. Similar Box and Whisker plots for %CV by assay for the analysis of the 42-sample subset and ordered by increasing NIST serum 25(OH)D₃ are provided in Supplemental Figure 2. When comparing the Box and Whisker plots for %CV for the 50-sample set (Figure 1A) versus the 42-sample set (Supplemental Figure 2A), there are no significant differences (compare also the %CV values in Table 2 for both sample sets). There was no apparent trend observed for %CV as a function of concentration of serum total 25(OH)D in the 50 single-donor samples (Supplemental Figure 2B). All 13 assays achieved the VDSP criterion of %CV less than 10%; however, for the IDS-EIA assay, only 56% of the individual measurements (i.e., 200 total measurements) were ≤ 10% (see Table 2).

Table 2.

Mean Coefficient of Variation (%) and Mean Percent Bias for Serum Total 25(OH)D based on Four Individual Replicate Measurements and Assessment of VDSP Performance Criteria by Assay

Assay Manufacturer	All 50 Donor Samples							42 Donor Samples (excluding 25(OH)D2 > 30 nmol/L)
	Total CV (%)					% Value ≤ 10%a	VDSP Criteria	Γ Total CV (%)					% Values ≤ 10%a	VDSP Criteria
	N	Mean	SD	Min	Max			N	Mean	SD	Min	Max
Abbott	50	2.71	1.4	0.5	5.9	100	Pass	42	2.61	1.3	0.5	5.9	100	Pass
Beckman Coulter	50	5.65	3.4	1.0	15.2	86	Pass	42	6.03	3.3	1.2	15.2	86	Pass
Biomérieux I	49	6.02	8.4	0.7	47.3	90	Pass	41	6.26	9.1	0.7	47.3	91	Pass
Biomérieux II	49	4.30	3.4	0.2	16.4	90	Pass	41	4.36	3.4	0.2	16.4	90	Pass
DiaSorin	50	2.65	1.2	0.9	5.6	100	Pass	42	2.67	1.2	0.9	5.6	100	Pass
DIAsource	50	6.48	3.1	1.0	18.6	90	Pass	42	6.61	3.3	1.2	18.6	88	Pass
Diazyme	50	2.89	1.6	0.0	11.4	96	Pass	42	3.25	2.6	0.3	11.4	95	Pass
Fujirebio Inc.	50	1.91	1.0	0.3	4.5	100	Pass	42	1.99	1.1	0.4	4.5	100	Pass
IDS-EIA	50	8.82	4.6	0.9	18.4	56	Pass	42	8.88	4.6	1.6	18.4	57	Pass
IDS-iSYS	50	5.50	2.6	0.6	12.5	94	Pass	42	5.71	2.7	0.6	12.5	93	Pass
Roche	50	3.43	1.9	0.7	8.2	100	Pass	42	3.54	2.0	0.7	8.2	100	Pass
Siemens	50	7.72	5.8	1.8	37.7	70	Pass	42	8.00	6.2	1.8	37.7	67	Pass
LC-MS/MS	50	4.50	1.7	0.7	10.6	98	Pass	42	4.48	1.8	0.7	10.6	98	Pass
Mean		4.81	3.1	0.7	16.3	90			4.95	3.3	0.8	16.3	90
	Mean Bias (%)					% Values ≤ 5%		Mean Bias (%)					% Values ≤ 5%
Abbott	200	− 2.89	17.8	−48.8	28.4	33	Pass	168	3.44	10.0	−17.9	28.4	39	Pass
Beckman Coulter	200	−1.98	18.7	−50.2	73.2	21	Pass	168	−3.27	19.7	−50.2	73.2	18	Pass
Biomérieux I	196	9.45	28.8	−58.8	121	12	Fail	164	18.6	24.3	−58.8	121	15	Fail
Biomérieux II	193	1.46	24.1	−44.3	59.4	16	Pass	162	8.03	19.3	−32.9	59.4	19	Fail
DiaSorin	200	−2.91	15.8	−34.0	38.2	24	Pass	168	2.04	13.5	−27.8	38.2	26	Pass
DIAsource	200	16.7	32.0	−36.9	151	15	Fail	168	22.3	30.6	−26.0	151	17	Fail
Diazyme	196	−2.78	20.7	−56.4	54.7	18	Pass	166	−2.04	22.1	−56.4	54.7	15	Pass
Fujirebio Inc.	200	−3.72	6.8	−27.8	12.2	49	Pass	168	−3.62	7.2	−27.8	12.2	46	Pass
IDS-EIA	200	−8.52	18.1	−42.5	66.9	17	Fail	168	−5.08	−17.3	−37.5	66.9	20	Fail
IDS-iSYS	200	19.5	16.8	−23.1	77.6	9	Fail	168	14.8	12.5	−23.1	42.0	11	Fail
Roche	200	−1.79	11.8	−28.2	53.8	31	Pass	168	−0.13	11.9	−28.2	53.8	34	Pass
Siemens	200	−0.47	20.1	−59.4	57.3	30	Pass	168	−3.44	19.5	−59.4	57.3	30	Pass
LC-MS/MS	199	1.78	6.4	−20.4	27.7	58	Pass	167	1.20	6.4	−20.4	27.7	60	Pass
Mean		1.83	18.3	−40.8	63.2				4.06	16.5	−35.9	60.4

^aPercentage of individual measurements ≤ 10% CV

Figure 1.

Box and whisker plots for the percent Coefficient of Variation (%CV) (A) and mean % bias (B) by test assay for the analysis of 50 single-donor samples. The box encompasses the 25^th to 75^th percentile of all results, the whiskers represent 1.5 × interquartile range above the 75^th percentile and 1.5 × interquartile range below the 25^th percentile. The bar in the box represents the median value (CV% or mean % bias) for each assay. The dots outside the box and whiskers plots represent individual measurements outside the whisker range. For plot A, CVs greater than 35% are excluded and the red line denotes the 10% VDSP criterion for %CV. Four points are excluded with %CVs of 38% for Siemens, 41% and 47% for Biomérieux I, and 67% for Biomérieux II. For plot B, biases greater than 100% are excluded and the red solid line denotes zero bias and the red dashed lines represent ± ≤5% bias. Six points are off the scale and excluded with % biases of 141%, 148%, 149%, and 151% for DIAsource and 120% and 121% for Biomérieux I. Plot C is box and whisker plots for the mean % bias by test assay for the analysis of the eight single-donor samples with concentrations 25(OH)D₂ > 30 nmol/L.

3.3. Measurement Bias

The mean % bias (compared to the NIST target values) for the results of the analysis of the 50- and 42-sample sets using all individual replicate measurement for each assay (n = ≈200) are summarized in Table 2, including the number of measurements for each assay, standard deviation of the mean % bias, minimum and maximum % bias values, and percentage of individual samples within ± 5% bias. Four of the 13 assays evaluated failed to meet the VDSP criterion of mean % bias ≤ ±5%, i.e., Biomérieux I, DIAsource, IDS-EIA, and IDS-iSYS with mean % biases of 9.4%, 16.7%, −8.5%, and 19.5%, respectively, for the 50-sample set. (Biomérieux II passed for the 50-sample set but failed for the 42-sample set.) These four assays failed the criterion even when the high 25(OH)D₂ concentration samples were excluded from the evaluation; however, the mean % biases for the IDS-EIA and IDS-iSYS assays decreased (with IDS-EIA almost achieving the criterion) whereas the mean % biases for the Biomérieux I and DIAsource assays showed a higher mean % bias for the 42-sample subset compared to the 50-sample set.

The mean % bias for each assay for the 50-sample set is shown in Figure 1B as Box and Whisker plots. Similar Box and Whisker plots are provided in Supplemental Figure S3 for the 42-sample subset; comparison of Figure 1B and Supplemental Figure S3A indicates that there are only minor differences. However, when only the eight samples with high concentrations of 25(OH)D₂ are included in the Box and Whisker plots (Figure 1C), significant differences are observed for the Abbott, Biomérieux (I and II), DiaSorin, DIAsource, IDS-EIA, and IDS-iSYS assays. The Fujirebio Inc. and the LC-MS/MS assays appear to have comparable behavior for the samples with high 25(OH)D₂ concentrations. The mean % bias results as Box and Whisker plots for all assays for the 50 single-donor samples as a function of increasing concentration of 25(OH)D are shown in Supplemental Figure 3B, with no apparent trend observed.

The % bias results with all four replicate measurements for each of the 50 single-donor samples are shown in Figure 2 for three assays (Abbott, LC-MS/MS, and Fujirebio Inc.). The results in Figure 2A indicate that the Abbott assay has a significant negative bias (> 30%) for 7 of the 8 samples with concentrations of 25(OH)D₂ > 30 nmol/L. Similar negative biases were observed for the Biomérieux, DiaSorin, and DIAsource assays (see Table 2) but not as pronounced as with the Abbott assay. The immunoassay with the largest percentage of sample measurements within ± 5% bias, Fujirebio Inc. at 49%, is shown in Figure 2B, and the assay performance does not appear to be affected by the high 25(OH)D₂ concentrations. However, the Fujirebio Inc. assay appears to have significant negative biases for low concentrations of 25(OH)D (< 40 nmol/L). A similar mean % bias plot for the LC-MS/MS assay is shown in Figure 2C. Similar plots of % bias for the remaining nine assays are provided in Supplemental Figures S4 to S6. The IDS-iSYS assay has a positive bias for the high concentration 25(OH)D₂ samples (Supplemental Figure S2B).

Figure 2.

Comparison of % bias from NIST target values for individual replicate measurements for 50 single-donor samples using the (A) Abbott (B) IDS-iSYS, (C) Fujirebio Inc., and (D) LC-MS/MS (C) assays. Red circles represent the samples with 25(OH)D₂ concentrations greater than 30 nmol/L. Red solid line represents the mean % bias for all 50 samples; black solid line represents the mean % bias for 42 samples excluding the samples with 25(OH)D₂ concentrations > 30 nmol/L. Dashed blue lines represent ±5% bias from NIST target values.

The four assays that do not meet the VDSP mean % bias criterion, i.e., Biomérieux (I and II), DIAsource, IDS-iSYS, and IDS-EIA, have significantly lower percentages of samples within ≤ ±5% bias (12% and 16%, 15%, 9%, and 17%, respectively) as would be expected (see Table 2). However, six of the assays that do meet the criterion of ≤ ±5% bias still have less than 33% of the individual sample measurements within ≤ ±5% bias. Only Fujirebio Inc. (49%) and the LC-MS/MS assay (58%) approach having approximately 50% of the samples within the ≤ ±5% bias criterion. The Centers for Disease Control and Prevention (CDC) Vitamin D Standardization – Certification Program (VDSCP) uses a criterion of ≤ ±5% bias for the mean of the replicate measurements for each single-donor sample rather than using the individual measurements. The percentage of individual sample measurements of serum total 25(OH)D within various limits of bias (5% to 50%) by assay are summarized in Supplemental Table S6 including the percentage of samples within ≤ ±5% mean bias using the mean of the four replicate measurements. Using the CDC approach does not significantly alter the percentage of values with bias ≤ ± 5% with only 6 of the 13 assays slightly increasing the percentage with ≤ ±5%. Only for the LC-MS/MS assay did the percentage ≤ ±5% bias increase significantly (from 58% to 78%) by using the mean rather than the individual measurements.

3.4. Regression Analysis of Test Assays vs. NIST Target Values

Using the individual replicate measurements for 25(OH)D (n = 200) and the NIST target value for each of the 50 single-donor samples, the ordinary least squares linear regression line and 95% prediction interval were calculated for each test assay [40]. The results of the linear regression analysis (slope and R²) are summarized in Table 3 for both the 50 single-donor samples and for the 42-sample subset. In comparing the linear regression analysis for the 50 samples versus 42 samples in Table 3, only minor differences (< 5%) are observed for the slopes for the LC-MS/MS, Beckman Coulter, and Fujirebio Inc. assays indicating that the samples with high levels of 25(OH)D₂ have minimal effect on the performance of these assays. The Diazyme and Roche assays have minor slope changes (<12%) while the remaining assays show a significant change in regression line slope (16% to 38%) with removal of the high 25(OH)D₂ samples.

Table 3.

Simple Least Squares Linear Regression Analysis for Assays using All Individual Measurements for Each Sample (n = 200)

Assay	50 Samples (nmol/L)						42 Samples excluding High 25(OH)D2 (nmol/L)						Difference 50 – 42 Sample Sets
	SLS Regression			95% PI			SLS Regression			95% PI
	Slope	int.a	R2	Minb	Maxb	Widthc	Slope	Int.a	R2	Minb	Maxb	Widthc	Slope	Widthc	R2
Abbott	0.838	9.01	0.699	−30.1	48.1	78.2	1.157	−7.33	0.958	−22.7	8.0	30.7	−0.321	52.4	−0.260
Beckman Coulter	1.141	−11.5	0.888	−40.3	17.3	57.6	1.089	−8.58	0.834	−38.8	21.6	60.4	0.040	37.5	0.041
Biomérieux I	0.955	10.9	0.621	−41.1	63.0	104.1	1.304	−7.13	0.853	−40.6	26.3	66.9	−0.351	40.4	−0.236
Biomérieux II	0.930	6.26	0.635	−42.7	55.2	97.9	1.284	−12.4	0.876	−42.0	17.1	59.1	−0.353	42.4	−0.245
DiaSorin	0.918	4.24	0.807	−27.7	36.2	63.9	1.114	−5.63	0.899	−29.2	17.9	47.1	−0.198	17.3	−0.094
DIAsource	1.208	−2.07	0.692	−59.4	55.2	114.6	1.514	−17.3	0.826	−61.2	26.5	87.7	−0.310	28.6	−0.137
Diazyme	0.964	0.67	0.817	−31.7	33.0	64.7	1.078	−5.57	0.855	−33.3	22.1	55.4	−0.092	0.2	0.002
Fujirebio Inc.	1.025	−3.57	0.987	−12.4	5.3	17.8	1.065	−5.68	0.986	−13.6	2.2	15.8	−0.050	1.6	0.000
IDS-EIA	0.762	10.2	0.740	−21.9	42.4	64.3	0.902	3.17	0.790	−26.3	−32.6	58.9	−0.142	5.7	−0.052
IDS-iSYS	1.309	−7.66	0.888	−40.7	25.4	66.1	1.095	3.17	0.922	−17.0	23.4	40.4	0.220	28.3	−0.039
Roche	0.999	−1.47	0.929	−21.1	18.2	39.3	1.089	−6.01	0.940	−23.5	11.4	34.9	−0.091	4.1	−0.011
Siemens	1.122	−9.09	0.843	−43.5	25.3	68.8	0.944	0.70	0.826	−26.7	28.1	54.8	0.178	14.5	0.013
LC-MS/MS	1.017	0.03	0.981	−10.0	10.0	20.0	0.998	0.80	0.980	−6.7	8.5	15.2	0.018	0.9	0.002

^aIntercept

^bMinimum and maximum y-intercept for the 95% Prediction Interval

^cWidth of the 95% Prediction Interval

The linear regression analysis plots for the 50- and 42-sample sets using the four replicate measurements are shown in Figures 3 and 4 for the Fujirebio Inc., DiaSorin, Abbott, and LC-MS/MS assays. These plots provide a convenient qualitative visual comparison of the differences in the performance of the various assays. Similar regression analysis plots for the remaining assays are provided in Supplemental Figures S7 to Figure S11. In addition, regression analysis plots using all individual replicate measurements are provided in Supplemental Figures S12 through S15 with the linear regression lines for the 50-sample and 42-sample sets and the identity line (y=x) on the same plot for easy comparison of the changes in slope. As shown in Figure 3 for the Fujirebio Inc. (A and B) and in Figure 4 for the LC-MS/MS (C and D) assays, there are only minor differences between the plots for 50- versus 42-sample sets. Similar minor reductions in prediction interval width are observed for the Beckman Coulter (Supplemental Figures S7A and S7B) and Diazyme (Supplemental Figure S9A and S9B) assays. For the DiaSorin (Figures 3C and 3D) and Abbott (Figures 4A and 4B) assays, however, the width of the prediction intervals decreases significantly with the removal of the high 25(OH)D₂ concentration samples and the slopes increase by 21% and 38%, respectively (see Table 3).

Figure 3.

Results for determination of serum total 25(OH)D in single-donor samples versus the NIST assigned target value for the Fujirebio Inc. (A and B) and DiaSorin (C and D) assays. The four replicate measurements for each of the 50 single-donor sample are represented by open black or red circles. Red circles represent the samples with 25(OH)D₂ concentrations > 30 nmol/L. The shaded area is the 95% prediction interval for the regression line. Plot on the left is for all 50 single-donor samples; plot on the right is for 42 single-donor sample subset excluding the 8 samples with 25(OH)D₂ > 30 nmol/L.

Figure 4.

Results for determination of serum total 25(OH)D in single-donor samples versus the NIST assigned target value for the Abbott. (A and B) and the LC-MS/MS (C and D) assays. The four replicate measurements for each of the 50 single-donor sample are represented by open black or red circles. Red circles represent the samples with 25(OH)D₂ concentrations > 30 nmol/L. The shaded area is the 95% prediction interval for the regression line. Plot on the left is for all 50 single-donor samples; plot on the right is for 42 single-donor sample subset excluding the 8 samples with 25(OH)D₂ > 30 nmol/L.

The removal of the high 25(OH)D₂ samples produces a similar significant reduction in the prediction interval width for the Biomérieux (I and II) (Supplemental Figures S7C, S7D, S8A, and S8B) and DIAsource (Supplemental Figures S8C and S8D) assays. For the remaining immunoassays, the prediction intervals width decreases slightly with the removal of the high 25(OH)D₂ samples i.e., IDS-EIA (Supplemental Figures S10A and S10B), IDS-iSYS (Supplemental Figures S10C and S10D), Roche (Supplemental Figures S11A and S10B), and Siemens (Supplemental Figures 11C and 11D).

3.5. Comparison to Other Studies and VDSP Intercomparison Study 1

A number of recent studies have assessed the performance of various 25(OH)D assays with particular emphasis on recovery for 25(OH)D₂ [8, 41, 42]. Freeman et al. [41] evaluated four 25(OH)D immunoassays (Siemens, DiaSorin, Roche, and Abbott) and compared them with an ID LC-MS/MS assay using samples from donors supplemented with vitamin D₂ over 6-month period. They found that the Siemens, DiaSorin, Roche, and Abbott assays were negatively biased relative to the ID LC-MS/MS assay by −5.7%, −20.3%, −12.1%, and −17.8%, respectively, for supplemented donor samples having median concentrations of 25(OH)D₂ of 57 nmol/L. Our results for the eight samples with 25(OH)D₂ concentrations > 30 nmol/L showed similar biases for the DiaSorin, Roche, and Abbott assays (see Figure 1C and Supplemental Table S5). Bjerg et al. [8] analyzed 200 patient serum samples using seven different assays for 25(OH)D (including Siemens, Roche, DiaSorin, and IDS-iSYS) and reported that all achieved the precision requirement of the VDSP (CV ≤ 10%); however, only the IDS-iSYS and DiaSorin assays achieved an accuracy bias of ≤ ±5 % when compared with results for the analysis of SRM 972a. In our study, Abbott, DiaSorin, and Roche met the VDSP bias criterion, whereas the IDS-iSYS assay failed with a significant overestimation of 25(OH)D₂ as illustrated in Figure 1C and 2B. Even though the Abbott assay met the bias criterion in our study, it had a significant underestimation (negative bias) for 25(OH)D₂ as shown in Figure 1C and Figure 2A. Garnett et al. [42] evaluated the Abbott and Roche assays for their recoveries of 25(OH)D₂ and 25(OH)D₃ and concluded that caution should be used in interpreting results using the Abbott assay in patients supplemented with vitamin D₂.

VDSP Intercomparison Study 1 was a multi-laboratory study with results from 8 immunoassays and 8 LC-MS/MS assays reported [3]. The comparison study reported in this paper was a single-laboratory study using 13 assays (11 unique immunoassays and an LC-MS/MS assay). In the first study, only 50% of the immunoassays met the criterion for CV ≤10% and only three of eight immunoassays achieved the ≤5% bias. The results from this intralaboratory study indicate that there was some improvement in the immunoassay performance with all assays evaluated within the CV ≤10% criterion and 9 of 13 immunoassays achieving the ≤ ±5% bias criterion. To assess the overall assay performance by combining both bias and precision, the mean % bias vs. the %CV for each assay in both Intercomparison Study 1 and this intralaboratory study is plotted in Supplemental Figure S15. Using this plot, it is easy to assess whether a laboratory’s performance meets the VDSP criteria of ≤ 10% for %CV and ≤ ±5% for mean % bias.

4. Conclusions

All 13 assays achieved the VDSP criterion of %CV ≤ 10% while only 9 assays satisfied the criterion of ≤ ±5% mean bias (i.e., Abbott, Beckman Coulter, Biomérieux I, DiaSorin, Diazyme, Fujirebio Inc., Roche, Siemens, and LC-MS/MS). However, 3 of the 9 assays meeting the bias criterion had only 16% to 24% of the individual measurements within ±5% bias indicating a potential need to strengthen the VDSP bias criterion. The presence of a significant number of single-donor samples with high levels of 25(OH)D₂ in this study provided a critical performance test for these assays with ten immunoassays exhibiting some change in response due to the high 25(OH)D₂ concentration samples with the Abbott, Biomérieux, and DIAsource assays having the largest deviations (25% to 38%). Two immunoassays (Fujirebio Inc. and Beckman Coulter) and the LC-MS/MS assay were only minimally affected by the presence of the high 25(OH)D₂ concentration samples. One interpretation of these deviations with high 25(OH)D₂ concentration samples would be that some assays do not have equal response or recovery for both 25(OH)D₂ and 25(OH)D₃. For the 12 immunoassays, the Fujirebio Inc. assay exhibited the lowest %CV and the highest percentage of individual measurements within ±5% mean bias. Expanded assay performance evaluation and comparison with 34 additional assays used in multiple laboratories Intercomparison Study 2 are detailed elsewhere [31, 32].

Highlights.

• Single-laboratory study assessed variability and bias for 12 assays for total serum 25(OH)D

• All 12 assays achieved ≤ 10% CV and 9 achieved ≤ ±5 % mean bias

• Ten immunoassays exhibited changes in response for samples with high concentrations of 25(OH)D₂