Lot-to-Lot Variability in HLA Antibody Screening Using a Multiplexed Bead Based Assay

Abstract

Background

Identifying antibodies to HLA (HLA-Abs) by solid phase assays is used to screen blood donors to mitigate TRALI risk. Various cutoffs for detection assays have been proposed in the literature, however, these do not take into consideration lot-to-lot variability of commercially available assays.

Methods

Samples from 93 non-transfused males were tested using five different lots of a multiplex bead-based HLA-Ab detection kit. A subset of 17 samples was tested on five days using a single lot. An additional 96 samples from donations with varied HLA-Ab levels were tested using kits from two different lots. Results were reported as an NBG (normalized background) ratio.

Results

For the 93 non-transfused donors, NBG values generated using the reference lot were significantly higher than those obtained with three of the four comparator lots. However, for the 96 samples with low, moderate, and higher level HLA-Abs, Class-I values were 1.4 times lower and Class-II values were 1.2 times lower using the reference versus comparator lot. For class-I antibodies the between lot SD was 1.36 (CI:1.19–1.60), while the between day SD was 1.27 (CI:1.08–1.52). Similarly, for class II antibodies the between lot SD was 0.81 (CI:0.70–0.95), while the between day SD was 0.50 (CI:0.43–0.60).

Conclusions

There is inter-lot variability in the tested HLA detection assay as well as significant bias between lots. It may be reasonable to develop a new cutoff when a new lot is obtained.

INTRODUCTION

Transfusion-related acute lung injury (TRALI), which was formally identified by Popovsky and Moore ^1,2, has become and continues to be the most common cause for transfusion related mortality in the United States since 2003 ^3–5. Pathogenesis of TRALI is part of ongoing research, with possible causes including soluble inflammatory mediators ^6,7, antibodies to human neutrophil antigens (anti-HNA) ^8–10, as well as antibodies to human leukocyte antigens (anti-HLA) ^2,11–13. Recognition of this association between antibodies and TRALI led to the publication of an AABB bulletin in 2006 that recommended that blood collection and transfusion centers implement actions to reduce the risk of TRALI ¹⁴. This included minimizing the transfusion of plasma-rich components from donors likely to be alloimmunized to leukocytes (HLA and/or HNA), such as females. Most donor centers now utilize predominantly male plasma; however, using a similar strategy for apheresis platelets would have a significant effect on platelet inventory. It is predicted that using only male platelet donors could result in loss of nearly a third of apheresis platelet collections ¹³. As only a fraction of female donors are alloimmunized, one strategy to minimize this loss is to test female donors who have been pregnant for the presence of antibodies to white cells ¹³. Currently, anti-HNA antibody detection is possible at few centers, with none using a high through put method ¹⁵. However, less technically demanding, high throughput HLA antibody assay platforms are available ^15,16 and are used at many centers to screen blood donors using a wide range of cutoffs ¹⁷. Results across platforms were found to be comparable using a single lot of reagents ¹⁶. As a part of the prospective TRALI study at our center, recipient and donor samples were tested prospectively as they arrived using multiplex bead-based assays on the Luminex platform ¹⁸. Samples were tested over a four year period and used five different reagent lots. Initial analysis of these demonstrated that there was lot-to-lot variability which can potentially effect donor screening ¹⁹. The aim of this manuscript is to provide evidence of lot-to-lot variability in multiplex bead-based assays.

MATERIALS AND METHODS

Samples

Specialized Center of Clinically Oriented Research (SCCOR) samples

Active surveillance of TRALI was conducted at two tertiary care medical centers: the University of California, San Francisco (UCSF) and the Mayo Clinic, Rochester, MN. Surveillance began on March 1, 2006 and ended on December 31, 2009 ¹⁵. All patients over six months of age were prospectively evaluated for hypoxia (PaO2/FiO2 ≤ 300 mmHg) within 12 hours after issue of any blood component from the blood bank, by electronic surveillance of arterial blood gas results and blood bank records in the hospital laboratory information system (Oztech Systems Inc, Burlingame, CA). Because the study was observational, both cases and controls were enrolled without written informed consent. The protocol was approved by the institutional review board at each institution. Serum samples from the same donation were stored at −20C in multiple aliquots. Serum samples from 93 male donors who were never transfused and for which extra aliquots were available were analyzed for lot-to-lot comparison. Of these, a subset of 17 samples with extra aliquots was tested later using lot #15 for five consecutive days for inter-day comparison. Thus all samples analyzed were subjected to one freeze-thaw cycle.

Leukocyte Antibody Prevalence Study (LAPS) samples

LAPS was a prospective cross-sectional six-center study conducted by the Retrovirus Epidemiology Donor Study – II (REDS-II) program of the National Heart, Lung, and Blood Institute. LAPS enrollment and study design have been previously described in detail ¹³. Briefly, donors consenting to the study provided a blood sample for anti-HLA Class I and II testing and a detailed history of pregnancy and transfusion. A total of 8171 (6011 females, 2160 males) donors were enrolled. Females and transfused males were intentionally oversampled. For this analysis, a total of 96 plasma samples were selected from different donations based on strength of HLA antibody on previous testing as described previously for REDS-II LAPS titration analyses ¹⁴. Using a set of lower NBG values previously suggested by the manufacturer, thirty-two samples were selected from donations that were designated as positive for Class I only (CL-I NBG >2.2 and CL-II NBG ≤2.2), 32 from donations that were designated as positive for Class II only (CL-II NBG >2.2 and CL-I NBG ≤2.2), and 32 from donations that were designated as positive for Class I and Class II (both CL-I and CL-II NBG >2.2). Within each of those three groupings, roughly equal numbers of samples were further selected based on two further parameters: anti-HLA antibody screening NBG values and the number of antibody specificities found in the HLA single antigen bead (SAB) test. Screening assay categories were: Low, which included donations with screening NBG values greater than 2.2 but less than or equal to 3SD (CL-I= 10.8; CL-II = 6.9); Medium, which included donations with screening NBG values greater than 3SD but less than or equal to 4SD (CL-I= 25.4; CL-II = 13.8); High, which included donations with screening NBG values greater than 4SD but less than or equal to 5SD (CL-I= 59.3; CL-II = 27.5); and Very High, which included donations with screening NBG values greater than 5SD. As previously reported, single antigen assay categories were based on the number of reactive SAB (1, 2 to 3, 4 or more) identified by the One Lambda LS1A04 or LS2A01 SAB assay using the following cutoffs for determining reactivity: CL-I Median Fluorescence Intensity (MFI) > 2500; CL-II MFI > 1500²⁰. For testing on both lots 13 and 15, a sample that had undergone one freeze-thaw cycle was used.

HLA antibody testing

SCCOR donor serum samples were tested on a periodic basis with the contemporary available lot of beads. Donor samples were screened for HLA Class I antibody (anti-HLA-Class I) and HLA Class II antibody (anti-HLA-Class II), using the LABScreen^™ Mixed (One Lambda Inc., Canoga Park, CA) on the Luminex platform and analyzed using HLA Fusion software (One Lambda) to determine the normalized background ratio (NBG) as per the manufacturer’s suggested protocol. The NBG ratio of the bead yielding the highest value from a given group of beads (e.g. class I beads) was considered as the test result ²¹. Samples with NBG ratio of ≥2.2 were considered as positive as per the manufacturer’s protocol. Samples initially tested on lot 13 (n=93) were later tested on lots 11, 12, 14, and 15. Testing on lots 11, 12, 14, and 15 was performed using the same lot of secondary antibody. For the inter-day comparison, a subset of 17 samples with additional aliquots was tested using lot 15 at a later period for five consecutive days. During this time the thawed aliquots were stored at 4C. LAPS samples were tested only using lots 13 and 15.

Statistical analysis

The data were summarized using means and standard deviations or medians along with the 25^th and 75^th percentiles, as appropriate. For both Class I and Class II, lot 13 was compared with other lots. Paired differences were calculated between the lots being compared, and the Wilcoxon signed-rank test was used to assess whether the differences were significantly different from zero. In light of multiple statistical testing, p-values less than 0.01 were considered statistically significant. Pearson correlation was used to calculate the R² value for the correlations between or within lots. Scatter plots were used to graphically compare each lot to the other along with an identity line (x=y) to illustrate the bias between the lots, if any. Bias for each comparison was calculated using the Bland-Altman method. The variability between days as well as between lots was assessed using random-effects models from which the variance components (along with their 95% confidence intervals) were estimated. The standard deviations (and their confidence intervals) were estimated by taking the square root of the variance component estimates. The residual variance in each model captured the day-to-day (or lot-to-lot) variability separately from the between-sample variance. Analyses of SCCOR samples were conducted using SAS version 9 (Cary, NC), and figures were generated using R (http://www.r-project.org/) or Excel. Analyses of LAPS samples were conducted using GraphPad Prism (La Jolla, CA).

RESULTS

Lot-to-lot variation in bead composition

The number of beads and number of distinct antigens displayed per bead across lots varied (Table 1)²². Lots 11–13 had eight beads with Class I antigens and three beads with Class II antigens, each with multiple cell lines. Lot 13 had six cell lines per bead for Class I as compared to seven cell lines per bead for lots 11 and 12. Each cell line represented six HLA antigens, resulting in a decrease in the total HLA specificities for lot 13 as compared to lots 11 and 12. Although Class I specificities for lots 14 and 15 were similar, some of the cell lines were different between the two lots. In addition lots 14 and 15 included cell lines expressing HLA-DP, thus enabling the detection of antibodies to HLA-DP antigen.

Table 1.

Characteristics of bead lots

	Class I					Class II
	# of beads	Cell lines/bead	Total Specificities			# of beads	Cell lines/bead	Total Specificities
			HLA-A	HLA-B	HLA-C			HLA-DR	HLA-DQ	HLA-DP
Lot#11	8	7	84	84	84	3	8	75	45	0
Lot#12	8	7	84	84	84	3	8	75	45	0
Lot#13	8	6	70	73	68	3	8	78	46	0
Lot#14	12	3	71	72	67	5	5	81	48	45
Lot#15	12	3	71	72	67	5	5	82	47	44

Distribution of Class I and II NBG ratios for non-transfused males

A total of 93 samples were included in the analysis, and the value of the bead with the highest NBG class I or II ratio for each sample was analyzed. After truncating the negative values to zero, we observed that the distributions were not normal (Table 2). The median values ranged from 0.9 to 1.2 and the interquartile ranges (Q3-Q1) were 0.8 to 1.8 for class I, and the median values ranged from 0.4 to 1.0 and the interquartile ranges (Q3-Q1) were 0.8 to 1.3 for class II.

Table 2.

Distribution of raw data (using maximum for the sample within each lot)

	Class 1 (N=93)	Class 2 (N=93)
Lot 11
Mean (SD)	1.5 (2.2)	0.6 (0.7)
Median	0.9	0.4
Q1, Q3	0.6, 1.4	0.0, 0.9
Range	(0.0–14.3)	(0.0–3.3)
Lot 12
Mean (SD)	1.5 (2.2)	0.6 (0.7)
Median	1.0	0.5
Q1, Q3	0.6, 1.5	0.0, 0.9
Range	(0.0–17.5)	(0.0–3.0)
Lot 13
Mean (SD)	2.1 (2.7)	0.7 (0.6)
Median	1.2	0.7
Q1, Q3	0.8, 2.4	0.2, 1.1
Range	(0.0–14.6)	(0.0–2.5)
Lot 14
Mean (SD)	2.1 (2.9)	1.1 (0.8)
Median	1.2	1.0
Q1, Q3	0.8, 2.2	0.6, 1.5
Range	(0.0–17.4)	(0.0–4.5)
Lot 15
Mean (SD)	1.4 (1.7)	0.6 (0.8)
Median	1.0	0.4
Q1, Q3	0.6, 1.5	0.0, 0.8
Range	(0.0–12.4)	(0.0–4.6)

Note: Negative values set to zero.

Comparison of Class I values between lots for non-transfused males

Lot 13 differed significantly from all lots with the exception of lot 14 (Table 3). The values generated using lot 13 was significantly higher compared to lots 11, 12, and 15 (see Table 3). Lot 14 tended to give slightly (although not significantly) higher results. The strongest linear associations among the lots were noted between lots 13 and 14 (R² = 0.907) and also between lots 11 and 12 (R² = 0.913). All remaining R² values were less than 0.90 (Fig. 1).

Table 3.

Comparisons to lot 13

	Class 1 (N=93)	Class 2 (N=93)
Lot 11 - Lot 13
Mean (SD)	−0.6 (1.31)	−0.1 (0.51)
Median	−0.3	−0.1
Q1, Q3	−0.9, 0.0	−0.3, 0.0
Range	(−6.9–5.6)	(−2.5–1.2)
Signed-Rank P-value	<0.001	0.006
Lot 12 - Lot 13
Mean (SD)	−0.6 (1.66)	−0.1 (0.45)
Median	−0.3	−0.1
Q1, Q3	−1.0, 0.0	−0.4, 0.0
Range	(−7.1–8.9)	(−1.2–1.8)
Signed-Rank P-value	<0.001	<0.001
Lot 14 - Lot 13
Mean (SD)	0.0 (0.87)	0.4 (0.54)
Median	0.0	0.3
Q1, Q3	−0.2, 0.2	0.0, 0.6
Range	(−3.6–4.8)	(−0.6–4.1)
Signed-Rank P-value	0.51	<0.001
Lot 15 - Lot 13
Mean (SD)	−0.7 (1.59)	−0.1 (0.58)
Median	−0.2	−0.1
Q1, Q3	−1.1, 0.0	−0.3, 0.0
Range	(−7.5–3.8)	(−2.5–2.2)
Signed-Rank P-value	<0.001	0.001

Note: Negative values set to zero prior to calculating differences

Figure 1.

Scatter plots demonstrating the correlation of Class I NBG ratios between lots for the SCCOR samples with identity (x=y) line. R-squared and p values (Wilcoxon signed-rank test) for each comparison are indicated in the corresponding boxes. Histogram represents the distribution of NBG ratios for each lot.

Comparison of Class II values between lots for non-transfused males

Similar to class I, all lots differed significantly from lot 13 (Table 3). Lot 13 yielded higher values compared to lots 11, 12, and 15, and lower values compared to lot 14. The linear associations were much weaker between class II lots compared to what was observed in class I (R² ranged from 0.11 to 0.799, Fig. 2).

Figure 2.

Scatter plots demonstrating the correlation of Class II NBG ratios between lots for the SCCOR samples with identity (x=y) line. R-squared and p values (Wilcoxon signed-rank test) for each comparison are indicated in the corresponding boxes. Histogram represents the distribution of NBG ratios for each lot.

Comparison of lots 13 and 15 using plasma samples from subjects with a wide range of HLA antibody strength

To replicate the above findings, lots 13 and 15 were compared using REDS-II LAPS samples that had been aliquoted and frozen previously. The samples were originally tested using lot 13, and when lot 15 became available these samples were re-tested using a replicate aliquot with the same freeze-thaw history. The samples were selected for comparison based on availability and to span a wide range of signal strength on the assay (see Methods). Consistent with the greater number of beads in lot 15 compared to lot 13, values for class I and class II antibodies fell predominantly above the line of identity upon testing with lot 15 (Fig. 3). Pearson correlation of test values between the lots showed R² values of 0.85 and 0.87 for class I and II, respectively. Bland-Altman analysis revealed a bias toward lower NBG ratios for class I (bias −23.1; 95% CI −79.8 to 33.6) and class II (bias −12.0; 95% CI −57.5 to 33.5). Of note, the direction of bias between lots 13 and 15 was opposite using the SCCOR vs. the LAPS samples. As the SCCOR samples were selected to contain very low values, we reanalyzed the LAPS samples including only those samples with NBG ratio <15 on original testing with Lot 13 to match the signal range seen in the SCCOR samples (Fig. 4). The bias was still negative for both class I and II antibodies in LAPS samples at the low end of the signal range, opposite to what was found in the SCCOR samples. This finding led us to question how much of the variability and bias we measured was due to between lot differences and how much due to inherent variability of the assay.

Figure 3.

Correlation of NBG ratios (Class I and II) for lots 13 and 15 for the LAPS samples. Left panels show scatter plots, with the line of identity shown in red. Right panels show Bland-Altman plots, with 95% confidence intervals of the bias shown in dotted lines.

Figure 4.

Bland-Altman plots for Class I and Class 2 samples between lot 15 and lot 13 with 95% confidence intervals of the bias shown in dotted lines. Plots on the left panel represent subset of samples with NBG ratio <15.0 and those on the right panel represent the SCCOR samples

Within lot variability

To understand the significance of between lot variability in the multiplex assay, it is important to know the within lot variability. Therefore we tested the samples from 17 of the 93 non-transfused males with detectable HLA antibodies in five separate assay runs using lot 15. The variability of the values for these samples from lot-to-lot was compared to the variability from day-to-day using the same lot. For class I antibodies the between lot SD was 1.36 (CI: 1.19–1.60), while the between day SD was 1.27(CI: 1.08–1.52). Similarly, for class II antibodies the between lot SD was 0.81 (CI: 0.70–0.95), while the between day SD was 0.50 (CI: 0.43–0.60). Although, the day-to-day variability of the assay appears to be approximately the same for class I antibodies and slightly lower for class II antibodies compared to the lot-to-lot variability, it is not possible to directly compare the between lot variability with the between-day variability as not all samples were run across all lots and additionally assays were run on two occasions that were months apart.

DISCUSSION

The interpretation of HLA antibodies detected by solid phase assays for TRALI is evolving, with studies suggesting establishment of a cut-off that is significantly higher than used in the organ transplant field, especially for highly sensitive methods using multiplexed bead based assays analyzed on a flow cytometer ^13,15,23,24. A recent study demonstrated that different platforms may yield variable results; however, comparisons can be made across platforms using statistical and mathematical calculations ¹⁴. The significant finding in this study is that for assays using microbeads on a flow analyzer results can vary significantly from one lot to another, but that this variability does not appear greater than the inherent day-to-day variability of the assay. A theoretical reason for variability from lot-to-lot is the significant change in the bead configuration from one lot to another (Table 1). For example, lots 11–13 had no DP antigen represented in them and thus would not detect DP antibodies that can be found in blood donors ²⁰ and are considered to be of clinical significance in transplantation ²⁵. For lots 11–13, 8 beads with class I antigens and 3 beads with class II antigens represent the spectrum of HLA antigens, while for lot 14 and 15, 12 beads with class I antigens and 5 beads with class II antigens constitute the complete set. This results in all eight beads representing HLA-A1 antigen in lots 11 and 12, while only five beads represent HLA-A1 antigens on lots 14 and 15. This probably leads to different signal strengths with the same amount of antibody as seen in this study.

This study possessed several limitations, which will be important to note prior to extrapolating these results to clinical blood bank practice. As there was limited availability of older lots it was not possible to test all samples on all lots. Another limitation of this study is the small number of samples that were tested. In addition, as there were many results with a negative value for the lots tested using non-transfused male samples, it was not possible to perform a multivariate analysis comparing each lot to one another. We thus compared lot 13 to other lots as the majority of our samples were run on lot 13.

In conclusion, there is significant variability in NBG ratios from the same sample when compared between different lots of multiplexed beads from the same manufacturer on a flow cytometric platform, and the degree of variability was of the same order of magnitude as within lot variability. Significant bias was detected using replicate samples tested with different lots, though the direction of bias differed when tested in two different laboratories. As these values are used to determine the assay cutoff, it may be reasonable to develop a new cutoff when a new lot is obtained.