Analytical assessment and validation of the ProteinSimple ELLA serum B-cell maturation antigen assay

Abstract

Objectives

Soluble B-Cell Maturation Antigen (sBCMA) is a degradation product of plasma cell-bound BCMA found in serum. Serum sBCMA concentrations correlate with bone marrow plasma cellularity, making it an attractive biomarker for monitoring plasma cell disorders, such as multiple myeloma. Here we evaluated the automated BCMA immunoassay for the ProteinSimple ELLA, for the analysis of sBCMA.

Design & methods

Inter and intra-run precision was assessed through replicate sBCMA measurements at 3 different concentration levels. Linearity was determined through serial dilution of a high sBMCA patient sample. Accuracy was assessed through split specimen analysis on two separate lots of reagents. Stability was assessed at 3 temperature levels over 14 days. Cross-reactivity was assessed on BCMA targeting and non-targeting chemotherapeutics. A reference range was established through the analysis of 146 healthy donor samples. The effect of endogenous interferents was assessed through spiking and recovery studies.

Results

Inter and intra-run precision studies afforded CVs of <10% at all three concentration levels. Analytical measurement range was confirmed from 0.1 to 7 ng/mL. Accuracy studies afforded a slope of 0.976, intercept of 1.22, R² of 0.996. Assayed sBCMA values were unaffected by endogenous interferents and non-BMCA targeting antibodies. BCMA targeting therapeutics negatively affected assayed sBCMA concentrations. The reference range was established at 19–58 ng/mL sBCMA is analytically stable.

Conclusions

The ProteinSimple ELLA sBCMA assay shows acceptable performance for the clinical assessment of sBCMA. The assay was highly affected by BCMA targeting therapeutics, thereby patients undergoing this therapy should not have their sBCMA levels assessed by this method.

Highlights

• •Complete analytical validation of the ProteinSimple ELLA BCMA assay.
• •Established an SBCMA reference interval at 19–55 ng/mL.
• •SBCMA assays are subject to interference from BCMA targeting chemotherapeutics.

1. Introduction

Multiple myeloma is a plasma cell dyscrasia characterized by the clonal expansion of terminally differentiated plasma cells and the uncontrolled secretion of monoclonal immunoglobulin (M-protein) [1]. Malignant plasma cells reside within the bone marrow, and disease progression can be monitored by the direct assessment of bone marrow cellularity through bone marrow biopsies [2]. Bone marrow biopsies are extremely invasive to the patient and preferentially avoided, as such, multiple myeloma is typically monitored indirectly, through measurement of plasma cell secreted M-Protein This is accomplished through a battery of laboratory testing including serum protein electrophoresis, (SPEP) serum immunofixation, (IFE) and free light chain analysis [3,4]. This extensive laboratory testing is time-consuming, labor-intensive, and requires considerable experience in their interpretation. Additionally, M-protein has a half-life of approximately 3 weeks, meaning treatment effectiveness will not be apparent for several weeks after the induction of a new therapy [5]. Furthermore, a separate population of multiple myeloma patients secrete no measurable M-protein, termed non-secretory multiple myeloma [6]. For these aforementioned diagnostic and analytical complexities, a single easy-to-measure biomarker to monitor multiple myeloma disease progression would be advantageous.

B-cell maturation antigen (BCMA) is a transmembrane protein highly expressed on mature plasma cells. BCMA is encoded by a TNFRSF17 gene located on the short arm of chromosome 16 (16p13.13) and is critical for B-cell differentiation into plasma cells as well as B-cell regulation, and survival [7]. BCMA is 184 residues in total length, consisting of a 130-residue transmembrane portion, and a 54-residue extracellular portion [8]. BCMA has two major ligands, B-Cell Activation Factor (BAFF) and A Proliferation Inducing Ligand (APRIL) [9,10]. The extracellular portion of BCMA can be cleaved by the action of γ-secretase, forming soluble (or serum) BCMA (sBCMA) [11].

BCMA is expressed solely on plasma cells, and BCMA expression is upregulated during multiple myeloma pathogenesis, making it is an attractive target for anti-B-cell therapeutics [12,13]. Increased levels of BCMA expression affords greater levels of γ-secretase mediated BCMA cleavage, increasing sBCMA concentrations [14,15]. SBCMA levels have been shown to trend with bone marrow plasma cell concentration in both multiple myeloma patients and non-secretary multiple myeloma patients [16]. Furthermore, in patients undergoing treatment for myeloma, changes in M-protein levels correlate with sBCMA levels [16]. As such, sBCMA can be used as a biomarker to monitor patients with multiple myeloma, potentially supplanting complicated laboratory testing and testing algorithms.

Current methods of measuring sBCMA concentration are limited to ELISA assays [17]. ELISA assays are generally manual procedures that are time consuming to perform, and have the propensity for operator error that decreases their analytical precision and accuracy compared to automated platforms. In addition, published evaluations of the analytical performance of these sBCMA ELISAs is limited. Multiple myeloma is the second most common blood cancer worldwide, thereby, integration of sBCMA into myeloma testing algorithms would require an assay platform with high sample throughput to be clinically useful [18]. As such, an analytically stable, automated platform to assess sBCMA concentration is needed to effectively integrate sBCMA values into myeloma clinical workflows.

The ProteinSimple ELLA is an automated microfluidics-based immunoassay system [19]. The assay reagents, including the capture and detection antibodies, as well as the assay fluorophores, are all contained within a single assay cartridge. The analytical process is nearly completely automated, apart from sample preparation and loading steps, of which 72 samples can be tested during each cartridge run. The analytical procedure is performed via automated microfluidics, and the assay signal is generated in the form of fluorescence, where relative fluorescence is proportional to sBCMA concentration via a cartridge-specific manufacturer-supplied internal calibration curve [19]. The high sample throughput of the ELLA makes it well-suited for integration into high-patient census myeloma clinics.

In this study, we performed a compressive validation of the sBCMA assay for the ProteinSimple ELLA evaluating precision, accuracy, linearity, stability, and analytical sensitivity, including limit of detection (LoD), quantification (LoQ) and blank (LoB). In addition, a reference range was established, and the analytical effect of chemotherapeutic drugs and common interferences were also assessed.

2. Methods

2.1. Reagents and general procedure

All sBCMA measurements were performed on the ProteinSimple ELLA platform using SinglePlex 16 or 72 well BCMA (TNRF117) reagent cartridges. All samples were diluted at a minimum of a 1:100 dilution, except where noted. All dilutions were performed using the manufacturer-supplied diluent, SD13. Samples beyond the analytical range of the analyzer were further diluted and re-assayed. After dilution, samples, as well as manufacturer-supplied wash buffer, were loaded into their respective wells within the assay cartridge. Each sample well requires 50 μL of diluted sample or control. The cartridge is then placed into the ELLA for analysis. The analytical process is fully automated, lasting approximately 72 min.

2.2. Precision

Both inter and intra-run precision was assessed by though replicate measurements of 3 levels of sBCMA. High, mid, and low sBCMA levels were obtained from three separate patients with varying endogenous sBCMA levels, Intra-run precision was assessed through 10 replicates on a single sBCMA cartridge, while inter-run precision was determined through 25 replicates across 16 different cartridges assayed on 8 separate days. Precision data was represented in terms of % coefficient of variation (CV).

2.3. Linearity

Assay linearity was confirmed using a patient serum sample containing an elevated endogenous sBCMA level. The patient serum was initially diluted at 1:300, then serially diluted at the following dilution factors: 1:4, 1:16, 1:64, 1:256, 1:1024, 1:3072, and 1:6144. The diluted samples were measured in triplicate on a single reagent cartridge. The experimentally obtained values were then compared to the expected values for each dilution, where the expected value was calculated with respect to the initial diluted (1:300) sample.

2.4. Analytical sensitivity

The LoB, LoD, and LoQ were all calculated using recommendations from CLSI guidelines EP17-A2. The LoB was determined by creating 5 sBCMA serum samples at a near zero value, while the LOD was determined by creating 5 low-concentration sBCMA samples. The LoQ was determined by creating 5 sBCMA samples near the expected LoQ value. Each sample was assayed for 3 replicates on 3 different days, affording 15 values for each concentration level (45 values in total). LoB and LoD results were calculated using a parametric approach, where the 95th percentile was determined to be LoB and LoD. The LoD was calculated similarly, again using a parametric analysis, where the mean value was determined as the LoD. The LoQ was calculated as described in EP17-A2 where the total imprecision is <10% CV.

2.5. Reference interval

The reference interval was established by analyzing 146 samples obtained from healthy volunteers for sBCMA concentration. Reference interval samples were obtained from Mount Sinai Hospital (MSH) employees through the MSH Employee Health Service. The sample cohort consisted of 53 males and 93 females, ages 23–77 years old, median age: 36. All patients were self-described as healthy, and were of sufficient health to pass a pre-employment health screen administered by MSH Clinicians. The reference interval was established following CLSI C28-A3c guidelines, a nonparametric central 95% reference interval, where the lower and upper limits were defined as the 2.5th and 97.5th percentiles, respectively.

2.6. Cross-reactivity with chemotherapeutics

Cross-reactivity was assessed in 5 monoclonal therapeutic antibodies: Daratumumab, AMG 701, Talquetamab, Isatuximab, and Belantamab. Residual antibody, reconstituted per the manufacturer’s instructions, was obtained from the MSH Infusion Service. Residual serum was obtained from a patient with a high endogenous sBCMA level. Antibody was then spiked into the serum at increasing concentrations, and allowed to incubate at room temperature for 20 min, the samples were then assayed for sBCMA concentration in triplicate and the mean sBCMA concentration was reported. Antibody concentrations were as follows, represented in terms of percent antibody to the total solution (vol/vol). Talquetamab, AMG 701: 2%, 5%, 8%, 10%, 50%; Daratumumab, Isatuximab: 5%, 10%; Belantamab: 1%, 5%. Percent recovery and cross-reactivity of the sBCMA samples was determined by comparing sBCMA concentrations of spiked samples to sBCMA concentrations obtained from the unspiked samples, compensating for expected dilutional effects.

2.7. Endogenous interferents

The effect of common interferents, hemolysis, lipemia, and icterus was assessed through a recovery study. High and low concentrations of the interferent were created from a concentrated interferent stock (Sun diagnostics; PN# Int-01) and SD13. 10ul of each interferent stock was combined into 90ul of a healthy donor serum sample containing endogenous sBCMA within our established reference interval (discussed below). A corresponding ‘normal’ sample was created by combining 10ul of SD13 into an additional 90ul aliquot of the normal donor serum. Each sample was assessed for sBCMA in triplicate and the mean sBCMA concentration was reported. The percent difference in sBCMA concentration between the interferent spiked sample and the unspiked sample was determined.

2.8. Stability

Stability was assessed at both within and above our established reference range at sBCMA concentrations of 26.4 ng/mL and 367.4 ng/mL. Stability samples were aliquoted and incubated at the following either 22 °C, 4 °C, or −70 °C. Samples were incubated for 24 h, 72 h, 7 days, and 14 days. After each incubation period, each aliquot was assessed for sBCMA concentration in triplicate and the mean sBCMA concentration was reported. Results are represented as percent change from the original sBCMA concentration.

2.9. Accuracy

Accuracy was determined by Alternative Performance Assessment. To our knowledge, there are no other laboratories measuring sBCMA on the ProteinSimple ELLA. Accuracy was assessed by measuring sBCMA concentration on 48 patient specimens on two different lots of simplex cartridges, each cartridge with its own distinct internal calibration curve. Regression analysis was performed to determine the correlation between the two lots of sBCMA reagents. In addition, to assess the comparability between the ELLA and BCMA ELISAs, we performed a split specimen analysis using 38 specimens on both the ELLA and the Invitrogen Human BCMA/TNFRSF17 ELISA Kit (Thermo-Fisher).

3. Results

Precision studies are summarized in Table 1, both inter and intra-run precision studies resulted in %CV’s of ≤10% for all concentrations examined. Regression analysis of the accuracy study, shown in Fig. 1, afforded a slope of 0.976, with an intercept of 1.22 and a correlation coefficient (R²) of 0.996. Accuracy specimens analyzed spanned a range of 8.2–756.8 ng/mL. The analytical measurement range was confirmed between 0.1 and 7 ng/mL, resulting in a slope of 1.002 with an intercept of 0.071 and an R² of 0.997, this data is shown in Fig. 2. The LoB was determined to be 0.005 ng/mL, while the limit of LoD and LoQ were established at 0.007 and 0.014 ng/mL, respectively. The reference interval was established at 19–58 ng/mL. Comparison studies between the ELLA and ELISA methodologies afforded a slope of 1.314, intercept of 138.5 and an R² of 0.59 (Supplemental).

Table 1.

Precision studies.

Precision Studies
Intra-Run Precision (n = 10)	Mean	CV
High	370.5 ng/mL	5.5%
Mid	135.9 ng/mL	3.3%
Low	25.8 ng/mL	6.9%
Inter-Run Precision (n = 25)	Mean	CV
High	336.0 ng/mL	10.0%
Mid	135.8 ng/mL	7.4%
Low	27.1 ng/mL	7.0%

Fig. 1.

Split specimen analysis was performed using two different lots of reagents, 1:1 line is shown.

Fig. 2.

SBCMA linearity study was performed through serial dilution using patient with an elevated endogenous sBCMA concentration, 1:1 line is shown.

The effect of common chemotherapeutics on assayed sBCMA are listed in Table 2. Results are notable for a nearly 100% recovery in spiking studies with Daratumumab, Isatuximab, and Talquetamab, while a <20% recovery was found in Belantamab and AMG 701 spiking studies. The effect of common endogenous interferents is shown in Table 3. The assay showed no major change in assayed sBCMA concentration due to hemoglobin, icterus, or lipemia. Stability studies are summarized in Table 4. All stability studies resulted in analyte recovery within 15%.

Table 2.

Cross-reactivity analysis.

Cross Reactivity Studies		sBCMA Concentration of 70.0 ng/mL
Drug Name	Drug Concentration (% of total solution)	Percent Cross -Reactivity	Percent Recovery
Daratumumab	5%	3%	103%
	10%	−4%	96%
Isatuximab	5%	2%	101%
	10%	−4%	97%
Talquetamab	2%	1%	101%
	5%	4%	105%
	8%	−1%	103%
	10%	0.4	105%
Belantamab	1%	−1875%	5%
	5%	−2477%	4%
	10%	−2168%	5%
AMG 701	2%	−474%	18%
	5%	−923%	10%
	8%	−966%	10%
	10%	−998%	10%

Table 3.

Effect of Endogenous interferents.

Endogenous Interference Studies		sBCMA Concentration of 23.6 ng/mL
Interferent	Final Interferent Concentration	% Change
Lipemia	79 mg/dL	7.3%
Triglyceride Spiked	212 mg/dL	3.6%
Icterius	0.9 mg/dL	1.1%
Bilirubin Spiked	6.3 mg/dL	0.4%
Hemoglobin	60 mg/mL	5.8%
Hemolysate Spiked	403 mg/mL	2.6%

Table 4.

Stability studies.

Stability Studies		sBCMA Concentration of 26.4 ng/mL	sBCMA Concentration of 367.4 ng/mL
Temperature	Time Point	% Change	% Change
22°C	24 Hours	101.4%	104.3%
	72 Hours	103.9%	94.1%
	7 Days	109.2%	104.8%
	14 Days	97.0%	92.8%
4°C	24 Hours	106.9%	99.7%
	72 Hours	109.5%	94.3%
	7 Days	101.7%	92.2%
	14 Days	102.7%	92.2%
−70°C	24 Hours	106.9%	99.7%
	72 Hours	109.5%	94.3%
	7 Days	101.7%	92.2%
	14 Days	102.7%	92.2%

4. Discussion

The ProteinSimple ELLA showed excellent precision at sBCMA concentrations both within and well above the normal reference interval, with resulting CV’s of less than <10% in both the inter and intra-run precision studies (Table 1). This level of precision is likely due to the assay being completely automated, apart from a manual pre-analytical sample dilution and introduction step, making it much more robust than typical ELISA assays.

Each ELLA cartridge contains a cartridge-specific internal calibration curve, in which the manufacturer claims a linear measuring interval between 1 and 10 ng/mL. Our linearity studies confirmed a smaller linear range of 0.1–7 ng/mL, whereas linearity studies at the high end of the manufacturer’s claimed range showed limited recovery. This smaller linear range should have little effect on the use of the assay clinically, as we have confirmed, with sample dilution, our CRR to be as high as 7000 ng/mL (Data not shown) In our experience, very few patients, including myeloma patients with active or progressive disease, will have an sBCMA level above our confirmed CRR. Conversely, we have confirmed our LoB, LoD, and LoQ to be acceptable for clinical use of the assay, as all three indices all well below our lower level of linearity of 0.1 ng/mL. This low level of measurable sBCMA could prove analytically useful when assessing minimal residual disease (MRD) status in myeloma patients, where low, or near zero sBCMA values could be indicative of complete myeloma remission. However, use of sBCMA in MRD testing could prove challenging in that in that our reference range studies shows that healthy patients have a relatively high level of endogenous sBCMA, discussed below. Furthermore, lot-to-lot accuracy studies showed acceptable accuracy spanning the entire measurement interval.

Reference interval studies for sBCMA in the literature are limited. There are no published sBCMA reference interval studies in the literature performed using the ELLA, apart from a manufacturer-performed study affording a normal reference interval of 22–41 ng/mL (n = 10) [20]. As such, we chose to establish a normal reference interval for sBCMA on the ELLA. Our reference interval studies established a surprisingly wide range for normal endogenous sBCMA levels at 19–58 ng/mL, with a median sBCMA value of 33.0 ng/mL (n = 146). Sex stratification of the interval yielded similar results, with males affording an interval of 20–54 ng/mL (n = 54) and females yielding an interval of 14–56 ng/mL (n = 92). Our results are similar to a prior published study using a different sBCMA ELISA assay, which established a normal sBCMA interval of <82.59 ng/mL with a median sBCMA value of 37.51 ng/mL [21].

Stability studies showed acceptable specimen stability at three different temperature levels for at least 14 days (Table 4). This level of analyte stability makes the ELLA sBCMA acceptable for use in a large outpatient health system or a reference lab setting, in that the analyte does not require immediate analysis, or esoteric specimen handling and transport conditions. Longer-term specimen stability is likely possible in that quality control material, supplied by the manufacturer, is stable for upwards of 6 months at −70 °C. Furthermore, the analyte was unaffected by two freeze/thaw cycles (data not shown).

The effect of endogenous and chemotherapeutic interferents on assayed sBCMA concentrations was quite varied. Although assayed sBCMA concentrations were unaffected by endogenous interferences, hemolysis, icterus, and lipemia, effects from common myeloma chemotherapeutics varied extensively.

Recovery studies were performed using both BCMA-targeting therapeutics and non-BMCA-targeting therapeutics. CD-38 targeting therapeutics, Daratumumab and Isatuximab, and the bi-specific Talquetamab, targeting GPRC5D and CD3, showed no interference on assayed sBCMA concentration, affording a nearly 100% recovery for all three antibodies. However, studies performed using BCMA targeting therapeutics, Belantamab and AMG 701, extensively interfered with assayed sBCMA values, yielding a <20% recovery for both antibodies at concentrations as low as a 2% v/v spike. SBCMA is a very small protein, 54 residues long, with limited epitopes for antibody binding. It is likely that both the therapeutic anti-BCMA antibody, and the capture antibody, immobilized within the ELLA cartridge, are competing for the same, or very similar, binding targets. Furthermore, steric hindrance between the ELLA capture and the therapeutic antibody might prevent effective binding of the capture antibody on the small sBCMA peptide [22,23]. Clinically, in a patient undergoing anti-BCMA myeloma treatment, the drug will always have the opportunity to bind sBCMA prior to analysis, thereby making this interference difficult to avoid. Although we were unable to test for interference in all BCMA targeting, it is reasonable to surmise, however not guaranteed, that other BCMA targeting therapeutics will behave in a similar manner. For this reason, we have chosen to include a comment to the clinician with every sBCMA result, that patients undergoing therapy with anti-BCMA therapeutics, should preferentially have their BCMA levels assayed by a different method, and that their results should be interpreted with caution.

There are few published reports on sBCMA analytical assessment in the literature. Numerous reports have been published on the efficacy of sBCMA as a tumor marker for myeloma, including risk stratification for progression from MGUS to myeloma, and for sBCMA use as a surrogate for SPEP and IFE in myeloma monitoring [16,17]. Additional studies have been performed suggesting that sBCMA can bind anti-BCMA therapeutics, limiting therapeutic effectiveness and yielding sBCMA analytical issues [24,25]. However, these studies however, have all used an sBCMA targeting ELISA to determine sBCMA concentrations. ELISA assays, though relatively simple to perform, are extremely slow and require numerous manual steps, decreasing both turn-around times and sample throughput, as well as introducing the possibility for operator analytical errors. Integration of such an assay into a large myeloma clinic would require excessive staffing, or the use of an automatic liquid handler, incurring additional costs and analytical complexities. The ELLA is at a distinct advantage in it circumvents all of these issues associated with manual ELISA systems, making it appropriate for use in a large-scale myeloma clinic. Our comparative studies between the ELLA and a BCMA ELISA shows poor correlation between the two assays, slope: 1.314, R²: 0.59, suggesting that BCMA values cannot be used interchangeably between assay methods (Supplemental).

5. Conclusion

The ProteinSimple ELLA BCMA assay is an acceptable platform for use in monitoring sBCMA levels. The minimal preparation and analytical time make it acceptable for use in large hospital clinics requiring a large sample throughput. The assay shows acceptable analytical performance in all indices assessed, however, the assay has the potential to be greatly affected by BCMA targeting therapeutics. Future studies will be performed to assess methods or workflows to circumvent this analytical interference, as well as to assess the interface in additional and newly developed anti-BCMA antibodies.

Funding

This work was funded by the Mount Sinai Hospital Clinical Chemistry Service.

CRediT authorship contribution statement

Daniel Conrad Kirchhoff: Conceptualization, Formal analysis, Supervision, Validation, Writing – original draft, Writing – review & editing, Data curation. Wei Zhang: Data curation. Athanasia Chandras: Data curation, Formal analysis. Damodara Rao Mendu: Formal analysis, Investigation, Project administration, Supervision, Visualization, Writing – review & editing.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Appendix A. Supplementary data

The following is the Supplementary data to this article:

Data availability

Data will be made available on request.