Flow Cytometry Quality Requirements for Monitoring of Minimal Disease in Plasma Cell Myeloma

Abstract

Current therapeutic approaches for plasma cell myeloma (PCM) attain an overall survival of more than 6 years for the majority of newly diagnosed patients. However, PFS and OS are the only accepted FDA clinical endpoints for demonstrating drug efficacy before they can be become frontline therapeutic options. There is, however, recognition that the increasing gap between drug development and approval for mainstream therapeutic use needs to be shortened. As such regulatory bodies such as the FDA are now considering whether biomarker response evaluation, as in measurement of minimal residual disease (MRD) as assessed by flow cytometry (FC), can provide an early, robust prediction of survival and therefore improve the drug approval process. Recently, FC MRD using a standardized eight-color antibody methodology has been shown to have a minimum sensitivity of 0.01% and an upper sensitivity of 0.001%. To ensure that all laboratories using this approach achieve the same levels of sensitivity it is crucially important to have standardized quality management procedures in place. This manuscript accompanies those published in this special issue and describes the minimum that is required for validating and quality monitoring of this highly specific test to ensure any laboratory, irrespective of location, will achieve the expected quality standards required.

Current therapeutic approaches for plasma cell myeloma (PCM) attain an overall survival of more than 6 years for the majority of newly diagnosed patients (1–3). Novel effective treatments are being developed at an unprecedented rate for these patients but the transition to mainstream availability is much slower (1–5). This is because randomized phase three clinical trials take years to show a benefit when using progression free survival (PFS) and overall survival (OS) as study endpoints. Currently, PFS and OS of patients with PCM are the only accepted FDA clinical endpoints for demonstrating drug efficacy before such therapies can be become frontline therapeutic options (6). However, there is recognition that this increasing gap between drug development and approval needs to be shortened (6). As a result, regulatory bodies are now investigating whether biomarker response evaluation, such as measurement of minimal residual disease (MRD) as assessed by flow cytometry, can provide an early, robust prediction of survival and therefore improve the drug approval process (7).

Over the past decade, several publications have emerged demonstrating enhanced prediction of outcome using flow MRD testing for multiple myeloma over conventional response assessments (8–13). In these studies, flow cytometry has been shown to be an independent predictor of progression-free (PFS) and overall survival (OS). However, for flow cytometry to be deemed to be clinically useful and acceptable as a surrogate for PFS and OS by the FDA, a standardized, reproducible assay that can be performed in multiple laboratories is required. A recently published survey of labs performing MRD testing for myeloma, however, revealed a striking heterogeneity in practices with major differences in antibody panels, gating strategies and event counts (14). Thus, for the FDA to accept flow-MRD as a surrogate for PFS and OS, a standardized approach in which different labs achieve comparable results is necessary.

Over the past decade, three publications have emerged demonstrating enhanced prediction of outcome using flow cytometry MRD over conventional response assessments (8,9,15). In these studies, flow cytometry demonstrated independent prediction of progression-free and often overall survival. One recent large study evaluated over 1,000 patients divided among three treatment arms used a six-color antibody panel testing for CD19, CD56, CD38, CD138, CD45, and CD27 to distinguish abnormal from normal plasma cells (8). They counted 500,000 bone marrow cells and required 50 or more abnormal plasma cells as a cutoff for MRD positivity. Taking this approach, they had a maximum detection sensitivity of 0.01% and found achievement of MRD negativity by flow cytometry was a powerful predictor of favorable PFS and OS outcomes. Furthermore, an international group of experts in flow cytometry and multiple myeloma from Australia, Europe and North America and also representing the International Clinical Cytometry Society (ICCS), National Institutes of Health (NIH) and the Food and Drug Administration (FDA) has been working on consensus guidelines for flow-MRD in multiple myeloma (7). This working group, building on the published studies showing correlation with outcomes, has established a standardized methodology, 8-color antibody panel with a minimum sensitivity of 0.01% and an upper sensitivity of 0.001%.

As PCM therapies become more effective, assessing treatment efficacy according to MRD levels becomes increasingly important (13,16–19). Standardization therefore of flow cytometric PCM MRD testing is vital to ensure better and uniform assessment of response and clinical prognostication. Accordingly, the need for harmonized flow cytometric approaches is required that not only provides backwards compatibility with established assays but as treatment strategies evolve will also offer sufficiently high enough sensitivity to remain relevant for the next decade (7).

The following document defines the approaches to acquisition, analysis and quality control for such a harmonized flow cytometry assay that offers a limit of detection and quantification in the same order of magnitude as high throughput sequencing (HTS). Such an approach has the additional benefits of internal sample quality checks, no mandatory requirement for pretreatment samples and a proven track record of predicting outcome in prospective clinical trials.

INSTRUMENT STANDARDIZATION AND QC

In addition to obtaining the correct sample, optimizing sample preparation, panel configuration, and undertaking the correct analysis and gating strategy (20–25); a well maintained, optimized, and standardized flow cytometer is critical in PCM MRD testing. This will ensure that for each specimen identical fluorescent and light scatter output signals are obtained for the data analysis that facilitates the identification of abnormal plasma cells (aPCs) in the sample.

Upon initial instrument installation of the flow cytometer, the manufacturer will optimize the laser alignment and establish optimum target values for light scatter and fluorescence. The testing laboratory will perform instrument quality control (QC) of the flow cytometer daily to verify that the instrument reproduces these target values. Flow cytometry instrument quality control consists of verifying the optical alignment of the instrument, that target values for light scatter and fluorescence are obtained with each use (standardization), and the removal of fluorescent spectral overlap is optimally achieved (compensation).

Instrument linearity in each fluorescent channel must be established and should be verified after major maintenance, at least bi-annually or if other regulatory bodies require at shorter frequency as part of the instrument QC program. This is often done with a mixture of beads at varying and known intensity levels such as Spherotech Rainbow beads, although other methods exist. The correlation coefficient (r²) between the bead MFI and the beads fluorochrome equivalents should be ≥0.99 (26).

Optical Alignment

Proper alignment of the sample stream with the laser(s) and respective optical detectors is critical to obtain accurate and reproducible results for PCM MRD. This is performed using alignment microspheres (beads) with a uniform signal for each light scatter and fluorescent parameter. The goal of alignment is to obtain the “brightest and tightest” signal with maxima signal and minimal variation. Peak or median channels for forward scatter (FSC) and side scatter (SSC) and all relevant fluorescent channels used in the respective protocol are captured along with their respective coefficient of variation (CV) values. Each peak channel must fall within the pre-established acceptable target range established at set-up (20).

Standardization of the flow cytometer is specific to each instrument protocol because it may be that different antibody panels are used as well as different target populations assessed. Thus, the goal of standardizing the flow cytometer daily is to verify that the populations being analyzed are always measured in the same location of the histogram, and exhibit the same level of fluorescence on a day-to-day basis. This is particularly critical for assessing plasma cell neoplasms, as subtle differences in light scatter and/or fluorescence are used to distinguish abnormal populations from their normal counterpart. The standardization procedure is undertaken by running stabilized microspheres (beads) with known fluorescence and light scatter characteristics. Each parameter is placed into pre-established channels (target channels established at instrument set-up) by adjusting the detector voltage settings. Voltage settings for each parameter are then captured and monitored for fluctuations. Longitudinal displays of changes in voltage can be graphically displayed in Levy-Jennings type plots to visualize shifts and/or trends in the instrument performance (21,27).

If multiple instruments are used within a laboratory to perform the same test, then they should be demonstrated to yield comparable results on a semiannual basis. This is best accomplished by running three samples in parallel on all instruments used to perform the assay. These samples should have normal plasma cells (nPC) and at least one, though ideally all should have an aPC population around the lower limit of detection (LLOD). For the flow cytometric PCM MRD assay, it is important to document the numbers of nPCs and aPCs detected on each instrument have a 10% CV which can vary up to a 30% CV at levels close to the LLOD. Antigen expression patterns (i.e., positivity and negativity) for each antigen characterized on nPCs and aPCs should also be compared across instruments and comparable separation between positive and negative populations on normal cells should be documented. To easily accomplish this and to standardize fluorescent intensities across instruments, the target values for the fluorescent microsphere beads discussed above for one instrument may be used to set comparable target values on another instrument.

Compensation

Compensation settings are established at assay validation to subtract the spectral overlap of the different fluorochromes in the assay. Compensation is mathematically linked to other instrument variables including fluorescence PMT voltages, gains, laser power, and optical filters. Therefore when any of these variables are altered, they will affect compensation. This may occur during maintenance or instrument repair or also if voltage settings are adjusted during the standardization process. Compensation verification is part of the daily assay quality control, but should also be optimized if voltage settings are altered significantly during standardization (20).

Carryover

Testing should be performed to measure the level of carryover from one sample to the next and provide evidence that carryover does not exceed specifications. This is particularly important for laboratories performing MRD and rare event analysis, specifically for those instruments using a carousel or multi-well sampler. This can be done by sequentially acquiring and analyzing replicates of a sample near the lower limit of detection (~0.01%) followed by a sample that contains a high level of PCM, and again followed by the low end specimen (23,28). Alternatively, beads maybe used as outlined in Gratama et al. (29). If carryover between samples is an issue, incorporating a wash step accomplished by running a tube containing sheath fluid or water between samples will usually resolve this problem.

ASSAY VALIDATION

Multiparametric flow cytometry is a powerful technology that facilitates analysis of a large number of events in heterogeneous cellular specimens and provides information on a cell-by-cell basis. Thus, flow cytometry is well suited for rare event detection in assays such as PCM MRD because it has the ability to acquire high numbers of cells very rapidly. However, before implementation in clinical practice, PCM MRD requires extensive analytical validation in order to establish performance specifications that meet the intended use of the assay.

Recently published practice guidelines for the validation of cell-based fluorescent assays describe in detail the differences between cell-based and traditional soluble analyte assay validations, as the former requires unique approaches to validation (22). In addition, these guidelines describe the different categories of bioanalytical assays and the respective strategies for designing method validation experiments (21,23,30,31). The PCM MRD assay is designed to detect the presence or absence of disease, and to enumerates the amount of disease (as a percent of nucleated cells) thus categorizing it as quasi-quantitative. This section will define strategies to establish performance specifications that will cover both the qualitative and quasi-quantitative aspect of this assay (31).

A reliable PCM MRD assay must be optimized and demonstrate interlaboratory reproducibility and concordance. However, currently no External Quality Assessment (EQA)/Proficiency Testing (PT) programs exist, thus until such programs become available sample exchanges between centers developing their PCM MRD test with a center with a well-established test should be undertaken to properly validate the assay. Thereafter, laboratories should exchange samples on a quarterly basis, review the results and discuss any variance to determine the root cause. Ideally, multiple laboratories would participate in the exchange to help achieve a consensus. To achieve global standardization, the ideal situation would be to have a clinically validated assay to which every center adheres. Once this validated panel has been developed, each testing laboratory must demonstrate concordance and establish the performance specifications of the assay using their laboratory equipment, staff and reagents. This analytical validation is the process of proving that the optimized and clinically validated procedure works as expected and consistently achieves the expected results for the intended use of the assay. Strategies for determining the following performance specifications based on the practice guidelines will be discussed in detail for accuracy, precision/reproducibility, analytical sensitivity, analytical specificity, and sample stability.

Finally, it is important to note that if a laboratory is validating a PCM MRD assay which is to be performed on multiple instruments, then as discussed below, the accuracy, precision/reproducibility, and analytical sensitivity of the assay should be confirmed on each instrument. Thereafter, samples should be run in parallel semi-annually on each instrument.

ACCURACY

Accuracy is defined as the closeness of agreement between the average values obtained from a large series of tests and an accepted reference standard (32). In flow cytometric cell-based assays there is no known standard that exists, thus an alternative approach is required to determine if the assay is detecting the appropriate value. The validation process must be able to detect the measurement at a level that is clinically required, defined as the LLOD (23). Strategies to ensure accuracy and the LLOD for the PCM MRD assay can be developed by creating samples with known amounts of PCs. This can be done by diluting bone marrow from a patient with PCM into a non-diseased bone marrow sample (spiking) such that the lowest level of detection is below the expected detection level of the assay. A minimum of five assays at each dilution point are then undertaken and the variance at each point plotted, this will also serve for calculating the intra assay CV% that should generally be <10%. However, for less abundant populations (such as MRD where the population is rare) the CV% will increase as the LLOD of detection is reached. In cases where the LLOD is 0.01% or lower a CV of ≤30% is acceptable (30). This dilution assay can also be used to demonstrate assay linearity by calculating the correlation coefficient (r²) between actual percentages of abnormal cells detected at each dilution versus the predicted value which should be greater than 0.95. Often, the first point is discarded because the total WBC count between the PCM free donor marrow and the patient with PCM may differ significantly. For example if the patient’s WBC count is half the healthy donors WBC count, the first 1:10 dilution (based on volume) will result in a 1:19.0 dilution of the patients actual cells. The second 1:10 dilution will result in 1:10.5 dilution of cells and the third a 1:10.1, etc.

The level at which two adjacent dilutions variance overlaps indicates the LLOD of the assay has been achieved and the higher level of these two points is deemed to be the LLOD. Additionally, these samples can be split with another laboratory and compared for equivalence and recovery. The recovery of the created amount of sample must be demonstrated within an adequate level of precision. Concordance between the results based on the presence or absence of MRD should be demonstrated as well as detection of the LLOD with a CV of <30% (30).

The difficulty of this process is procuring a sample that contains a significant amount of aPCs that can be used in the spiking experiments. It is critical to ensure accurate calculation of the end concentration of such cells, and importantly, perform accurate reverse pipetting during the spiking process.

Precision/Reproducibility

Due to the quasiquantitative nature of the assay, establishing precision is required specifically at the LLOD (30). Inter and intra-assay precision can be determined by spiking samples with known amounts of PCM into nondiseased bone marrow, running them in triplicate, and calculating the %CV. As discussed earlier, a desirable target for assay precision is generally a CV of <10%; however for rare event analysis such as MRD where the CV will increase as the LLOD is reached, an assay precision of ≤30% is acceptable (30).

Analytical Sensitivity

Analytical sensitivity is defined as “how well an assay performs with known positive samples defining the limit of detection.” Routine assays for the detection of plasma cells have a sensitivity of approximately 0.1% (24). With the advances in rare event analysis by flow cytometry, it is possible to detect small abnormal populations of specific cell types as long as an adequate number of events are acquired (for a 10% CV a minimum of 100 events must be collected for each target population) and appropriate analytical procedures are used to distinguish these cells and avoid false positive events. Sensitivity is critical in MRD assays as it defines the LLOD. The PCM MRD assay detects a small amount of aPCs at levels of 0.01% or less in a background of other cells including nPCs. This requires differentiation between the nPCs and aPCs. Levels for aPCs have been reported as low as 0.005% to 0.01% (10). Based on Poisson distribution, in order to obtain a sensitivity of 0.005%, acquiring at least 2,000,000 events with a minimum of 100 aPCs is required. To obtain a LLOD of less than this, more cells must be collected. The minimal and ideal levels of sensitivity for the PCM MRD assay are detailed in the companion Data Analysis and Reporting manuscript also found in this special issue (33).

In summary, one must be able to verify that the assay can recover at least a sensitivity of 0.005% with a CV of ≤30%; therefore the LLOD should have the ability to accurately and reproducibly recover a minimum of 1 myeloma cell in 20,000 nucleated cells.

Analytical Specificity

Analytical specificity is defined as how well an assay performs with known negative samples. As described by Rawstron et al. (34) and others in this issue, aPCs have a specific abnormal immunophenotype that is different from nPCs. A number of factors influence analytical specificity of the assay; the markers used to define the population of interest, the gating strategy and, interactions between reagents. In order to establish the specificity of the PCM MRD assay, one must assess material that contains the disease (ideally near the LLOD) and samples that do not have the disease. Results are compared and should agree with the expected results (23). Analytical specificity = TN/TN + FP where TN = true negative and FP = false positive. In establishing the analytical specificity a minimum of 20 “normal” bone marrow aspirates from patients without hematological disease in the bone marrow should be assayed to establish the analytical specificity and these should all be negative or at least fall below the LLOD.

Stability (Sample and Cocktail)

Plasma cells are fragile and can deteriorate rapidly after sample collection or due to adverse sample handling. If there is likely to be a delay in testing the diagnostic or MRD samples of >24 h for samples collected in EDTA or 48 h for samples collected in sodium heparin, as could be the case in a reference laboratory setting, one must verify that the assay results (both qualitatively and quantitatively) are not altered over time (21). This can be done by testing specimens for PCM MRD (ideally at the LLOD) within ≤6 h of the draw and then subsequently every 6 to 12 h. Qualitative equivalency of each marker as well as the ability to detect positivity at the LLOD will determine sample stability. It is highly recommended and some agencies require that samples that are tested >24 h from the draw include a viability assessment. Samples with viabilities less than 85 to 90% should not be reported, however, exceptions are made for precious, difficult to obtain and irreplaceable samples such as a bone marrow (21). If a sample with low viability is reported, then a disclaimer stating the viability was below the laboratory’s acceptable standard should accompany it. This is because low viabilities are associated with (a) nonspecific binding of antibodies to dead cells which can make it difficult to interpret staining patterns and accurately quantify populations, (b) reduced expression of CD138 on aPCs (35), and (c) decreased frequencies of fragile cell populations including both nPCs and aPCs. Alternatively, a proprietary cell-stabilizing reagent may be used, however, the reagent must be shown to have proven ability to stabilize the samples without affecting antigen integrity over the period that samples will be in transit (plus at least one extra day to allow for testing).

The use of premixed antibody cocktails in the PCM MRD assay can simplify and standardize the assay set up and minimize potential errors in pipetting. If one chooses to utilize such cocktails for the assay, validating that the results are equivalent for the cocktail over time is recommended. This can be done by comparing results while using the cocktail and comparing them to the same sample results (ideally a sample at the LLOD) while using freshly pipetted single antibodies. Comparing results for categorical concordance will define the stability of the cocktail over time.

The following specifications are not applicable for an analytical validation of MRD by flow cytometry:

• While a reference range does not need to be established because patients with a negative result will not have residual aPCs present, patients without hematological disease in the bone marrow must be run to validate the assay. As discussed under analytical specificity, a minimum of 20 samples is required to demonstrate they fall below the LLOD.

• Reportable range is defined as the acceptable limits that each of the reportable analytes has met the analytical precision requirements. This is addressed with the analytical sensitivity and the lower limit of detection (LLOD).

EXTERNAL AND INTERNAL QUALITY ASSESSMENT

Quality control of the assay should be subject to all normal local laboratory procedures for quality control, a full review of this approach has been detailed at length recently (25). This entails having an auditable Internal Quality Control (IQC) system whereby all aspects of the procedure are controlled so as to facilitate immediate identification of problem areas. It should be stressed that IQC encompasses training and competency assessment together with achieving appropriate accreditation to national and wherever possible international standards (e.g., ISO 15189) and does not just cover the technical aspects of the particular test (34). A robust IQC system provides an “early warning system” to potential problems. Any anomalies it flags should be documented and investigated fully before proceeding with testing patient material. A common IQC material for flow cytometers is the use of microbeads that can be used to monitor compensations, fluidics, laser, and PMT voltages. Results from the use of beads should be plotted on a Levy–Jennings type plot and any noticeable drift should be investigated immediately. To do this, boundaries of acceptability should be established and all staff be made aware and to understand what these boundaries mean. It is also important that the instrument should be checked with beads and stabilized samples following instrument servicing to re-establish optimal settings. Furthermore, the laboratory undertaking the test should be fully conversant with the technical protocols designed for PCM flow cytometric testing both for diagnostic and PCM MRD.

At the present time, no daily quality controls exist for quality controlling the PCM MRD assay procedure. Thus it is appropriate in such circumstances to run a full process control that, whilst not being from a MM patient, does have assigned values for a given set of CD antigens, for example, CD3, CD4, etc. Such samples can be used to provide the operator with important information regarding staining, lysing, acquisition and analysis of antigens. This will allow day-to-day monitoring of the instrument, operator and technique and will facilitate consistency checks by plotting percent positive and MFI values on Levy Jennings charts (25). It is also important to have assay specific QC checks whilst running clinical samples. This may involve using a Delta check or Levy Jennings like assessment for samples on a cumulative basis, running samples with known values at the start of each working day or each batch.

A properly controlled assay should include a review of internal normally defined cell populations as internal positive and negative controls within each patient sample. For example, CD45 fluorescence intensity on lymphocytes should be at least 10 times brighter than on erythroid cells; CD19 on B cells at least 10 times brighter than on NK cells; and CD56 intensity on NK cells at least 10 times greater than on B cells. CD138 expression on plasma cell should be compared with expression on CD191 CD81 bright progenitor cells, CD117 expression on myeloblasts can be compared with lymphocytes, etc. By using normal cell populations found in each sample the laboratory can establish acceptable staining patterns for each antibody. Similarly, each lot of antibody and laboratory developed pre-mixed reagent cocktail should be fully validated as described above comparing the fluorescent intensities in parallel between the new and in use lot. This may not always be possible with normal samples for reagents such as CD117 and CD138, in which case the samples being run in parallel should be spiked with a commercial cell preparation, cell line or positive patient sample established to be positive for the antigen. This QC check is particularly important when using reagents and cocktails that require less stable fluorochromes.

However, whilst no specific PCM control exists, the patient sample itself will have inbuilt internal controls that can also be used to verify staining processes have worked efficiently. Normal leucocytes will always be present, albeit sometimes in low numbers, in any bone marrow specimen and these can be used to assess the level of positive/negative staining and therefore verify that the staining process has worked. Expected results for internal controls must be documented as part of this process.

Currently there are no External Quality Assessment (EQA)/Proficiency Testing (PT) programs for PCM MRD, although organizations such as UK NEQAS for Leucocyte Immunophenotyping are examining possible approaches to undertaking such EQA/PT. However, whilst ISO 17043 standards state that the EQA/PT matrix should resemble as closely as possible the material used in the test procedure (35), the practicalities of developing such a program will be limited by the availability of suitable material and should not be underestimated. This is because both PCM diagnosis and PCM MRD testing are undertaken on bone marrow samples. Therefore, the development (and availability of a suitable volume) of EQA/PT material mimicking such a matrix may be difficult to develop. In such instances compromises may have to be made where material, whilst not being exactly identical to patient test material, might have to be used. Thus, whilst it is desirable to have samples that have antigens present at levels usually encountered in PCM, with sample limitations, having a matrix that contains cells that express antigens usually found in plasma cell myeloma cases but not necessarily having the antigen profile of an aPC may have to be used. Studies to develop such a material are currently in progress.

It is also important to have assay specific QC checks whilst running clinical samples. This may involve using a Delta check or Levy Jennings like assessment for samples on a cumulative basis, running samples with known values at the start of each working day or each batch. In addition, acceptability limits for each test should be defined and cross checks included within a given panel.

SUMMARY

Any new assay (including PCM MRD) requires analytical validation to establish the performance specifications and verify that they meet the intended use of the assay. As described here, the validation must demonstrate reproducibility, stability and include method comparison together with establishing the analytical sensitivity (LLOD) and specificity of the assay. Ongoing quality control (both internal and external) is absolutely critical to ensure that once the assay is running in the clinical setting, these performance specifications remain stable and any deviations from the expected values are documented and investigated accordingly. No clinical results must be issued if any problems are identified until the issue(s) are resolved.