Abstract
The HIV Vaccine Trials Network (HVTN) is a global network of 28 clinical trial sites dedicated to identifying an effective HIV vaccine. Cryopreservation of high-quality peripheral blood mononuclear cells (PBMC) is critical for the assessment of vaccine-induced cellular immune functions. The HVTN PBMC Quality Management Program is designed to ensure viable PBMC are processed, stored and shipped for clinical trial assays from all HVTN clinical trial sites. The program has evolved by developing and incorporating best practices for laboratory and specimen quality and implementing automated, web-based tools. These tools allow the site-affiliated processing laboratories and the central Laboratory Operations Unit to rapidly collect, analyze and report PBMC quality data. The HVTN PBMC Quality Management Program includes five key components: 1) Laboratory Assessment, 2) PBMC Training and Certification, 3) Internal Quality Control, 4) External Quality Control (EQC), and 5) Assay Specimen Quality Control. Fresh PBMC processing data is uploaded from each clinical site processing laboratory to a central HVTN Statistical and Data Management Center database for access and analysis on a web portal. Samples are thawed at a central laboratory for assay or specimen quality control and sample quality data is uploaded directly to the database by the central laboratory. Four year cumulative data covering 23,477 blood draws reveals an average fresh PBMC yield of 1.45×106 ±0.48 cells per milliliter of useable whole blood. 95% of samples were within the acceptable range for fresh cell yield of 0.8–3.2×106 cells/ml of usable blood. Prior to full implementation of the HVTN PBMC Quality Management Program, the 2007 EQC evaluations from 10 international sites showed a mean day 2 thawed viability of 83.1% and recovery of 67.5%. Since then, four year cumulative data covering 3338 specimens used in immunologic assays shows that 99.88% had acceptable viabilities (>66%) for use in cellular assays (mean, 91.46% ±4.5%), and 96.2% had acceptable recoveries (50%–130%) with a mean of recovery of 85.8% ±19.12% of the originally cryopreserved cells. EQC testing revealed that since August 2009, failed recoveries dropped from 4.1% to 1.6% and failed viabilities dropped from 1.0% to 0.3%. The HVTN PBMC quality program provides for laboratory assessment, training and tools for identifying problems, implementing corrective action and monitoring for improvements. These data support the benefits of implementing a comprehensive, web-based PBMC quality program for large clinical trials networks.
1. Introduction
Clinical evaluation of experimental vaccines has increasingly involved more complex assays to measure vaccine-induced immune responses. Within the arena of HIV vaccine development, these assays often generate data that inform key product advancement decisions. This critical role places a tremendous amount of scrutiny on the quality of the assays and, in turn, the specimens utilized in those assays. Measurements of vaccine-induced antibodies rely on serum samples that are relatively easy to prepare and are stable when stored at −80°C. Traditionally, vaccine-induced antibodies were evaluated as a measure of vaccine response; more recently, an enhanced understanding of cellular responses and their related role in protection against infection or disease progression has raised interest in developing HIV vaccines that can also elicit cellular responses. Cellular samples require more complex processing methods and are highly susceptible to loss of viability and function unless processed, cryopreserved, and thawed with strict adherence to handling and cold chain requirements.
Studies conducted by Bull, et al. and Kierstead, et al. revealed that the duration of time from venipuncture to deep cold cryopreservation can significantly affect cell viability and the functional immune responses measured by later assays (Bull et al., 2007; Kierstead et al., 2007). Their studies compared functional immune responses from peripheral blood mononuclear cells (PBMC) processed and preserved at different time points following single blood draw. Samples processed beyond the 24- hour mark showed a significant drop in functional immune assay responses of up to 20% (Bull et al., 2007). Earlier studies had established that PBMC viabilities ≥70% had consistent functional immune responses suitable for functional analysis (Weinberg et al., 2000). HVTN immunological assays had shown reliable results with as low as 66% viability; this was the lower limit for viability allowed during immune assay validation (Horton et al., 2007). The HVTN program follows the time parameters established by Bull and Kierstead, using thawed recoveries and viabilities as quality markers.
These findings were significant to clinical research because they established key performance indicators with a direct correlation to the validity of the studies in which they were used, and they altered how PBMC-based research was managed by the HVTN, particularly in the United States where processing after overnight shipment of whole blood was the norm. The findings defined the optimum timeframe for specimen processing and cryopreservation as less than eight hours from venipuncture. Prior to these results, overnight shipping to a central processing site had been the standard approach to minimize on-site and staff cost for multi-site trials for the HVTN within the United States. The risk of up to a 20% decrease in functional response outweighed the benefit of related cost savings. As soon as the preliminary results of the Bull, et al. optimization studies were available to the HVTN in early 2004, the network decided to implement local processing at or near each clinical site. The first sites to make the change began operating processing labs in 2005. The first protocol to have PBMC specimens processed locally, exclusively under this new eight-hour local processing model, was HVTN 054 (Peiperl et al., 2010).
Local specimen processing laboratories (SPL) with locally trained staff needed to be standardized and brought into compliance with the high standards for human clinical trials research. The new PBMC program included establishment of local processing centers, training, certification and external evaluation. Weinberg’s studies and others on PBMC processing observed that the level of experience and skill of laboratory staff performing cryopreservation is a major determinant of the viability and functionality of cryopreserved cells, especially when PBMC are isolated using the Ficoll overlay method (Weinberg et al., 2000; Dyer et al., 2007; Weinberg et al., 2007; Sigma-Aldrich, 2011; Aziz et al., 2013). This emphasized the critical need to rapidly implement a comprehensive PBMC quality program that included staff training and quality management systems that would mitigate risk to successful vaccine development.
The PBMC Quality Management Program developed by the HVTN was designed, implemented, and continues to be managed by the HVTN Laboratory Operations Unit; it ensures that each SPL maintains robust sample quality and a level of control and compliance sufficient to support regulatory requirements and standards for human clinical research. The program evolved by incorporating best practices, addressing problematic areas, and providing rapid feedback on specimen quality to the laboratories. The program provided the tools needed by both the processing laboratories and the HVTN to monitor workloads, detect problems, analyze trends and summarize specimen quality by selected site, technician, or protocol. The PBMC Quality Management Program includes five key components: 1) Review of Facilities; 2) Training and Certification; 3) Processing internal quality control (IQC); 4) HVTN external quality control (EQC); and 5) Assay specimen quality check (AQC).
Here we show examples of how this system can help identify potential problems and how close monitoring of processing data across all clinical sites, technicians and protocols can lead to a very high processing success rate even in a globally distributed network. We also show how automated analysis tools can provide valuable information for both the laboratory management and the central Laboratory Operations Unit for short term and long term monitoring of laboratory quality.
2. Methods
2.1 PBMC Standard Operating Procedures (SOP)
The principle for freezing and thawing cells is to maintain a unidirectional temperature gradient (Riccio et al., 2002; Bull et al., 2007; Dyer et al., 2007; Smith et al., 2007; Mallone et al., 2011).
2.1.1 Blood collection, PBMC isolation and cold chain management
Use of HANC (HIV/AIDS Network Coordination) Cross-Network PBMC Processing SOP or the HVTN version of the HANC Cross-Network PBMC Processing SOP was implemented in April 2009. Briefly, blood is collected in either sodium heparin or ACD BD Vacutainer tubes. Processing and initiation of cryopreservation takes place within 8 hours of specimen collection. PBMC are isolated using either a frit barrier separation tube with density gradient media (such as Accuspin) or a manual density gradient over/underlay technique. The isolated PBMC are washed in Hank’s Balanced Salt Solution (HBSS) or phosphate buffered saline (PBS) without calcium or magnesium, counted and re-suspended in a chilled (4°C) cryopreservation solution of 10% endotoxin free dimethyl sulfoxide (DMSO) and 90% fetal bovine serum (FBS). Aliquots are immediately (within 10 minutes of exposure to DMSO) placed in rate-controlled cooling containers (such as StrataCooler, Mr. Frosty or biocision CoolCell) until the cycle is complete (time depends on device used) and stored between −64°C and −90°C for cryopreservation. Once cryopreservation is complete, all handling of PBMC aliquots, including transfer from the rate controlled devices to freezer boxes or transfer of freezer boxes to dry ice shippers, is performed in dry ice.
Samples are shipped weekly on dry ice to the central specimen repository (CSR). The CSR transfers PBMC aliquots to liquid nitrogen for long-term vapor storage between −150°C and −190°C. Once placed in liquid nitrogen, all subsequent handling and transport is conducted using liquid nitrogen vapor baths and liquid nitrogen dry shippers. Upon request from the HVTN Laboratory Center, PBMC are shipped to the immune monitoring laboratories (IML) in liquid nitrogen dry shippers. The IML handles PBMC samples in liquid nitrogen vapor baths and stores them in liquid nitrogen vapor phase to maintain cells below re-crystallization temperatures of −90°C to −130°C until thawed for testing.
2.1.2 PBMC Thawing Protocol
PBMC aliquots are retrieved from liquid nitrogen vapor phase storage and held in a liquid nitrogen vapor bath pending a controlled thaw in a 37°C water bath. R10/Benz media (RPMI 1640, HEPES buffer, FBS, L-glutamine, Penicillin-Streptomycin, Benzonase) is added to the cells immediately after thawing. The thawed cell mixture is centrifuged, the cell pellet is re-suspended in R10 only and incubated overnight at 37°C and 5% CO2. At initial thaw (day 1) and after overnight incubation (day 2), cells are counted on the Guava PCA bench top analyzer using the Guava ViaCount Reagent to determine total cell numbers (recovery) and cell viability.
2.2 Component 1: Specimen Processing Laboratory (SPL) Facilities and Operations
28 site-affiliated laboratories were included in this analysis. Coded identifications are used throughout to protect their identity. Pre-determined key processing facility characteristics are first evaluated in an effort to standardize and optimize specimen processing and management; potential issues identified during the review are resolved before progressing to staff training. Criteria evaluated at this step include compliance with NIH National Institute of Allergy and Infectious Disease Division of AIDS (DAIDS) Good Clinical Laboratory Practice (GCLP) guidelines (or equivalent), specimen transport time, workflow issues, equipment shortages, cold-chain practices and backup systems. This list is compiled from analysis of marginal and failed EQC specimens within our affiliated laboratories..
Ideally, the processing laboratory facility should be located within close proximity (less than 2 hours transport time) to the clinic to allow frequent specimen deliveries throughout the business day. Scheduling tools between the clinic and lab are useful for managing workloads and tracking late or missing specimens. Frequent clinic deliveries optimize staff utilization and minimize the time from specimen collection to cryopreservation. The laboratory space should include a receiving area for checking and staging specimens. At least two workstations are recommended to allow the processing lab to function even if equipment for one workstation fails or is undergoing maintenance. The workflow should allow specimen movement from receiving to processing to freezing and eventually shipping with minimal interruption. SPLs are required to have backup power generators and/or CO2 freezer backup systems. All PBMC are shipped to the central repository weekly to minimize the quantity of specimens held at the clinical sites.
The DAIDS sponsored Laboratory Data Management System (LDMS) is used for logging specimen demographics, labeling, storage, and shipping. LDMS data is transmitted to the HVTN SDMC and linked to the web-based HVTN PBMC Quality Management Program. Potential problems or issues identified in the review are addressed prior to staff training. The HVTN also maintains an ongoing dialog with the SPLs to immediately identify and address changes to the laboratory facilities.
2.3 Component 2: Staff Training and Certification
Training for PBMC processing, PBMC certification, and HVTN protocol operations are addressed once facility issues identified during site visits are resolved. Technicians that will process PBMC for HVTN trials must successfully complete the training and certification program. After an orientation meeting, a site-specific training program is tailored based on prior staff experience in specimen processing and clinical trials. Related training documents are provided to guide the site through certification. The initial training environment (e.g., the SPL, another HVTN certified laboratory, HVTN core laboratory, or via conference call/web meeting) is also selected based on the level of prior staff experience. For example, with prior approval by the HVTN, a technician at an HVTN-certified SPL may be trained by a senior staff member within the lab that is in good standing with the HVTN PBMC program.
Training is designed to mimic HVTN protocol specimen processing, including the use of the HVTN version of the DAIDS Cross-Network PBMC Standard Operating Procedure (SOP) and related protocol documents; trainees are tasked with specimen receiving, processing, LDMS data entry, labeling, temporary storage, cold-chain management, shipping and IQC review. Each trainee is first required to conduct at least one practice run in which ~100 mL of fresh blood is drawn, processed and shipped. The worksheets from the practice run are submitted to the HVTN for review prior to beginning the actual certification run. The certification run requires processing of two separate samples of ~100 mL fresh blood. Cryopreserved specimens and completed worksheets from the certification runs are sent to HVTN Laboratory Operations Unit for evaluation of proper packaging and documentation as well as PBMC recovery and viability after thawing. Certification documents are issued for the technician if results of the thawed specimens meet acceptance criteria, the processing documents are complete and shipping meets International Air Transport Association and HVTN criteria. The technician will remain certified as long as they process HVTN specimens monthly and related IQC and EQC results meet established evaluation criteria. If the technician becomes de-certified, they must re-complete the certification requirements. Laboratories that relocate or undergo an extensive remodel must also be recertified.
2.4 Component 3: Internal Quality Control (IQC)
Internal quality indicators and evaluation tools are available to the SPLs online through the Atlas web portal programmed and supported by the Statistical Center for HIV/AIDS Research and Prevention (SCHARP). While there is no practical quality control reagent available during the separation and cryopreservation tasks that would indicate successful completion, the SPL may employ process controls or indicators as broad guidelines. These indicators include the fresh cell yield (number of harvested PBMC per milliliter of fresh whole blood), the total time to cryopreservation (time from specimen collection to initiation of cryopreservation) and the actual processing time (time from initiation of blood processing to cryopreservation). Initially the percentage of viable fresh cells was also included as an indicator; however, this value did not show significant variation and did not correlate with problems identified subsequent to thawing.
Results for the three internal indicators are exported from the specimen data entry results program (LDMS) to Frontier Science (http://www.fstrf.org) by the SPLs as part of routine weekly workflow. These results are then imported by the HVTN Statistical Data Management Center (SDMC) into the web-based Atlas Specimen Management program. Results are viewable in table, Levy-Jennings and box plot formats by both the processing laboratories and HVTN Laboratory Operations Unit. Out-of-specification results are automatically flagged for review. Site-specific summaries, processing workload statistics, indicator trend analyses, and technician comparison reports are available to the individual laboratories and the HVTN Laboratory Operations Unit; additional comparison summaries across all laboratories are available to the HVTN Laboratory Operations Unit. Routine reviews and related investigations are managed on-line via the Atlas portal. Issues requiring retraining are resolved by conference call or site visits.
2.5 Component 4: External Quality Control (EQC)
While IQC offers broad indicators that are immediately actionable, a more detailed external evaluation of cryopreserved PBMC collected in the context of HVTN clinical protocols is vital. An initial competency assessment is not sufficient to monitor ongoing PBMC quality in a manner that reflects the performance of tasks when a laboratory is working under the challenges of an actual clinical trial. For this reason, a small quantity of protocol PBMC are sacrificed for external quality evaluation at intervals that allow timely investigation and corrective action. Because the laboratories do not know in advance which specimens will be selected for EQC, any bias related to preferential treatment of specimens is eliminated. Generally, efforts are made to select samples that cover all active sites and each technician monthly. Criteria for selection of specimens to be evaluated include in order of priority: 1) poor previous EQC results 2) unusual IQC results 3) shipping irregularity 4) new technician monitoring 5) random coverage of shipments. In an average year, samples are selected from approximately 8–12% of the total time points at which PBMC are collected in HVTN trials. The specimens are handled in the same manner as protocol specimens, including shipping and cold-chain management. The specimens are thawed and evaluated by the HVTN PBMC QA Laboratory in Seattle for viability and recovery immediately upon thawing (Day 1). The specimens are then re-evaluated for viability, recovery and cell loss after overnight incubation at 37°C and 5% CO2 (Day 2). This process mimics the procedure used when PBMC are thawed for protocol-mandated cellular assays (e.g., intracellular cytokine staining (ICS), ELISpot, and Multiplex Bead Array).
Each EQC evaluation may include as many as 32 samples. Internal PBMC controls are included at the beginning and end of each evaluation batch. Each lot of control samples is prepared from approximately 400 mL of fresh whole blood following the HVTN version of the DAIDS Cross-Network PBMC SOP. Parallel testing is performed to bridge each new lot of controls. Control results are scrutinized for acceptability prior to releasing the evaluation results to the processing laboratories. The EQC evaluation results are uploaded to the Atlas Specimen Management program by the HVTN PBMC QA Laboratory. The HVTN Laboratory Program completes an initial review with comments prior to releasing the reports to the individual laboratories. The individual laboratories then complete an internal review of the evaluation results along with any investigations and corrective actions. HVTN reviews any investigations and corrective actions before accepting the report. The dual review allows the HVTN Laboratory Program to accept the final action or request more detail. Via the Atlas portal, the SPL and HVTN Laboratory Program may view site-specific evaluation reports, summary reports in Levy-Jennings or boxplot format, and technician comparison reports; additionally, the HVTN Laboratory Program has access to management reports that provide summaries across all sites by date range or by protocol.
2.6 Component 5: Assay Specimen Quality Control (AQC)
Assay Quality Control (AQC) provides an assessment of the quality of the PBMC that were used in the clinical trial cellular assays. Viability and recovery data is captured when cryopreserved PBMC are thawed and counted in preparation for the assays. To accomplish this assessment, an electronic upload tool was developed that allows direct transfer of files containing the cell count and viability data obtained from a Guava cell counter to the SDMC database. This data is then automatically transferred into the Atlas Specimen Management program, where it can be displayed in reports by protocol or by SPL. AQC results provide information on the quality of specimens used in the cellular assays as well as feedback on the effectiveness of the EQC program; comparisons can be made between vials obtained from the same venipuncture that were thawed separately for EQC and AQC.
3. Results
3.1 Component 1: Specimen Processing Laboratory (SPL) Facilities and Operations
Laboratory facilities and operations are critical for the processing and handling of high quality PBMC in a clinical trial environment. We have found that poor laboratory design can lead to problematic workflow, such as an inadequate receiving area that led to specimen mix up. Poor freezer placement has led to excessively high room temperature and increased internal freezer temperatures. In addition, limited freezer capacity can result in frequent access, resulting in fluctuating internal temperatures. Excessive workload with limited staffing can result in delayed initiation of processing and extended total time to PBMC cryopreservation. Poor communications between the SPL and clinic regarding scheduling can result in delayed specimen processing or loss of specimens. Equipment failures can lead to interruption of processing and compromised specimen storage. As such, proactively planning for replacing aging equipment is part of the HVTN’s program.
While efforts are made to establish facilities and systems prior to laboratory activation that can prevent these occurrences, continuous monitoring, communications with central Laboratory Operations and the EQC testing program have been successful at identifying problems that arise subsequent to activation.
3.2 Component 2: Staff training and certification
The HVTN requires a minimum of two certified technicians per site to manage a typical HVTN workload. Three or more may be required for sites that are involved in more than the average number of protocols. 290 technicians have been certified over a seven year period to maintain an average of 90 active certified technicians across all sites. Currently, 89 technicians are actively certified across 28 SPLs.
In prior years technicians required an average of 1.8 attempts at proficiency testing to become qualified, frequently due to them skipping the practice processing runs that are required prior to actual proficiency testing. Recently, we have implemented a requirement that worksheets from practice runs are submitted to the Laboratory Operations Unit for review and proof that they were completed. In the six month period since implementing this requirement, the average number of proficiency test attempts before successful certification has improved slightly to 1.5, and we expect this to improve further. This emphasizes that practicing the HVTN PBMC processing method on actual blood specimens, in addition to training, is important to successfully pass proficiency testing. Reducing the number of proficiency runs translates into a significant cost savings for the HVTN in reagents, shipping and labor. Certification costs currently run approximately $1800 per proficiency submission for US laboratories and $2600 for non-US laboratories; the difference is driven by shipping of international specimens for assessment in the United States. The costs for certification of technicians in Southern Africa may be reduced with the opening of the Cape Town HVTN Immunology Laboratory, which will allow for fewer international shipments.
3.3 Component 3: Internal Quality Control
Internal quality indicators have been helpful in identifying operational or technical challenges in the SPLs. Faulty specimen processing technique may result in an out-of-tolerance cell yield. Issues with communications between the clinic and laboratory regarding scheduling of blood draws may be indicated by an extended total time from collection to freeze. Unanticipated workload issues may be reflected in longer processing times from start to freeze. Site summary reports are available to laboratory management for cell yields, total time to freeze and processing time. These reports may be displayed for a specific date range covering all laboratory technicians or for individual technicians. The summary reports allow the site (and the HVTN Laboratory Operations Unit) to monitor trends and overall performance, and evaluate individual technician performance. While these indicators do not guarantee high viability or recovery, they can help identify problems that contribute to marginal or failed specimen quality. Unlike EQC results, which provide feedback as much as six to eight weeks after the processing event, the IQC markers can be monitored in real time by the processing laboratory staff.
Cell yields below the expected range may indicate poor specimen quality, cell counting issues, poor cell harvesting technique or equipment problems. The cell yield is defined as the number of cells harvested per millimeter of useable whole blood processed. The expected range for healthy adults is between 0.8 and 3.2×106 cells/ml, and is used as a marker to identify potential problems. If the cell yield is outside the expected range, the sample is stored with a comment and still used for protocol testing. If the cell yield is extremely low (≤0.4×106 cells/ml), a replacement specimen may be requested to ensure sufficient quantity of cells for testing.
The technicians enter the processing data into LDMS along with the specimen demographics for tracking and labeling at the time of processing. LDMS data is uploaded to a central database at the HVTN Statistical and Data Management Center for access and analysis on the Atlas web portal. The results are available for review by the SPL managers within 24 hours of export. This rapid turnaround makes IQC indicators useful for early detection of potential problems. Figure 1 is an example of the trend analysis from a monthly IQC report for fresh cell yield for SPL #10, and shows that 7 times within November 2012 the cell yield was below the lower limit of 0.8×106 cells/ml. The full report, which includes a line listing for all specimens processed and technician identification, also indicates that the SPL manager has reviewed the processing paperwork for these PBMC samples and commented on their findings (not shown).
Figure 1. Site #10, Fresh Cell Yield Trend Analysis for November 2012.
Each point represents the cell yield for the processing of a single blood draw. Participant identification was removed from the x-axis to protect blinding. All samples processed within the month are shown. Dotted lines represent the expected range of 0.8 – 3.2×106 cells/ml of usable whole blood.
Technician Comparison reports are useful for management and staff to visualize differences in quality between the technicians at a single site. A review of this report for November revealed that one technician was responsible for 6 of the 7 specimens with low yields (Figure 2). This technician was new to processing that month, and the SPL manager took steps to review the processor’s technique and make corrections, resulting in improvements in subsequent months (Figure 3, showing all technicians). This extended view is valuable to the central Laboratory Operations Unit when evaluating overall site performance and trends.
Figure 2. Site #10, Fresh Cell Yield Report for November 2012.
Each box plot represents the cell yield data for a single technician from November 2012 for SPL #10. Boxes represent the 25 and 75 percentile, whiskers extend to the most extreme data point within 1.5 interquartile ranges (IQR) and the horizontal line in the box indicates the median cell yield for the month. Dotted lines indicate the expected range of 0.8 – 3.2×106 cells/ml of usable whole blood.
Figure 3. Site #10, 12 month Fresh Cell Yield Report for all technicians.
This report includes specimens processed May 2012 through April 2013. Each box plot represents the cell yield data from a single month for SPL #10. Boxes represent the 25 and 75 percentile, whiskers extend to the most extreme data point within 1.5 interquartile ranges (IQR) and the horizontal line in the box indicates the median cell yield for the month. Dotted lines indicate the expected range of 0.8 – 3.2×106 cells/ml of usable whole blood.
The Atlas program provides the SPL with charting capabilities for the Total Time to Freeze and the Processing Time. These indicators can also help detect problems that may affect PBMC quality. The Total Time to Freeze represents the time elapsed from collection of the blood specimens to the initiation of cryopreservation using rate-controlled freezing. Total times are required to be less than 8 hours. The processing time represents the time elapsed from the initiation of processing (i.e., opening the blood tubes in the biosafety cabinet to initiate processing steps) to the initiation of cryopreservation using rate-controlled freezing. Processing times are required to be less than 4 hours.
Total times in excess of 8 hours suggest possible specimen transport delays from the clinic to the processing laboratory, scheduling problems between the clinic and the processing laboratory, or excessive workload for the level of staffing within the laboratory. Figure 4 provides a summary of the Total Time to Freeze for a 12 month period for SPL #3. Dotted lines mark the expected range of 85 minutes to 480 minutes; 85 minutes was selected as a minimum total time when the transportation time is assumed to be negligible. This report shows that the 8 hour time limit was exceeded 9 times in this period. In follow up to these findings, the SPL and the clinical site indicated there were transportation delays and scheduling problems which led to extended Total Time To Freeze for these nine samples. As a result, a new courier service was implemented and a review of the scheduling tool shared by the clinic and laboratory occurred in January 2013, which resolved the issues for this site.
Figure 4. Site #3, 12 month Time to Freeze Summary Report.
This report includes specimens processed May 2012 through April 2013. Each box plot represents the time to freeze data from a single month for SPL #3. Boxes represent the 25 and 75 percentile, whiskers extend to the most extreme data point within 1.5 interquartile ranges (IQR) and the horizontal line in the box indicates the median time to freezing for the month. Dotted lines indicate the expected range of 85 – 480 minutes.
Processing times in excess of 4 hours may suggest an inexperienced processing technician, workflow interruptions, or equipment problems within the SPL. Figure 5 provides a summary of the processing time for a 12 month period for SPL #3. This chart shows that 7 samples exceeded the processing time limit in four months. Discussions with the laboratory indicated that these events were likely due to staff shortages in July 2012, the introduction of an inexperienced technician in September 2012, and elevated work volumes in November 2012. Corrective actions were implemented in all cases and subsequent months reflect the effects of re-scheduling and retraining; an additional outlier in March 2013 was due to an equipment failure.
Figure 5. Site #3, 12 month Processing Time Summary Report.
This report includes specimens processed May 2012 through April 2013. Each box plot represents the processing time data from a single month for lab #3. Boxes represent the 25 and 75 percentile, whiskers extend to the most extreme data point within 1.5 interquartile ranges (IQR) and the horizontal line in the box indicates the median processing time for the month. Dotted lines indicate the expected range of 85 – 240 minutes.
Like the cell yield, the total and processing time expected ranges are guidelines. Times that exceed these ranges may indicate system problem(s) that need addressing. PMBC specimens that exceed the expected guidelines are stored with comments and may still be used for protocol testing if the thawed viability on day 2 is >66%. If the times grossly exceed the expected ranges, such as a total time >12 hours or a processing time >6 hours, a replacement specimen may be requested.
Management reports are available on the Atlas portal for the Laboratory Operations Unit to allow comparison of IQC data across all labs over a specified date range. Figure 6 provides an example of an IQC Fresh Cell Yield summary report for all SPLs across a one year period. Similar reports are available for Total Time to Freeze and Processing Time. Outlying data may be indicative of a problem at a particular SPL, flagging it for prompt follow up by the Laboratory Operations Unit.
Figure 6. All Sites, 1 Year PBMC Fresh Cell Yield Report.
This report includes specimens processed September 2012 through August 2013. Each box plot represents the cell yield data from a single site for this period. Boxes represent the 25 and 75 percentile, whiskers extend to the most extreme data point within 1.5 interquartile ranges (IQR) and the horizontal line in the box indicates the median cell yield for the month. Dotted lines indicate the expected range of 0.8 – 3.2×106 cells/ml of usable whole blood. The histogram at left shows the overall distribution of cell yields for all sites during this period. Three sites (#6, #18, and #22) are not shown because they did not process specimens for the HVTN during this period.
Between August 2009 and August 2013 over 23,000 blood draws were processed for PBMC cryopreservation. Overall, 95.4% of these samples met the acceptance criteria for fresh cell yield, 99.4% met the criteria for Time to Freeze and 99.3% met the criteria for Processing Time (Table 1).
Table 1.
IQC indicators for samples collected between August 2009 and August 2013.
| Expected Range | Number of Specimens |
Mean | % within acceptable range |
|
|---|---|---|---|---|
| Cell Yield, cells ×106/mL whole blood | 0.8 – 3.2 | 23,477 | 1.45 | 95.4 % |
| Total Time to Freeze, minutes | <480 | 23,477 | 227.9 | 99.4 % |
| Processing Time, minutes | <240 | 23,477 | 122.3 | 99.3 % |
3.4 Component 4: External Quality Control (EQC)
EQC evaluations include PBMC recovery and viability immediately after thawing (Day 1) and following overnight incubation (Day 2). Results are typically posted twice monthly via the Atlas web portal for joint review by the HVTN Laboratory Operations Unit and each SPL. In addition to the individual evaluations, Levy-Jennings and box-plot summary reports are available to the SPLs for monitoring their own overall performance and trends. The summary reports may also be displayed by individual technician for training, troubleshooting or competency assessment requirements.
The most common problem identified in follow up to out-of-specification EQC results is poor cold-chain management. Cold-chain management issues can include exposed storage locations in freezers (e.g., repeated opening of freezer compartment) and exposure during cryovial transfer from the rate-controlled freezing unit to storage, or from storage to packing for shipment. These issues can be corrected by thoughtful designation of freezer storage space and use of dry ice during transfer steps for samples stored at −80°C or LN2 vapor bath during transfer steps for samples stored in LN2. Cold-chain management, training or re-training may also be needed. Additional problems revealed by EQC testing are frequently attributed to less experienced technicians. Corrective actions may include supervision by a certified technician during processing, retraining by more experienced SPL staff or HVTN Laboratory Operations personnel, and evaluating additional EQC specimens for three months following retraining. From August 2009 to present, failed recoveries dropped from 4.1% to 1.6% and failed viabilities dropped from 1.0% to 0.3%. We attribute these improvements to the application of corrective actions.
Some protocols require collection of PBMC from HIV-infected individuals, which may affect PBMC quality. Table 2 summarizes a limited data set taken in 2010 from three South African sites comparing mean PBMC viabilities and recoveries between infected and uninfected specimens. Based on these specimens, mean PBMC viabilities and recoveries from HIV-infected individuals are, on average, only slightly lower than those from uninfected individuals. A larger comparison would be needed prior to changing acceptance criteria for samples from infected participants.
Table 2.
EQC Comparison between PBMC from South African HIV infected and uninfected individuals.
| HIV Status | N | Viability | Recovery | ||
|---|---|---|---|---|---|
| Day 1 | Day 2 | Day 1 | Day 2 | ||
| SA Sites-HIV Neg | 51 | 92.5% | 91.6% | 89.0% | 74.4% |
| SA Sites-HIV Pos | 71 | 88.6% | 89.2% | 83.9% | 73.1% |
EQC data collected on all sites over a four year period assessing 3412 specimens show that 99.3% of the samples met the viability acceptance criteria of 66% for assay use (Figures 7A and 7C). In addition, 97.9% were above the preferred level of viability of 74%. Table 3 shows results for Day 1 and Day 2 PBMC recovery and viability for the same four year period. A higher percentage of specimens met the expected viability criteria than the expected recovery criteria. This is in part due to recovery being highly affected by counting methods and precision in aliquotting during processing.
Figure 7. All sites, 1 Year EQC and AQC Day 2 Viability Reports.
These reports include specimens processed September 2012 through August 2013. PBMC samples were thawed and incubated overnight at 37°C in 5% CO2 for the Day 2 EQC (A) and AQC (B) viability assessments. Boxes represent the 25 and 75 percentile, whiskers extend to the most extreme data point within 1.5 interquartile ranges (IQR) and the horizontal line in the box indicates the median viability for the site. The upper line indicates the preferred minimum viability, the lower line indicates the minimum viability for acceptance in a functional assay, and the shaded area is considered acceptable but marginal. C) Statistics for EQC and AQC results indicate similar trends in viability.
Table 3.
EQC results for August 2009 through August 2013 across all HVTN processing laboratories*.
| Thawed PBMC | Expected Range | Number of specimens |
Mean | % within acceptable range |
|---|---|---|---|---|
| Recovery Day 1 | 70–130% | 3420 | 99.0% | 92.9% |
| Recovery Day 2 | 50–130% | 3412 | 85.8% | 96.2% |
| Viability Day 1 | >80% | 3420 | 93.9% | 99.5% |
| Viability Day 2 | >75% | 3412 | 88.79% | 99.4% |
Includes analysis of specimens from both HIV-uninfected and infected participants.
3.5 Component 5: Assay Quality Control (AQC)
The HVTN Laboratory Program uses AQC data to evaluate the quality of specimens actually used in the clinical trial immunologic assays. As with EQC, the AQC results include PBMC recovery and viability assessed immediately after thawing (Day 1) and following overnight incubation (Day 2). Day 2 viability provides the most useful marker of specimen quality since acceptability criteria are applied to this indicator during the assays (any sample with <66% Day 2 viability is excluded from assay testing). Through the Atlas program, AQC results can be reviewed for a given time period across all sites (Figure 7B) or for a specific protocol (Figure 8).
Figure 8. AQC Day 2 Viability Report for protocol HVTN 205.
This report includes specimens processed September 2012 through August 2013. Only sites that processed for this protocol are shown. PBMC samples were thawed for immunologic assays and Day 2 viability was assessed following overnight incubation at 37°C and 5% CO2. Boxes represent the 25 and 75 percentile, whiskers extend to the most extreme data point within 1.5 interquartile ranges (IQR) and the horizontal line in the box indicates the median viability for the site for the specified period. The upper line indicates the preferred minimum viability, the lower line indicates the minimum viability for acceptance in a functional assay, and the shaded area is considered acceptable but marginal.
The ability to compile all AQC data for a given time period allows the HVTN to evaluate the overall quality of specimens used in immunologic testing. Figure 7C shows that 99.9% of specimens thawed for immunologic assays met the acceptance criteria for use. The ability to compile all AQC data for a given protocol allows the HVTN to provide evidence to the protocol teams and vaccine developers showing the overall quality of all specimens thawed for assay in their protocol, as shown for HVTN 205 in Figure 8. The data clearly show that the overall viability of the samples used for this study was very high (mean, 99.2%) and there were only 4 specimens in the marginal range for viability.
Because some EQC samples are selected as a result of suspected problems at a particular site, the overall EQC results may be slightly worse than that of AQC samples, for which there is no bias in the selection process. Figure 7C compares Day 2 viabilities from EQC and AQC over a common timeframe. As expected, these data show that the AQC data has slightly higher average Day 2 viability (90.67% vs. 88.11%) and lower variability than the EQC data.
4. Discussion
For many years laboratories conducting cellular immunogenicity assessments for vaccine-induced responses in clinical trials lacked high level quality assurance practices, despite the fact that experimental vaccines were in part advanced in clinical trial phases based on data from these assays. This is in part because these assays do not fall within the scope of the established FDA laboratory quality guidelines (GLP, GMP, CLIA). In addition, there is a common misconception that immunologic assays do not have to be validated until the particular vaccine product reaches phase 3 testing. This may hold true for assessments of some drug products, but this approach is not logical in the field of HIV vaccine development. HIV vaccines are generally advanced from phase 1 to phase 2b (efficacy) based on safety and immunogenicity. Since most HIV vaccines tested in recent years have been shown to be safe, the decisions on product advancement and/or down-selection have been primarily based on immunologic data. This has led to increased validation of immunologic assays used to assess vaccine-induced responses as well as the implementation of quality practices in the labs conducting these assays for early phase clinical trials. While cellular immunologic assays are generally conducted on cryopreserved specimens long after collection and processing, it is important consider that the immunologic assay actually begins with sample collection, and the quality of the downstream data is dependent on the quality of the specimen that is thawed for assay. With this in mind it becomes necessary to employ high level, documented laboratory quality practices at the point of specimen collection and maintain this level of quality through processing, shipping, handling, storage and assay of the specimens. This requires extensive laboratory training and oversight at the point of collection and processing by those responsible for the multi-site studies. Here we show that implementing a multifaceted program utilizing personal interactions, networked databases and automated analysis tools can result in the collection, storage and shipping of large quantities of high quality cellular specimens from multiple globally distributed clinical sites while minimizing the number of central oversight personnel needed to implement and oversee quality at the processing laboratories and repositories.
The five component PBMC quality program implemented by the HVTN has allowed the network to successfully oversee processing of high quality PBMC specimens at 28 globally distributed clinical sites contributing to the successful evaluation of over 35 HIV vaccine clinical trials. The ability to electronically store, in a single database, data on specimen quality indicators from both freshly processed cells and thawed specimens has provided the HVTN with the ability to monitor the quality of over 23,000 fresh PBMC specimens and over 6000 cryopreserved samples that were thawed for external quality control or for immunologic assays in a highly cost effective manner. The implementation of this comprehensive program has led to an improvement in the percentage of specimens meeting acceptance criteria for immunologic assays. The provision of web-based tools to the laboratory managers for the analysis of IQC data has improved the internal oversight of quality practices within the laboratories. The example shown in Figures 1 and 2 illustrates a case in which a problem was identified through observation of cell yield trends using these tools and was addressed early within the laboratory. This improvement of internal oversight within the laboratories helps to minimize the required oversight and follow up by the central Laboratory Operations Unit.
Data compiled prior to full implementation of the HVTN PBMC Quality Management Program indicated that the mean Day 2 thawed viability of PBMC samples was only 83.1% and a the mean Day 2 thawed recovery was 67.5%, both well below our current acceptance criteria for cell recoveries after thawing. Key to the immunologic evaluation of vaccine products in HVTN clinical trials, we have shown here that 99.88% of cellular specimens collected over the last four years and thawed for immunologic assays in HVTN clinical trials met the acceptance criteria for use in the cellular assays. with a mean viability of 91.46% ±4.5%.
In recent years funding levels have been flat or even decreasing for most research groups and the HVTN likewise has limited funds to spend on QA programs. However, we feel that all components of our program our necessary for our success. That being said, smaller organizations with even greater resource limitations resources may need to work up to a full PBMC QA program as we did. At a minimum, particularly for sites that are new to processing, it is very important to assess the physical facility, train technicians on the methods for processing and handling and have them participate in proficiency test. If the sites are more experienced and have good GCLP practices in place, monitoring IQC and EQC are more helpful to maintain the acquisition of high quality specimens. Note that beyond the cost of developing the analytical web portal, Components 3 and 5 do not add significant expense to the program. The IQC (Component 3) requires a small amount of time by the site staff to enter and review the data as well as some time in the central Laboratory Operations Unit to review and approve monthly IQC reports. The AQC (component 5) data is automatically collected during the assay process and therefore adds very minimal cost.
While hands on training and diligence in oversight are ongoing requirements, the web-based reporting and analytical tools described here add a high level of efficiency to the HVTN PBMC quality program. We are confident that this robust program is sufficient to handle the current workload as well as the expected expansion in clinical sites that will be need to execute future large scale efficacy studies in international settings. A similar program could be broadly applicable to any multicenter setting that requires high quality cellular samples for clinical trial evaluation.
Acknowledgments
Research reported in this publication/press release was supported by the National Institute of Allergy and Infectious Diseases of the National Institutes of Health under award number UM1AI068618. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
We thank Roy Lewis and Joanne Wiesner for their PBMC assessment expertise and Stephen Voght for assistance with manuscript preparation. We also thank the Deb Bassuk, April Randhawa, Scott Langley, Tobin Stelling, Lloyd Albin, Laura Saganic, Sravani Cheeti and Sara Shoemaker of the Statistical Center for HIV/AIDS Research and Prevention for their work in data management and programming the PBMC quality tools within the Atlas web portal.
Footnotes
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
References
- 1.Aziz N, Margolick JB, Detels R, Rinaldo CR, Phair J, Jamieson BD, Butch AW. Value of a quality assessment program in optimizing cryopreservation of peripheral blood mononuclear cells in a multicenter study. Clin Vaccine Immunol. 2013;20:590–595. doi: 10.1128/CVI.00693-12. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Bull M, Lee D, Stucky J, Chiu YL, Rubin A, Horton H, McElrath MJ. Defining blood processing parameters for optimal detection of cryopreserved antigen-specific responses for HIV vaccine trials. J Immunol Methods. 2007;322:57–69. doi: 10.1016/j.jim.2007.02.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Dyer WB, Pett SL, Sullivan JS, Emery S, Cooper DA, Kelleher AD, Lloyd A, Lewin SR. Substantial improvements in performance indicators achieved in a peripheral blood mononuclear cell cryopreservation quality assurance program using single donor samples. Clin Vaccine Immunol. 2007;14:52–59. doi: 10.1128/CVI.00214-06. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Horton H, Thomas EP, Stucky JA, Frank I, Moodie Z, Huang Y, Chiu YL, McElrath MJ, De Rosa SC. Optimization and validation of an 8-color intracellular cytokine staining (ICS) assay to quantify antigen-specific T cells induced by vaccination. J Immunol Methods. 2007;323:39–54. doi: 10.1016/j.jim.2007.03.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Kierstead LS, Dubey S, Meyer B, Tobery TW, Mogg R, Fernandez VR, Long R, Guan L, Gaunt C, Collins K, Sykes KJ, Mehrotra DV, Chirmule N, Shiver JW, Casimiro DR. Enhanced rates and magnitude of immune responses detected against an HIV vaccine: effect of using an optimized process for isolating PBMC. AIDS Res Hum Retroviruses. 2007;23:86–92. doi: 10.1089/aid.2006.0129. [DOI] [PubMed] [Google Scholar]
- 6.Mallone R, Mannering SI, Brooks-Worrell BM, Durinovic-Bello I, Cilio CM, Wong FS, Schloot NC T-Cell Workshop Committee, I.o.D.S. Isolation and preservation of peripheral blood mononuclear cells for analysis of islet antigen-reactive T cell responses: position statement of the T-Cell Workshop Committee of the Immunology of Diabetes Society. Clin Exp Immunol. 2011;163:33–49. doi: 10.1111/j.1365-2249.2010.04272.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Peiperl L, Morgan C, Moodie Z, Li H, Russell N, Graham BS, Tomaras GD, De Rosa SC, McElrath MJ Network, N.H.V.T. Safety and immunogenicity of a replication-defective adenovirus type 5 HIV vaccine in Ad5-seronegative persons: a randomized clinical trial (HVTN 054) PLoS One. 2010;5:e13579. doi: 10.1371/journal.pone.0013579. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Riccio EK, Neves I, Banic DM, Corte-Real S, Alecrim M, Morgado M, Daniel-Ribeiro CT, Ferreira-da-Cruz Mde F. Cryopreservation of peripheral blood mononuclear cells does not significantly affect the levels of spontaneous apoptosis after 24-h culture. Cryobiology. 2002;45:127–134. doi: 10.1016/s0011-2240(02)00121-9. [DOI] [PubMed] [Google Scholar]
- 9.Sigma-Aldrich. Histopaque-1077 Product Information. In, Vol. 2013. 2011. [Google Scholar]
- 10.Smith JG, Joseph HR, Green T, Field JA, Wooters M, Kaufhold RM, Antonello J, Caulfield MJ. Establishing acceptance criteria for cell-mediated-immunity assays using frozen peripheral blood mononuclear cells stored under optimal and suboptimal conditions. Clin Vaccine Immunol. 2007;14:527–537. doi: 10.1128/CVI.00435-06. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Weinberg A, Louzao R, Mussi-Pinhata MM, Cruz ML, Pinto JA, Huff MF, de Castro AC, Sucupira MC, Denny TN. Quality assurance program for peripheral blood mononuclear cell cryopreservation. Clin Vaccine Immunol. 2007;14:1242–1244. doi: 10.1128/CVI.00187-07. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Weinberg A, Zhang L, Brown D, Erice A, Polsky B, Hirsch MS, Owens S, Lamb K. Viability and functional activity of cryopreserved mononuclear cells. Clin Diagn Lab Immunol. 2000;7:714–716. doi: 10.1128/cdli.7.4.714-716.2000. [DOI] [PMC free article] [PubMed] [Google Scholar]