A Biospecimen Proficiency Testing Program for Biobank Accreditation: Four Years of Experience

Abstract

Biobanks produce and distribute biospecimens, ensuring their fitness for purpose and accurately qualifying them before distribution. In their efforts toward professionalization, biobanks can nowadays seek certification or accreditation. One of the requirements of these standards is regular participation in Proficiency Testing (PT) programs. An international PT program has been developed and provided to biobanks and other laboratories that perform specific tests to qualify different types of biospecimens. This PT program includes biospecimen testing schemes, as well as biospecimen processing interlaboratory exercises. This PT program supports the development of biobank quality assurance by providing the possibility to assess biobank laboratory performance and useful insights into biobank laboratory method performance characteristics and thus fulfill the demands from accreditation authorities.

Introduction

A biobank is an infrastructure for collecting, processing, storage, and distribution of biological samples and associated data in compliance with standard operation procedures (SOPs), for use in research. In the last 15 years, biobanks have rapidly evolved due to fast-growing technology in high-throughput omics analysis. As a consequence, biospecimens and their associated data are being requested by research organizations and industries with increased frequency.

To meet the demands of access to sufficient number of samples of adequate and proven quality, biobanks must ensure that their biospecimens and their derivatives (DNA, RNA, protein, metabolites, etc.) are adequately described in terms of concentration, purity, viability, integrity, identity, etc. Each biobank defines its own specimen preparation protocols without necessarily having validated the method and without prior comparison with other methods, which may lead to discrepancies in the specified quality and integrity of individual samples.¹ Most biobanks perform some testing on the samples or rely on external parties/laboratories for such testing and characterization. Current biobank certification or accreditation standards include national programs, international certification to ISO9001 (Quality Management Systems—Requirements),² and accreditation to ISO17025 (General Requirements of the Competence of Testing and Calibration Laboratories)³ or ISO15189 (Medical Laboratories Requirements for Quality and Competence).⁴ Certain national certification standards include requirements for method validation (e.g., NF S 96-900 Qualité des centres de ressources biologiques (CRB)—Système de management d'un CRB et qualité des ressources biologiques d'origine humaine et microbienne⁵). Proficiency Testing (PT) schemes support such validation; furthermore, participation in a PT program is a normative requirement of ISO17025, ISO15189, and the College of American Pathologists (CAP) biorepository accreditation standard.^6–8

Bearing this requirement for PT schemes in mind, a multischeme international PT program in compliance with ISO17043 standards (Conformity Assessment—General Requirements for Proficiency Testing),⁹ also called an External Quality Assessment (EQA) program, was designed under the auspices of the International Society for Biological and Environmental Repositories (ISBER) and the Integrated Biobank of Luxembourg (IBBL) and has been offered annually to biobanks, biobank-subcontractor laboratories, research organizations, and bioservice providers, who may have a parallel biobanking activity, since 2011.¹⁰ The aim of the PT program is to assess laboratory performance for specific measurements or tests, to foster and monitor a laboratory`s continuing performance, and to stimulate the standardization of procedures to achieve more uniform results.¹¹ The PT program was designed to include both processing and characterization/testing schemes. Distinguishing between the processing schemes and testing schemes allows the PT provider to specifically pinpoint potential sources of error in the participant's routine workflow. Therefore, the goals of the biorepository PT program are to assess (1) the efficiency of laboratories that process biomaterial (i.e., “Processing Schemes”) and (2) the accuracy and precision of laboratories that perform tests characterizing biomaterials used in research (i.e., “Testing Schemes”). Finally, PT programs provide evidence of the reproducibility and robustness of the methods.

The PT schemes correspond to routine workflows carried out in a laboratory and include the most widespread commonly applied assays used across biobanks (Fig. 1). In the future, as more and novel biospecimen quality control assays are developed, these will be progressively implemented into new schemes.

FIG. 1.

Activities undertaken by biobanks from the collection of a specimen to the distribution of samples. The proficiency/competency of processing and testing methods performed by the laboratory is assessed through participation in, respectively, Processing and Testing Schemes. PT, Proficiency Testing; QC, Quality Control.

Materials and Methods

Logistical support for PT programs

The biorepository PT program is developed and maintained by IBBL and overseen by an international Advisory Group (AG). IBBL is responsible for all the PT program-related activities and is accountable for the compliance of the program with ISO/IEC 17043:2010 standards.⁹ A project manager is responsible for the day-to-day management (e.g., managing the logistics of registrations, communicating with participants, collecting, collating, and analyzing the results, publishing the final reports, and managing all the internal PT-related documentation and records). Members of the scientific staff are responsible for preparing the samples to be tested/processed, assessing their homogeneity and stability, and shipping them to the participants.

The AG is an external and independent group of experienced professionals who support the PT programs` operations. The AG is composed of ISBER members, with relevant technical knowledge and experience. At least three laboratories accredited/certified to ISO17025,³ Clinical Laboratory Improvement Act (CLIA), or equivalent for the methods foreseen in the PT programs are represented within the AG. The AG acts “internally” in PT program design and implementation (e.g., in the proactive identification and resolution of difficulties associated with the preparation and maintenance of homogeneous PT items or in the provision of stable assigned values) and “externally” in supporting and mentoring the PT participants.

The biorepository PT programs are implemented annually, with each program containing one or more “schemes.”

Testing and processing items

Each annual PT program includes schemes for the testing and/or processing of items (e.g., nucleic acids, cells, whole blood), prepared by IBBL in compliance with ISO/IEC 17025:2005.³ IBBL's internal procedures describe the preparation of bulk material, homogeneity and stability testing, establishment of reference values, preparation of aliquots, labeling, and storage conditions before shipment to participants (Fig. 2). Collection of specimens is performed with approval of the Ethics Committee (Comité National d'Ethique de Recherche [CNER] Reference 201107/02).

FIG. 2.

General workflow for preparation, shipment, and reporting of testing and processing schemes. DNA quantification and purity and DNA extraction from whole blood schemes are shown as examples. Items in italics correspond to the items shipped to participants. * Only applicable when reference laboratories determined the assigned value. When the assigned value corresponded to the consensus mean of participants, it was determined during data analysis.

Homogeneity and stability

In each PT program, IBBL demonstrates the homogeneity and stability of the items. To ensure appropriate processing or testing and reliable results, the items must be adequately homogeneous and stable in their final packaging under the storage and transport conditions.

Information on items provided to participants

The participating laboratories receive detailed information and instructions on how the test items were prepared, safety information, storage conditions, and other factors that could influence the testing/processing, which data to submit and timelines of return of data. Participants are instructed to use their routine procedures when participating in the PT program.

Assigned value, total uncertainty, and PT standard deviation

To assess participants' proficiency, both a target value and an accuracy range must be defined. Laboratory methods have intrinsic variability (operator, day, etc.), which makes it impossible to know the exact expected value. As per standard ISO/IEC 13528:2005 (Statistical Methods for Use in Proficiency Testing by Interlaboratory Comparison),¹² several methods allow the determination of the assigned value (X_CRM), its total uncertainty (u_CRM), and the PT standard deviation (σ_p):

• • In certain schemes, sampling aliquots of PT items are analyzed by expert laboratories, which typically belong to the AG. X_CRM corresponds to the robust average of the results determined by these laboratories (at least 18 measurements per laboratory). u_CRM is based on the measurement, homogeneity, and stability uncertainties. When the consensus value measured by expert laboratories is used as X_CRM, σ_p is defined by the AG as the performance level that should be achieved by the participants in a particular assay.

• • If specific technologies/kits are included in a scheme, items are sent to the technology/kit manufacturer, who performs at least 18 measurements on each PT item. X_CRM for that testing method will be calculated based on the mean of these measurements.

• • For other schemes, the robust average of participants' results is implemented as X_CRM when at least two successive rounds of the scheme have shown that the mean of participants' measurements is within the limits of X_CRM ± 0.5 σ_p. In this case, u_CRM is determined with the following formula:

S_R² is the reproducibility variance and S_r² is the repeatability variance.

When the consensus mean of all the participants is used as X_CRM, σ_p corresponds to the reproducibility standard deviation of all participants after outliers have been removed. Therefore, each participant contributes to the estimation of the target value and range of accuracy.

Participants' registration, shipment of items, and submission of results

The PT programs have been enhanced by the implementation of validated software allowing an efficient planning, organization, and analyses of the interlaboratory tests. This software is linked to a web-based platform, which is accessed by the participants for registration and result submission. During the registration process, a unique code is automatically assigned to each participant to ensure confidentiality.

PT items are prepared and packed according to ICAO/IATA regulations, and shipped either at room temperature or on dry ice depending on their type and stability.

In the Testing Schemes, participants test these items according to their routine method(s). In the Processing Schemes (DNA and RNA extraction), the participants perform the extraction and return DNA or RNA to IBBL. The IBBL laboratory then tests all the samples in a single run (isochronous testing) to measure the appropriate parameter(s) (e.g., yield).

Participants' results are collected through the web-based platform, which contains scheme-specific data-entry questionnaires. During the data submission process, participants enter the result for the measured parameter (e.g., concentration, integrity, percentage of viable cells) and complete a short questionnaire to provide additional information such as the dates of reception and testing/processing, the method/equipment used, the equipment performance verification, and any issue encountered during the testing/processing. Since the 2014 program, participants can enter replicate measurements for certain schemes, which allow for the assessment of their internal precision and accuracy.

Data analysis and reporting

A statistical design is decided in advance to meet the objectives of each scheme, based on the nature of the data (quantitative or qualitative), the statistical assumptions, the sources of uncertainty, and the expected number of participants. Each participant's performance is assessed in two calculative steps: (1) individual performance statistics and (2) performance assessment against peers. Based on participants' performance, IBBL and the AG provide expert comments to the participants concerning potential sources of error and educational feedback for continuous improvement of the performance.

Performance statistics calculation

The statistical analysis is performed using a validated software. Outliers are determined by the Cochran and/or Grubbs' test. Although outliers are excluded from the summary statistics, they are taken into account in the individual participant's statistics. The software produces a graph presenting the results from all participants and provides descriptive statistics (mean, median, standard deviation, and range).

Interlaboratory performance assessment

Participants' results are converted into z-scores (Table 1) based on the equation z = (X − X_CRM)/σ_p, where X_CRM is the assigned value and σ_p is the fitness for purpose-based PT standard deviation.

Table 1.

z-Scores and Relative Performance as Shown in Reports Provided to Participating Laboratories

Distance from assigned value (z-score)	Consensus score	Performance
<1 standard deviation	0	Very satisfactory
>1 standard deviation and <2 standard deviations	1	Satisfactory
>2 standard deviations and <3 standard deviations	2	Questionable
>3 standard deviations	3	Requiring action

In Testing Schemes, a graphical summary of all coded participants' results (X_CRM ± tolerance limits) and a graphical overview of z-scores provide the comparison of each participant's performance against X_CRM and that of their peers. These graphs enable participants to visualize their results in relation to those obtained by other participants.

In Processing Schemes, the mean and standard deviation of all parameters are determined based on participants' results and converted into z-scores. A graphical summary of all participants' results (participants mean ± 3 standard deviation) provides the comparison of performance against peers.

For each scheme, the participant's performance in terms of z-scores is graphically presented over years.

Intralaboratory performance assessment with precision and accuracy estimation (Youden plots, Lischer plots)

In each report, Youden plots (Fig. 3) are generated to compare the results obtained by each participant for two different items in the same round. Youden plots are a simple, but highly informative method for comparing both the intralaboratory variability and the interlaboratory variability, and pinpoint systematic bias and random errors. In addition, if multiple measurements are reported by a participant, the independent analysis of the participant's uncertainty of measurement provides the “laboratory internal standard deviation.” In this case, Lischer plots (Fig. 4) are generated to analyze both the accuracy and the precision of the laboratories that carried out replicate measurements for the same item. Lischer plots indicate the intralaboratory variability for a method, which allows for the identification of irregularities between different analyses performed in a laboratory for the same parameter, for example, performed by different operators, on different pieces of equipment or on different days.

FIG. 3.

Example of Youden plot displaying the comparison between Tube B (X-axis) and Tube C (Y-axis). Points correspond to Participants' results. Points that lie near the 45° reference line, but far from the crossing of the assigned values, indicate large systematic error. Points that lie far from the 45° line indicate large random error. Points outside the rectangle indicate large total error.

FIG. 4.

Example of Lischer plot displaying the intralaboratory variability for a method. Lischer plots are constructed by plotting the standard deviation (Y-axis) against the corresponding laboratory mean value (X-axis) of all results. A confidence level of 95% is statistically calculated. The location of laboratory (points) and method combinations inside the 95% area indicates the absence of significant differences for the test results within a laboratory.

Results

Global evolution from 2011 to 2014

The number of schemes and the number of participants per scheme for each program are shown in Table 2. Due to import restrictions, China was not able to participate in the 2014 program, which explains the recent significant reduction in the number of participants from Asia (Fig. 5).

Table 2.

Schemes Implemented from 2011 to 2014

	Scheme	2011	2012	2013	2014	Total
Testing schemes	DNA quantification and purity	32	32	63	50	177
	RNA integrity	24	25	45	30	124
	Cell viability		12	23	17	52
	Tissue histology		18	21	10	49
	Tissue antigenicity		4			4
Processing schemes	DNA extraction from whole blood			23	48	71
	DNA extraction from FFPE cells				13	13
	RNA extraction from whole blood				18	18
Total		56	91	175	186	508

FFPE, formalin-fixed paraffin embedded.

FIG. 5.

Geographical distribution of participants over the four annual PT programs.

In the DNA Quantification and Purity Scheme, participants received two or three samples of DNA and were asked to assess the concentration (μg/mL) and purity (260/280 ratio) using their routine method (spectrophotometry, spectrofluorometry, Trinean spectrophotometry with cDrop software, or “other”).

In the RNA Integrity Scheme, participants received two or three samples of RNA and were asked to measure the integrity (using Agilent Bioanalyzer, Biorad Experion, ScreenTape R6K, QiaXcel, Fragment Analyzer, or “other”).

In the Cell Viability Scheme, participants received two or three aliquots of cells and were asked to measure the viability (%) (using trypan blue or flow cytometry).

In the Tissue Histology Scheme, participants received one, two, or three hematoxylin and eosin slide images taken from human tumor formalin-fixed paraffin-embedded (FFPE) blocks and were asked to assess the proportion of normal tissue and viable tumor tissue areas, by visual examination.

In the DNA Extraction from Whole Blood Scheme, participants received one PAXgene Blood DNA tube containing whole blood from which they were required to extract the DNA. Extracted DNA was subsequently returned to IBBL, which performed spectrophotometric assessment of yield (μg DNA/mL blood) and purity (260/280 ratio) as quality metrics.

In the RNA Extraction from Whole Blood Scheme, participants received one PAXgene Blood RNA tube containing whole blood from which they were required to extract the RNA. Extracted RNA was subsequently sent back to IBBL, which performed spectrophotometric assessment of yield (μg RNA/mL blood) and purity (260/280 ratio), and RNA Integrity Number (RIN) as quality metrics.

In the DNA Extraction from FFPE Cells Scheme, participants received two FFPE sections of 20 μm thickness from which they were required to extract DNA. Extracted DNA was then returned to IBBL, which performed spectrophotometric assessment of yield (μg DNA/20 μm FFPE slice) and purity (260/280 ratio) as quality metrics.

Evolution of statistics from 2011 to 2014

When σ_p is set by the AG, there is a risk that the predefined σ_p value is below the method's reproducibility among the laboratories that are participating in the scheme. However, the correlation between the participants' standard deviation and the σ_p demonstrates that the AG has always defined achievable levels of performance, in accordance with the methods (Supplementary Tables S1–S5; Supplementary Data are available online at www.liebertpub.com/bio).

As expected, most participants used spectrophotometry to measure DNA concentration in the DNA Quantification and Purity Scheme. Based on participants' data submission, the most commonly used spectrophotometers are the NanoDrop™ (Thermo Scientific) or (multimodal) absorbance readers (e.g., Biotek). Commonly used spectrofluorometers are Qubit^® (Life Technologies) and, again, multimodal absorbance readers (e.g., Biotek). Microfluidics Lab On Chip method was used by only one participant over the 4-year period. The use of the Trinean spectrophotometer with cDrop software increased during this time period.

Spectrofluorometry for DNA quantification has provided lower concentration values and higher variability (standard deviation) compared to spectrophotometry over the 4-year time period. Reproducibility of spectrofluorometric measurements was lower with higher coefficients of variation. Trinean spectrophotometry with cDrop software generally returned lower and more accurate DNA concentration values, that is, very low standard deviation. The PT program results confirmed that Trinean spectrophotometry with cDrop software provides more accurate data than simple spectrophotometry and data that are equivalent to spectrofluorimetric data.

Comparison of ratio measurements made by spectrophotometry and Trinean spectrophotometry with cDrop software did not show a significant difference, either in measured ratio or standard deviation.

In the RNA Integrity Schemes, participants submitted their results in units corresponding to their test method (RIN, RQI, SDV, etc.). For statistical data analysis, all units were transformed into RIN, based on verified conversion formulae. Most participants used the Bioanalyzer (Agilent Technologies) to assess RNA integrity (99 over the 4 years). The Experion (Bio-Rad) was used by few participants (5 over the 4 years) and the Fragment Analyzer™ (Advanced Analytical Technologies), QIAxcel (Qiagen), and ScreenTape R6K (Agilent Technologies) methods each had only one participant during this time period.

In the Cell Viability Scheme, participants obtained similar results by using either trypan blue or flow cytometry.

Participants mainly used column-based methods (magnetic bead based or silica membrane based) for nucleic acids extraction from whole blood or FFPE cells (Supplementary Tables S1–S5). Regarding the extraction principle, several participants reported results under “other,” although they used magnetic bead-based or silica membrane-based methods. Slightly more participants used automated (13 participants in 2013 and 24 participants in 2014) than manual (8 participants in 2013 and 23 participants in 2014) extraction methods in the DNA Extraction from Whole Blood Scheme. In 2014, automated extractions led to significantly higher DNA yields (t-test, p = 0.0386). In contrast, when extracting RNA from whole blood, manual processing (12 participants) was more used than automated (6 participants) in 2014, and no significant difference in RNA yields was observed when comparing automated and manual extractions.

In general, there was a high variability among participants and methods for all parameters measured in Processing Schemes. Moreover, variability was high in each particular method, and therefore, no statistically significant differences between methods could be seen. For instance, in 2014, no significant differences in nucleic acid yields were observed between the different methods (one-way ANOVA, DNA Extraction from Whole Blood: p = 0.955; RNA Extraction from Whole Blood: p = 0.288; DNA Extraction from FFPE Cells: p = 0.371). The same observation was made for the nucleic acid purity (OD 260/280) (one-way ANOVA, DNA Extraction from Whole Blood: p = 0.556; RNA Extraction from Whole Blood: p = 0.086; DNA Extraction from FFPE Cells: p = 0.699).

Performance evolution

On average, more than 70% of participants successfully passed the proficiency tests, in all schemes. The only exception to this average was observed in the DNA Quantification and Purity Scheme of 2014, in which 60% of participants obtained passing rates (Fig. 6).

FIG. 6.

Participants' successful results (Bars) and satisfaction (Curve). The pass rates were determined based on participants with z-scores between −2 and 2 (Very Satisfactory and Satisfactory Results). The satisfaction was determined based on customer satisfaction surveys conducted after each round of the program, in which the participants were asked their overall level of satisfaction for the program. DNA, DNA Quantification and Purity Scheme; RNA, RNA Integrity Scheme; CELL, Cell Viability Scheme; THIS, Tissue Histology Scheme; DNABLD, DNA Extraction from Whole Blood Scheme; RNABLD, RNA Extraction from Whole Blood Scheme; DNAFFC, DNA Extraction from FFPE Cells. FFPE, formalin-fixed paraffin embedded.

Approximately 50% of the participants dropped out of the PT program after the first year of their participation (Fig. 7). This observation does not take into account the laboratories who registered for the first time during the 2014 PT program (25 new participants in 2014).

FIG. 7.

Number of laboratories participating in one, two, three, or four PT programs. Participation was not necessarily consecutive. The pie chart includes all new Participants in the 2014 PT program.

Laboratories that have participated in PT schemes over several years have seen global improvement in their performance in terms of their z-scores (Fig. 8).

FIG. 8.

Evolution of Participants' performance in terms of z-scores after 1, 2, 3, or 4 years of participation in the (a) DNA Quantification and Purity Scheme (Assessment of DNA concentration by spectrophotometry), (b) DNA Quantification and Purity Scheme (Assessment of DNA 260/280 ratio by spectrophotometry), (c) RNA Integrity Scheme (Assessment of RNA Integrity using the Bioanalyzer).

Discussion

Comparison of testing methods

Assessment of DNA quantification and purity

Downstream analyses on DNA samples range from PCR amplification to next-generation sequencing. Each of these analyses requires accurately quantified high-quality DNA (not contaminated by upstream reaction reagents).^13,14

As observed in the four annual PT programs, spectrophotometry is the most commonly used method by biobank laboratories for DNA quantification and purity assessment. NanoDrop spectrophotometers are most commonly used to assess DNA concentration and purity (ratio 260/280), possibly due to their ease of use, low sample-volume requirement, and speed of analysis. However, spectrophotometry is not selective as all nucleic acids and potential contaminants absorbing light at 260 nm are detected (e.g., free nucleotides, RNA, single-stranded and double-stranded DNA, and some proteins and salts). This leads to significant overestimation of DNA concentration, hence inaccurate results.¹⁵ Moreover, the sensitivity of spectrophotometric methods is generally inadequate, preventing quantification of DNA samples of low concentrations.^13,16

On the other hand, UV-induced fluorescence methods using intercalating dyes, such as PicoGreen, are selective, precise, and sensitive methods to quantify double-stranded DNA. The intercalating dye selectively interacts with double-stranded DNA, which avoids interference of other nucleic acids (RNA, single-stranded DNA). Compared to spectrophotometry, spectrofluorometry can measure very low DNA concentrations (10 pg/μL), but does not provide information on DNA purity and possible contaminants.¹⁷ In the PT program, spectrofluorometry had higher variability, as shown by all four programs, and was often associated with systematic errors.

Microfluidics Lab On Chip technologies provide reliable qualitative information on molecular integrity, but less accurate/precise data on concentration. They are generally more expensive than other platforms. There were too few participants using this method to enable us to draw meaningful conclusions about its functionality.

Assessment of RNA integrity

The Bioanalyzer and Experion are two automated platforms that use “lab-on-chip” technology based on electrophoresis. RNA quality is expressed in RINs for the Bioanalyzer and in RNA Quality Index (RQI) numbers for the Experion, both ranging from 1 (RNA completely degraded) to 10 (RNA completely intact). It has been suggested that the Experion returns more reproducible results with a higher sensitivity than the Bioanalyzer due to its more automated sample preparation and chip loading procedure.^18,19

However, due to the low number of participating laboratories using the Experion and other alternative methods, a platform comparison becomes problematic. A comparison between the Agilent and Qiaxcel technologies has shown that Qiaxcel is more precise for certain types of RNA samples.²⁰ All four annual PT programs showed that the Agilent Bioanalyzer is associated with random errors, especially between RINs 4 and 7.

RNA integrity may influence downstream results such as gene expression analyses. Therefore, a precise assessment of RNA integrity is of major importance and biorepositories are encouraged to evaluate RNA quality using methods other than spectrophotometry, which only provides partial information on protein and organic solvent contamination.²¹

Assessment of cell viability

Cell viability is an important quality attribute of cryopreserved cells for many downstream applications such as functional tests, establishment of cell lines, or immunophenotyping.²² The most common methods for cell viability assessment are trypan blue exclusion and flow cytometry. Both methods are based on plasma membrane integrity and use dyes that penetrate into cells with compromised membrane integrity (trypan blue dye) or intercalate DNA (propidium iodide). Manual trypan blue exclusion is a time-consuming method when a large number of samples are analyzed, and being highly operator dependent, often provides results of low precision and reliability. Cellometer (Nexcelom Bioscience) is based on the same principle as trypan blue, but is automated and more precise.²³ Flow cytometry gives a high-speed analysis of cells in suspension, with a distinction between viable and nonviable cells. Flow cytometry requires highly trained operators and expensive systems.²³ Small variations have been observed, as trypan blue dye generally provides higher viability proportion than flow cytometry.²⁴ These discrepancies may be explained by the individual uncertainties of microscopic examination and the lower sensitivity of optical microscopy to detect unstained cells. Both factors may then lead to overestimation of viable cells by trypan blue determination.²³ However, when a sufficiently high number of cells were counted, good agreement has been shown between trypan blue exclusion and flow cytometry measurements.^23,25 This was confirmed by the results of the PT program, as no significant difference was observed between trypan blue and flow cytometry results.

The PT program results suggest that both dye methods are comparable and can provide suitable results on cell viability.

Assessment of viable tumor content

While PT schemes for clinical histopathology exist, these focus on histopathological diagnostic tests and not on assessment of viable tumor content. The latter measurement represents a critical annotation of tumor tissue samples provided by tumor banks. The percentage of tumor content often determines the suitability of a tissue sample for downstream molecular analyses, for example, next-generation sequencing.

Comparison of processing methods: nucleic acid extraction

Nucleic acid extraction is a crucial step in biobank laboratories, as the output samples constitute the starting material of many applications.²⁶ It is important that contamination and degradation of DNA and RNA samples are avoided, as the quality and integrity of nucleic acids are likely to directly influence the results of downstream research studies.^27,28

Hence, processing schemes for DNA and RNA are very important and informative as they assess the ability of laboratories to achieve the best yield, quality, and integrity of DNA and RNA, and will pinpoint suboptimal laboratory workflows or the need of training.

Whole blood

Protocols for extracting nucleic acids from whole blood can be divided into column-based and solution-based methods. Column-based methods can be subdivided into silica membrane based and magnetic bead based, both involving three steps: binding using adsorbent surfaces, washing, and eluting the nucleic acids. Both column-based methods are fast, convenient, relatively inexpensive, and efficient methods of nucleic acid purification. Magnetic bead-based extractions are also easily transposable to automated platforms. Chaotropic salts such as guanidium thyocyanate are commonly used in nucleic acid purification as they disrupt membranes and denature RNases.^29,30 Solution-based extraction methods lead to higher yields of DNA, but also to a higher variability. On the other hand, column-based methods lead to slightly lower yields, but have a more consistent performance, as shown by the PT program results.

When comparing manual and automated extraction, laboratories participating in the PT scheme largely used automated extraction in the DNA Extraction from Whole Blood Scheme, increasing the reproducibility of results (yield and purity) and saving operator time. Automated extractions enable standardized sample processing, limiting errors and contamination. Therefore, although manual nucleic acid extractions provide reliable and suitable results when purifying smaller numbers of samples, biobanks prefer automated extraction when high throughput is required to increase the workflow efficiency and standardization, while decreasing variability.^31,32

Similar quality (OD 260/280) was observed for manual and automated DNA extractions from whole blood. However, in the RNA Extraction from Whole Blood Scheme, a lower purity was observed for participants using phenol/Trizol-based extraction, which could be explained by a probable contamination by organic solvents. The results obtained by participants using the column-based methods were consistent with previous data showing that similar RNA quality, in terms of purity and RIN, was obtained independently from the extraction method (silica membrane vs. magnetic beads).^33,34

FFPE cells

DNA extracted from FFPE tissues is suitable for PCR, microarray, and sequencing analyses, although it is often difficult to amplify long fragments due to DNA fragmentation and cross-linking.³⁵ Formalin causes cross-linking between nucleic acids and proteins, which can prevent amplification.³⁶ Since factors such as the type of tissue, block size, fixation time, block age, and storage temperature can also influence DNA degradation, the choice of DNA extraction method must prioritize both the quantity and quality of DNA.^37,38 FFPE sections require deparaffinization, heating (to reverse formalin cross-links), and protein digestion before the DNA can be purified; each process potentially causing further degradation and contamination of the purified DNA.

A specific national scheme on DNA extraction from FFPE has recently been described in Germany. The results of this scheme showed great heterogeneity in the quality of extracted DNA.³⁹

The PT results confirmed this heterogeneity among participants and showed no significant difference in terms of quality or quantity of the extracted DNA between the different extraction methods.

Why should biobank laboratories participate in PT programs?

Participation in PT programs brings many advantages to biobank or biobank-subcontractor laboratories such as confirming/improving performance, identifying testing or processing problems, comparing methods and procedures, determining method precision and accuracy, comparing operator capabilities, educating staff, and instilling confidence in staff and external users of laboratory services.⁴⁰ PT participation also helps laboratories assess the strengths and weaknesses of their procedures by comparing their reproducibility with those used by their peers. By providing useful information on method-trueness (especially where Reference Materials do not exist), the results following participation in PT programs can also be used to support analytical method validation.^41,42 PT programs can also highlight potential technical issues with either equipments or reagents. Laboratories that have the highest bias also have the largest z-scores (3 < z-score < −3) and should immediately take corrective actions.

The most common error sources that were found during the 4 years of PT programs were linked to equipment calibration, operator training, and sample dilution factors.

For all these reasons, PT programs represent a very important part of a biobank laboratory's quality management system. All laboratories should implement a complete set of measures to ensure that they always achieve and provide high-quality data and monitor the reliability of these data. This includes the use of SOPs and validated protocols, the use of internal quality controls (reference materials, control charts, etc.), participation in PT programs, and certification/accreditation to a recognized standard (ISO/IEC). It has already been shown that laboratories participating in PT programs have robust procedures for internal quality controls and obtain better z-scores.⁴³

The importance of participating in PT programs is actively supported by ISO standards. As stated in ISO 13528:2005,¹² “a single z-score falling outside the range from −3 to 3 (“Requiring Action”) in a round or two z-scores in the range −2 to −3 and 2 to 3 (“Questionable”) in successive rounds should be taken as evidence that an anomaly has occurred that requires investigation.” Moreover, it is recognized that recurrent participation leads to improved performance in PT programs,⁴¹ which was confirmed by our results in the DNA Quantification and Purity and RNA Integrity Schemes (Fig. 8). This has been explained by the identification and correction of problems by laboratories, the adoption of more accurate and reproducible methods, and improvement in technical training.⁴⁴

To ensure the consistency of the SOPs and their quality management system, many biobanks are seeking certification and/or accreditation. PT programs represent valuable tools supporting certification/accreditation. Indeed, a laboratory participating in a PT program will likely instill confidence in its customers, regulators, and accreditation bodies. As per ISO/IEC17011 (Conformity assessment—General requirements for accreditation bodies accrediting conformity assessment bodies),⁴⁵ accreditation bodies are strongly encouraged to take into account a laboratory's participation and performance in PT programs.

There is a growing and urgent need for guidelines to eventually harmonize results of processing and testing methods in biobank- and biobank-subcontractor laboratories. The World Health Organization Good Clinical Laboratory Practice (GCLP/08)⁴⁶ already recommends that clinical laboratories should participate in PT programs to ensure that their data are generated with timeliness, accuracy, and clinical relevance, to increase confidence of third parties in generated results and ensure that human specimens are tested accurately and reliably.⁴⁷ Research laboratories and biobank laboratories often lack the external oversight on their routine procedures that are conferred by PT programs. While no internationally recognized standard on biobanking practices currently exists, a newly constituted ISO Technical Committee (ISO TC276) is expected to address these issues.

Conclusion

Biobank laboratories should always seek the highest quality of data to annotate and qualify their biospecimens. In this context, recurrent participation in PT schemes enables an objective and independent assessment of a laboratory's performance to guarantee this high quality. As part of their quality management system, biobank laboratories should implement a complete set of quality control measures, including the use of internal quality controls and participation in PT programs.

Schemes being currently developed include testing schemes (RNA quantification and purity, Hemoglobin quantification in plasma, Hemoglobin quantification in CSF, and CD154 (sCD40L) quantification in serum) and processing schemes (Viable PBMC isolation and RNA extraction from FFPE cells).

Acknowledgments

We are grateful to William Mathieson for critical reading of the article and Francesca Poloni and Michele Zink for initiating the PT program project management. We are grateful to Christian Bläul and the QuoData scientific and technical team for software development and validation.

Author Disclosure Statement

No conflicting financial interests exist.