Standardization of Nucleic Acid Tests: the Approach of the World Health Organization

The first World Health Organization (WHO) international standards (ISs) for nucleic acid amplification techniques were established two decades ago, with the initial focus on blood screening for three major viral targets, i.e., hepatitis C virus, hepatitis B virus, and human immunodeficiency virus 1. These reference materials have subsequently found utility in the diagnosis and monitoring of a wide range of infectious diseases in clinical microbiology laboratories worldwide.

ABSTRACT

The first World Health Organization (WHO) international standards (ISs) for nucleic acid amplification techniques were established two decades ago, with the initial focus on blood screening for three major viral targets, i.e., hepatitis C virus, hepatitis B virus, and human immunodeficiency virus 1. These reference materials have subsequently found utility in the diagnosis and monitoring of a wide range of infectious diseases in clinical microbiology laboratories worldwide. WHO collaborating centers develop ISs and coordinate international studies for their evaluation. The WHO Expert Committee on Biological Standardization is responsible for the endorsement of new standardization projects and the establishment of new and replacement ISs. Potencies of ISs are defined in international units (IU); the reporting in IU for assays calibrated with an IS (or secondary standards traceable to the IS) facilitates comparability of results for different assays and determination of assay parameters such as analytical sensitivities.

INTRODUCTION

Nucleic acid amplification technology (nucleic acid testing [NAT] or nucleic acid amplification testing [NAAT]) has become a staple in both clinical microbiology laboratories and blood-screening centers for the detection of microbial pathogens, particularly viruses. This was not the case more than two decades ago, with the transmission of hepatitis B virus (HBV), hepatitis C virus (HCV), and human immunodeficiency virus 1 (HIV-1) to recipients of therapeutic plasma derivatives or blood components, when it was realized that closing the serological window using NAT improved blood safety. In the following years, considerable effort was invested in the implementation of NAT screening for blood and plasma donors and the introduction of this technology for diagnostic testing in clinical microbiology laboratories, using both commercial tests and laboratory-developed tests (LDTs). However, assay sensitivities and specificities varied widely between laboratories, contamination by amplicons was problematic, and assays lacked standardization. During this time, the World Health Organization (WHO), as the global institution for setting standards for health systems, was requested to establish internationally accepted reference materials, e.g., international standards (ISs), for NAT assays. The ISs are measurement standards with defined concentrations of specific analytes that enable the comparison of results among different assays and different laboratories. These reference materials initially were prepared from viremic plasma donations (reflecting the type of sample being tested) and freeze-dried. The complex nature of donor and clinical samples, such as plasma or serum samples, means that nucleic acid measurement of a specific pathogen cannot be determined by physicochemical methods. Before nucleic acid concentrations can be determined, samples must be extracted and undergo in vitro amplification and detection; therefore, results cannot be simply reported in International System of Units (SI)-related units such as kilograms or moles. For WHO ISs representing complex biological materials, the WHO took the approach of adopting international units (IU); IU have been used to define potencies of all ISs for NAT-based assays. In this review, we discuss the steps involved in prioritization and in the preparation and characterization of WHO ISs, their establishment and replacement, and realization of their value in harmonizing results among different assays and different laboratories.

SETTING PRIORITIES FOR NAT STANDARDIZATION

An international working group on standardization of genomic amplification techniques (SoGAT) was established in 1995, on behalf of the WHO, and has since been coordinated by the National Institute for Biological Standards and Control (NIBSC) (United Kingdom). Initially, the focus was to standardize NAT assays for blood-borne pathogens important in the field of blood safety; however, standardization was also essential for the diagnosis and monitoring of infectious diseases in clinical settings. WHO ISs for pathogens such as HCV, HBV, and HIV-1 have been widely used in microbiology laboratories as well, and new standards have been prepared for increasing numbers of clinically important pathogens.

The first WHO IS for NAT assays, established in 1997, was HCV (1), followed by HBV and HIV-1 in 1999 (2, 3). Subsequently, ISs were established for other blood-borne viruses, including parvovirus B19 (B19V), hepatitis A virus (HAV), HIV-2, hepatitis E virus (HEV), and hepatitis D virus (HDV) (4–8) as well as human cytomegalovirus (CMV) and Epstein-Barr virus (EBV) (9, 10). Several of those standards, such as those for HCV, HBV, and HIV-1, have been essential for introducing regulatory requirements for testing of blood and plasma donations, as well as being used by clinical microbiology laboratories for determination of viral loads. In the field of transplantation, ISs have been prepared for CMV, EBV, BK virus (BKV), JC virus (JCV), and human herpesvirus 6b (HHV-6b) (9–13). Other ISs established include ones for the parasites Plasmodium falciparum and Toxoplasma gondii (14, 15), as well as a standard for Mycoplasma species (16). More recently, emerging diseases have been addressed with the establishment of ISs for Zika virus (ZIKV) and chikungunya virus (CHIKV) (17, 18). Slightly different types of WHO standards, termed reference reagents (RRs), have been prepared for Ebola virus (EBOV) (19) and the four dengue virus serotypes (DENV) (20). Although initially developed for vaccine studies, ISs have been prepared for human papillomavirus 16 (HPV-16) and HPV-18 (21); in this case, the ISs were based on plasmids representing the viral genomes, due to lack of native or cultured source materials. Current WHO ISs and RRs for NAT assays are shown in Table 1.

TABLE 1.

Current viral and microbial WHO ISs and RRs for NAT assays^a

Preparation (unitage)	Standard (code no.)	Materialb	GenBank accession no.	Year of establishment	Reference
BKV DNA (10,000,000 IU/vial)	First IS (14/212)	Cultured BKV diluted in buffer-human serum albumin-trehalose		2015	11
CHIKV RNA (1,250,000 IU/vial)	First IS (11785/16)	Cultured and heat-inactivated R91064 strain diluted in human plasma	KJ941050	2017	18
DENV-1 RNA (13,500 units/vial)	First RR	Cultured and heat-inactivated Hawaii strain diluted in human plasma	KM204119	2016	20
DENV-2 RNA (69,200 units/vial)	First RR	Cultured and heat-inactivated New Guinea C strain diluted in human plasma	KM204118	2016	20
DENV-3 RNA (23,400 units/vial)	First RR	Cultured and heat-inactivated H87 strain diluted in human plasma	KU050695	2016	20
DENV-4 RNA (33,900 units/vial)	First RR	Cultured and heat-inactivated H241 strain diluted in human plasma	KR011349	2016	20
EBOV NP-VP35-GP (32,000,000 units/vial)	First RR (15/222)	Lentiviral vector with EBOV np-vp35-gp genes in buffer-human serum albumin-trehalose	KT186367	2015	19
EBOV VP40-L (50,000,000 units/vial)	First RR (15/224)	Lentiviral vector with EBOV vp40-L genes in buffer-human serum albumin-trehalose	KT186368	2015	19
EBV DNA (5,000,000 IU/vial)	First IS (09/260)	Cultured EBV B95-8 strain diluted in buffer-human serum albumin-trehalose	V01555	2011	10
HAV RNA (15,451 IU/vial)	Third IS (15/276)	Viremic human plasma	KY003229	2017	49
Human CMV DNA (5,000,000 IU/vial)	First IS (09/162)	Cultured Merlin strain diluted in buffer-human serum albumin-trehalose	AY446894	2010	9
HBV DNA (477,500 IU/vial)	Fourth IS (10/266)	Viremic human plasma representing HBV genotype A2, HBsAg subtype adw2	KY003230	2016	47
HCV RNA (100,000 IU/vial)	Fifth IS (14/150)	Viremic human plasma representing HCV genotype 1		2015	44
HDV RNA (287,500 IU/ml)	First IS (7657/12)	Viremic human plasma	HQ005369	2013	8
HEV RNA (125,000 IU/vial)	First IS (10/6329)	Viremic human plasma representing HEV genotype 3a	AB630970	2011	7
HIV-1 RNA (125,893 IU/vial)	Fourth IS (16/194)	Cultured and heat-inactivated subtype B isolate diluted in human plasma	KJ019215	2017	52
HIV-2 RNA (1,000 IU/vial)	First IS (08/150)	Cultured and heat-inactivated CAM2 strain diluted in human plasma	KU179861	2009	6
HHV-6B (56,234,132 IU/vial)	First IS (15/266)	Cultured HHV-6B strain Z-29 diluted in buffer-human serum albumin-trehalose	AF157706	2017	13
HPV-16 DNA (5,000,000 IU/vial)	First IS (06/202)	HPV-16 plasmid DNA diluted in buffer-trehalose	K02718	2008	21
HPV-18 DNA (5,000,000 IU/vial)	First IS (06/206)	HPV-18 plasmid DNA diluted in buffer-trehalose	X05015	2008	21
JCV DNA (10,000,000 IU/vial)	First IS (14/114)	Cultured JCV diluted in buffer-human serum albumin-trehalose		2015	12
Mycoplasma DNA (100,000 IU/vial)	First IS (8293/13)	Cultured Mycoplasma fermentans in Mycosafe Friis medium		2013	16
B19V DNA (705,000 IU/vial)	Third IS (12/208)	Viremic human plasma representing B19V genotype 1		2013	54
Plasmodium falciparum DNA (500,000,000 IU/vial)	First IS (04/176)	Parasitemic human blood		2006	14
Toxoplasma gondii (500,000 IU/vial)	First IS (10/242)	T. gondii tachyzoites obtained from infected mice and diluted in buffer-trehalose		2014	15
ZIKV RNA (25,000,000 IU/vial)	First IS (11468/16)	Cultured and heat-inactivated PF13/251013-18 strain diluted in stabilizer	KX369547	2016	17

^aDetails of the standards are available on the WHO website (https://www.who.int/biologicals/reference_preparations/en/) and on the respective websites of the WHO CCs (http://www.nibsc.org/, https://www.pei.de, and https://www.fda.gov/).

^bSequences are unavailable for some ISs.

The SoGAT group has met at least annually since it was established, collectively identifying priority pathogens for which there is a need for NAT standardization and coordinating international studies to develop and to evaluate these materials. The need for specific standards is determined through discussions with the scientific and medical community worldwide through the SoGAT forum and through WHO programs in disease areas such as malaria and tuberculosis, with input from manufacturers of in vitro diagnostic devices (IVDs) and the three official WHO collaborating centers in the fields of blood and IVDs, namely, the NIBSC, the Paul Ehrlich Institute (PEI) (Germany), and the U.S. Food and Drug Administration (FDA) Center for Biologics Evaluation and Research (CBER) (USA).The SoGAT meetings allow for the discussion of results from international collaborative studies prior to submission and review by the WHO Expert Committee on Biological Standardization (ECBS). The ECBS plays a formal role in the establishment of ISs and related reference materials, and committee members are scientific experts from national control agencies, research institutes, academia, and public health bodies. All new proposed international standardization projects are subject to review by the ECBS before endorsement. Occasionally, special topics have been discussed at extraordinary SoGAT meetings; examples include addressing the problems with detection of different genotypes of B19V and how to improve standardization (22).

TYPES OF WHO REFERENCE MATERIALS

ISs and their role.

ISs are measurement standards and are assigned an internationally agreed unitage in IU (23). The potencies of ISs are determined by consensus means through international collaborative studies, using a range of methods typically in routine use by participating laboratories. In the case of NAT assays, potencies are determined by a combination of endpoint dilution analysis for qualitative assays and, for example, “copy numbers” or “genome equivalents” for quantitative assays. Although IU are arbitrary in theory, in practice they correspond to the mean overall potency (“NAT-detectable units”) reported by participating laboratories. Adoption of IU also avoids the issue of copy number, the definition of which is assay dependent and which implies, misleadingly, that material is traceable to an SI unit. Repeatedly, during studies to evaluate new ISs, quantitative reports of concentrations of samples in copy numbers typically vary over several orders of magnitude. This finding demonstrates that copy number is not a robust measure that can be compared readily between laboratories; the use of IU allows better comparison of results.

WHO ISs are considered the highest-order, international, conventional calibrators, in accordance with ISO guidelines (24). The principal uses of ISs are for the calibration of secondary standards (Fig. 1), traceable in IU, and for the evaluation of critical assay parameters such as analytical sensitivities and quantification range, including upper and lower limits of quantification. The preparation and calibration of secondary standards are described in detail elsewhere (25). Uncertainty values are not assigned to WHO ISs, since IU are arbitrary units and variance is associated with that of the vial content.

FIG 1.

Hierarchy of standards. The relationship between ISs and secondary and tertiary standards is shown, together with their uses.

In Europe, the new regulation on medical IVDs stipulates the design requirements for calibration of assays using “reference materials of a higher metrological order” (26). Furthermore, the regulation requires metrological traceability of values assigned to calibrators and control materials using reference materials of a higher order, which should be communicated to users. In addition, the common technical specifications state that WHO ISs should be included in the performance evaluation and the reporting of test results in IU for “high-risk” IVDs (e.g., for quantitation of HIV-1, HBV, or HCV) (27). Furthermore, regulatory requirements for testing of biologics may define minimal sensitivity for suitable assays based on WHO ISs. Examples include national requirements for blood screening markers (e.g., HIV-1 RNA and HCV RNA in Germany) and European regulation of plasma derivatives (e.g., HCV RNA in manufacturing plasma pools).

Representatives of the U.S. FDA CBER participate on a regular basis in the international standardization efforts undertaken by the WHO. In contrast to the European Union, there is no legal requirement in the United States to use WHO ISs for assay calibration; however, panel members used by the FDA CBER for lot release of NAT assays have been calibrated against WHO ISs (28, 29).

When an IS is established for the first time, it is designated the first IS; upon its replacement, it is termed the second IS, the third IS, and so on, with each subsequent standard replacing its predecessor as the highest-order reference standard. Replacement of ISs is discussed in more detail below.

Reference reagents and international reference panels.

In addition to WHO ISs, other types of standards are established by the WHO ECBS, including RRs and international reference panels (IRPs). Both RRs and IRPs are prepared and evaluated using principles similar to those used for WHO ISs. The IRPs consist of different genotypes or important strains of pathogens with diverse global distribution; examples of such panels include those for HIV, HBV, B19V, and HEV (Table 2) (30–35). The role of IRPs is to help ensure consistent detection of pathogen variants, particularly when they are used for assay validation purposes. They have been important tools for improvements in assay performance when detection of specific variants has been suboptimal. Usually, no unitage is assigned to members of IRPs. However, the data on assay performance are included in the collaborative study reports published on the WHO website, providing a range of potencies reported for individual panel members.

TABLE 2.

Current IRPs for NAT assays (viral markers)

Panel	Standard (code no.)	Materiala	Year of establishment	Reference
HBV genotypes (15 members)	First IRP (5086/08)	Viremic plasma diluted in pooled human plasma; HBV genotypes A1 (n = 2), A2, B2 (n = 2), B4, C2 (n = 3), D1 (n = 2), D3, E, F2, and G	2009	33
HEV genotypes (11 members)	First IRP (8578/13)	Viremic plasma donations and stool samples diluted in pooled human plasma; HEV genotypes 1a, 1e, 2a, 3b, 3c, 3e, 3f/l, 3 ra, 4c, and 4g	2015	35
HIV-1 subtypes (10 members)	Second IRP (12/224)b	Cultured and heat-inactivated HIV-1 subtypes A, B, C, D, AE, F, G, AG-GH, N, and O diluted in human plasma	2012	31
HIV-1 circulating recombinant forms (10 members)	First IRP (13/214)	Cultured and heat-inactivated HIV-1 CRFs and subtype variants diluted in pooled human plasma	2013	32
B19V genotypes (4 members)	First IRP (09/110; CBER B19V genotype panel 1)	Viremic plasma donations diluted in pooled human plasma; B19V genotypes 1a1, 2, and 3a and negative plasma control	2009	34

^aSequence details for IRP members are available in the supplemental material. CRF, circulating recombinant forms.

^bThe second HIV-1 IRP consists of the same strains as the first IRP; strains included in the first IRP were not subjected to head inactivation.

In the case of RRs, these are usually interim standards with a unitage defined in units rather than IU. Upon further characterization, RRs may be established as ISs and the unitage defined in IU. Examples of RRs include NAT standards for EBOV, which were established in response to the Ebola crisis in 2014 and were based on recombinant lentivirus vectors to avoid biosafety issues (19). More recently, four RRs have been established for DENV-1 to DENV-4. Because of the genetic differences between the types, it was not possible to select a single strain as an IS; consequently, each type has a separate unitage (20).

PREPARATION AND ESTABLISHMENT OF WHO REFERENCE MATERIALS

Characterization and preparation of candidate standards.

The processes involved, from the identification of the scientific need to develop a standard through establishment of the standard and ultimately its replacement, are shown in Fig. 2. The procedure to establish WHO standards is extremely rigorous (23) and is undertaken by one of the three WHO Collaborating Centers on behalf of the WHO.

FIG 2.

Process for the development of WHO ISs, RRs, and IRPs. The procedure is shown from the identification of a scientific need to develop a standard to the establishment of the standard and ultimately to its replacement. cIS, candidate IS.

The development of a new standard starts with the identification and preparation of a suitable stock material, which may be viremic plasma (for example, for HCV, HBV, and HEV), parasitemic whole blood (for Plasmodium falciparum) (14), or pathogens propagated in culture. More rarely, animals have been used as alternative starting materials when sources of native materials are unavailable or not of insufficiently high titer, and an example of this is the propagation of Toxoplasma gondii tachyzoites in mice (15). HPV ISs have been based on the preparation of plasmid DNAs diluted in human genomic DNA (21). An estimate is made of the concentration of the stock material, and identity testing is performed (e.g., by sequence analysis). When material has been obtained from blood or plasma, donations are screened to ensure the absence of blood-borne pathogens other than the target in question. Whenever possible, strains are selected to reflect those with widespread distribution and global importance. Occasionally, materials may be inactivated, depending on feasibility combined with biosafety concerns. Such procedures should be validated; however, this may not be possible for some pathogens for which suitable cell culture systems are not available. To facilitate worldwide distribution, WHO standards are usually lyophilized; therefore, formulation is an important factor to consider. Formulation is fairly straightforward when viremic plasma is used and the standards will be further diluted in the matrix when used in the recipient laboratories. However, when testing of certain pathogens can be performed with different types of matrices (e.g., whole blood, urine, cerebrospinal fluid [CSF], and plasma), cultured viral and microbial strains have been formulated in solutions containing excipients (e.g., buffers, sugars, and/or stabilizers) that allow further dilution of the standard into the appropriate type of matrix. The final formulation should not cause any interference with the NAT assays, such as decreases in extraction efficiency or inhibition of amplification.

When the bulk standard preparation is dispensed into either vials or ampoules, the coefficient of variation of the filled volume is determined. Several thousand vials/ampoules are usually prepared. After lyophilization, the ampoules or vials are back-filled with nitrogen and the homogeneity of the lyophilized material is determined, with sampling across the batch. Testing is performed for residual moisture and oxygen, which may affect product stability, and accelerated stability (at higher temperatures) and real-time stability are determined, to confirm that the reference material can be shipped at ambient temperatures worldwide, without loss of potency under normal storage temperatures (typically −20°C) over the life of the IS.

Commutability.

Commutability is a property of a reference material that is demonstrated by the closeness of agreement between the results obtained for the reference material and the results obtained for clinical specimens, when comparatively tested in different assays (36, 37). In other words, in order to be suitable as an assay calibrator, the reference material should not behave differently than clinical specimens. Commutability is demonstrated by testing the different materials (reference material and clinical specimens) in multiple assays. Matrix is an important aspect of commutability. ISs are designed to reflect as closely as possible the specimens tested in routine diagnosis or blood screening. For example, human plasma and serum are very common types of sample matrices tested in blood-screening and clinical laboratories, and several ISs are derived from viremic donations or contain culture-derived virus diluted in plasma. In addition, the strain of pathogen (i.e. the analyte) used for the IS is usually selected to represent the most commonly circulating variant.

Commutability is an important precondition for the ability of the calibrant to harmonize different assays, and it is addressed by inclusion of clinical specimens, as far as possible, in the international collaborative study. The impact of different extraction systems (reagents and equipment) on the extraction efficiencies for different matrices is another factor to be addressed in commutability studies. In the case of CMV, noncommutability of the IS has been demonstrated for some assays (38). Commutability is particularly complex in the case of CMV and is affected by features such as the physical form of viral DNA in the IS (virion-associated DNA), compared to that found in transplant patients, which is highly fragmented (39, 40). Furthermore, during amplification/detection reactions, amplicon length affects viral load determinations (40). With the development of additional ISs for clinical pathogens, the challenge of commutability becomes even more complex, with quantitative values being reported for multiple types of sample matrices, including urine, CSF, and stool. CSF is a matrix with a low protein content that is difficult to obtain in large volumes and is not easy to evaluate in collaborative studies or in formal commutability investigations. Stool is another challenging sample type, in which the matrix contains inhibitors and sample extraction is not well standardized.

International collaborative studies.

Candidate ISs, RRs, and IRPs are evaluated in international collaborative studies. Participants volunteering to take part in these studies include blood centers, reference laboratories, clinical microbiology laboratories, manufacturers of diagnostic kits and medicinal products, and regulatory organizations. Typically, 15 to 25 laboratories are involved in such a study. The assays included in the studies are ones used throughout the world and include commercially available tests as well as LDTs. The studies investigate the potency of the candidate materials, including clinical comparator samples as well as related reference materials and calibrators; potencies are determined using qualitative or quantitative assays, as described above. One of the major aims of each study is to provide a basis for assignment of unitage to the standard; the unitage assignment is usually based on the combined mean potency for all of the assays included in the study. Expressing results of the study samples against the candidate IS can greatly reduce variation in the measured potencies reported by participants, and the harmonization effect (see below) is an important factor reviewed by the ECBS to demonstrate the utility of a new IS. The studies themselves allow head-to-head comparisons of assays used throughout the world and provide information on sensitivity (based on endpoint analysis of qualitative assays) and variability in quantification.

Statistical analysis of the study data forms the basis for the final report, which includes a proposal for the unitage for the IS. Participants are invited to comment on the report and are asked whether they agree with the proposed unitage. The final report is made available on the WHO website for public review before the annual meeting of the ECBS. In the case of IRPs, no unitage is assigned to the panel members; however, details may be included in the report, with the range of potencies observed. Subsequent to the establishment of a standard or panel, the custodian laboratory has responsibility for the storage of each batch under controlled conditions, monitoring of stability, and coordination of worldwide distribution.

REPLACEMENT OF WHO ISS

Although several thousand vials are prepared for each standard, when they are nearing exhaustion it is essential to replace the preparation. Replacement projects are prioritized by the WHO. An important aspect of replacement of one standard with the next is maintaining the continuity of the IU in order to ensure that tests can be compared over time. Details of the NAT standards that have been replaced are shown in Table S1 in the supplemental material. Since it was established in 1997 (1), the HCV IS has been replaced four times (41–44). Replacement ISs have been prepared for HBV (45–47), HAV (48, 49), HIV-1 (50–52), and B19V (53, 54). In each case, replacement preparations have been evaluated in parallel with the previous IS, using qualitative endpoint assays and quantitative assays (within the linear range) and covering appropriate dilutions. With each subsequent IS, the possibility exists for drift in the IU; this may be exacerbated by issues with assay features included in collaborative studies, such as primer/probe mismatches affecting quantification, which emphasizes the need for good characterization of starting materials. An example is the study to establish the third IS for B19V (54), in which the new B19V viremic plasma donation used for the third IS was underquantified by the cobas TaqScreen DPX test, probably due to a mismatch between the primers/probe and the sequence of the ISs (55), affecting the assigned unitage.

ASSAY HARMONIZATION USING WHO ISS

Relative potencies.

During the establishment of WHO ISs, one of the criteria for acceptance of a new standard is the demonstration that, when results of testing are expressed relative to the candidate IS, an improvement in the agreement observed among assays and laboratories is seen. An example of this is shown in Fig. S1. An HEV sample that was included in the collaborative study to establish the HEV IS was evaluated using a mixture of qualitative and quantitative NAT assays; the reported potencies (Fig. S1, upper) showed a wide variation in titers over several orders of magnitude. When these potencies were expressed against the WHO IS (PEI code 6329/10), the agreement between laboratories was markedly improved, with variation being reduced to ∼1 log₁₀ unit and with a typical reduction in the associated standard deviation (SD).

External quality assessment programs.

External quality assessment (EQA)/proficiency testing (PT) programs can be very helpful in generating data on the implementation of WHO ISs by participating laboratories in a large number of countries. In some cases, WHO ISs have been included directly in EQA studies. For example, the first IS for ZIKV was made available by the WHO in July 2016, prior to formal establishment by the ECBS in October 2016, and was introduced as a consequence of the public health emergency of international concern (56). The first ZIKV IS has been included in all of the ZIKV EQA/PT programs provided by Quality Control for Molecular Diagnostics (QCMD) since 2016 (57).

Data analysis from QCMD EQA/PT schemes demonstrate that, when an IS has been established for a specific target pathogen, the observed variation (SD) based on the geometric mean of the log₁₀ viral load results is noticeably smaller (Table S2). This observation is based on results reported in IU per milliliter for duplicate panel members. In contrast, for pathogen targets for which an IS has been established only recently or for which there is no IS (and reporting of results is often in different types of units), the SDs are much greater (Table S2). In addition, when there is a known clinical need for pathogen quantitation, the IS and IU per milliliter are more readily accepted.

In the case of CMV for example, in early EQA/PT studies performed prior to 2004, the majority of assays performed by laboratories participating in the CMV EQA program were qualitative (Fig. S2). For quantitative assays performed prior to the establishment of the first CMV IS in 2010 (9), laboratories reported results in copies per milliliter or other units of measurement, such as genome equivalents per milliliter, as observed for the data reported in international EQA/PT schemes. Over the past 8 years, the number of laboratories reporting in IU per milliliter has increased significantly, from 0% to 50% of the data sets returned within the annual international EQA/PT schemes run by QCMD (Fig. S3). For CMV viral load testing, the increase in reporting in IU correlates with an increase in the use of commercial assays by participants in the QCMD studies (Fig. S4). In a recently published EQA study, the variation in results reported in IU per milliliter was lower than that for results reported in copies per milliliter, demonstrating that the use of the CMV WHO IS improves the reproducibility and comparability of CMV viral load results across laboratories (58). Consequently, the recently revised international guidelines on the management of CMV in solid organ transplantation recommend that all results should be reported as IU per milliliter (59). More significant improvements in results have been reported for EBV with the use of the IS (60).

PREQUALIFICATION OF IN VITRO DIAGNOSTIC DEVICES

International reference preparations play an important role in the WHO prequalification program for IVDs. In this program, IVDs targeting low- and middle-income countries (LMIC) are independently assessed by the WHO, since LMIC themselves rarely have the regulatory capacity to assess the quality and suitability of IVDs offered to the national market. In WHO prequalification studies, ISs may be used for comparative evaluation of essential assay features such as sensitivity, limit of detection, and range of quantitation. Furthermore, IRPs covering different variants (e.g., genotypes or recombinants) are important for the detection of strains that are more prevalent in certain regions. The outcome of performance evaluation studies initiated on behalf of the WHO prequalification program for IVDs is published together with a list of IVDs deemed suitable by the WHO for the intended purpose.

STRATEGIC ADVISORY GROUP OF EXPERTS ON IN VITRO DIAGNOSTICS

In 2017, the WHO established the Strategic Advisory Group of Experts on In Vitro Diagnostics (SAGE IVD). SAGE IVD recently published the first model list of essential diagnostics, including several NAT assays for markers including HBV, HCV, HIV, Mycobacterium tuberculosis, and HPV (61). The elaboration of the list is aimed at improving access to IVDs that are considered to be essential for achieving universal health coverage and strengthening health systems. The IVD list is akin to the WHO essential medicines list and will be important to ensure accurate diagnosis prior to treatment decisions and inform countries about the selection, procurement, and usage decisions for IVDs.

STANDARDS CURRENTLY UNDER DEVELOPMENT

Standards currently under development are shown in the Table S3 and include viral and parasitic markers as well as a standard for M. tuberculosis, reflecting the global burden of disease and the increasing use of molecular testing for this pathogen.

CONCLUSIONS

Significant progress in NAT standardization has been made over the past 2 decades, in the context of screening for blood-borne markers as well as use in clinical diagnostic laboratories. The development of WHO standards and other reference materials (ISs, RRs, and IRPs) has facilitated these efforts, also enabling the introduction of regulations for the detection of blood-borne pathogens in the fields of transfusion and blood product safety for markers such as HCV, HBV, HIV, HAV, B19V, and (more recently) HEV, by setting thresholds and control concentrations defined in IUs. For clinical laboratories, HCV, HBV, and HIV-1 standards have been important for viral load determinations for diagnosis and treatment monitoring; in relation to transplantation, standards established for CMV, EBV, HEV, BKV, JCV, and HHV-6b are used for expression of viral loads in IU. The use of IU improves agreement and allows comparability of data among laboratories, allows the introduction of regulations for blood screening using NAT assays, and informs clinicians regarding patient testing and monitoring of therapeutic interventions. International clinical guidelines (e.g., for CMV and HEV in the transplantation setting) encourage reporting in IU, supporting accuracy in viral load reporting and harmonization efforts (59, 62). These efforts are supported by the availability of secondary standards and controls traceable in IU, as well as calibrated assays.

Because of their biological nature, WHO standards control for the entire NAT process, including nucleic acid extraction. Organizations such as the National Institute of Standards Technology (NIST) in the United States take a different approach and produce standard reference materials (SRMs) for a small number of viral markers, including a bacterial artificial chromosome (BAC) containing the genome of the CMV Towne strain and a linearized plasmid DNA control for BKV. These SRMs are added directly to the amplification/detection reaction without undergoing prior extraction and are intended to be used for the calibration of controls and standards. Some organizations provide in vitro transcribed RNAs (IVTs); like the NIST materials, these materials do not control for the extraction part of the NAT assay. In a study organized by kit manufacturers, a partial HCV IVT was evaluated in a study comparing amplification methods; however, it was not found to perform better than the biological standard (63). During the study to establish the first WHO IS for CMV, the candidate standard, based on a clinical strain (Merlin) propagated in cell culture, was evaluated in parallel with a BAC containing the entire Merlin genome. Participants added the BAC directly to the amplification reactions. Expression of potencies of other cultured virus preparations against the candidate IS showed a marked reduction in variation among laboratories; when the results were expressed relative to the BAC, however, no improvement was observed, compared to the absolute mean estimates (9). In the study to establish the first WHO IS for ZIKV, expression of clinical samples and biological reference materials yielded an improvement in the agreement of results among laboratories. In the study, two related IVTs were included, one containing several assay target sequences in a single transcript and the other containing a mixture of the individual IVT RNAs. Expression of one IVT preparation against the other resulted in harmonization; however, expression of clinical samples or biological reference materials against the IVTs failed to produce any improvement (17). These studies demonstrate the importance of controlling the extraction step in the NAT procedure, and they emphasize the advantage of the approach taken by the WHO, compared with the use of (bio)synthetic types of reference materials. The latter may be easier to replace, however, compared to sourcing new viremic donations for some WHO ISs.

Sequence data are available for most WHO ISs, RRs, and IRPs (Table 1; also see Tables S4 to S7 in the supplemental material), sometimes indicating sequence heterogeneities in comparison with clinical isolates (e.g., sequence deletions or sequence duplications in culture-based materials). Using next-generation sequencing data, even subpopulations of sequence variants are being detected, as was reported recently for the ISs for BKV and JCV (64, 65). Passage of the strains in cell culture resulted in heterogeneous DNA populations, the reason for which is not understood and which could affect some specialized assays (64, 65), although both preparations were shown to harmonize assay performance successfully in the collaborative studies (11, 12) and in independent studies (66). These observations demonstrate the importance of thorough characterization of the starting materials used for standard preparation. Methods such as digital PCR are useful in the characterization process, for understanding the relationships of IU/copy number ratios for specific methods and for estimating potency during the development of new ISs or when no standard exists. In the case of the first WHO IS for HAV, the IU/copy number ratio was determined to be 1:14 using digital PCR (S. Baylis, unpublished data); the low IU value was likely a consequence of the low sensitivity of assays used by participants in the original collaborative study (5).

With the absence of reference methods to define nucleic acid contents of microbial pathogens in complex biological matrices, the validity of the WHO approach in the development of reference standards and the harmonization of NAT assays is confirmed. The challenge for the development of such standards remains meeting the clinical need in a timely manner while maintaining rigorous procedures in the establishment process. Adequate commutability of ISs is essential, particularly for clinical settings, and may affect the treatment of patients and hinder the introduction of clinical practice guidelines. Inclusion of sufficient clinical materials in studies to evaluate commutability remains a problem, in terms of volume and transfer agreements, and the support of the wider scientific community in these efforts is essential to fully realize the potential of the WHO standardization efforts.

ADDENDUM IN PROOF

At the WHO ECBS meeting in Geneva, Switzerland (29 October to 2 November 2018), the second WHO IS for HIV-2 (code number 16/296) and the first IS for adenovirus (code number 16/324) were established.