The importance of using WHO International Standards to harmonise SARS-CoV-2 serological assays

Abstract

The COVID-19 pandemic led to the rapid development of tests to diagnose SARS-CoV-2 infection and ascertain the prevalence of infection, along with the formulation of various treatments and vaccines. Globally, over 220 anti-SARS-CoV-2 serological assays have been developed for laboratory use, and many of these assays are currently used to assess immune responses against SARS-CoV-2. However, because these assays were independently developed by different manufacturers with different target antigens, immunoglobulin detection, technologies, and data reporting approaches, the results are not directly comparable, making it challenging to draw conclusions regarding immune responses at the population level. With deficiencies in assay validation, standardisation, and harmonisation, the inability to use and compare large datasets is becoming a major issue as serological data continue to increase. To help in addressing this issue, WHO established the first International Standard for the anti-SARS-CoV-2 immunoglobulin in late 2020. In this Personal View, we define the WHO International Standard for the anti-SARS-CoV-2 immunoglobulin, summarise the uses of primary versus secondary serology standards, recommend the use of such standards for data harmonisation, and list guidance and resources for using serology standards to improve data comparability.

Introduction

SARS-CoV-2 serological assays

The COVID-19 pandemic led to an urgent development of tests to diagnose and ascertain the prevalence of infection, along with treatments and vaccines. This pandemic, caused by SARS-CoV-2, resulted in about 771 million confirmed cases and 6·9 million deaths as of Oct 25, 2023. Serological assays gained substantial importance in public health,¹ and the US Food and Drug Administration (FDA) took unprecedented steps to promote the development and approval of high-quality tests during the health crisis.² On Feb 29, 2020, the FDA instituted new policies allowing new serological assays intended for measuring SARS-CoV-2 antibodies to be marketed without formal FDA approval, provided they had been adequately validated and proper notifications had been made to the FDA.^1–3 However, the FDA noted variations in the performance of the tests entering the market.¹ In April, 2020, the FDA, National Cancer Institute, US Centers for Disease Control and Prevention, National Institute of Allergy and Infectious Diseases, Frederick National Laboratory for Cancer Research, National Institutes of Health Clinical Center, and Biomedical Advanced Research and Development Authority (all based in the USA) established a collaboration to independently evaluate the performance of different serological assays—particularly sensitivity and specificity estimates—with the aim of ensuring assay reliability.¹ Notably, only 27 assays of the 91 assay evaluations performed through this collaboration met the performance standards set by the FDA, indicating that substantial variability occurred in the performance characteristics between different serological assays.¹ On a global scale, different research groups also recognised the need for assessing SARS-CoV-2 serological assays and performed independent comparative studies, finding varying sensitivity and specificity results.^4–7

Currently, over 220 commercial SARS-CoV-2 serological assays have been developed for laboratory use.⁸ Overall, these assays fall into broad categories on the basis of the target antigen (eg, spike protein, S1 or S2 domains of the spike protein, nucleocapsid [N], or receptor binding domain [RBD]), immunoglobulin (eg, total antibody, IgG, or IgM), and technology involved (eg, direct ELISA or chemiluminescence immunoassay).⁸ Theoretically, all these assays can measure immune responses against SARS-CoV-2. However, these assays were developed with different target antigens, immunoglobulin detection, and technologies, which introduces difficulties in comparing the results from different studies and drawing conclusions regarding population-level immune responses. Even within the same study, results cannot be directly compared if different assays are used, resulting in a loss of statistical power and an inability to draw overarching conclusions.^9,10

Comparing results across assays

Amid the global efforts to control COVID-19, the inability to use and compare available data (due to inadequate assay standardisation and harmonisation) is becoming a major challenge. As SARS-CoV-2 continued to spread and evolve, scientific research necessitated real-time adaptation, using the samples and assays available at that time. Consequently, as the technology progressed and manufacturing faced challenges in keeping pace, numerous different assays were being simultaneously used to answer similar questions. To optimise the use of the data, it was evident that an International Standard for serology-based assays was urgently required to enable assay calibration and the comparison of results obtained across different studies and vaccines, mirroring previous circumstances in the context of other infectious diseases.

The WHO International Standard for anti-SARS-CoV-2 immunoglobulin

What is an International Standard?

Biological International Standards are characterised samples designed, developed, stored, and distributed by WHO through the Medicine and Healthcare Products Regulatory Agency’s National Institute for Biological Standards and Control (NIBSC), endorsed by its Expert Committee on Biological Standardization (ECBS), and assigned an arbitrary unit.¹¹ These International Standards are meant to serve as primary standards for calibrating secondary or national standards. These secondary standards can then be used to calibrate tertiary standards or directly used in assays against which the target samples are measured.^12,13 International or primary standards should have specific characteristics to be considered effective. They should (1) have been developed on the basis of scientific evidence, (2) show stability over a practical length of time without degradation, (3) account for all associated uncertainties, and (4) accurately represent the analyte being measured.¹²

The WHO International Standard: development and roll-out

Recognising the need for an International Standard to harmonise the increasing abundance of COVID-19 serological data, WHO established the first WHO International Standard for anti-SARS-CoV-2 immunoglobulin (NIBSC code 20/136) on Dec 10, 2020, and released it on Dec 18, 2020. The International Standard was assigned a potency of 250 IU/lyophilised ampoule for neutralizing antibody activity.^10,11 Additionally, the International Standard could be used to compare binding antibody assays (provided the assays detect antibodies of the same class and against the same antigen), using the reference’s designated 250 binding antibody units (BAU)/lyophilised ampoule.

The first WHO International Standard for anti-SARS-CoV-2 immunoglobulin was created from a pool of plasma samples collected from 11 convalescent patients following confirmed SARS-CoV-2 infections during the first COVID-19 wave in 2020.¹² The pooled candidate sample was then characterised and evaluated for suitability as an International Standard in neutralisation assays, ELISAs, and rapid tests (among other methods) by 44 independent laboratories across 15 countries.¹² Given that International Standards comprise meticulously characterised patient samples, it is important to recognise them as finite resources. Therefore, such International Standards should be used with proper care to ensure their preservation for as long as possible. Unfortunately, owing to the high demand and improper use as a primary calibration standard, the first WHO International Standard has already been depleted.

The role of secondary standards: preserving the International Standard as a long-term resource

Secondary standards are crucial for preserving International Standards as finite resources. Secondary standards, such as the US SARS-CoV-2 serology standard or the Chinese national-standard SARS-CoV-2-neutralising antibody, should (1) be calibrated against the WHO International Standard, (2) resemble the analyte being measured as closely as possible(although secondary standards offer more flexibility than primary standards), (3) be traceable, and (4) show stability and commutability across different assay types.^11,13,14 The main purpose of a primary standard is to calibrate secondary standards, whereas the secondary standards find application in various experiments including sample measurements, assay calibration, as run controls, and for calibrating tertiary standards.¹¹ The proper use of a standard material is essential for maintaining the international unit across as many studies as possible. However, in the rush to obtain rapid results, many laboratories directly used the International Standard in their experiments for purposes other than calibration, resulting in severe depletion of the first WHO International Standard for anti-SARS-CoV-2 immunoglobulin. Consequently, a second WHO International Standard for anti-SARS-CoV-2 immunoglobulin (NIBSC 21/340) had to be developed in 2022 (and proposed to the ECBS between Oct 24 and Oct 28, 2022) with the goal of maintaining the continuity and harmonisation potential of assays already standardised against the first International Standard.¹⁵ As the first WHO International Standard for anti-SARS-CoV-2 immunoglobulin was being depleted, the US SARS-CoV-2 serological standard (developed by pooling samples from four convalescent donors, evaluated in a harmonisation study across eight independent laboratories in the USA, and calibrated against the first WHO International Standard for anti-SARS-CoV-2 immunoglobulin) was considered to be a crucial resource for numerous different laboratories worldwide.¹⁴

Assay harmonisation: is it feasible?

The role of the WHO International Standard for anti-SARS-CoV-2 immunoglobulin is to facilitate harmonisation and comparisons of pre-existing COVID-19 serological data. However, before this strategy can be broadly applied, scientific evidence should be gathered to show that harmonisation with the WHO International Standard is both accurate and feasible.

Evaluating the feasibility of harmonisation

To date, two studies^16,17 have been conducted to evaluate the feasibility of using the International Standard for data harmonisation. The overall results of the HARMONY study, which was conducted through the WHO Solidarity II programme,¹⁶ support the use of the International Standard to harmonise results from different assays that measure the same immunoglobulin class and SARS-CoV-2 antigen. The feasibility and utility of this method was also reflected by the Frederick National Laboratory Harmonization Study,¹⁷ which compared the performance of commercial (ie, MSD, Roche, Abbott, Siemens, Ortho Clinical Diagnostics, and Kantaro), in-house developed, and neutralising antibody-measurement assays across 17 independent institutions and used the International Standard to harmonise the data. Although some degree of variability was present, the results showed that the International Standard could be used to harmonise IgG, pan-IgG, and total-IgG data for different antigens (including SARS-CoV-2 spike, RBD, and N) from different assays.¹⁷

Harmonisation is feasible and practical

The results from the two above studies^16,17 show that harmonisation was successful using the International Standard or the US SARS-CoV-2 serology standard, enabling data comparisons across assays and laboratories. Harmonised measures of the US SARS-CoV-2 serology standard obtained through ELISA (both commercial and in-house developed) strongly matched the known calibrated BAU values of the US SARS-CoV-2 serology standard. Although substantial variability was noted, neutralisation assays successfully measured the known calibrated neutralisation activity of the US SARS-CoV-2 serology standard in IU after data harmonisation. However, because of the potential influence of the antigen and immunoglobulin of interest, WHO recommends that the antigen and immunoglobulin be specified when reporting the results of binding antibody assays so that comparisons can be drawn between different assays with the same specifications.^11,12

Notably, when calibrating candidate secondary standards relative to International Standards, each antigen should be handled independently as done for other standards, such as the US SARS-CoV-2 serology standard.¹⁴ Because existing primary and secondary serological standards and most commercial serological assays are based on the original SARS-CoV-2 strain, it is essential to systematically assess the consistency of assays calibrated against these standards when measuring antibody concentrations in serum samples from individuals infected with different SARS-CoV-2 variants.

Controversy surrounding the International Standard design

Despite the established use of serological International Standards in studies of immunoglobulins against other viruses such as human papillomavirus, HIV, and influenza virus,¹⁸ some controversy exists regarding the current design of the WHO International Standard for the anti-SARS-CoV-2 immunoglobulin.

Different units for the same type of measurement

One point of controversy specifically relates to the question of whether the WHO International Standard for anti-SARS-CoV-2 immunoglobulin can function as a standard with measures in two different units (IU and BAU).¹⁹ Hansen and colleagues,¹⁹ in their letter published in Clinical Chemistry Laboratory Medicine in 2022, indicated that using different units to describe the same type of measurement was against the International Organization for Standardization (ISO) convention. Hansen and colleagues advocated for standardising all units as IU, with additional specification as either binding antibodies or neutralising antibodies. However, this argument does not account for the fact that although the International Standard primarily serves as a standard for measuring neutralising activity in IU, WHO acknowledged that the International Standard design also enables its use as a reagent for calibrating direct binding antibody assays.^20,21 Consequently, the International Standard was also assigned a BAU value (purposely chosen to differentiate it from IU) when used to calibrate ELISAs.^20,21 Notably, as pointed out by Knezevic and colleagues,²⁰ WHO International Standards and NIBSC research reagents are not named according to ISO conventions.

Measuring affinity versus quantity

Some researchers have noted another limitation concerning the WHO International Standard. Plebani and colleagues²² propose that the actual requirement lies in establishing a standard for measuring antibody affinities towards SARS-CoV-2, rather than focusing on a standard for measuring neutralising activities. In their letter response to Hansen and colleagues, Plebani and colleagues²² pointed out that antibodies obtained from individuals at different stages of a disease have different affinities and present with different antibody-class ratios, resulting in little comparability between antibodies collected from different serum samples. Different assay types (eg, traditional ELISAs and lateral flow assays) and methods of analysis (including 5-parameter logistic curve fit, parallel-line, ratio-to-negative, and ratio-to-positive methods) present additional variables, as different protocols detect different antibody classes at varying efficiencies.²² Consequently, they argue that the International Standard should be developed on the basis of the overall functionality of the measured substance or the total antibody affinity, as opposed to the concentration of the neutralising antibodies, which does not reflect the actual biological activity.²² However, calibration and standardisation using the first International Standard is not intended to deliver an absolute analytical concentration. Instead, International Standards measure relative antibody activities (whether they are neutralising or binding antibodies). Additionally, the practicality of assay calibration should be considered, especially considering the scientific evidence that using a standard to calculate relative potencies is feasible.^16,17

Finally, as discussed above, both WHO and the Frederick National Laboratory have conducted studies to investigate the feasibility and utility of using the WHO International Standard for harmonising serological data.^16,17 The results of both studies showed that existing data could be reliably compared after harmonisation against the WHO International Standard.^16,17 Additionally, most assays currently in use for evaluating anti-SARS-CoV-2 humoral immune responses measure neutralising or total binding antibody concentrations, not binding affinities. Therefore, the current needs of the scientific community are addressed with these units of measurement.

International Standards should be a standard-use tool

The WHO International Standard for anti-SARS-CoV-2 immunoglobulin and new SARS-CoV-2 variants

A valid concern persists regarding the long-term applicability of the WHO International Standard for anti-SARS-CoV-2 immunoglobulin, as SARS-CoV-2 has continued to mutate and develop new variants with considerable antigenic differences.²³ Consequently, regular testing and monitoring of the International Standard for neutralising antibody activity against emerging variants is crucial. Preliminary results suggested that the first and second WHO International Standard for anti-SARS-CoV-2 immunoglobulin do contain antibodies that remain capable of neutralising variants alpha (B.1.1.7), beta (B.1.351), and gamma (P.1).^10,15 These results highlighted the robust nature of the International Standard and enable the standards to continue to serve as a calibrant.^10,15 Additionally, a new reference panel for SARS-CoV-2 variants of concern (NIBSC 22/270) that specifically contains antibodies targeting and capable of neutralising alpha, delta, gamma, and omicron variants has since been developed.^15,24

Assay results should be harmonised whenever possible

With the ongoing COVID-19 pandemic and over 220 commercial COVID-19 serological assays developed for laboratory use, the need to harmonise and compare data between studies is immediate and pertinent. Ideally, all SARS-CoV-2 serological data would be calibrated against the International Standard, with vaccine developers reporting clinical trial antibody data in IU or BAU.¹⁰ Additionally, regulatory officials, clinical trial sponsors, vaccine-development funders, and other organisations should request that all submitted data be reported in IU for neutralisation assays to ensure comparability between results. Finally, for optimal adoption of the use of standards, medical journals should require, as appropriate, the reporting of results in IU and BAU for publication. This standardisation is crucial, as without it, determining a meaningful correlate of protection threshold remains challenging.

A compilation of guidance and resources for assay developers is presented in the panel.

Conclusions

As the COVID-19 pandemic evolves, the ability to compare data across studies becomes increasingly important. The WHO International Standard for anti-SARS-CoV-2 immunoglobulin is a reliable tool for harmonising serological neutralisation data and calibrating binding antibody data. The use of this tool (and other aforementioned tools) and the adoption of IU and BAU as units of measurement of SARS-CoV-2 serology will advance our understanding of immune responses to SARS-CoV-2 and vaccines and will facilitate efficient evaluation and implementation of protective thresholds for antibody treatments and vaccines on a global scale.

Panel: Guidance and resources for assay developers.

Resources for assay development and usage

• WHO has developed The Unity Studies: Early Investigation Protocols, which are part of a global sero-epidemiological-standardisation initiative aimed at increasing evidence-based knowledge for public health actions. The protocols are available to promote standardised epidemiological, molecular, and serological methods and facilitate international comparisons between studies conducted in different laboratories, even those in different countries.²⁵

• The US Centers for Disease Control and Prevention has designed and validated a laboratory serological assay designed to estimate the percentage of the US population previously infected with SARS-CoV-2 and is partnering with the Frederick National Laboratory for Cancer Research (FNLCR) in performing independent evaluations of commercial antibody tests.^26,27 The results are updated monthly. A list of all serological assays currently with an US Food and Drug Administration (FDA) emergency use authorisation is also available.²⁸

• The US Pharmacopeia has published guidance on the design and development of biological assays.²⁹

• The European Medicines Agency and the FDA have also published assay-development guidance documents.^30,31

Resources for standards and reagents

• WHO has published guidance for the proper use of primary and secondary standards.³²

• WHO has provided descriptions of International Standards and other SARS-CoV-2 calibration reagents.³³

• A sample of the US SARS-CoV-2 serological standard can be requested from the FNLCR.³⁴

Resources for collaboration

Serology laboratories can also seek to join global collaborations to gain access to resources and protocols.

• The Solidarity II Collaboration seeks to enable standardisation of serological assays worldwide by sharing well characterised serum panels, providing access to high-quality antigens for use in serological assays, and developing standardised serological assays.³⁵

• The FNLCR also runs the coordinating center for a collaborative Serological Sciences Network (SeroNet), which can connect members or requesters to resources, protocols, and standards.³⁶