Multilaboratory Assessment of Epstein-Barr Virus Serologic Assays: the Case for Standardization

IgA antibodies targeting Epstein-Barr virus (EBV) have been proposed for screening for nasopharyngeal carcinoma (NPC). However, methods differ, and the antigens used in these assays differ considerably between laboratories. To enable formal comparisons across a range of established EBV serology assays, we created a panel of 66 pooled serum samples and 66 pooled plasma samples generated from individuals with a broad range of IgA antibody levels.

ABSTRACT

IgA antibodies targeting Epstein-Barr virus (EBV) have been proposed for screening for nasopharyngeal carcinoma (NPC). However, methods differ, and the antigens used in these assays differ considerably between laboratories. To enable formal comparisons across a range of established EBV serology assays, we created a panel of 66 pooled serum samples and 66 pooled plasma samples generated from individuals with a broad range of IgA antibody levels. Aliquots from these panels were distributed to six laboratories and were tested by 26 assays measuring antibodies against VCA, EBNA1, EA-EBNA1, Zta, or EAd antigens. We estimated the correlation between assay pairs using Spearman coefficients (continuous measures) and percentages of agreement (positive versus negative, using predefined positivity cutoffs by each assay developer/manufacturer). While strong correlations were observed between some assays, considerable differences were also noted, even for assays that targeted the same protein. For VCA-IgA assays in serum, two distinct clusters were identified, with a median Spearman coefficient of 0.41 (range, 0.20 to 0.66) across these two clusters. EBNA1-IgA assays in serum grouped into a single cluster with a median Spearman coefficient of 0.79 (range, 0.71 to 0.89). Percentages of agreement differed broadly for both VCA-IgA (12% to 98%) and EBNA1-IgA (29% to 95%) assays in serum. Moderate-to-strong correlations were observed across assays in serum that targeted other proteins (correlations ranged from 0.44 to 0.76). Similar results were noted for plasma. We conclude that standardization of EBV serology assays is needed to allow for comparability of results obtained in different translational research studies across laboratories and populations.

INTRODUCTION

Assays that measure antibody responses to Epstein-Barr virus (EBV) have become increasingly important tools for studying and diagnosing nasopharyngeal carcinoma (NPC) and for other research (1, 2). Several studies have shown that individuals with elevated levels of antibody responses against EBV antigens (particularly IgA responses) are at increased risk for the development of NPC (3–15). In areas of NPC endemicity, such as Southern China, EBV IgA antibody testing has been proposed for general population screening to triage individuals for further clinical evaluation aiming at the early detection and treatment of NPC (4, 7, 16, 17). However, recent studies have elucidated the underlying (epitope) complexity of anti-EBV antibody responses, and this needs to be considered in order to achieve standardization within the community (2).

IgA antibodies against EBV capsid antigen (VCA-IgA) and EBV nuclear antigen 1 (EBNA1-IgA) are the two EBV serological markers most frequently considered for screening purposes (4, 7, 16–18). However, several assays that measure VCA-IgA and EBNA1-IgA exist, and efforts to standardize these EBV assays have been limited, making it difficult to compare results across studies that utilize different assays. To date, no studies have directly compared VCA-IgA or EBNA1-IgA results from the various assays used in different laboratories globally in order to define interassay agreement or assess whether the same humoral immune response is being measured by each assay. Since such markers have been proposed for use in early-detection NPC screening programs, understanding the relationship between existing commercial and research assays is necessary to enable interpretation of the published literature. Evaluation of the correlation and percentage of agreement between assays represents an important initial step toward the standardization for assays intended for clinical use.

To measure agreement between assays measuring antibodies against EBV, we conducted a study in which pools of serum and plasma from individuals with a range of expected antibody levels were created and were blindly distributed to six different laboratories for testing. We initially focused on assays that measure antibodies against VCA and EBNA1, because those are the two main EBV antigens targeted for antibody tests considered for EBV screening purposes. Here we describe the various laboratories’ methods and correlation/agreement between assays. For completeness, we also included assays that measure antibodies against other EBV proteins (e.g., early D antigen [EAd] and Zta) in order to understand the correlations between assays that measure antibodies against these different proteins.

MATERIALS AND METHODS

Source population.

The panel of EBV serology standards was created by capitalizing on biospecimen resources from ongoing and completed studies conducted in Taiwan (10, 19) between 1991 and 2016. Serum and plasma samples were prepared within 24 h of collection and were stored frozen at –80°C until analysis. These studies were reviewed/approved by the National Cancer Institute Special Studies Institutional Review Board and the National Taiwan University Institutional Review Board. Written informed consent was obtained for all participants.

Creating pools for testing.

To create a resource with sufficient volume to permit testing by multiple assays in multiple laboratories, pooling of samples across individuals was required. We created both serum and plasma pools, with different individuals contributing samples for serum pools and plasma pools because of limited specimen availability from the previous studies. To ensure that a broad distribution of IgA antibody responses was retained after pooling, blood samples from individuals with similar expected IgA responses were pooled whenever possible. IgA antibody titers at collection were retrieved from participants’ medical files or experimental records, based on different IgA assays in routine clinical use at the time each of the studies was conducted. Briefly, a total of 66 pooled serum samples and 66 pooled plasma samples were generated from an average of 2 individuals (range, 1 to 5 individuals), of which 22 pooled serum or plasma samples were created from (i) NPC patients (representing samples with potentially elevated IgA antibody titers) and non-NPC patients with known high levels of IgA antibodies against EBV, (ii) general-population controls from a previously conducted NPC case-control study (representing samples expected to have low IgA antibody titers) and hospital outpatients with known low levels of IgA antibodies against EBV, and (iii) unaffected individuals from an ongoing multiplex family study of NPC (representing individuals at high risk of developing NPC).

Plate batching of pools.

Participating laboratories were provided with one aliquot (range, 25 μl to 150 μl) of each sample without knowledge of whether the sample came from a high-risk or a low-risk pool. We also included approximately 20% randomly selected, blinded duplicate samples (n = 14) to assess within-assay intraclass correlation coefficients (ICCs) and coefficients of variation (CV). All samples were randomly distributed on the plate and were sent to participating laboratories in individual cryovials.

Assays performed.

Six independent laboratories agreed to test serum and/or plasma specimens using research or commercial assays (enzyme-linked immunosorbent assay [ELISA] or Luminex assays). Of the 26 assays, two VCA-IgA assays (A2.1 and A2.2) and two EBNA1-IgA assays (A9.1 and A9.2) were commercial assays purchased from the same company but tested in different laboratories with different predefined positivity cutoffs. No special instructions were given to the laboratories regarding the handling or testing of these specimens. The details of each assay, including information on sample dilution, antigens targeted, amino acid sequences, and whether the assays were designed to capture IgA alone, IgG alone, or IgG, IgA, and IgM (IgG/IgA/IgM), are provided in the supplemental methods in Text S1 and in Table S1 in the supplemental material. In total, we included eight assays designed to measure antibodies against VCA, of which six were designed to detect IgA, one was designed to detect IgG/IgA/IgM, and one was designed to detect IgG. We included nine assays designed to measure antibodies against EBNA1, of which six were designed to detect IgA, two were designed to detect IgG/IgA/IgM, and one was designed to detect IgG. Nine assays were designed to measure antibodies against other antigens (i.e., EA-EBNA1, Zta, and EAd), of which two assays were designed to detect IgA against EA-EBNA1 combined, four were designed to detect antibodies against EAd (two for IgA, one for IgG/IgA/IgM, and one for IgG), and three were designed to detect antibodies against Zta (two for IgA and one for IgG/IgA/IgM).

Statistical analysis.

We first utilized the blinded duplicate pools included in our panel to estimate the reproducibility of the 26 assays performed as part of our effort. For each specimen type (i.e., serum or plasma), assays were clustered according to their Spearman correlations using unsupervised hierarchical clustering with Euclidean distance and complete linkage (20). Correlation coefficients of >0.7, between 0.5 and 0.7, and <0.5 were considered to indicate strong, modest, and weak correlations, respectively (21). We also estimated percentages of agreement and kappa values for assay pairs using predefined positivity cutoffs for IgA assays, since these IgA assays have been proposed for screening for NPC (Table S1).

Analyses were performed using R Statistical Software (R Foundation for Statistical Computing, Vienna, Austria). All statistical tests were two-sided, and a P value of <0.05 was considered statistically significant.

RESULTS

After quality control, we excluded from further consideration six assays evaluating serum (i.e., assays A18, A19, A21, A22, A23, and A24) and five evaluating plasma (i.e., assays A3, A9.2, A14, A23, and A24) with ICCs of <0.8 or CV of >20% (Table 1). Among assays measuring antibodies against VCA, we included eight assays (six for IgA, one for IgG, and one for IgG/IgA/IgM) evaluating serum and seven assays (five for IgA, one for IgG, and one for IgG/IgA/IgM) evaluating plasma. Among assays measuring antibodies against EBNA1, we included nine assays (six for IgA, one for IgG, and two for IgG/IgA/IgM) evaluating serum and seven assays (five for IgA, one for IgG, and one for IgG/IgA/IgM) evaluating plasma. Among assays measuring antibodies against other antigens (i.e., EAd and Zta), we included two (both detecting IgA) for serum and six (three detecting IgA, one detecting IgG, and two detecting IgG/IgA/IgM) for plasma in the analysis. The average response levels are summarized in Table 1, and results stratified by our three predefined groups are shown in Table S2 in the supplemental material.

TABLE 1.

Summary of titers for assays testing anti-EBV antibodies^a

Assay	Antigen	Antibody type	Method	Unit	Antibody level in:
					Serum		Plasma
					Medianb (IQR)	Min–max	Medianb (IQR)	Min–max
VCA
A1	VCA-p18	IgA	ELISA	OD	4.28 (5.61)	1.25–22.58	2.41 (3.63)	0.53–17.12
A2.1	VCA	IgA	ELISA	Relative OD	1.18 (2.92)	0.19–14.71	0.7 (2.15)	0.15–13.5
A2.2	VCA	IgA	ELISA	OD	0.92 (1.98)	0.19–10.8	0.66 (2.06)	0.2–11
A3c	VCA	IgA	ELISA	OD	0.07 (0.16)	0.01–0.9	N/A	N/A
A4	VCA-p18	IgA	Luminex	MFI	1,737 (2,726.75)	15–10,615	1,105 (2,517.25)	6–10,231
A5	VCA-p18	IgA	Luminex	MFI	1,682.5 (2,932.5)	102–16,524	1,082 (2,317)	45–12,190
A6	VCA-p18	IgG	Luminex	MFI	11,466.25 (3,455.62)	1,256.5–21,006	13,130.25 (6,511.5)	1,661–19,609
A7	VCA-p18	IgG/IgA/IgM	Luminex	MFI	3,065 (2,764)	377–13,253	2,444 (2,073.5)	164–7,492
EBNA1
A8	EBNA1	IgA	ELISA	OD	1.12 (6.6)	0.7–25.5	0.87 (4.04)	0.44–17.65
A9.1	EBNA1	IgA	ELISA	Relative OD	0.44 (2.58)	0–5.07	0.25 (2.01)	0.00–5.04
A9.2c	EBNA1	IgA	ELISA	OD	0.13 (1.09)	0–2.65	N/A	N/A
A10	EBNA1	IgA	Luminex	MFI	58 (1,335.5)	5–1,987	38 (1,353.75)	35,796
A11	EBNA1	IgA	Luminex	MFI	277 (1,434.75)	52–4,994	185.5 (1,137.5)	19,725
A12	EBNA1	IgA	Luminex	MFI	119 (966.88)	26–4,131.5	71.5 (658.25)	30–3,403
A13	EBNA1	IgG	Luminex	MFI	10,569.75 (4,737.62)	345–16,185	10,938 (7,585.25)	290–17,357.5
A14c	EBNA1	IgG/IgA/IgM	Luminex	MFI	3,109 (2,162)	89–17,673	N/A	N/A
A15	EBNA1	IgG/IgA/IgM	Luminex	MFI	7,741.5 (3,384.75)	836–17,545	5,816.5 (6,303)	257–17,412
Other antigens
A16	EAd	IgA	Luminex	MFI	147 (2,003.75)	1–13,243	48.5 (2,046.25)	1–13,982
A17	EAd	IgA	Luminex	MFI	96 (75.25)	37–2,654	62.75 (85.25)	29–14,150
A18c	EAd	IgG	Luminex	MFI	N/A	N/A	90.75 (610)	29–7,846
A19c	EAd	IgG/IgA/IgM	Luminex	MFI	N/A	N/A	309.5 (1,518)	1–15,095
A20	Zta (ZEBRA)	IgA	Luminex	MFI	30.5 (237)	1–5,024	18 (537.25)	1–8,591
A21c	Zta (ZEBRA)	IgA	ELISA	OD	N/A	N/A	0.08 (0.11)	0.02–1.73
A22c	Zta (ZEBRA)	IgG/IgA/IgM	Luminex	MFI	N/A	N/A	268.5 (892)	1–6,309
A23c	EA-EBNA1	IgA	ELISA	OD	N/A	N/A	N/A	N/A
A24c	EA-EBNA1	IgA	ELISA	OD	N/A	N/A	N/A	N/A

^aAbbreviations: EBV, Epstein-Barr virus; EAd, early D antigen; EBNA1, EBV nuclear antigen 1; ELISA, enzyme-linked immunosorbent assay; MFI, median fluorescence intensity; OD, optical density; VCA, EBV capsid antigen; IQR, interquartile range; min, minimum; max, maximum.

^bThe median level is based on 66 pooled samples.

^cResults for assays with intraclass correlation coefficients (ICC) of <0.8 or coefficients of variation (CV) of >20% are presented as not available (N/A).

Antibodies against VCA.

The correlations between assays measuring antibodies against VCA in serum are presented in Fig. 1A. A total of three clusters were identified. Correlations tended to be higher within than across immunoglobulin classes (clusters 1 and 2 versus cluster 3; cluster 3 represented IgG and IgG/IgA/IgM). IgA-only assays grouped into two clusters: clusters 1 and 2. Cluster 1 included three research assays measuring the same antigen (VCA-p18 [assays A1, A4, and A5]; sequences given in Fig. S1 in the supplemental material) with a median Spearman coefficient of 0.85 (range, 0.85 to 0.87). Cluster 2 included two commercial assays: assays A2.1/A2.2 and A3 (assays A2.1 and A2.2 were purchased from the same company but tested by two different labs), with a median Spearman coefficient of 0.71 (range, 0.64 to 0.97). Weak-to-moderate correlations were observed among IgA assays across clusters 1 and 2, with a median Spearman coefficient of 0.41 (range, 0.20 to 0.66). The lowest correlation was observed between assays A2.2 and A5 (Spearman coefficient, 0.20).

FIG 1.

Unsupervised hierarchical clustering based on the Spearman correlation coefficient for assays measuring anti-EBV antibodies in serum. (A) Antibodies against VCA; (B) antibodies against EBNA1; (C) antibodies against Zta (ZEBRA), EAd, VCA, and EBNA1. Red indicates a strong positive correlation, and blue indicates a weak correlation.

Among IgA-only assays, the percentages of agreement for serum differed considerably, from 12% to 98% (kappa values ranged from −0.03 to 0.9 [Table 2]). Higher percentages of agreement were observed between assays that clustered together in Fig. 1 (e.g., between assays A2.1 and A2.2 [95%] and between assays A4 and A5 [98%]). In contrast, lower percentages of agreement were observed between assays that clustered separately in Fig. 1 (e.g., between assays A1 and A3 [12%] and between assays A3 and A5 [15%]).

TABLE 2.

Percentages of agreement (kappa) for assays detecting IgA antibodies against VCA in serum^a

Assay	% of agreement (kappa) with assay:
	A1	A2.1	A2.2	A3	A4	A5
A1b	100	52 (N/A)	47 (N/A)	12 (N/A)	95 (N/A)	97 (N/A)
A2.1		100	95 (0.9)	61 (0.2)	50 (–0.03)	52 (0.002)
A2.2			100	65 (0.3)	48 (0.02)	50 (0.05)
A3				100	17 (0.01)	15 (0.009)
A4					100	98 (0.8)
A5						100

^aVCA, Epstein-Barr virus capsid antigen.

^bAll samples were defined as positive by assay A1. No kappa value could be estimated, so results are presented as not available (N/A).

Antibodies against EBNA1.

The correlations between assays measuring antibodies against EBNA1 in serum are presented in Fig. 1B. Again, among three clusters that were identified, correlations tended to be higher within than across immunoglobulin classes (cluster 1 versus clusters 2 and 3; clusters 2 and 3 represent IgG/IgA/IgM and IgG). In contrast to the observations made for VCA, all IgA-only assays grouped into a single cluster (sequences are given in Fig. S2 in the supplemental material), with a median Spearman coefficient of 0.79 (range, 0.71 to 0.89). However, a wide range of percentages of agreement (29% to 95%, with kappa values ranging from 0.1 to 0.9 [Table 3]) was observed for these IgA assays.

TABLE 3.

Percentages of agreement (kappa) for assays measuring IgA antibodies against EBNA1 in serum^a

Assay	% of agreement (kappa) with assay:
	A8	A9.1	A9.2	A10	A11	A12
A8	100	67 (0.7)	76 (0.9)	73 (0.8)	62 (0.2)	68 (0.7)
A9.1		100	88 (0.8)	94 (0.8)	29 (0.1)	92 (0.9)
A9.2			100	94 (0.9)	41 (0.2)	89 (0.7)
A10				100	35 (0.2)	95 (0.7)
A11					100	30 (0.1)
A12						100

^aEBNA1, Epstein-Barr virus nuclear antigen 1.

Antibodies against other EBV antigens (Zta and EAd).

To understand the correlations between assays measuring antibodies against distinct EBV proteins (i.e., Zta, EAd, VCA, and EBNA1), we compared the results of assays targeting Zta and EAd (sequences are given in Fig. S3 in the supplemental material) with those of representative assays targeting VCA and EBNA1. Specifically, for this evaluation, we included one IgA assay from each of the two clusters identified for VCA-IgA (assays A1 and A2.1) and one assay from the single cluster identified for EBNA1-IgA (assay A8). The correlations between those assays in serum are shown in Fig. 1C. Weak-to-moderate correlations were observed for IgA assays, with a median Spearman coefficient of 0.60 (range, 0.44 to 0.76).

Results in plasma.

Correlations similar to those observed with serum samples were observed with plasma samples when comparisons were made across assays, and the results are presented in Fig. S4 and Tables S3 and S4 in the supplemental material.

DISCUSSION

IgA antibodies against EBV VCA and EBNA1 have been proposed to facilitate the diagnosis and early detection of NPC in high-incidence regions (9, 17, 18). However, very little effort has been made to standardize the assays being considered for such programs and to understand the similarities and differences in their performance. Here we report the first study to directly compare assays designed to measure these antibodies. Although we observed high correlations and levels of agreement between some assays, our results demonstrate wide variability among the assays evaluated when they were compared with respect to both antibody levels and serostatus. Such variability could be caused by differences in the antigens targeted, the detection methods, and the dynamic ranges of the assays. These findings highlight the need for more-formal attempts to validate and standardize EBV serology assays that are being considered or used for population screening or clinical diagnosis aimed at the early detection of NPC.

In the present study, clear differences were observed among assays designed to detect antibodies against VCA. Although a low level of agreement between assays designed to measure different Ig classes (IgG versus IgA) was expected (22), two distinct clusters of IgA assays were noted. For these two clusters, good agreement was noted for assays contained within a cluster while poor agreement was observed for assays across clusters. The high correlation within clusters is likely explained by the sharing of antigens/epitopes targeted by these assays (e.g., assays A1, A4, and A5 targeted VCA-p18, one of six proteins making up the EBV viral capsid), although in some instances (assays A2.1, A2.2, and A3), we could not confirm this fact, since information on target probes was not disclosed by the assay developer/manufacturer. The EBV VCA is a complex containing the major capsid protein (p160; BcLF1), the small capsid protein (VCA-p18; BFRF3), the scaffold protein (VCA-p40; BdRF1), the tegument protein p23 (BLRF2), and glycoproteins gp125/110 (BALF4) and gp350/220 (BLLF1) (2). The immunodominant and virus-specific antigenic domain of VCA-p18 has been mapped and is located in its C terminus (amino acids [aa] 110 to 176), whereas the location of the immunodominant domain is less clear for other VCA complex proteins (2). It is expected that different VCA components will contain distinct immunodominant domains, induce different levels of antibody response, and have different diagnostic performance. Moving forward, it will be important to report the probe sequences used to measure EBV VCA antibodies so as to facilitate the interpretation of results across studies.

For EBNA1, we noted poor agreement for assays designed to detect different Ig classes but better agreement for assays designed to detect IgA, suggesting that these assays target similar epitopes. In fact, review of the probe sequences used to capture antibodies against EBNA1 revealed overlap across all assays for aa 382 to 404. This is consistent with reports that an immunodominant epitope of the EBNA1 protein (BKRF1, the major antigenic component of the EBNA complex), is located within aa 390 to 450 (2, 13, 23). Nonetheless, it should be noted that despite the high correlation observed for EBNA1-IgA assays, the range of positive percent agreements between these assays was wide, suggesting various sensitivities or thresholds for defining a positive response. The seropositivity cut point we applied for each assay was predefined by the assay developers/manufacturers. These different assay positivity rates further highlight the need for careful validation and standardization of these assays in the future.

The moderate correlations for assays measuring IgA antibodies against different EBV proteins (VCA, EBNA1, EAd, and Zta) were included in this report for completeness and provide a useful benchmark for evaluation of the levels of agreement for VCA and EBNA1 assays. The rates of agreement across protein targets were consistent with previous findings (22). Elevated levels of anti-EBV antibodies could indicate ongoing viral lytic activity (reactivation) and a potential lack of control over the virus in general. Noteworthy is the fact that levels of agreement observed across proteins (expected to be modest) overlap those noted within proteins (expected to be high for well-standardized and characterized assays), again highlighting the need for further assay standardization in the future.

The strengths of our study included the careful selection of pools meant to represent the entire expected range of antibody levels, direct comparison of assays using these pools, and the inclusion of many assays and laboratories. However, our results should be interpreted in light of some limitations. First, serum and plasma samples were not collected from the same individuals, precluding formal comparison and correlation of the antibody levels in serum and plasma based on paired samples. Second, information on the nature of the EBV antigen used was missing for a few assays, precluding further exploration of the factors causing variability across different assays.

In conclusion, using a carefully defined panel of serum and plasma samples distributed among multiple reference laboratories, we report high agreement for some assays designed to measure antibodies against the same EBV antigens. However, we also observed considerable variability in the agreement between assays designed to measure antibodies against EBV VCA and EBNA1, both with respect to their correlation and with respect to their reported positivity rates. Our study highlights the need for more-systematic standardization of these assays and for the development of an international standard for measuring these antibody responses in serum or plasma. Such efforts are prerequisites for the formal evaluation and quantitation of the performance of these assays in clinical practice or for population-based screening aimed at the early detection of NPC in high-incidence regions.