Correlation between serum bactericidal activity against Neisseria meningitidis serogroups A, C, W-135 and Y measured using human versus rabbit serum as the complement source.

Abstract

The surrogate of protection against invasive meningococcal disease is the presence of serum bactericidal activity (SBA) at a titer ≥4 in an assay using human serum as the complement source (hSBA). However, for various practical and logistical reasons, many meningococcal vaccines in use today were licensed based on a modified SBA assay that used baby rabbit serum as the complement source (rSBA). To assess the strength of correlation between the two assay systems for serogroups A, C, W-135 and Y, we analyzed a subset of samples from adolescent subjects enrolled in a Phase II study of Novartis’ MenACWY-CRM conjugate vaccine vs. an ACWY polysaccharide vaccine; samples were analyzed in parallel using hSBA and rSBA. We compared geometric mean titers (GMTs), calculated Pearson correlation coefficients between paired hSBA and rSBA results, and calculated sensitivity/specificity and likelihood ratios for an rSBA ≥8 or ≥128 for classifying hSBA ≥4, taking hSBA as the ‘gold standard’. Correlations between hSBA and rSBA ranged from 0.46 to 0.78 for serogroup C, but were weaker for serogroups A, W-135 and Y (range -0.15 to 0.57). In post vaccination samples, nearly all subjects had rSBA titers ≥8, though up to 15% remained seronegative by hSBA. In post vaccination settings, rSBA titers at ≥8 or ≥128 was highly sensitive for an hSBA titer ≥4, but non-specific. In conclusion, results generated by rSBA did not accurately classify serostatus according to hSBA for serogroups A, W-135 and Y.

INTRODUCTION

The serologic surrogate of protection against invasive meningococcal disease is the presence of serum bactericidal antibody (SBA) activity.[1] Evidence supporting this includes the therapeutic efficacy of hyper-immune horse serum for treating meningococcal disease in the pre-antibiotic era,[2] the inverse relationship between meningococcal disease incidence and rising bactericidal activity as a function of age,[3] and the increased risk of meningococcal disease among individuals with congenital hypo-gammaglobulinemia or any of several defects in the complement cascade.[4] However, the most direct evidence remains the studies by Goldschneider et al, in which the presence of measurable meningococcal serogroup C bactericidal activity among military recruits at the start of basic training was associated with lack of susceptibility to meningococcal disease.[3, 5] Subsequent to Goldschneider, the SBA assay has served as the basis for licensure of virtually all meningococcal vaccines.

The theoretical basis for the SBA assay is straightforward. Serum that has been depleted of innate complement activity (e.g., through heat inactivation), but which may contain bactericidal antibodies, is combined in vitro with a standardized preparation of meningococci in mid-log growth phase in liquid media. This is then supplemented with exogenous complement from serum that was either depleted of IgG or from a donor lacking bactericidal antibodies against the serogroup being tested, and the survival of meningococci assessed across serial dilutions of patient serum.

Goldschneider’s serum bactericidal assay used human serum as the complement source (hSBA), selected by screening for donors whose serum did not lyse the test strain, or prepared by adsorbing serum against the test strain polysaccharide. More recently however, assays using rabbit serum as the exogenous complement source (rSBA) have become a prevalent platform, due largely to the greater ease in obtaining serum without need for human donors. Nonetheless, Goldschneider et al’s findings remain the only direct link between a specific marker of immunity, in this case an hSBA titer against serogroup C of ≥4, and a clinical endpoint, i.e., the occurrence of invasive meningococcal disease in the military recruits.

Prior to licensure of serogroup C-conjugate meningococcal vaccines in the UK, analyses of the relationship between hSBA and rSBA results supported the use of rSBA as the basis for licensure.[6] At a population level, the subsequent near elimination of serogroup C meningococcal disease in the UK clearly validated the efficacy of these vaccines, but has also been taken as evidence validating the rSBA assay itself. Under the assumption that the observed correlation between rSBA and hSBA would apply to other serogroups, rSBA results have since been used to infer vaccine efficacy against serogroups A, W-135 and Y.

To better understand the relationship between rSBA and hSBA results, particularly for non-C serogroups, we analyzed data from a clinical trial conducted to support licensure of a novel quadrivalent A, C, W-135 and Y conjugate meningococcal vaccine. In this current analysis, data for a subset of trial subjects were generated in parallel using both hSBA and rSBA assays. This allowed us to address the following questions:

1) What is the strength of correlation between hSBA and rSBA results for each of the four vaccine serogroups, and does it vary between significantly between serogroup?

3) Does the strength of correlation vary as a function of whether the response is measured before (i.e. at baseline) or at time points after vaccination?

4) Is the strength of correlation influenced by the kind of vaccine used (in this case purified polysaccharide vs. protein-polysaccharide conjugate)?

MATERIALS AND METHODS

Overview

Serum samples were obtained from subjects enrolled in Novartis’ trial V59P6, a Phase II, randomized, single-blind, controlled multicenter trial comparing the safety and immunogenicity of one dose of quadrivalent Neisseria meningitidis serogroups A, C, W-135, and Y CRM₁₉₇ conjugate vaccine (Menveo®, Novartis Vaccines and Diagnostics) (MenACWY-CRM) or a quadrivalent A, C, W-135, and Y polysaccharide vaccine (Menomune®, sanofi pasteur) (MPSV-4), among healthy adolescents aged 11-17 years. The primary and key secondary endpoints were generated using hSBA, as previously reported.[7] Only results from subjects who had received the final non-adjuvanted formulation of MenACWY-CRM were included in this analysis. Subjects provided blood samples at baseline, 1 month after vaccination, and 12 months after vaccination. For the purposes of this analysis, subject samples from the MenACWY-CRM and MPSV-4 groups were randomly selected to undergo further testing by rSBA. At any given time point, samples used for this analysis were those that had paired results both by hSBA and rSBA.

Serologic Methods

The hSBA was performed by Novartis Vaccines serology laboratory in Marburg, Germany, using serum from donors who were screened for the absence of intrinsic toxicity to N. meningitidis or group-specific SBA activity as described previously.[8] The rSBA was performed by the Vaccine Evaluation Unit, Health Protection Agency, Manchester Royal Infirmary, United Kingdom, as previously described.[9]

Statistical Methods

Since the study population size was fixed in this analysis, no a priori sample size assumptions were made. SBA titers were expressed as the reciprocal of the last dilution that led to ≥50% killing of meningococcal test strains for each serogroup. Titers below the limit of detection were assigned a value of ‘2’ for calculation of GMTs. For hSBA, results were expressed as the proportion with a titer ≥4; for rSBA, results were expressed as the proportion with a titer ≥8 or ≥128. The choice to evaluate rSBA at both the ≥8 and ≥128 thresholds was in response to an earlier correlational analysis where the higher threshold was more specific for classifying hSBA serostatus.[10] The percentage of subjects and associated 95% Clopper-Pearson confidence intervals (95% CIs) were computed for each serogroup and for each vaccination group. Geometric Mean Titers (GMTs) were constructed by exponentiation (base 10) of the least square means of the logarithmically transformed (base 10) titers, and 95% CIs obtained from an Analysis of Variance (ANOVA) model.

We used several approaches for assessing the concordance between rSBA and hSBA results. We generated scatter plots for titers generated by hSBA and rSBA and calculated Pearson correlation coefficients for each pairwise comparison of results from matched samples. In addition, we assessed the operating characteristics of rSBA as a ‘test’ for correctly classifying the paired hSBA result as <4 or ≥4. For this analysis, we treated hSBA as the ‘gold standard’, and assessed rSBA as the predictor variable at threshold titers of ≥8 and ≥128. A similar analysis, comparing the proportion of pre/post samples demonstrating a four-fold rise in titer by rSBA and hSBA was abandoned after a preliminary analysis of the data revealed that of subjects with a paired pre and post vaccination sample only one subject failed to demonstrate a four-fold rise following either vaccination.

For each pairwise set of comparisons, we calculated sensitivity and specificity and positive and negative likelihood ratios (LR+ and LR−). Likelihood ratios provide an indexed measure of the accuracy of a positive or negative test result by combining the sensitivity and specificity of the test into a single statistic. The higher the LR+ (or lower the LR-), the more likely it is that the test result correctly agrees with the gold standard; by definition an LR at or near 1.0 represents a useless test, since the post-test results do not update pre-test odds of the hypothesis being correct (in this case that a given subject does, or does not, have an hSBA ≥4).

RESULTS

A total of 68 MenACWY-CRM and 70 MPSV-4 subjects (N=138), who had been enrolled in the US, provided serum for analysis that had matched hSBA and rSBA results for at least one of the three time points. Among MenACWY-CRM and MPSV-4 subjects, respectively, mean ages were 13.9 (SD 1.7) and 14.1 (SD 1.9) years; 58% and 64% were male; and 90% and 93% were Caucasian.

Baseline, 1-month and 12-month post vaccination hSBA and rSBA GMTs are summarized in Figures 1a and 1b. Universally, post vaccination rSBA GMTs exceeded hSBA GMTs for all serogroups. For MenACWY-CRM vaccinees, at 1 month after vaccination the lowest GMTs by hSBA were for serogroup A (37) and highest for serogroup C (89). By contrast, by rSBA, serogroup C had the lowest GMTs at 1 month after vaccination (1791), and very high results for serogroup A (8159).

Figure 1. Geometric mean titers (GMTs) at baseline, 1 month or 12 months after vaccination with MenACWY-CRM or MPSV-4, as measured by SBA using human (hSBA) or rabbit (rSBA) serum as the complement source.

summarizes the geometric mean titers for meningococcal serogroups A, C, W-135 and Y, as measured by hSBA (Figure 1a) and rSBA (Figure 1b). For each serogroup, results are presented in pairs, representing subjects who received MenACWY-CRM or MPSV-4, and by time period: baseline, 1 month after vaccination, and 12 months after vaccination. GMTs are expressed on a log scale, with the precise value displayed over the top of each histogram.

Figure 2 summarizes the correlation coefficients between paired samples when measured using the hSBA and rSBA assays for each serogroup across the three time points and for the two test vaccines. Serogroup C displayed the highest degree of correlation between hSBA and rSBA for both vaccines and across all time points, with correlation coefficients between ~0.5 and 0.8. For the other serogroups, the correlations tended to be quite poor, particularly at the baseline assessments. For example, among MPSV-4 recipients, the serogroup A coefficients were below 0.2 for all time points. For serogroup W-135, there was an inverse correlation between in the baseline results for subjects who were to receive MPSV-4. For both vaccines and for all serogroups, the strength of correlation improved between the baseline and 1 month after vaccination time points. Conversely, the strength of correlations tended to decline at the 12 month assessments, particularly for serogroups W-135 and Y among the MPSV-4 recipients. Lastly, the strength of correlations tended to be stronger among subjects vaccinated with MenACWY-CRM than with MPSV-4.

Figure 2. Correlation coefficients between hSBA and rSBA assessments among subjects who were vaccinated with MenACWY-CRM or MPSV-4.

provides the correlation coefficients on the Y-axis for each of the four serogroups across each of the three time points (baseline, 1 month after vaccination and 12 months after vaccination). Results are displayed separately for subjects who received MenACWY-CRM on the right, and MPSV-4 on the left. The lines connecting the points on the graph are provided solely for visual ease to link a given serogroup’s responses over time, not to imply probable correlations in the intervening time periods.

Figures 3a-3d present representative scatter plots for hSBA and rSBA titers one month after vaccination with MenACWY-CRM. Notably, samples lacking protective immunity by hSBA were often seropositive when measured by rSBA, i.e., ≥8. Overall, 15% of serogroup A samples (10/69) and 8.7% of serogroup C samples tested seropositive by rSBA but seronegative by hSBA. In some cases, the degree of discordance was striking. For example, one sample that tested seronegative for serogroup A by hSBA had a paired rSBA titer exceeding 65,000 (Figure 3a).

Figure 3. Scatter plots for paired results as measured by hSBA and rSBA, assessed 1 month after vaccination with MenACWY-CRM.

provides a representative sample of the matched hSBA and rSBA results from paired samples. Each point on the scatter plots represents the result of a single sample, for a single serogroup, at a single point in time (in this case, 1 month after vaccination), represented simultaneously by hSBA and rSBA. For example, the point at the bottom most left of figure 3a was a sample with a titer of 256 by rSBA and <4 by hSBA. The superimposed rectangles identify samples with discordant serostatus, i.e., where the hSBA was <1:4 but the rSBA was ≥ 8.

Table 1 provides the operating characteristics of using rSBA titers either at the ≥8 (Table 1a) or ≥128 (Table 1b) thresholds for classifying a given sample’s serostatus by hSBA, i.e., an hSBA titer ≥4 or <4, among subjects vaccinated with MenACWY-CRM. Overall, rSBA titers at both thresholds were highly sensitive for predicting hSBA seropositivity, but had poor specificity, particularly for the two post-vaccination time points. For all post vaccination assessments, sensitivity and specificity were at or close to 100% and 0%, respectively. The corresponding LR+s were 1.0, indicating that subjects with a positive rSBA result were no more likely to have a positive hSBA result than subjects with a negative rSBA result. Since all rSBA samples were positive at this time point, the predictive value of a seronegative rSBA result could not be determined, so the LR- was mathematically undefined [‘zero’ in the denominator of the equation LR- = (1-sensitivity)/(specificity)]. Shifting to the higher rSBA threshold of 1:128 did not improve the operating characteristics of the rSBA assay for classifying hSBA (Table 1b).

Table 1.

Operating characteristics of rSBA serostatus as a test for correctly classifying hSBA serostatus, according to either of two threshold values for rSBA.

Table 1a: Classification of hSBA (≥1:4 or < 1:4) by rSBA at a threshold titer of 1:8

Baseline (pre-vaccination)
Serogroup	Sensitivity	Specificity	LR+	LR−
A	100%	65%	2.9	0.0
C	37%	92%	4.6	0.7
W-135	50%	93%	7.1	0.5
Y	15%	94%	2.5	0.9
1 month after vaccination
Serogroup	Sensitivity	Specificity	LR+	LR−
A	100%	0%	1.0	Undefined*
C	100%	0%	1.0	Undefined*
W-135	100%	0%	1.0	Undefined*
Y	100%	0%	1.0	Undefined*
12 months after vaccination
Serogroup	Sensitivity	Specificity	LR+	LR−
A	100%	0%	1.0	Undefined*
C	86%	33%	1.3	0.5
W-135	100%	0%	1.0	Undefined*
Y	100%	22%	1.3	0.0

Table 1b: Classification of hSBA (≥1:4 or < 1:4) by rSBA at a threshold titer of 1:128

Baseline (pre-vaccination)
Serogroup	Sensitivity	Specificity	LR+	LR−
A	100%	79%	4.8	0.0
C	21%	98%	10.5	0.8
W-135	50%	93%	7.1	0.5
Y	5%	94%	0.8	1.0
1 month after vaccination
Serogroup	Sensitivity	Specificity	LR+	LR−
A	100%	0%	1.0	Undefined*
C	98%	17%	1.2	0.1
W-135	100%	0%	1.0	Undefined*
Y	100%	0%	1.0	Undefined*
12 months after vaccination
Serogroup	Sensitivity	Specificity	LR+	LR−
A	100%	3%	1.0	0.0
C	74%	50%	1.5	0.5
W-135	100%	0%	1.0	Undefined*
Y	100%	22%	1.3	0.0

Data are from subjects vaccinated with (or about to be vaccinated with) MenACWY-CRM. Sensitivity and specificity and likelihood ratios were calculated from a 2 x 2 matrix in which hSBA was treated as the gold standard, and rSBA as the predictor variable. rSBA serostatus was evaluated using either of two cut points: a titer ≥1:8 or <1:8 (Table 1a); or ≥1:128 or <1:128 (Table 1b).

^*Undefined due to zero in the denominator of the equation: [LR− = (1-Sensitivity)/(Specificity)]Abbreviations: LR+ ‘positive likelihood ratio’; LR− ‘negative likelihood ratio’

Abbreviations: LR+ ‘positive likelihood ratio’; LR− ‘negative likelihood ratio’

DISCUSSION

This analysis is one of few to have assessed the concordance between rSBA and hSBA results from matched samples. It is also the only to consider serogroups A, C, W-135 and Y responses in the same analysis, or to use multiple statistical approaches to test associations. This is relevant given that the stronger correlations observed previously for serogroup C, and echoed in our results, were assumed to apply also to the other serogroups. However, our results challenged this assumption. For serogroups A, W-135 and Y, paired hSBA and rSBA results generated from the same samples correlated poorly, and rSBA did not reliably classify hSBA serostatus. This was regardless of which vaccine was used, or the timing relative to vaccination. Moreover, samples lacking detectable hSBA activity were frequently positive by rSBA, in some cases with rSBA GMTs in the tens of thousands range. Consistent with our results, Findlow et al observed a weak correlation (r=0.5) between rSBA and hSBA for serogroup A among subjects who were vaccinated with a serogroup A conjugate vaccine, and a poorer correlation following vaccination with a serogroup A/C polysaccharide vaccine (r=0.34). Consistent with our results, several of Findlow et al’s subjects were seronegative by hSBA but had rSBA GMTs in the 100s to 1000s range.[11]

With the exception of one MenC-CRM conjugate vaccine, which was tested using hSBA, the licensure of MenC conjugate vaccines in the UK used rSBA assays. This was supported by in vitro data demonstrating a high degree of concordance between results generated by rSBA and hSBA[12]. Historically, serogroup A and C vaccines have shown excellent clinical effectiveness[13, 14] and vaccination leads to a predictable rise in rSBA titers.[15] However, it does not necessarily follow that the rise in rSBA titer mediates the efficacy of the vaccine.

The weak correlation between rSBA and hSBA could be interpreted in two principal ways.

The first assumes that the high proportion of subjects who had rSBA titers ≥8 (or ≥ 128) but low or undetectable titers by hSBA, indicates that hSBA is overly conservative, with a high rate of false negatives, i.e., the hSBA assay is specific but insensitive. Supporting this interpretation, sera from some adults lacking measurable SBA are still able to lyse meningococci in vitro.[16] Further, C6-depleted serum, which cannot trigger formation of the membrane attack complex, but can still opsonize via activation of C3, supports the killing of meningococci by polymorphonuclear cells.[17] In vitro data suggest that opsonophagocytic activity is important for protection against invasive meningococcal disease.[18, 19] Conversely, epidemiologic data suggest that deficiencies in antibody-independent mechanisms, notably the mannose-binding lectin pathway, predispose to meningococcal disease.[20, 21]

The second interpretation of these results is more conservative. If one accepts an hSBA titer ≥4 as the ‘gold standard’ surrogate of protection, then rSBA appears to generate many false positives, particularly for serogroups A, W-135 and Y: i.e., the rSBA assay is sensitive but not specific.

Our data cannot resolve which of these explanations is correct. Nor are the two interpretations necessarily mutually exclusive. Yet, from a public health perspective, the risk of underestimating protection through a less sensitive assay seems preferable to overestimating protection rates. To this point, following the UK Men C vaccination campaign, Auckland et al analyzed cases of Men C disease occurring in individuals within the target age ranges for vaccination. Among 465 individuals with confirmed Men C disease, 53 were true vaccine failures. Of these 53, 41% of these individuals had acute rSBA titers ≥8,[22] and therefore should have been protected. While the authors hypothesized that these individuals’ titers were measured just after seroconversion, no direct evidence was presented to support this theory.

As with previous reports, GMTs measured by the rSBA assay exceeded those generated by hSBA. This difference has been attributed previously to the species-specific ability of human factor H binding protein (fHBP) to bind factor H, an inhibitor of the alternative pathway of complement [23, 24]. The inability of meningococci to bind and inactivate non-human factor H renders them more sensitive to complement-mediated bacteriolysis by non-human complement.[25] Yet, if this finding was simply a matter scale, the hSBA and rSBA correlations should have been strong. Instead, with the notable exception of serogroup C, the strength of correlation was poor and varied significantly and without a clear pattern between serogroups A, W-135 and Y, between the pre- and post-vaccination assessments, and between the two test vaccines. Overall, our data suggest that the hSBA and rSBA assays measure different aspects of the immune responses, or are sensitive to subtle interactions between an individual subject’s serum and the two complement sources. These could include: blocking antibodies; non-specific interactions between patient antibodies and complement proteins or with non-bactericidal antibodies competing for shared epitopes; differential avidity of a subject’s antibodies to capsular or sub-capsular epitopes; or interactions with regulatory or counter-regulatory components of the complement cascade, such as anti-idiotypic antibodies.[26]

Our analysis has several limitations. First, the sample size was defined post hoc, rather than based on statistical assumptions. Second, the samples we tested all came from one population: US adolescents enrolled in a single clinical study. Results might vary across different age groups or ethnic populations.[27, 28] Conducting similar analyses in different populations and with a larger sample size would enhance the strength and generalizability of our findings. That said, the size of our cohort was quite similar to that used in the original rSBA/hSBA validation analysis for serogroup C.[6] Lastly, our analysis only considered subjects who received a single dose of study vaccine. It would be instructive to conduct a similar analysis among subjects receiving multiple doses of vaccine over time,[29] such as infants or toddlers receiving priming and booster doses of vaccine.

Our results conflicted with a previous report from Santos et al. That study only assessed serogroup C responses among a population of children aged 1-5 years who received a single dose of MenC-CRM (Menjugate, Novartis Vaccines, Cambridge, MA, USA) or MPSV-4.[10] In that analysis, the specificity of rSBA for predicting hSBA improved at a higher threshold titer of ≥128, whereas in our analysis the specificity of rSBA did not improve. Subjects in both studies received a single dose of vaccine. Although results often differ across laboratories using apparently similar methodology, it should be emphasized that the serogroup C hSBA assay used by Novartis in our current analysis was developed by transferring and revalidating Santos’ original serogroup C assay. Similarly, all rSBA results in both studies were performed at the same laboratory (Britain’s Health Protection Agency Laboratory). This makes it less likely that differences can be explained by inter-laboratory variability. By contrast, one notable distinction between Santos’ results and ours was the age of the subjects: children aged 1-5 years in Santos, adolescents aged 11-17 in the current analysis. Differential immunogenicity as a function of age could well explain the different magnitude of serogroup C GMTs observed in the two studies (116 in Santos et al vs. 1791 in this analysis). Simultaneously, it suggests an explanation for why the rSBA in the current study was not discriminative in our analysis: the rSBA GMTs exceeded the ≥128 threshold by a full log order, so differences in immune responses were obscured by a significant ceiling effect.[10]

CONCLUSIONS

Our analysis demonstrated that with the notable exception of serogroup C, rSBA and hSBA results were poorly correlated and the presence or absence of rSBA titers ≥8 did not identify subjects classify hSBA serostatus in adolescent vaccinees. While our results support earlier concerns that rSBA at titers of ≥8 were not discriminative, in this population, our analysis found no evidence that increasing to the higher threshold of ≥128 improved the operating characteristics of rSBA for predicting hSBA serostatus. Whether adopting an even higher threshold titer could improve the specificity of the rSBA assay remains an open question.