Two-Component Cluster Analysis of a Large Serodiagnostic Database for Specificity of Increases of IgG Antibodies against Pertussis Toxin in Paired Serum Samples and of Absolute Values in Single Serum Samples

Abstract

Measuring IgG antibodies against pertussis toxin (IgG-Ptx) with an enzyme-linked immunosorbent assay (ELISA) can be used to diagnose pertussis infection; however, the cutoff points are not unanimously defined. To determine the diagnostic specificity of increases of IgG-Ptx in paired sera and of absolute values in single serum samples, we applied a two-component cluster analysis to serum samples of patients suspected for pertussis, whose sera had been submitted to a routine diagnostic laboratory between 2003 and 2009, and had been assayed with an in-house IgG-Ptx ELISA calibrated with the international FDA lot 3 IgG-Ptx reference serum. Children eligible for the acellular pertussis vaccination were excluded to avoid interference from a vaccine-induced IgG-Ptx rise. Binary distribution mixtures were fitted to the data. Receiver operating characteristic (ROC) curves were calculated for absolute values in single samples (n = 14,452) and increases in paired samples (n = 2,455). For both parameters, two subpopulations could be identified: a population with high reactivity (persons with pertussis infection) and a population with low reactivity (persons without pertussis infection). For absolute values in single samples, the area under the curve (AUC) of the ROC curve was 0.993 and the optimum cutoff (with the highest cumulative value of specificity plus sensitivity) was 67.7 IU/ml (95% confidence interval, 63.9 to 74.1; sensitivity, 96.4%; specificity, 95.7%). A previously determined diagnostic cutoff of 125 IU/ml was associated with a sensitivity of 88.1% and a specificity of 98.8%. For increases in paired sera, the AUC was 0.999 and the optimum cutoff was 3.1-fold (95% CI, 2.8 to 3.4; sensitivity, 99.6%; specificity, 99.2%). Given the methodology of this study, estimates of sensitivity probably are overrated (because pertussis patients without IgG-Ptx response are not detected), but estimates of specificities can be considered very accurate.

INTRODUCTION

Despite a high coverage of their national immunization programs, many countries experience an increase in the incidence of pertussis, especially among adolescents and adults (5, 8). The gold standard for laboratory diagnosis of pertussis is the culture of Bordetella pertussis. However, both the culture and the pertussis PCR have a low sensitivity, especially when applied late in the disease (27). Serology is the most sensitive laboratory method for the diagnosis of pertussis, but because completion of the acute immune response requires several weeks, optimal sensitivity is reached relatively late in the disease (10, 27). Consensus has been reached that for the serological diagnosis of pertussis, measurement with an enzyme-linked immunosorbent assay (ELISA) of IgG antibodies to pertussis toxin (IgG-Ptx) is the method of choice (13). An international IgG-Ptx standard serum is commercially available, allowing uniform quantitation in IU/ml (31). Diagnosis is based on a significant increase of IgG-Ptx in paired sera or on high levels of IgG-Ptx in single samples (22, 24). High levels of IgG-Ptx are diagnostic because the peak levels induced by infection, which are reached within 4 to 8 weeks after the onset of infection, persist only temporarily (7, 10, 26, 28). Interference of the high IgG-Ptx levels induced by the vaccination of children with acellular pertussis vaccines is limited because vaccine-induced peak levels in children also persist only temporarily (11, 14, 16, 18, 20, 29).

Diagnostic cutoff points for serology have not been unanimously determined. For increases in paired sera the proposed diagnostic cutoff points range from 1.5- to 4-fold (1, 2, 10, 15, 25), and for absolute values in single serum samples the proposed diagnostic cutoff points range from 50 to 200 IU/ml (3, 10, 17, 19, 30, 32).

In this study we determined the diagnostic specificity of increases of IgG-Ptx in paired sera and of absolute values in single serum samples through the application of two-component cluster analysis to a large routine diagnostic database of IgG-Ptx in single and paired serum samples of patients suspected of having pertussis. In all those sera, IgG-Ptx was measured with an ELISA calibrated with the international IgG-Ptx standard serum (31). We show that for increases in paired sera, as well as for absolute values in single serum samples, two subpopulations can be identified: a highly reactive population (persons with pertussis infection) and a population with low reactivity (persons without pertussis infection), and that for both parameters the overlap is minimal (absolute values) or even practically absent (increases). As far as we are aware, this is the first study in which extensive data on the specificity of increases of IgG-Ptx in paired sera are presented.

MATERIALS AND METHODS

Serological testing.

The National Institute for Public Health and the Environment (RIVM) performs pertussis serology for patients suspected of recent pertussis infection. Routinely submitted sera from patients who were clinically suspected of pertussis are assayed with an in-house ELISA consisting of the measurement of IgG antibodies against purified Ptx (10, 21, 22). Since 1 October 2003, this ELISA has been applied with two modifications: patient sera are tested in 1:400 dilution, and antibody-binding activity in patient sera is calculated relative to the antibody-binding activity in a reference serum that has been calibrated with the international CBER/FDA (lot 3 derived) IgG-Ptx reference serum (31). The interassay coefficient of variation of the ELISA is <20% (as assessed at values between 20 and 200 IU/ml). All results were registered in an electronic database.

Data selection.

From the database, all samples from persons tested from 1 October 2003 until 31 December 2009 were selected. For each serum sample, the following data were extracted from the database: unique patient number; date of birth; date of blood sampling; IgG-Ptx test result; and, if known, date of the onset of disease. Based on the unique patient number, subsequent samples from one patient were matched. From this first data selection we obtained two data sets, one to evaluate the single-sample cutoff levels and one to study significant dynamics (fold increase in paired serum samples) indicative of infection.

To assess the validity of a cutoff level in single serum samples (data set A), we selected all patients with a known date of onset of disease, and from those we selected the ones who were sampled within 100 days after the onset of illness, as we assumed that by that time the antibody levels in all immune responders should have reached their highest level and would not yet be declining.

To study the fold increase in paired sera (data set B), we selected all patients with two serum samples available and with a second sample taken within 10 to 28 days after the first sample. In this selection, knowledge of the date of onset of disease was not required.

Besides natural infection, recent vaccination with an acellular pertussis vaccine may also cause elevated IgG-Ptx levels, while the whole-cell vaccine that was used in the Netherlands until recently has been shown to induce no or very low levels of IgG-Ptx (4). In the Netherlands, vaccination with an acellular vaccine was introduced for 4-year-olds in 2001, and in 2005 the whole-cell vaccine used for infants was replaced by an acellular vaccine. To avoid potential interference of IgG-Ptx induced by vaccination with an acellular pertussis vaccine, we excluded from both data sets (A and B) children who belonged to the cohorts eligible for either an acellular booster vaccination at 4 years of age (i.e., those born after 1 January 1998 and aged 4 years or older) or a vaccination with an acellular pertussis vaccine in infancy (i.e., born after 1 January 2005). Consequently, data set A contained 14,452 patients with a mean age of 28 years (standard deviation [SD] = 21.1), and data set B contained 2,455 patients with a mean age of 32 years (SD = 22.0).

Due to the use of only one dilution of patient serum, the IgG-Ptx assay has an upper limit of detection of 400 IU/ml. The lower detection limit of the assay is 5 IU/ml. Censoring was accounted for in the procedures for fitting binary distribution mixtures. Because the upper censoring level was so low as to remove almost any information on the shape of the positive component distribution, we used a small set of 56 known positive sera of patients (mean age, 20.0 years; SD = 14.6) from a household transmission study (9) with a test result of >400 IU/ml that had been fully titrated (see Fig. S1 in the supplemental material) to determine the shape of the positive component distribution.

Data analyses.

Before analysis, antibody levels were transformed to a log base 2 scale.

To discriminate between negative (“baseline”) and positive sera, binary distribution mixtures were fitted to the age-stratified data. Any observed (log) titer was assumed to have originated either from a (log) normal distribution LN(μ₁, σ₁) with probability 1− P or from a (log) normal distribution LN(μ₂, σ₂) with probability P. The parameters for either component (μ₁, σ₁) and (μ₂, σ₂), as well as the prevalence P, were estimated by maximizing the likelihood function,

$$\ell ({\mu }_1,{\sigma }_1,{\mu }_2,{\sigma }_2,\mathit{p})=\underset{\mathit{n}=1}{\overset{\mathit{N}}{\mathrm{\Pi }}}\phi ({\mathit{X}}_{\mathit{n}}|{\mu }_1,{\sigma }_1)(1-\mathit{p})+\phi ({\mathit{X}}_{\mathit{n}}|{\mu }_2,{\sigma }_2)\mathit{p}$$

for a set of log-transformed serum antibody titers {X₁, X₂, …, X_N}, where φ() is the density of the normal distribution. Contributions of censored observations (titers of >400 IU/ml or <5 IU/ml) were reweighed by replacing the density φ() with a corresponding cumulative distribution term in the above likelihood function.

For estimation of the fold increase in paired sera, a similar procedure was used, with the difference in log titer X_2,n − X_2,n as an observed change in serum antibody titer. Here, censoring could not be accounted for in a simple manner; therefore, for any titer of >400 IU/ml, a random sample was taken from the fitted positive component distribution for that data set and substituted in place of the censored observation.

The specificity and sensitivity for any cutoff point can be calculated from the two component distributions representing the positive and negative subpopulations, for both single serum samples and for paired sera of the seroconverted subjects.

Using the two fitted components, receiver operating characteristic (ROC) curves were constructed, and the area under the ROC curve (AUC) was calculated.

RESULTS

IgG-Ptx single serum cutoff.

Figure 1A shows the distribution of log-transformed IgG-Ptx titers in single serum samples and the two fitted components. Figure 2A shows the corresponding ROC curve, and Table 1 shows the cutoff and sensitivity for different levels of chosen specificity. The diagnostic cutoff with the highest cumulative value of specificity plus sensitivity, in the following called the “optimum cutoff,” was 67.7 IU/ml (95% confidence interval [CI], 63.9 to 74.1) with a sensitivity of 96.4% (95.9 to 96.9) and a specificity of 95.7% (95.3 to 96.1). The internationally used cutoffs of 100 and 125 IU/ml have sensitivities of 92.0% and 88.1%, respectively, and specificities of 98.0% and 98.8%, respectively. The cutoff of 94 IU/ml as found by Baughman et al. (3) has a sensitivity of 92.9% and a specificity of 97.7% in our model.

Fig 1.

(A) Density distribution of log₂(IgG-Ptx) concentrations in single serum samples (bars) obtained within 100 days after the onset of illness (n = 14,452) and the fitted negative (gray line) and positive (black line) components. The dark gray bar illustrates censored data, and the light gray bars illustrate their presumed distribution, as sampled from the positive component of the binary mixture. (B) Density distribution of the log₂(fold-increase) in paired serum samples with an IgG-Ptx concentration between 5 and 25 IU/ml in the first serum sample (n = 1,316). Lines indicate the fitted negative (gray line) and positive (black line) components.

Fig 2.

(A). ROC curve for the model fitted in Fig. 1A (absolute values of IgG-Ptx in single serum samples). (B) ROC curve for the model fitted in Fig. 1B (increases of IgG-Ptx in paired sera). Se, sensitivity; 1-Sp, 1-specificity.

Table 1.

Cutoffs and sensitivities in single serum samples obtained within 100 days after onset of illness for different levels of specificity

Specificity (%)	Estimated cutoff (IU/ml)	Sensitivity (%)
90	41.1	99.0
95	62.3	97.1
97.5	89.4	93.6
98	99.6	92.1
99	136.1	86.3

We also applied the model to the data stratified by the duration of reported illness, but this did not influence the optimum cutoff level, and the AUC was identical for all subgroups and the group in total (see Table S1 in the supplemental material). We also did not find a difference in optimum cutoff level when data for subgroups were analyzed that were stratified according to age (<1 to 9 years, n = 3,445; 10 to 19 years, n = 3,045; 20 to 39 years, n = 3,286; 40 to 59 years, n = 3,313; >60 years, n = 1,363), except for the >60-year age category, where the optimum cutoff was higher (96.1 IU/ml) than was the overall optimum cutoff (67.7 IU/ml).

Increase in IgG-Ptx in paired sera.

In the analysis of changes of IgG-Ptx in paired sera, serum pairs with IgG-Ptx in the first sample of >199 IU/ml were excluded since at such high (diagnostic) values, possible further dynamics are without additive diagnostic value. The remaining paired sera were divided into subgroups depending on the IgG-Ptx value in the first sample: <5 IU/ml, 5 to 24 IU/ml, 25 to 49 IU/ml, 50 to 99 IU/ml, and 100 to 199 IU/ml (Table 2). The best distinction between the low- and high-value clusters was found in the first four subgroups (AUC, 0.999), being somewhat lower for the serum pairs with an IgG-Ptx value between 100 and 199 IU/ml. The optimum cutoffs were similar for the first three subgroups: 3.3-fold, 3.1-fold, and 2.8-fold. At the higher IgG-Ptx concentrations in the first serum sample, the optimum cutoff was considerably lower. We selected the subgroup of paired sera with IgG-Ptx values in the first serum sample between 5 and 24 IU/ml for a more extensive presentation of the results: the distribution of changes of IgG-Ptx in those paired sera and the two fitted components in Fig. 1B and the corresponding ROC curve in Fig. 2B. The optimum cutoff in that subgroup was 3.1-fold (95% CI, 2.8 to 3.4) with a sensitivity of 99.6% (99.2 to 99.7) and a specificity of 99.2% (98.2 to 99.5). At lower cutoffs, e.g., 2-fold or 1.5-fold, specificity rapidly declined (Table 3). In this subgroup, stratification by age yielded similar results per age category except for the >60-year age category, where the optimum cutoff was slightly higher (4.5-fold increase) than in the total sample (3.1-fold increase).

Table 2.

Optimum cutoffs (fold increase), sensitivities, and specificities estimated with a mixture model for paired sera, stratified by IgG-Ptx level in the first serum sample

IgG-Ptx concn (IU/ml) in first serum sample	No. of samples	AUC	Optimum cutoff (fold increase)	Specificity (%)	Sensitivity (%)
<5	473	0.999	3.3	99.4	99.8
5–24	1,316	0.999	3.1	99.2	99.6
25–49	339	0.999	2.8	99.3	99.2
50–99	253	0.999	2.3	98.9	98.4
100–199	74	0.989	1.5	94.9	94.7
<5–199	2,455	0.924	2.1	84.4	97.7

Table 3.

Specificities and sensitivities for various fold increases of IgG-Ptx, estimated with a mixture model on paired sera with IgG-Ptx levels between 5 and 25 IU/ml in the first sample

Fold increase in IgG-Ptx	Specificity (%)	Sensitivity (%)
1.0	51.1	100.0
1.25	70.5	100.0
1.50	83.0	100.0
2.0	94.7	99.8
3.0	99.4	99.3
4.0	99.9	98.2
6.0	100.0	94.5
8.0	100.0	89.4

DISCUSSION

A cluster with low reactivity and one with high reactivity for absolute values in single serum samples, as well as for increases in paired sera, were found using a binary mixture model applied to IgG-Ptx values in the sera of patients suspected of having pertussis as gathered in a large routine serodiagnostic database, resulting in ROC curves with an AUC of 0.993 for absolute values in single serum samples and 0.999 for increases (changes) in paired sera. We expect that the application of our method to similar IgG-Ptx databases of other laboratories will yield the same results.

We considered the positive clusters to consist of patients with a very recent or actual IgG immune response to Ptx caused by pertussis. The assessment of sensitivities of diagnostic cutoffs in these positive clusters probably yielded an overestimation of true sensitivity because possibly not all patients with pertussis exhibit an IgG-Ptx immune response (10, 27). For a nonbiased assessment of the sensitivity of pertussis serology, prospective studies are necessary in which data from pertussis PCR and culture of Bordetella pertussis, vaccination status, and clinical symptoms and duration thereof have also been gathered (27, 29). We considered the negative clusters to consist of patients without a recent or actual IgG immune response to Ptx, i.e., patients with a coughing disease caused by respiratory infections other than pertussis or by other diseases. These negative clusters are very well suited for assessment of the specificity of diagnostic cutoffs. Possible contamination of the negative cluster with pertussis patients without an immune response to Ptx does not affect this suitability.

Our study is unique with respect to assessment of the specificity of diagnostic cutoffs for the increase of IgG-Ptx in paired sera. In most studies focusing on paired sera, the diagnostic cutoff for increases in IgG-Ptx is chosen in relation to the accuracy of the serological assay that is used. Traditionally, a cutoff of a 4-fold increase is used (1, 2, 10). However, the accuracy of the IgG-Ptx ELISA is such that lower increases, i.e., 2-fold or even 1.5-fold increases, have been taken to be diagnostic for pertussis (15, 25). However, in our study increases or decreases of 1.5-fold or 2-fold fell within the distribution of the cluster with low reactivity, indicating that changes of such magnitude are not characteristic of a specific immune response and may have other etiologies. In subgroups of paired sera with nondetectable or detectable but relatively low IgG-Ptx values in the first serum sample, the distinction between the cluster with low reactivity and the cluster with high reactivity was sharpest, yielding optimum cutoffs varying between 2.8- and 3.3-fold increases. In subgroups of paired sera with relatively high IgG-Ptx values in the first serum sample, the distinction between the cluster with low reactivity and the cluster with high reactivity was less sharp and optimum cutoffs were lower. This is probably due to contamination with patients with pertussis, in whom the IgG-Ptx already or almost had reached its peak so that a further increase was absent or modest. The results in the various subgroups may be translated in the following diagnostic rule: IgG-Ptx increases in paired sera are diagnostic for actual pertussis when the increase is ≥3-fold to a level of at least 20 IU/ml or ≥2-fold to a level of >100 IU/ml.

The IgG-Ptx immune response induced by infection requires several weeks to reach its peak and declines again after that peak (26). We expected the distinction between the cluster with low reactivity and the cluster with high reactivity to be blurred in the subgroup of sera from patients with the shortest duration of disease at the time of sampling. Surprisingly, the distinction between the positive and negative clusters in early disease was as sharp as it was in later disease. One explanation may be that many sera in the positive clusters contained IgG-Ptx levels of >400 IU/ml, i.e., above the upper limit of quantitation of the ELISA used in routine practice. To those values of ≥400 IU/ml, a distribution of exact values was given as found in 56 sera of pertussis patients with IgG-Ptx concentrations of >400 IU/ml in which the exact value had been specified by complete titration. This high-value distribution had a rather strong influence on the total distribution in the positive cluster. It may be that, in reality, the distribution of values of >400 IU/ml was variable depending on the duration of disease, with higher levels occurring later in the disease, and that application of the indirectly derived standard high-value distribution has exaggerated the distinction between the positive and the negative clusters, specifically in those with a short duration of disease (28).

In most studies on the diagnostic value of IgG-Ptx concentrations in single serum samples, population sera have been used for the assessment of specificity using the 95th percentile as the diagnostic cutoff (13). This may lead to an underestimation of specificity because population sera are contaminated with the sera of individuals with pertussis (3, 10). Baughman et al. (3) applied mixture models on population-based serum samples assuming a combination of four groups of exposure. For IgG-Ptx, they found four clusters of which the highest sera (4.2%) were hypothesized to be from individuals with recent B. pertussis infection. The 99th percentile of IgG-Ptx in the sera was 232 IU/ml, but after subtraction of the cluster of highest values this amounted to 94 IU/ml, and the authors proposed to use this value as the diagnostic cutoff. A point of criticism of this method is that infection-induced high IgG-Ptx levels decline over time in a continuous manner (26); therefore, clustering of high IgG-Ptx values that are associated with recent infection with B. pertussis only occurs when there is clustering of the time interval between the onset of infection and the sampling of serum. The sera used by Baughman et al. were sampled over a period of 3 years from a population in which pertussis was endemic; thus, it is highly unlikely that the clustering of time intervals since infection was present. Presumably, the application of this method to other sets of population sera will not yield similar clustering of IgG-Ptx values.

The positive and negative clusters of IgG-Ptx in single serum samples obtained within 100 days after the onset of disease consisted of large numbers resulting in a smooth ROC curve and small 5 to 95% confidence intervals. Thus, cutoffs for IgG-Ptx can be chosen depending on the a priori chance of pertussis. However, in routine settings, the a priori chance of pertussis may be as low as 10 to 20%, and in such settings a standard diagnostic cutoff associated with high specificity should be used. Serodiagnosis of pertussis, a self-limiting disease, usually occurs relatively late in the disease, and at that time the possibilities for treatment and prevention of transmission are often practically absent. In contrast, a false-positive serodiagnosis may unduly delay the diagnosis of other causes of prolonged coughing. From that point of view, a false-positive diagnosis of pertussis potentially is more damaging to the patient than is a false-negative diagnosis. Still, general practitioners should actively ask patients with persistent cough for a minimum of 1 week and possibly pertussis whether he or she has contact with infants or pregnant women, as young children are at the highest risk for severe disease, which may then be prevented by antibiotic prophylaxis. Information on infection control measures, such as cover-your-cough campaigns, will help to reduce exposure and transmission (6).

We prefer to keep using our previously established diagnostic cutoff of 125 IU/ml (10, 12, 23), which in this study had a specificity of 98.8%. Values of >62 IU/ml and <125 IU/ml (specificity, 95 to 98%) can then be reported as “suspect for pertussis; investigation of a second serum sample obtained within 2 to 4 weeks after the first is advised.” Interestingly, in population sera of >9-year-olds sampled in the same years, as was the patient sera from this study, the diagnostic cutoff of 125 IU/ml had a specificity of 96.6% (8), underscoring the problem of using population sera for the assessment of specificity.

Because in children the rate of decay of IgG-Ptx after vaccination with an acellular pertussis vaccine is similar to the rate of decay of IgG-Ptx after infection with B. pertussis, the time window after vaccination during which the use of IgG-Ptx for diagnosing pertussis is compromised is limited. However, as shown recently by Dalby et al. (7), the decay of IgG-Ptx induced by booster vaccination of adults with an acellular pertussis vaccine is approximately twice as slow as it is after infection with B. pertussis. In the Netherlands and probably in most countries, booster vaccinations beyond childhood are still rare, but their application may increase and should be reckoned with in the serodiagnosis of pertussis based on measurement of IgG-Ptx.