Reliability of Helicobacter pylori and CagA Serological Assays

Abstract

Background serological assays for Helicobacter pylori are commonly used without knowledge of reliability. This information is needed to define the ability of serological tests to determine either new cases of infection or loss of infection in longitudinal studies. We evaluated the reproducibility and the interrelationships of serological test results for H. pylori and cytotoxin-associated gene product A (CagA) enzyme-linked immunoassays within a subset of participants in a population-based study. Stored samples from 1,229 participants in the third U.S. National Health and Nutrition Examination Survey were replicate serologically tested for H. pylori and CagA. Overall disagreement was 3.4% between duplicate tests for H. pylori (or 2.3% if equivocal results were disregarded). Six percent of samples positive on the first test had an immune serum ratio at least 30% lower on repeat testing. The odds ratio for H. pylori seropositivity on retesting was 2.8 (95% confidence interval [CI] = 1.8 to 4.5) when CagA serology was positive versus when it was negative. CagA antibody was found among 47.8% of H. pylori-equivocal and 7.0% of H. pylori-negative samples. CagA-positive yet H. pylori-negative samples were more likely to occur among Mexican Americans (odds ratio, 5.2; 95% CI = 2.4 to 11.4) and non-Hispanic blacks (odds ratio, 5.5; 95% CI = 2.3 to 13.0) than among non-Hispanic whites. Relying on repeated H. pylori serological tests over time to determine infection rates may result in misinterpretation due to limits in test reproducibility. CagA testing may have a role in verifying infection.

Helicobacter pylori serology has often been used to characterize risk factors, prevalence, incidence, and loss of infection in various settings where endoscopy and breath tests have not been readily available (8, 21, 29). Such studies often involve large numbers of participants who may be geographically dispersed. The results of serological tests have done much to establish H. pylori as a causative agent of peptic ulcer disease and gastric cancer (13, 17). Serological detection of the cytotoxin-associated gene product A (CagA) of H. pylori appears to correlate with further increases in risk for peptic ulcer disease and gastric cancer (3, 11, 25).

H. pylori serological tests have been evaluated against criterion biopsy standards, including histology, culture, and Campylobacter-like organism tests (1, 6, 7, 19). We are unaware, however, of detailed investigation into the reliability of a commonly used H. pylori serological test. By “reliability” the ability to obtain a reproducible result with repeated testing is meant. Because unreliable serology cannot accurately reflect true infection, such information would help define the limit of accuracy of serological testing. We evaluated the reproducibility and the interrelationships of serological test results for H. pylori whole-cell and CagA enzyme-linked immunosorbent assays (ELISAs) in a large subset of participants in a U.S. population-based study.

MATERIALS AND METHODS

Sample selection.

The third national health and nutritional examination survey (NHANES 3) was conducted by the National Center for Health Statistics of the Centers for Disease Control and Prevention. The study was approved by the National Center for Health Statistics Institutional Review Board, and signed, informed consent was obtained from all participants. Details of the survey and results of H. pylori whole-cell testing have been published previously (9). A total of 7,465 interviewed and examined participants from phase 1 (1988 to 1991) with available serum were tested for H. pylori infection (78.7% of interviewed participants). The initial 900 samples received by the testing laboratory were serologically tested in duplicate for the presence of H. pylori and CagA. (One sample had insufficient serum for duplicate CagA testing, resulting in the number of samples for duplicate testing for the presence of CagA being reduced to 899.) They were subsequently supplemented with an additional 329 replicates based on potentially discrepant initial results, including those H. pylori positive and CagA negative (119 samples), H. pylori equivocal and CagA positive (60 samples) or negative (30 samples), and H. pylori negative and CagA positive (120 samples). The reason for the selection of these samples was not divulged to the testing laboratory.

Serological methods.

H. pylori serological testing was performed at Vanderbilt University using a commercial immunoglobulin G (IgG) ELISA (Pyloristat; Wampole Laboratories, Cranbury, N.J.). Each 96-well plate contained the manufacturer's three cutoff controls, negative, high-positive, and low-positive controls, and three positive and two negative controls provided by the testing laboratory. For each specimen, an immune status ratio (ISR) was calculated by dividing the specimen optical density by the mean optical density of the three cutoff controls. Specimens were considered negative if the ISR was 0 to 0.90, equivocal if it was 0.91 to 1.09, and positive if it was at least 1.10. The manufacturer reported a coefficient of variation of 15.8% for a sample with a low-positive ISR (ISR = 1.31) that underwent 30 repeated measures over 3 days (Wampole Laboratories H. pylori IgG ELISA package insert, Wampole Laboratories, 1997). CagA status was determined by an ELISA based on the presence of serum IgG antibodies against orv220, a 65,000-bp recombinant CagA truncated protein purified from Escherichia coli (4). CagA IgG was considered positive if the absorbance index was at least 0.35 (16). Duplicate testing was performed on separate days without regard to the results of the initial test.

Statistical methods.

Mexican Americans and non-Hispanic blacks were over sampled in NHANES 3. Results of NHANES are generally sample weighted to reflect the U.S. population distribution. Such national prevalence estimates are calculated using SUDAAN, a family of statistical procedures for analysis of data from complex sample surveys (B. Shah, B. Barnwell, and G. Bieler, SUDAAN user's manual, release 7.0, Research Triangle Institute, Research Triangle Park, N.C.). This approach was used for the estimate of the proportion of the U.S. population with discrepant H. pylori and CagA results. All other results were reported without regard to the sample weights because the subset of samples that underwent replicate testing was not randomly chosen from those eligible for testing and because we were not attempting to make prevalence estimates generalizable to the U.S. population. Comparisons of means were performed by paired and unpaired t tests. Mantel-Haenszel tests or logistic regression analyses were used to determine adjusted odds ratios and tests for trends.

RESULTS

Nine hundred consecutive samples were retested without regard to the results of the initial test. In aggregate, results of the first and second tests were nearly identical. Thus, the mean ISR difference between the first and second tests was 0.005 (P = 0.83). The numbers of samples that were positive in the first and second tests were 517 and 516, respectively. Individual differences between the results of the two tests are shown in Table 1. Overall disagreement between the first and second tests was 3.4% (95% CI, 2.4 to 4.9%). When the 33 samples with equivocal results on either test were excluded, overall disagreement between tests fell to 2.3% (95% CI, 1.4 to 3.5%). Samples positive for the H. pylori whole-cell ELISA in both tests had significantly higher mean ISRs on the first test (3.01) than samples that were positive in the first test only (1.52) (P < 0.001). Likewise, the mean ISR in the first test was higher for samples that were equivocal or negative in the first test and positive in the second test (0.62) than for samples that were equivocal or negative in both tests (0.31) (P < 0.001).

TABLE 1.

Results of initial and repeated tests for H. pylori antibody (n = 900)

Sample group resulta	No. of samples	Disagreement on repeat tests
		No. dis- agreeing/ total no.	% Dis- agreeing	95% CI
All samples
Positive-positive	501	16/517	3.1	1.8%-5.0%
Positive-equivocal/negative	16
Equivocal/negative-equivocal/ negative	368	15/383	3.9	2.2%-6.4%
Equivocal/negative-positive	15
Samples with equivocal results excluded
Positive-positive	501	10/511	2.0	1.0%-3.7%
Positive-negative	10
Negative-negative	346	10/356	2.8	1.4%-5.3%
Negative-positive	10

^aResults are presented for initial test-repeat test for each sample group as indicated. Lower portion of table presents results for sample groups excluding those with equivocal results (n = 867).

Because decline in ISR has been correlated with successful H. pylori eradication, we further characterized quantitative differences in ISR between initial and repeated tests (Table 2). On dividing positive ISR levels into approximate thirds, samples in the lower third proved more likely to be negative on repeat testing (8.2%) than samples in the upper two-thirds (0.6%). Fourteen of the 16 initially positive samples that were negative or equivocal on repeat testing were among this lowest third of samples with positive H. pylori ISR values. Of the 517 samples that were positive on initial testing, on repeat testing 31 (6.0%) had ISRs at least 30% lower, 17 (3.3%) had ISRs at least 40% lower, and 10 (1.9%) had ISRs at least 50% lower. The proportion of samples whose ISRs declined a certain percent was unassociated with the initial ISRs of those samples. Demographic features of age, sex, and ethnicity were not found to be associated with H. pylori antibody test reproducibility (Table 3).

TABLE 2.

Decline in H. pylori EIA ISR among samples that initially tested positive for H. pylori according to level of the initial ISR (n = 517)

All samples or samples grouped by indicated initial ISR	No. of samples	% With antibody not detected on repeated testa	% With indicated decline of ISR on repeated test
			≥30%b	≥40%c	≥50%d
All	517	3.1	6.0	3.3	1.9
3.36-8.70	176	0.6	6.8	4.0	2.3
2.27-3.35	171	0.6	4.1	1.8	0.6
1.10-2.26	170	8.2	7.1	4.1	2.9

^aP value for trend, <0.0001.

^bP value for trend, 0.94.

^cP value for trend, 0.95.

^dP value for trend, 0.66.

TABLE 3.

Reproducibility of H. pylori EIA test results according to age, sex, and ethnicity (n = 900)

Demographic feature	No. of samples	% Disagreement between initial and repeat test	P value
Age (yr)
20-39	388	3.4	0.80
40-59	217	3.2
≥60	295	3.7
Sex
Male	429	2.8	0.31
Female	471	4.0
Ethnicity
Non-Hispanic white	239	5.0	0.32
Non-Hispanic black	142	3.9
Mexican American	488	2.7
Other	21	0

Next we compared H. pylori and CagA serological results. In the full national sample, 50.8% (95% CI = 47.9 to 53.6%) of the H. pylori-positive samples were CagA positive, but 47.8% (95% CI = 33.2 to 62.3%) of the equivocal H. pylori samples and 7.0% (95% CI =6.0 to 8.2%) of the negative H. pylori samples were also CagA positive. As a result, 24.6% of all CagA-positive results occurred among H. pylori-equivocal or -negative specimens. Among the CagA-positive samples, the mean CagA absorbance index was higher among H. pylori-positive samples (0.75) than among negative samples (0.59 [P < 0.001]) but no higher than among equivocal samples (0.77 [P = 0.37]).

To examine further the discordance between H. pylori and CagA results, repeat testing was performed on 329 samples that were H. pylori positive and CagA negative, H. pylori equivocal, or H. pylori negative and CagA positive, which resulted in a total of 1,228 samples that were replicate tested for both H. pylori and CagA antibody. The mean H. pylori ISR was higher if CagA antibody was detectable than if it was not, except among the fewer H. pylori-equivocal samples, which had a narrow range of ISRs (Table 4). Furthermore, if the initial CagA result was positive, the repeat H. pylori result was more likely to be positive than if the CagA result was initially negative. Thus, the odds ratio for a positive H. pylori result when CagA was initially positive was 4.8 for initially positive H. pylori tests, 5.5 for initially equivocal H. pylori tests, and 7.0 for initially negative H. pylori tests. Overall, upon controlling for the initial H. pylori ISR level, the odds ratio for H. pylori positivity on retesting was 2.8 with 95% CI of 1.8 to 4.5% (P < 0.0001) when initial CagA was positive versus when it was negative.

TABLE 4.

Agreement between initial and repeat H. pylori serologies and repeat CagA serology according to initial CagA antibody status (n = 1228)

Initial test					Repeat test H. pylori serology results			H. pylori odds ratio (95% CI)	Repeat CagA results
H. pylori serology	No. of results	CagA serology	Mean H. pylori ISR	P value	Positive (%)	Equivocal	Negative		Positive (%)	Negative
Positive	303	Positive	3.05	<0.002	299 (98.7)	3	1	4.8 (1.6-14.1)	286 (94.4)	17
	333	Negative	2.72		313 (94.0)	6	14		25 (7.5)	308
Equivocal	68	Positive	1.01	0.40	44 (64.7)	11	13	5.5 (2.4-12.8)	64 (94.1)	4
	44	Negative	1.00		11 (25.0)	19	14		6 (13.6)	38
Negative	140	Positive	0.49	<0.001	18 (12.9)	10	112	7.0 (2.8-17.2)	96 (68.6)	44
	340	Negative	0.25		7 (2.1)	4	329		22 (6.5)	318

CagA antibody test reproducibility for was 93.4% if the H. pylori result was positive, 91.1% if it was equivocal, and 86.3% if it was negative (right-hand two columns of Table 4). Seventy-one samples were reproducibly H. pylori negative yet CagA positive in both initial and replicate testing. These samples were compared with 309 samples that were reproducibly H. pylori and CagA negative by using logistic regression analysis that included ethnicity, age, and sex as potential determinants of CagA seropositivity. The CagA-positive samples were more likely to have come from Mexican Americans (odds ratio, 5.2; 95% CI = 2.4 to 11.4) and non-Hispanic blacks (odds ratio, 5.5; 95% CI = 2.3 to 13.0) than from non-Hispanic whites. Age and sex were unrelated to CagA seropositivity (P > 0.05).

DISCUSSION

Replicate testing of serum samples drawn from a nationally representative population-based study provided information on the reliability of H. pylori enzyme immunoassay (EIA) tests commonly used in epidemiological studies. In such studies, the only practical tests presently available for establishment of H. pylori status are serological or urea breath tests. For detection of past H. pylori infection, serological testing of stored serum is often the only option. Hundreds of epidemiological studies have used serological tests (8), but reproducibility has received little attention. The issue of reliability is critical in longitudinal studies in which serologically positive or negative groups are followed for changes in antibody status. The present study found that 3.9% of H. pylori-negative or -equivocal samples were positive on repeated testing (and a slightly lower percentage if equivocal results were ignored). In a longitudinal study, it would be expected that at least a small proportion of a cohort would appear to seroconvert without actually developing new infection. If the true conversion rate were low, false-positive results could be quite influential. For example, frequent testing of uninfected cohorts of adults in developed countries would likely result in falsely high infection rates (8, 24).

Reproducibility can define the limits of sensitivity and specificity of a diagnostic test. If samples were to vary frequently between positive and negative on repeated testing, then neither sensitivity nor specificity would be high. Good reproducibility, however, does not necessarily mean the test is sensitive and specific. A test could repeatedly give the same result and yet be inaccurate because it measures something other than the condition of interest. The degree to which lack of reliability contributes to lower sensitivity and specificity can be estimated by calculating test accuracy—the proportion of samples measured correctly. The manufacturer has reported sensitivity of 92% and specificity of 97% (Wampole Laboratories H. pylori IgG ELISA package insert, Wampole Laboratories, 1997). Independent unpublished results from the testing laboratory revealed a sensitivity of 98% and specificity of 91% among 23 confirmed H. pylori-positive patients and 43 H. pylori-negative patients. As noted in a review paper, median sensitivity for the Pyloristat test was 96% and median specificity was 88% among 14 patient groups (18). Given this range of sensitivity and specificity and the prevalence of infection of about 30% found in the adult U.S. population (9), the overall accuracy would be between approximately 90 and 95%. With a reproducibility of 96.6% as determined in this paper, between one-third and two-thirds of inaccurate test results could be attributed to errors in reproducibility.

Loss of antibody does not necessarily mean loss of infection, as shown by the 3.1% of samples that initially tested H. pylori positive but tested negative or equivocal in repeat testing. Most specimens that appeared to lose antibody in repeat testing had relatively low ISRs on the first test. Some authors have evaluated decline in ISR of at least 30 to 50% as a substitute for invasive testing to determine the effectiveness of therapy against H. pylori (10, 15, 26). Using these criteria, the present study indicates that about 2 to 6% of infected patients tested would appear to lose infection, the erroneous diagnoses being due solely to fluctuation in the test results. Higher initial ISR did not correlate to a greater proportional drop in the repeat ISR. Test reproducibility also did not vary by sex, age, or ethnic group, which suggests that reliability of this serological assay differs little across population settings in the United States.

Perhaps uniquely in a large, national study, CagA antibody testing was performed irrespective of the results of the H. pylori testing, which allowed several interesting observations. First, average H. pylori ISR levels were higher when CagA was present. This finding is consistent with the greater immune stimulus and higher H. pylori antibody levels among cagA-positive strains (27). Second, an initial CagA-positive test predicted H. pylori positivity on repeat testing, even after accounting for the initial H. pylori ISR level. If CagA serology improves the reproducibility of H. pylori serology, it might also improve the sensitivity and specificity of H. pylori antibody tests. This possibility should be tested against biopsy results. Third, a high proportion of CagA-positive tests was found among the H. pylori-negative specimens, results that persisted for most samples in repeat testing. In our study, 7% of the H. pylori-negative samples were CagA positive, compared with 8% in a study from northern California and 16% in an international study (25, 31). The authors of the latter study suggested that the discrepant samples came from a mixture of infected and uninfected persons, because mean pepsinogen levels were higher than those of other H. pylori-negative samples but lower than those of H. pylori-positive samples. Such discrepant antibody tests have recently been attributed to the relative insensitivity of whole-cell H. pylori antibody tests developed in one geographic region when used in other regions, perhaps as a result of small differences in multiple binding sites when using local antigens (2).

Our fourth observation regarding CagA was that reproducibly H. pylori-negative, CagA-positive antibody results occurred much more frequently among Mexican Americans and non-Hispanic blacks than among non-Hispanic whites. It is possible that this discrepancy could be due to H. pylori serology being less sensitive among Mexican Americans and non-Hispanic blacks. While it has been shown that H. pylori serological tests developed in Western countries are less accurate when used in Asia (5, 12, 20, 22, 30) and among children from other countries (14), we are unaware of even indirect evidence of ethnic differences in the accuracy of tests when used within the country where the test was developed; yet such differences in accuracy appeared to be the case among Mexican Americans and non-Hispanic blacks. If CagA-related H. pylori infection were underestimated in these groups, then a distorted understanding of the consequences of infection might occur. These populations have higher rates of gastric cancer than non-Hispanic whites, and CagA appears to be a risk factor for gastric cancer (23, 25, 28).

The present study had the advantage of being able to examine the reliability of H. pylori serology drawn from a national sample that was not biased by the selection of patients with H. pylori-related disease. Thus, its results are relevant to population-based epidemiological studies, particularly those that repeat testing over a period of years. The principal limitation was an inability to evaluate the clinical or biological significance of the results of serological retesting. Subsequent studies are needed to establish whether the serological patterns of H. pylori and CagA observed in this study have broader clinical and public health significance.