Operational Performance of the Onchocerca volvulus “OEPA” Ov16 ELISA Serological Assay in Mapping, Guiding Decisions to Stop Mass Drug Administration, and Posttreatment Surveillance Surveys

Abstract.

Onchocerciasis is a neglected tropical disease targeted for elimination. The World Health Organization (WHO) has developed guidelines for the verification of onchocerciasis elimination that include entomological and epidemiological criteria. The latter require demonstrating with statistical confidence that the infection prevalence in children is less than 0.1%, necessitating an assay with a high degree of specificity. We present an analysis of the performance of the Onchocerciasis Elimination Program for the Americas (OEPA) version of the Ov16 enzyme-linked immunosorbant assay (ELISA) when used under operational conditions. In Africa and Latin America, the assay demonstrated 99.98% specificity in 69,888 children in 20 foci where transmission was believed to be interrupted. The assay produced a prevalence estimate equal to that of skin snip microscopy when applied in putatively hypo-endemic zones of Ethiopia. The OEPA Ov16 ELISA demonstrated the specificity required to be effectively deployed to verify transmission elimination under the WHO guidelines, while exhibiting a sensitivity equivalent to skin snip microscopy to identify hypo-endemic areas.

Onchocerciasis (river blindness) is one of humanity’s neglected tropical diseases that may be eliminated by mass chemotherapy in the coming years.¹ It is caused by the filarial parasite Onchocerca volvulus which is transmitted by some species of Simulium black flies. In 2001, the World Health Organization (WHO) issued guidelines for the verification of elimination of onchocerciasis transmission, which were revised in 2016.² Both versions describe a four-stage elimination process: 1) launching mass drug administration (MDA) with ivermectin; 2) suppression of parasite transmission; 3) interruption of transmission, when MDA is discontinued and a 3- to 5-year posttreatment surveillance (PTS) period is launched; and 4) demonstration that transmission has not recrudesced during the PTS period, at which time transmission elimination may be declared. When all its transmission zones have been declared eliminated, a country may request WHO verification.

The WHO provides specific guidance on entomological and epidemiological surveys to be conducted during the final two stages of elimination. Both components are equally important in the “stop MDA” surveys, which require that programs demonstrate that the upper bound of the 95% confidence interval (CI) (95% ULCI) of the prevalence of O. volvulus infection in children under the age of 10 years is less than 0.1%, and that the 95% ULCI of the prevalence of vectors carrying infective-stage O. volvulus larvae is less than 0.05%.² “Post MDA” surveys rely primarily on entomological data, although serological data may also be collected.²

The WHO guidelines recommend the use of assays to detect the presence of immunogobulin G4 antibodies to Ov16, a 16 kDa antigen present in all stages of the parasite’s lifecycle.³ The strategy is to measure Ov16 antibody prevalence in children, who, having been born since the advent of the elimination program, should not be exposed to the parasite if the program is successful. The Ov16 antigen has two characteristics that are useful for this purpose. First, it is expressed in developing parasites and may elicit an antibody response before the appearance of skin microfilaria.⁴ Second, an Ov16 response may develop in situations that ultimately do not result in a patent infection (e.g., unsuccessful or single-sex infections). Ov16 antibodies may therefore represent a more sensitive and timely indicator of ongoing parasite transmission than detection of patent infections using classic skin snips read for microfilaria by light microscopy.

Where a 0.1% infection prevalence must be excluded, it is of utmost importance to maximize the specificity of the assay to achieve the highest positive predictive value possible. This can be carried out at the expense of sensitivity because clinical decisions are not made on the basis of individual test results, and therefore a relatively poor sensitivity can be compensated for by adjusting the sample size upward by a factor that is roughly the sample size necessary to achieve a goal using an assay with perfect sensitivity divided by the actual sensitivity.⁵ For example, if one is to say with 95% confidence that less than 0.1% of a population is positive, one must test 3,000 individuals and have none test positive if one is using an assay with a sensitivity of 100%. If one reduces the sensitivity to 99%, one must test 3,030 individuals and have none test positive to meet this criterion. However, if one reduces the specificity from 100% to 99%, one will be faced with a 1% false positive rate, and one will have to be able to say with 95% confidence that a 1.1% positive rate in the test population (1% false positive rate plus 0.1% true positive rate) is significantly higher than the 1% false positive rate. To do so will require testing more than 63,000 individuals.⁵

Before the deployment of the Ov16 ELISA, the Onchocerciasis Elimination Program for the Americas (OEPA) supported laboratory studies to operationalize the assay. Panels of known positive and negative samples were used to set a cutoff necessary to produce the degree of specificity necessary to meet the demands of the WHO threshold for verifying transmission elimination. A detailed version of the resultant “OEPA” Ov16 ELISA protocol has been published.⁶ However, the actual specificity and sensitivity of this assay has not been examined under operational conditions.

The OEPA Ov16 ELISA, coupled with polymerase chain reaction (PCR) confirmation of skin biopsies (snips) to detect patent infections in Ov16-positive individuals, has been used for a decade as a tool in successful stop-MDA surveys in Latin America and Africa.^7–22 To evaluate the specificity of the OEPA Ov16 ELISA, we collected and analyzed the raw data from all successful stop-MDA surveys conducted by Carter Center–assisted laboratories in Latin America and Africa where individuals positive in the Ov16 ELISA were subject to confirmation using skin snip PCR, following WHO recommendations. Data from one PTS study from Sudan, which included a serological component, was also included.²³ We felt that testing previously endemic populations was an ideal way to assess assay specificity, compared with calculating specificity using non-endemic individuals. All data were collected through routine surveillance activities conducted by the respective countries’ ministries of health, and thus was not considered as human subjects research.

Results from testing 69,888 children resident in 20 onchocerciasis transmission zones (foci) from Africa (10 foci) and Latin America (10 foci) were included in the study. Of these, 13 individuals were positive in the Ov16 ELISA (Table 1). All were tested with the recommended confirmatory assay (O-150 PCR analysis of DNA extracted from skin snips²⁴); none were positive. If we assume the worst-case scenario in which all 13 of these children represented false positives, the field specificity of the assay was 99.98%.

Table 1.

Summary of Ov16 serosurveys in stop MDA and PTS surveys

Country	Focus	Year	No. tested	Ov16 positive	Reference
Sudan	Abu Hamad (stop MDA)	2009	6,756	0	12
	Abu Hamad (PTS)	2014	5,266	1	23
Uganda	Mount Elgon	2015	3,072	0	16
	Imaramagambo	2015	3,256	0	19
	Itwara	2015	3,045	0	13
	Kashoya Kitomi	2017	3,018	0	20
	Maracha Terego	2011	6,634	0	–
	Mpamba Nkusi	2015	3,048	0	17
	Nyamugasani	2011	1,437	2	–
	Obongi	2011	3,308	3	21
	Wambabya	2017	3,079	0	–
Mexico	Southern Chiapas	2010	4,230	2	14
	Oaxaca	2008	242	0	8
	Northern Chiapas	2006	305	0	9
Ecuador	Esmeraldas	2009	2,012	5	15
	Central Endemic Zone	2010	3,417	0	18
	Santa Rosa	2004	3,232	0	7
	Huehuetenango	2007	3,118	0	10
Venezuela	North Central	2008–2010	2,089	0	11
	North East	2012	3,994	0	–
Colombia	–	2007	64	0	22
Total	–	–	69,888	13	–

PTS = posttreatment surveillance.

We then evaluated the sensitivity of the Ov16 ELISA assay using data collected as part of a mapping exercise to identify previously undiscovered onchocerciasis foci in Ethiopia. This activity, conducted by the Ethiopian Ministry of Health with assistance from The Carter Center, used a combination of standard skin snips (read by light microscopy to detect microfilaria) and Ov16 ELISA of resident adults (aged 18–90 years) in untreated districts adjacent to areas that had been previously identified as hyper- or meso-endemic for onchocerciasis. The data collected were part of the routine surveillance activities conducted by the Ethiopian Ministry of Health and thus not considered as human subjects research. The results are summarized in Table 2. Seven (0.7%, 95% CI: 0.2–1.2%) of the 1,026 individuals tested were positive by skin snip, whereas 12 (1.2%, 95% CI: 0.5–1.8%) were positive in the Ov16 ELISA. Although Ov16 prevalence was roughly twice the standard skin snip prevalence, the estimates were not significantly different (P > 0.05; χ² test). Only three individuals were positive in both assays. Taking the skin snip as the gold standard, the Ov16 assay exhibited a sensitivity of 43%.

Table 2.

Performance of the Ov16 ELISA vs. standard skin snip microscopy in putatively hypo-endemic zones of Ethiopia

	Skin snip positive	Skin snip negative	Total
Ov16 positive	3	9	12
Ov16 negative	4	1,010	1,016
Total	7	1,019	1,026

These data suggest that the Ov16 ELISA using the OEPA protocol exhibits a high degree of specificity under operational conditions. The assay, when applied to populations where transmission was believed to have been interrupted, was 99.98%. This calculation assumes that all of the positives seen were indeed false positives. However, it is possible that the some of these persons classified as false Ov16 positives were true positives. First, it is well known that microscopic examination of single skin snips is an insensitive indicator when applied to individuals with low-density infections.²⁵ One would expect to encounter low-density infections in children who had likely been treated with ivermectin and as a result had suppressed skin microfiladermia. Second, it is possible that some of the ELISA-positive individuals were exposed but had not developed a patent infection and were, therefore, negative by skin snip PCR. Thus, 99.98% must be considered the lower limit of the operational specificity of the OEPA Ov16 ELISA.

When judged against the standard microscopic examination of skin snips, the OEPA Ov16 ELISA had a sensitivity less than 50%. However, on a population basis, the Ov16 ELISA predicted a prevalence almost twice that of standard skin snip microscopy (although the difference was statistically insignificant). If it is assumed that all of the positives detected by the Ov16 ELISA were true positives (a reasonable assumption, given the high specificity of the assay when tested on endemic negatives), the Ov16 assay is likely to be a somewhat more sensitive indicator than skin snip microscopy for the detection of infection in low-prevalence situations.

Both the skin snip and Ov16 ELISA are insensitive assays when applied to areas where infection prevalence is low. The lack of sensitivity can be somewhat compensated for by increasing the sample size tested. For example, the current WHO guidelines call for testing 3,000 children and finding all to be negative to conclude that transmission has been interrupted in a given focus and treatment may be discontinued.² However, this estimate assumes an assay sensitivity of 100%. Accounting for a lower sensitivity (say 43% as estimated against the abovementioned skin snip) will require testing roughly 7,000 individuals.²⁶ Testing such a large number of individuals will mean that the assay would reach its limits imposed by assay specificity. For example, if one assumes that the specificity calculations presented previously are correct, one would expect to encounter a false positive roughly once in every 5,300 individuals tested. Therefore, one would expect to find at least one false positive in every group of the roughly 7,000 individuals that would need to be tested to meet the current WHO guidelines, when using a test with 43% sensitivity. Thus, a confirmatory assay would still be necessary to weed out false positives, even though the Ov16 ELISA exhibits a very high specificity. Currently, the WHO guidelines recommend the skin snip PCR assay as the confirmatory assay to be used in conjunction with the Ov16 ELISA.² But, the skin snip PCR assay suffers from some significant drawbacks when used as a confirmatory assay under the conditions specified by the WHO guidelines. First, the Ov16 ELISA the guidelines recommend applying the Ov16 ELISA to measure exposure in children, as a surrogate for measuring incidence. The reasoning behind this is that children born after effective control is implemented should not be exposed to the parasite. However, microfilaria infection intensities in children are likely to be quite low, and sensitivity of the skin snip is poor in low-density infections.²⁵ Furthermore, children older than the age of five are eligible to receive ivermectin, and such treatment would further suppress skin microfliadermia, further reducing the sensitivity of the skin snip PCR assay.

Ideally, a confirmatory assay to replace the skin snip would exhibit a high degree of sensitivity and specificity, would detect exposure as does the Ov16 ELISA, but be independent from the Ov16 assay. Adaptation of LIPS technology to the detection of antibodies to Ov16 and the use of other antigens has shown promise in this regard.^27,28 Ideally, these assays may be available to replace the skin snip PCR as a confirmatory assay in the near future.

Finally, the current WHO elimination guidelines leave little choice but to deploy a test parameterized toward the highest possible specificity to meet the stringent cutoff of an ULCI of < 0.1%. Recent modeling suggests that this cutoff is too stringent under most endemiological conditions.²⁹ If this were the case, biasing the OV16 cutoff toward maximum specificity can be relaxed, resulting in considerable improvement in the test’s sensitivity, which is understandably often criticized. Revising the cutoff would have several practical advantages, such as allowing less stringent assay conditions and a decrease in the number of samples that would need to be tested to verify that transmission had been interrupted. We recommend that the necessary studies be performed to support revising the current WHO guidelines for onchocerciasis elimination serological breakpoints.

Disclaimer: The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government.