First International Quality Assurance Study on the Rapid Detection of Viral Agents of Bioterrorism

Abstract

We have conducted an international quality assurance study of filovirus, Lassa virus, and orthopox virus PCR with 24 participants. Of the participating laboratories, 45.8 and 66.7% detected virus in all plasma samples, which contained ≥5,000 and ≥100,000 copies per ml, respectively. Sensitivity levels were not significantly different between viruses. False-negative results were attributable to a lack of sensitivity.

Variola major virus, filoviruses, and Lassa virus are classified as category A bioterrorism agents (7). Attacks with these viruses might be carried out by liberation of weaponized material or dispersion of the agents (especially smallpox) through infected patients. Diagnostic capacities have recently been implemented by many countries to enable rapid detection of these special pathogens. PCR is widely used for testing (3, 6), but because of the restricted availability of virological and clinical material, the evaluation and standardization of test procedures is difficult. To provide insight into the level of diagnostic proficiency of responsible laboratories, we have conducted an external quality assurance (EQA) study through the European Network for Diagnostics of Imported Viral Diseases (ENIVD).

A total of 28 civilian and military laboratories from 17 countries were invited to participate in the study. Selection of invitees was based on the register of ENIVD members as well as on literature contributions relevant to the topic. The study was announced as an EQA study on diagnostic proficiency that would include the certification and publication of results in a comparative and anonymous manner. Of the invited laboratories, 24 from 14 (mainly European) countries enrolled in the study (see the acknowledgments section). All participants performed routine diagnostic services for the detection of orthopox virus, filovirus, and Lassa virus (12 laboratories), filovirus and Lassa viruses only (2 laboratories), or orthopox viruses only (10 laboratories).

Three test panels, containing a total of 33 lyophilized human plasma samples spiked with cell culture-derived and sequence-confirmed strains of orthopox virus, filovirus, and Lassa virus (including 5 samples negative for virus), were distributed. Samples positive for virus contained virus concentrations on the order of 10² to 10⁷copies per ml (Table 1). The virus stock solutions used for generating the samples were proven to be noninfectious by cell culture after heat inactivation for 1 h at 56°C and gamma irradiation with 30 kGy. After dilution, aliquoting, and lyophilization, the expected DNA/RNA concentrations in resuspended samples were confirmed by real-time PCR (Table 1). The integrity of virus particles was tested by monkeypox virus electron microscopy, yielding detectable particles down to a concentration of 1,600 copies per ml. The participants were asked to analyze the material with the molecular methods they would routinely use in suspected human infections. Details about the methodologies used were also requested.

TABLE 1.

Results by sample^a

Virus	ENIVD sample	Strainb	Copies/ml expected	Copies/ml determinedc	% Of laboratories achieving positive resultsd
Ebola	F6	Mayinga	—e	2.2E7	92.9
	F9	Mayinga	2.2E6	2.7E6	85.7
	F4	Mayinga	2.2E5	1.6E5	78.6
Marburg	F5	Musoke	—	3.6E7	78.6
	F3	Ratayczak	—	3.9E4	64.3
	F10	Ratayczak	3.9E3	3.9E3	57.1
	F18	Ratayczak	3.9E2	<2,647f	35.7
Lassa	L11	AV	4.0E5	5.4E5	85.7
	L4	AV	4.0E4	2.6E4	78.6
	L8	AV	4.0E3	2.6E3	50
	L17	AV	4.0E2	<2,445f	21.4
	L6	Josiah	4.8E6	3.8E6	71.4
	L9	Josiah	4.8E5	7.0E5	85.7
	L3	Josiah	4.8E4	1.2E5	71.4
	L2	Josiah	4.8E3	7.2E3	50
Orthopox	P12	MPX Lam87	5.0E7	3.2E7	100
	P1	MPX Lam87	5.0E6	3.4E6	95.6
	P10	MPX Lam87	5.0E5	5.6E5	81.8
	P5	MPX Lam87	5.0E4	8.0E4	68.2
	P14	MPX Lam87	5.0E3	8.0E3	59.1
	P3	MPX Lam87	5.0E2	1.6E3	45.5
	P6	MPX Lam87	5.0E1	<1,000f	27.3
	P7	CML	8.8E6	8.8E6	95.6
	P9	CPXV 81/02	—	6.0E5	86.4
	P15	CPX Brithon	—	2.4E6	95.6
	P13	VAC	—	1.0E6	90.9
	P11	VAC mod.g	—	1.6E5	86.4
	P4	ECT	—	3.6E6	72.7

^aAll samples available on request through ENIVD (www.enivd.de).

^bOrthopox virus species: monkeypox virus (MPX), vaccinia virus (VAC), cowpox virus (CPX), camelpox virus (CML), mousepox virus (ectromelia virus [ECT]).

^cMeasured by real-time PCR to exclude degradation of virus in plasma after heating and irradiation.

^dCoefficients of correlation of detection rates with virus concentration in samples: filovirus panel, r = 0.914; Lassa virus panel, r = 0.85; and orthopox virus panel, r = 0.914 (P values for the correlations were 0.004, 0.0081, and ≤0.00009, respectively). Positive test results for the total panel for Ebola and Marburg virus, Lassa virus, and orthopox virus, 70, 73, and 78%, respectively; positive test results for samples with ≤10⁴ virus copies, 46, 40, and 44%, respectively.

^e—, virus concentration originally determined in this sample.

^fVirus detected but below linear quantification range of assay.

^gmod., modified.

The cumulative fraction of positive-testing results for all virus-containing samples in the three panels (filoviruses, Lassa virus, and orthopox viruses) ranged between 70 and 78% (Table 1), without significant differences between panels (chi-square test; P = 0.23). Detection rates in samples containing low virus concentrations ranged between 40 and 46%, and differences between panels were also not significant (Table 1) (P = 0.86 in chi-square test). The proportion of laboratories successfully detecting virus in a sample correlated significantly with the virus concentration (Table 1). False-negative reactions were therefore mainly attributable to lack of sensitivity.

As a criterion for good diagnostic sensitivity of individual laboratories, we required all samples that contained a virus concentration of >5,000 copies/ml to return positive test results; this concentration is above the analytical sensitivity limit of published PCR tests for the pathogens of interest (3, 4, 6). Less than half (45.8%) of the participating laboratories fulfilled this criterion, and even when only the samples containing 100,000 or more DNA/RNA copies per ml were taken into account, still only 66.7% of laboratories could detect virus in these samples.

To determine factors critical for sensitivity, the methods used by the participants were compared. For extraction of nucleic acids, most participants used one or more of the following methods: viral RNA Mini kit, DNA tissue and blood Mini kit, and RNeasy kit (all Qiagen), High Pure and Magnapure kits (Roche), and a procedure based on chaotropic salts and glass beads (2). Results for all methods in testing samples containing ≤100,000 copies of virus per ml were subjected to an analysis of variance (Statgraphics version 5.0 software package; Statistical Graphics, Jena, Germany). No significant deviances from the grand mean at the 95% confidence level were observed for any method except with a Qiagen RNeasy kit. This method yielded a significantly lower rate of detection, but it should be noted that the manufacturer does not specify the kit for application to plasma samples. Because of the multitude of PCR tests used (12 published and 11 unpublished protocols), a reasonable statistical evaluation of these protocols was not possible. However, laboratories could be classified according to the use of in-house versus published methods as well as according to the use of conventional versus real-time PCR protocols. The use of real-time PCR versus conventional PCR in at least one test per laboratory, but not the use of in-house versus published methods, was identified to have a significant effect on a laboratory's overall sensitivity (better outcome in laboratories using real-time PCR [t test; P = 0.0069]). This may be attributable to the fact that real-time PCR is still a new technology preferentially applied in laboratories with a high level of expertise in PCR technology.

In the samples negative for virus included in the study, one laboratory reported two false-positive results (1× orthopox viruses and 1× Lassa virus). Such results are especially critical because of the serious public health disconcertion they can entail in a diagnostic situation.

In conclusion, this first voluntary quality assurance study suggests that most participants are able to correctly detect important viral agents of bioterrorism whereas a small but significant fraction of laboratories should review and optimize their diagnostic protocols. This is especially true since some aspects that complicate molecular diagnostics have not yet been taken into account in this preliminary study. As an example, a sample quality much worse than that of the test plasma used here might occur when real clinical cases have to be examined, requiring especially good sensitivity for compensation (5). On the other hand, experiences with EQA programs in other fields of molecular diagnostics have shown that results rapidly improved in subsequent studies (8). Well-characterized evaluation samples are an essential prerequisite for improving methods, and such material has been generated here for the first time for the viruses of interest. All samples are available through the ENIVD, and the results generated by participants in this study will be a valuable basis for others willing to establish or improve tests. Given the demand for biological preparedness (1, 7), regular participation in EQA programs will become more and more important for laboratories worldwide.