Comprehensive Laboratory Evaluation of a Specific Lateral Flow Assay for the Presumptive Identification of Francisella tularensis in Suspicious White Powders and Aerosol Samples

Abstract

We conducted a comprehensive, multi-phase laboratory evaluation of the Tularemia BioThreat Alert^® (BTA) test, a lateral flow assay (LFA) for the rapid detection of Francisella tularensis. The study, conducted at 2 sites, evaluated the limit of detection (LOD) of this assay using the virulent SchuS4 strain and the avirulent LVS strain of F. tularensis. In 6-phase evaluation (linear dynamic range and reproducibility, inclusivity, near-neighbor, environmental background, white powder, and environmental filter extract), 13 diverse strains of F. tularensis, 8 Francisella near neighbors, 61 environmental background organisms, 26 white powders, and a pooled aerosol extract were tested. In the 937 tests performed, the Tularemia BTA demonstrated an LOD of 10⁷ to 10⁸ cfu/mL, with a sensitivity of 100.00%, specificity of 98.08%, and accuracy of 98.84%. These performance data are important for accurate interpretation of qualitative results arising from screening suspicious white powders in the field.

Tularemia is a zoonotic disease caused by Francisella tularensis, a Gram-negative facultative intracellular bacterium. F. tularensis is one of the most infectious pathogens known, with an estimated ID₅₀ for humans of <10 colony forming units (cfu).^1-3 There are 2 primary subspecies of F. tularensis that vary in virulence: F. tularensis subsp. tularensis (type A) and F. tularensis subsp. holarctica (type B).⁴ Infection with as few as 25 aerosolized organisms is established with F. tularensis subsp. tularensis.⁵ Humans can become infected through diverse environmental exposures (eg, blood-feeding arthropods, direct contact with an infected animal, or indirectly via tools used for animal dressing) and can develop severe and sometimes fatal illness; however, they do not transmit their infection to others.⁶ Infection can occur through inhalation or inoculation of the skin or mucous membranes. When bacteria enter through the skin or oral mucous membranes, enlarged and tender regional lymph nodes will be noted on physical examination.⁴ Primary clinical forms of tularemia vary in severity and presentation according to the virulence of the infecting strain, inoculum size, and site of inoculation. Primary disease includes ulceroglandular, glandular, oculoglandular, oropharyngeal, pneumonic, typhoidal, and septic forms.⁶ The incubation period for tularemia is 3 to 5 days (range 1 to 14 days) and is characterized by an abrupt onset, with fever, headache, chills and rigors, generalized body aches, coryza, and sore throat.⁶ Before the use of antibiotics, the fatality rate for tularemia caused by type A strains was 5% to 15% and, in the more severe respiratory form, 30% to 60%; currently, the fatality rate is <2%.⁶ Tularemia caused by type B strains is generally nonfatal but may have a protracted course with complications.⁴

F. tularensis has long been considered a potential biological weapon. The Japanese purportedly studied this organism at their germ warfare research unit (Unit 731) operating in Manchuria between 1932 and 1945.⁷ This microorganism was also weaponized by the Soviet Union and included strains that were engineered to be resistant to antibiotics and vaccines.⁸ F. tularensis was developed as a nonlethal agent by the US military through devices that would disseminate aerosols of F. tularensis.⁹ WHO estimated¹⁰ that the release of 50 kg of F. tularensis by an aircraft along a 2-km line upwind of a population center of 500,000 would result in 30,000 deaths and 125,000 people incapacitated. Because of prior weaponization, low infectious dose, dissemination potential, public health impact and needs for broad-based public health preparedness efforts (eg, improved surveillance, laboratory diagnosis and stockpiling of specific medications), F. tularensis was assigned to Category A¹¹ and is a Tier 1 select agent.¹²

The environmental niche occupied by F. tularensis is not well characterized. The bacterium can grow in vitro on rich laboratory media, but its nutritional requirements make it unlikely that it is a free-living microorganism in nature.¹³ Infected rodents, hares, and rabbits are important sources of human infection;¹⁴ however, they may not be the true reservoirs of infection, because, in these species, tularemia is an acute infection. Outbreaks of human disease often parallel outbreaks of tularemia in animals.¹³ Several outbreaks of tularemia due to type B strains have been associated with contaminated water supplies^15,16 Water contamination could result from the presence of infected urine, feces, or carcasses; however, it could also be due to the presence of organisms in the cysts or trophozoites of fresh water amoebae.¹⁷ F. tularensis is often difficult to isolate from environmental samples,¹⁸ but a selective medium has been developed for the isolation of F. tularensis and its near neighbors.¹⁹ To complicate matters, a number of Francisella-like bacteria have been identified in environmental samples (eg, soil, water, air)^20,21 and ticks,²² indicating considerable diversity within the Francisellaceae and suggesting that these organisms are more common and more widely distributed than previously thought. The presence of these near neighbors has complicated the detection of F. tularensis on filters from environmental aerosol collectors using real time PCR assays.^23,24

A biological attack involving F. tularensis might involve dispersal of the agent by aerosol.^25,26 Other modes of delivery could mimic the 2001 anthrax attack, which used the mail to disseminate spores of Bacillus anthracis.^27,28 During the 2001 anthrax attack, many public health laboratories and first responders were inundated with suspicious white powders because of fear and panic among the public.²⁹ When first responders encounter unknown white powders in the field, it is important to quickly evaluate them for the presence of biological threat agents to support appropriate public safety actions such as evacuation, closure of facilities to prevent additional exposure, decontamination of potentially exposed individuals, collection of samples for law enforcement and public health purposes, containing the material as appropriate to prevent secondary dissemination, and expediting the transfer of samples to the nearest laboratory response network (LRN) laboratory for immediate testing.

In order to support first responders with the appropriate tools to carry out their mission, there is a need to develop, evaluate, and validate rapid screening tools for testing suspicious white powders for the presence of biological threat agents of concern. A number of biodetection technologies are available for use by first responders for this purpose, including rapid antigen detection assays.³⁰

The purpose of this study was to evaluate the limit of detection, sensitivity, specificity, reproducibility, and limitations of an LFA for F. tularensis (Tularemia BTA Test, Tetracore^®, Inc.). The goal of this study was to determine whether the Tularemia LFA can provide reliable results, so that appropriate and effective decisions can be made by first responders to support public safety actions and avoid unnecessary fear, panic, and costly disruptions to society. This study was designed to provide an understanding of assay performance, including the likelihood of a false-negative result (ie, assay is negative but the analyte is present at a concentration above the limit of detection), a false-positive result (ie, assay is positive but the target analyte is not present in the sample), and the robustness and reproducibility of this assay for use in the field. This study was designed and executed through an interagency collaboration with participation from subject matter experts from the Department of Homeland Security (DHS), the Department of Health and Human Services (HHS), the Department of Justice (DOJ), the United States Department of Agriculture (USDA), and the United States Secret Service (USSS).

Materials and Methods

Biosafety Considerations

Strains used in this study were handled with appropriate biosafety conditions in accordance with Biosafety in Microbiological and Biomedical Laboratories (BMBL, 5th ed)³¹ and Federal Select Agent Regulations.

Tularemia BTA Test and BTA Reader MX

Tularemia BTA Kit, BioThreat Alert Reader MX (BTA Reader MX), and Tetracore BTA Buffer were obtained from Tetracore, Inc. (Rockville, MD). The performance of the Tularemia LFA and reader was evaluated at 2 test sites: samples containing viable virulent strains (including SchuS4) of F. tularensis (a Tier 1 Select Agent) and near neighbors were evaluated at the Centers for Disease Control and Prevention (CDC), and all other samples and the avirulent F. tularensis live vaccine strain (LVS) were evaluated at Omni Array Biotechnology (Rockville, MD). Samples for analysis were prepared at the CDC, Lawrence Livermore National Laboratory (LLNL), and Omni Array Biotechnology. Samples were diluted and analyzed and results were captured both visually and with the BTA Reader MX according to directions provided by the manufacturer—that is, between 15 and 30 minutes after adding the sample (150 μL) to the sample well of the lateral flow strip. The BTA Reader MX measures the ratio of incident light and absorbing light intensities on the surface of the lateral flow test strip. The resulting ratio, converted into a BTA Reader MX value by the instrument, is expressed without units. Samples with BTA reader MX readings of <200 were considered negative, and LFA tests on which the control line failed to develop were noted and discarded. The study consisted of 7 phases, which are described below. For Phases 1, 2, and 3, at least 1 negative control (BTA buffer) and 1 positive control (F. tularensis LVS, 10⁶ to 10⁷ cfu/mL) were tested each day of the study. For Phases 4, 5, and 6, at least 4 negative control (BTA buffer) and 2 positive control (F. tularensis LVS, 10⁶ to 10⁷ cfu/mL) test were run at each test site during each day of the study.

Phase 1: Linear Dynamic Range and Repeatability Study

The linear dynamic range and repeatability of the Tetracore Tularemia BTA test was determined using suspensions of F. tularensis SchuS4 in BTA buffer at the following concentrations: 10³ to 10⁴ cfu/mL, 10⁴ to 10⁵ cfu/mL, 10⁵ to 10⁶ cfu/mL, 10⁶ to 10⁷ cfu/mL, 10⁷ to 10⁸ cfu/mL, and 10⁸ to 10⁹ cfu/mL. For preparation of cell suspensions, F. tularensis strains were subcultured from frozen stocks onto cysteine heart agar containing 9% sheep blood (CHAB) and incubated at 35°C for 24 hrs. Isolates were subsequently subcultured 1 to 2 times using well-isolated colonies and minimal growth times (24 hours) to ensure maximum viability. A bacterial suspension was prepared in 0.85% sterile saline and lightly vortexed to ensure homogeneity. The density of this stock suspension was adjusted with sterile saline to an absorbance of 0.7 (1.4 x 10¹⁰ cfu/mL) at 600 nm, using a Microscan turbidity meter (Dade Behring, Inc., Deerfield, IL). The cfu/ml for a F. tularensis cell suspension with an OD₆₀₀ of 0.7 was determined by colony counts, and this absorbance subsequently used for preparing suspensions of known concentrations.

Suspensions for testing were prepared by performing 10-fold dilutions of the stock suspensions in BTA buffer. The diluted suspensions were quantified by spreading 100 μl onto CHAB, in triplicate, and counting colonies after incubation for 48 hours at 35°C. The diluted suspensions were lightly vortexed and immediately tested by adding 150 μL of each concentration to the sample well of a test. Results were read visually and with BTA MX Readers. The lowest concentration of bacteria that yielded positive results in 5 out of 5 LFA tests (LOD) was further evaluated for repeatability with an additional 123 tests; results were read visually and with 1 of 2 BTA MX Readers.

Linear dynamic range samples for the F. tularensis LVS strain were prepared using stock suspensions of F. tularensis LVS in BTA buffer at the following concentrations: 10³ to 10⁴ cfu/mL, 10⁴ to 10⁵ cfu/mL, 10⁵ to 10⁶ cfu/mL, 10⁶ to 10⁷ cfu/mL, 10⁷ to 10⁸ cfu/mL, and 10⁸ to 10⁹ cfu/mL. Positive control samples containing F. tularensis LVS strain were prepared at 10⁶ to 10⁷ cfu/mL. Each dilution was tested in triplicate by 2 operators. The diluted suspensions were gently vortexed before testing and immediately tested by adding 150 μL of each concentration to the sample well of a test. Results were read visually and with 2 BTA MX Readers.

Phase 2: Inclusivity Panel

To determine whether this assay could detect diverse strains of F. tularensis, 13 additional strains (Table 1) were tested. Colonies, grown overnight on CHAB plates, were selected and suspended in BTA buffer to a final concentration of 10⁸ to 10⁹ cfu/mL (1 log above LOD). A 150-μL volume of each suspension was tested 5 times.

Table 1.

Francisella tularensis Strains (N = 13) Used for Testing

S. No.	Species	Strain Name	Other Identifier	Location of Origin	Source	Year	Tree Code
1	Francisella tularensis subsp. tularensis	SchuS4	FSC237; NR 3015 FRAN016; DD 201 FRAN031 = SchuS4 Derivative USAMRIID 1944; Scherm	Ohio	Human	1941	A1a
2	Francisella tularensis subsp. tularensis	MA00-2987	NR 3017	Massachusetts	Human	2000	A1b
3	Francisella tularensis subsp. tularensis	ATCC 6223	FSC 230; B-38; FRAN001; DD 506; CCUG 2112; GIEM Schu	Utah	Human	2002	A2
4	Francisella tularensis subsp. tularensis	WY96-3418	FRAN072; NR 3016	Wyoming	Human	1996	A2a
5	Francisella tularensis subsp. tularensis	CO01-3713		Colorado	Rabbit	2001	A2b
6	Francisella tularensis subsp. holarctica	LVS	FRAN 004; ATCC 29684; FSC 155; DD 445	Russia	Water rat	1968?	B
7	Francisella tularensis subsp. holarctica	OR96-0246		Oregon	Primate	1996	B4
8	Francisella tularensis subsp. holarctica	KY99-3387		Kentucky	Human	1999	B2
9	Francisella tularensis subsp. holarctica	JAP	FRAN 024; FSC 022; Ebina	Japan	Human	1950	B
10	Francisella tularensis subsp. holarctica	RC503	FSC 257; GIEM 503/840	Russia	Tick	1949	B3
11	Francisella tularensis subsp. holarctica	SP03-1781	MO01-1673; SP98-2108; GA02-5387	Missouri	Human	2001	B2
12	Francisella tularensis subsp. holarctica	CA97-0657		California	Human	1996	Not tested
13	Francisella tularensis subsp. mediasiatica	FSC 147	GIEM 543	Kazakhstan	Midday gerbil	1965	N/A

Phase 3: Near Neighbor Panel

In order to understand the specificity of the Tularemia BTA test, 8 near neighbors (Table 2) were grown overnight on CHAB agar plates. Colonies were selected and suspended by vortexing in BTA buffer and diluted to a concentration of 10¹⁰ to 10¹¹ cfu/mL (3 logs above LOD). A 150-μL volume of each suspension was tested 5 times.

Table 2.

Francisella tularensis Near Neighbors (N = 8) Used for Testing

S. No.	Species	Strain Name	Other Identifier	Source	Location	Year	ANI to F. tularensis
1	Francisella novicida-like	TX07-6608		Seawater	Houston	2007	98%
2	Francisella novicida	GA99-3548	D9876	Human lymph node	Louisiana	1977	98%
3	Francisella philomiragia-like	TX07-7310		Seawater	Houston	2007	80%
4	Francisella philomiragia	ATCC 25015	97-11; Jensen O#319L	Muskrat	Utah	1969	83%
5	Francisella noatunensis noatunensis	DZM 18777	FSC774; FSC775	Fish	Norway	2006	82%
6	Francisella noatunensis orientalis	LMG24544	DSM 21254; Ehime-1; Ottem-Ehime 1; FSC771; PQ/AL 1105; NVI5887; JA12-2011	Three-lined grunts	Japan	2006	82%
7	Francisella hispaniensis	DSM 22475	FSC 454; CCUG 58020; FhSp1; FnSp1; F62	Human blood	Spain	2003	91%
8	Francisella cantonensis	FSC 996	08HL01032	Air-conditioning system	China	2008	79%
9	Francisella halioticida	DSM 23729	LMG 26062			2012
10	Francisella spp Wolbatchia persica	ATCC VR-331
11	Francisella Warm Springs	Tetracore Strain

Phase 4: Environmental Background Panel

Table 3 shows the information about the 61 strains of diverse environmental background organisms used in the study.³² Each of the microorganisms was inoculated onto optimal medium and incubated under appropriate conditions for 24 to 48 hours. A single, isolated colony was selected and inoculated onto a second agar plate and incubated for 1 to 6 days, depending on the organism and its growth rate. Plates were then sealed with parafilm, coded, and shipped to Omni Array Biotechnology. For testing, colonies were suspended in 4 mL BTA Buffer to a final density of 10⁹ to 10¹⁰ cfu/mL (2 logs above LOD). Once suspended, 150 μL of each cell suspension was added to the sample well of a test. Each organism was tested once by 5 different operators.

Table 3.

Environmental Background Panel

S. No.	Organism	Strain Name
1	Acinetobacter calcoaceticus	ATCC 14987; HO-1; NBRC 12552; NCIMB 9205; CIP 66.33; DSM 1139; LMG 1056
2	Acinetobacter haemolyticus	ATCC 17906; NCTC 10305; 2446/60; DSM 6962; CIP 64.3; NCIMB 12458
3	Acinetobacter radioresistens	ATCC 43998; DSM 6976; FO-1; CIP 103788; LMG 10613; NCIMB 12753
4	Aeromonas veronii	ATCC 35622; CDC 140-84
5	Bacillus cohnii	ATCC 51227; DSM 6307; LMG 16678
6	Bacillus horikoshii	ATCC 700161; DSM 8719; JP277; PN-121; LMG 17946
7	Bacillus macroides (aka Lineola longa; Bacillus sp.)	ATCC 12905; 1741-1b; DSM 54; NCIB 8796; NCIM 2596; NCIM 2812; LMG 18474
8	Bacillus megaterium	ATCC 14581; 7051; CCUG 1817, CIP 66.20, DSM 32, LMG 7127, NCIB 9376, NCTC 10342, NRRL B-14308
9	Bacteroides fragilis	ATCC 23745; ICPB 3498, NCTC l0581
10	Brevundimonas diminuta	ATCC 11568; DSM 7234; CCUG 1427, CIP 63.27, LMG 2089, NCIB 9393, NCTC 8545, NRRL B-1496, USCC 1337
11	Brevundimonas vesicularis	ATCC 11426; CCUG 2032, LMG 2350, NCTC 10900
12	Burkholderia cepacia	ATCC BAA-245; KC1766; LMG 16656; J2315; CCUG 48434; NCTC 13227
13	Burkholderia stabilis	2008724195; LMG 14294; CCUG 34168, CIP 106845, NCTC 13011; ATCC BAA-67
14	Chromobacterium violaceum	ATCC 12472; NCIMB 9131; NCTC 9757; CIP 103350; DSM 30191; LMG 1267
15	Chryseobacterium gleum	ATCC 29896; CDC 3531; NCTC 10795; LMG 12451; CCUG 22176; CDC 3531
16	Chryseobacterium indologenes	ATCC 29897; CDC 3716; NCTC 10796; CCUG 14483; CIP 101026; LMG 8337
17	Citrobacter brakii	ATCC 10053
18	Citrobacter farmeri	ATCC 31897; FERM-P 5539; AST 108-1
19	Clostridium butyricum	CDC 11875; ATCC 19398; NCTC 7423; VPI 3266; CCUG 4217; CIP 103309; DSM 10702; LMG 1217; NCIMB 7423
20	Clostridium perfringens	ATCC 12915; NCTC 8359; 3702/49; CIP 106516
21	Clostridium sardiniense	ATCC 33455; VPI 2971; DSM 2632; BCRC 14530
22	Comamonas testosteroni	ATCC 11996; 567201; FHP 1343; NCIMB 8955; CIP 59.24; NCTC 10698; NRRL B-2611; DSM 50244; LMG 1800; CCUG 1426
23	Deinococcus radiodurans	ATCC 35073; NCIMB 13156; UWO 298
24	Delftia acidovorans	ATCC 9355; LMG 1801; CCUG 1822; CIP 64.36; NCIMB 9153; NRRL B-783
25	Dermabacter hominis	ATCC 49369; DSM 7083; NCIMB 13131; CIP 105144; CCUG 32998; S69
26	Enterobacter aerogenes	ATCC 13048; CDC 819-56; NCTC 10006; DSM 30053; CIP 60.86; LMG 2094; NCIMB 10102
27	Enterobacter cloacae	ATCC 10699; NCIMB 8151; CCM 1903
28	Enterococcus faecalis	ATCC 10100; NCIMB 8644; P-60
29	Escherichia coli O157:H7	ATCC 43895; CDC EDL 933; CIP 106327; O157:H7
30	Flavobacterium mizutaii	ATCC 33299; CIP 101122; CCUG 15907; LMG 8340; NCTC 12149; DSM 11724; NCIMB 13409
31	Fusobacterium nucleatum subsp. nucleatum	ATCC 25586; CCUG 32989; CIP 101130; DSM 15643; LMG 13131
32	Jonesia denitrificans	ATCC 14870; CIP 55.134; NCTC 10816; DSM 20603; CCUG 15532
33	Klebsiella oxytoca	ATCC 12833; FDA PCI 114; NCDC 413-68; NCDC 4547-63
34	Klebsiella pneumonia subsp. pneumonia	ATCC 10031; FDA PCI 602; CDC 401-68; CIP 53.153; DSM 681; NCIMB 9111; NCTC 7427; LMG 3164
35	Kluyvera ascorbata	ATCC 14236; CDC 2567-61; CDC 0408-78; DSM 30109; CCUG 21164; CIP 79.53
36	Kluyvera cryocrescens	ATCC 14237; CDC 2568-61; CCUG 544; NCIMB 9139; NCTC 10484
37	Kocuria kristinae	ATCC 27570; DSM 20032; NRRL B-14835; CCUG 33026; CIP 81.69; LMG 14215; NCTC 11038
38	Lactobacillus plantarum	ATCC BAA-793; LMG 9211; NCIMB 8826
39	Listeria monocytogenes	ATCC 7302; BCRC 15329
40	Microbacterium sp.	ATCC 15283; MC 100
41	Micrococcus lylae	ATCC 27566; CCUG 33027; DSM 20315; NCTC 11037; CIP 81.70; LMG 14218
42	Moraxella nonliquefaciens	ATCC 17953; NCDC KC 770; NCTC 7784; CCUG 4863; LMG 1010; BCRC 11071
43	Moraxella osloensis	ATCC 10973; CDC Baaumamnn D-10; LMG 987; CCUG 34420
44	Myroides odoratus	ATCC 29979; NCTC 11179; LMG 4028; DSM 2802; CIP 105169
45	Mycobacterium smegmatis	ATCC 20; NCCB 29027
46	Neisseria lactamica	ATCC 23970; CDC A 7515; CCUG 5853; CIP 72.17; DSM 4691; NCTC 10617
47	Pseudomonas aeruginosa	ATCC 15442; NRRL B-3509; CCUG 2080; DSM 939; CIP 103467; NCIMB 10421
48	Pseudomonas fluorescens	ATCC 13525; Migula biotype A; NCTC 10038; DSM 50090; NCIMB 9046; NRRL B-2641; LMG 1794; CIP 69.13; CCUG 1253
49	Ralstonia pickettii	ATCC 27511; CCUG 3318; LMG 5942; CIP 73.23; NCTC 11149; DSM 6297; NCIMB 13142; UCLA K-288
50	Rhodobacter sphaeroides	ATCC 17024; ATH 2.4.2
51	Riemerella anatipestifer	ATCC 11845; CCUG 14215; LMG 11054; MCCM 00568; NCTC 11014; DSM 15868
52	Shewanella haliotis (Pseudomonas putrefaciens)	ATCC 49138; AmMS 201; ACM 4733
53	Shigella dysenteriae	ATCC 12039; CDC A-2050-52; NCTC 9351
54	Sphingobacterium multivorum	ATCC 33613; CDC B5533; NCTC 11343; GIFU 1347
55	Sphingobacterium spiritivorum	ATCC 33300; DSM 2582; LMG 8348
56	Staphylococcus aureus subsp. aureus	ATCC 700699; CIP 106414; Mu 50, MRSA
57	Staphylococcus capitis	ATCC 146; NRRL B-2616; BCRC 15248
58	Stenotrophomonas maltophilia	ATCC 13637; NCIMB 9203; NCTC 10257; NRC 729; CIP 60.77; DSM 50170; LMG 958; NRRL B-2756
59	Streptococcus equinus	ATCC 15351; 7H4; NBRC 12057; IFO 12057
60	Streptomyces coelicolor	ATCC 10147; DSM 41007; NIHJ 147; NBRC 3176
61	Vibrio cholerae	ATCC 14104; BG29

Phase 5a: White Powder Panel

A stakeholder panel consisting of representatives from industry, the first responder community, state public health laboratories, CDC, DOD, EPA, FBI, and other federal entities identified 26 white powders (shown in Table 4) that were commonly encountered by first responders and LRN reference laboratories.³³ These materials were evaluated for their ability to affect the performance of the assay. Five milligrams of each of the 26 white powders were sent to the test sites. After the addition of 500 μL of BTA buffer (final concentration = 10 mg/mL), each tube was vortexed for 10 seconds. The suspension was allowed to settle for at least 5 minutes, and then 150 μL of the supernatant was removed and added to a test. Each powder was tested once by 5 operators.

Table 4.

White Powder Panel

S. No.	Material	Source
1	Dipel (Bacillus thuringiensis)	SummerWinds Nursery, Palo Alto, CA
2	Powdered milk	Raley's Grocery Store, Pleasanton, CA
3	Powdered coffee creamer	Raley's Grocery Store, Pleasanton, CA
4	Powdered sugar	Raley's Grocery Store, Pleasanton, CA
5	Talcum powder	Raley's Grocery Store, Pleasanton, CA
6	Wheat flour	Van's, Livermore, CA
7	Soy flour	Van's, Livermore, CA
8	Rice flour	Ranch 99, Pleasanton, CA
9	Baking soda	Target Stores, Livermore, CA
10	Chalk dust	Target Stores, Livermore, CA
11	Brewer's yeast	GNC Stores, Livermore, CA
12	Drywall dust	Home Depot, Livermore, CA
13	Cornstarch	Raley's Grocery Store, Pleasanton, CA
14	Baking powder	Raley's Grocery Store, Pleasanton, CA
15	GABA (Gama aminobutyric acid)	Sigma-Aldrich Corp, St. Louis, MO
16	L-Glutamic acid	Sigma-Aldrich Corp, St. Louis, MO
17	Kaolin	Sigma-Aldrich Corp, St. Louis, MO
18	Chitin	Sigma-Aldrich Corp, St. Louis, MO
19	Chitosan	Sigma-Aldrich Corp, St. Louis, MO
20	Magnesium sulfate (MgSO4)	Sigma-Aldrich Corp, St. Louis, MO
21	Boric acid	Sigma-Aldrich Corp, St. Louis, MO
22	Powdered toothpaste	Walmart Pharmacy, Livermore, CA
23	Popcorn salt	Raley's Grocery Store, Pleasanton, CA
24	Baby powder	Target Stores, Livermore, CA
25	Powdered infant formula, iron fortified	Target Stores, Livermore, CA
26	Powdered infant formula, low iron	Target Stores, Livermore, CA

Phase 5b: White Powders Spiked with F. tularensis LVS

The white powders were also evaluated for their ability to interfere with, or inhibit, the detection of F. tularensis in the assay. After the addition of 450 μL BTA buffer to 5 mg of each of the white powders (final concentration = 10 mg/mL), 50 μL of a suspension of F. tularensis strain LVS (10⁸ to 10⁹ cfu/ml) was added to the tube and vortexed for 10 seconds. The spiked powder suspension was allowed to settle for at least 5 minutes, and then 150 μL of the supernatant was removed and added to the test. Each spiked powder was tested once by 5 different operators.

Phase 6a: Environmental Filter Extract

Thirty BioWatch filters that had been subjected to 24 hours of environmental aerosol collection were extracted by shaking with phosphate-buffered saline containing 0.1% Tween-20 (PBST) and pooled. The protein concentration was adjusted to 6 μg protein/μL with PBST containing 1% BSA (PBSTB), then shipped to the testing site. Protein concentrations were determined using Bradford Assay Reagent (Pierce Chemical Company, Rockford, IL) using a standard curve prepared with bovine serum albumin (EM Sciences, Cole-Parmer, Vernon Hills, IL).

A 500-μL volume of the pooled environmental filter extract containing 6 μg protein/μL was added to 500 μL BTA buffer. After mixing for 10 seconds, the suspension was allowed to settle for at least 5 minutes followed by removal of 150 μL of supernatant for testing. Each filter extract was tested 5 times, once by 5 different operators.

Phase 6b: Environmental Filter Extract Spiked with F. tularensis LVS

A 1.0-mL volume of pooled filter extract was added to a pellet containing 10⁸ to 10⁹ cfu/mL of F. tularensis strain LVS. After mixing for 10 seconds, the suspension was allowed to settle for at least 5 minutes followed by removal of 150 μL for testing. The spiked filter extract was tested 5 times, once by 5 different operators.

Statistical Analysis

Dot density plots, titration curves, and Receiver Operator Characteristic Curves (ROC) based on BTA Reader MX values were generated using GraphPad Prism version 7.01 for Windows (GraphPad Software, La Jolla, CA, www.graphpad.com). BTA test values were used for generating the interactive dot plots of LFA sensitivity and specificity calculations and assay performance evaluation using MedCalc Statistical Software version 17.2 (MedCalc Software bvba, Ostend, Belgium; https://www.medcalc.org; 2017).

Results

In this study, a total of 937 tests were performed, consisting of 380 positive tests and 557 negative test results. Thirty-eight positive control LFAs were run using a suspension of F. tularensis strain LVS containing 10⁶ to 10⁷ cfu/mL or 10⁹ to 10¹⁰ cfu/mL), and 36 negative control LFAs were run (using just BTA buffer as the sample) during the course of this study. All positive control and negative control samples tested in each phase gave expected results.

The number of LFA tests performed in each phase of the evaluation is shown in the Table 5. In Phase 1 (range finding and repeatability studies), a total of 168 tests were performed; 30 samples were tested at CDC using the virulent strain, F. tularensis SchuS4, for determining the LOD. All samples tested at a concentration >10⁷ cfu/mL to10⁸ cfu/mL were positive. Fifteen samples were tested at Omni Array Biotechnology using the vaccine strain F. tularensis LVS, and the LOD was determined as 10⁶ cfu/mL to 10⁷ cfu/mL. In Phase 1 repeatability testing, 123 tests were performed with F. tularensis SchuS4 at a concentration of 10⁷ to 10⁸ cfu/mL. Of these, 121 were positive as expected. The 2 remaining tests were visually positive and BTA reader negative. When the 2 test cassettes were read on a second BTA reader, both of them showed positive result. In Phase 2 (inclusivity), a total of 65 tests were performed, of which all 65 tests were visually positive as expected. Four tests were BTA reader negative, and when tested on a second BTA reader were positive. In Phase 3 (near neighbor), a total of 55 tests were performed, and all were visually negative as expected. Five tests were BTA reader positive, but when tested on a second BTA reader were negative. In Phase 4 (environmental background), 305 tests were performed, of which 295 were negative and 10 were positive based on both visual and BTA reader results. False-positive test results were observed with all 5 replicates, Myroides odoratus, and Staphylococcus aureus. In Phases 5 and 6, 260 tests were performed, of which 130 were negative and 130 were positive, as expected, based on visual and BTA reader results.

Table 5.

Details of the Number of Samples Tested, including the positive and negative controls by Ft LFA testing in each of the 6 phases

Test Phase	Positive Controls Tested	Negative Controls Tested	Number of Samples Tested	Total Tests Performed
Phase 1: Linear dynamic range and reproducibility testing	5	8	168	181
Phase 2: Inclusivity panel	5	5	65	75
Phase 3: Near-neighbor panel	5	5	55	65
Phase 4: Environmental background panel	5	5	305	315
Phase 5a: White powder panel	10	5	130	145
Phase 5b: White powders spiked with F. tularensis LVS panel	5	5	130	140
Phase 6: Environmental filter extract panel	3	3	10	16
Total tested	38	36	863	937

Before analyzing the linear dynamic range using BTA reader values, visual reading data were tabulated and a probit regression was performed to determine the concentration of F. tularensis SchuS4 and LVS strains that would correspond to a probability of detection of 0.95. These concentrations were estimated LODs (Figure 1). For SchuS4, the estimated LOD is 4.3 × 10⁶ cfu/mL (6.4 × 10⁵ cfu/assay), and for LVS, the estimated LOD is 4.3 × 10⁵ cfu/mL (6.4 × 10⁴ cfu/assay).

Figure 1.

Probit Regressions for the F. tularensis SchuS4 and LVS Strains. The curves are calculated probability of detection as a function of bacteria concentration. The estimated limit of detection is calculated by finding the bacteria concentration with a probability of detection at 0.95. For SchuS4, the LOD is 4.3 × 10⁶ cfu/mL (6.4 × 10⁵ cfu/assay), and for LVS the LOD is 4.3 × 10⁵ cfu/mL (6.4 × 10⁴ cfu/assay).

The true LOD of the assay was determined using the BTA reader values and the designated cutoff at 200. The LOD had to be a concentration where every replicate test produced a positive result above the cutoff of 200. The linear dynamic range study found that the lowest concentration of F. tularensis strain SchuS4 that gave a consistent positive result was 10⁷ cfu/mL to 10⁸ cfu/mL (Figure 2). The F. tularensis strain LVS was also tested, and the LOD was found to be approximately 1 log lower, at 10⁶ cfu/mL to 10⁷ cfu/mL. The 2 strains had different reactivity profiles when tested, and this can be seen in Figure 2. The SchuS4 strain has a lower BTA Reader MX value consistently through the various concentrations but demonstrates a steady increase in BTA Reader MX value as the concentration of F. tularensis cells increases. Conversely, the LVS strain has a significantly higher BTA Reader MX value at a concentration of 10⁶ cfu/mL and higher. However, there is a possible Hook effect after 10⁷ cfu/mL, where the BTA Reader MX value is at 10⁸ cfu/mL. The LOD that was determined for the SchuS4 strain was used as the concentration to assess repeatability, in which 123 tests were performed.

Figure 2.

Range-finding for SchuS4 and LVS Strains. These 2 curves are the titrations performed for the 2 F. tularensis strains (SchuS4 and LVS). The SchuS4 strain curve was generated with an average of 5 tests per concentration, while the LVS strain curve was generated with an average of 3 tests per concentration. The error bars are standard deviations of the BTA Reader MX values.

Sensitivity, specificity, and accuracy were used to measure performance of this assay, ascertaining whether, based on visual reads, the test could properly discriminate between samples with the analyte present versus samples where the analyte is absent. Each test result can be placed in 1 of 4 categories: true positive (TP, F. tularensis antigen present and test positive), true negative (TN, F. tularensis antigen absent and test negative), false positive (FP, F. tularensis antigen absent and test positive), and false negative (FN, F. tularensis antigen present and test negative). Sensitivity is defined as the proportion of true positives that are correctly identified by the test and is calculated as:

$$Sensitivity=100\%\times \frac{TP}{TP+FN}$$

Specificity is defined as the proportion of true negatives that are correctly identified by the test and is calculated as:

$$Specificity=100\%\times \frac{TN}{TN+FP}$$

Accuracy is the overall probability that a F. tularensis test correctly classifies the presence of this bacteria in the test sample and is calculated as:

$$Accuracy=100\%\times \frac{TP+TN}{TP+FN+TN+FP}$$

Table 6 is a 2x2 contingency table that shows the totals for each category and the resulting sensitivity (100%), specificity (98.1%), and accuracy (98.86%) of this assay.

Table 6.

2x2 Contingency Table

Test Result	Ft Present	Ft Absent	Total
Positive	342	10	352
Negative	0	511	511
Total	342	521	863
Parameter	Value (%)		Confidence Interval (%)
Sensitivity	100.00		98.93–100.00
Specificity	98.08		96.50–99.08
Accuracy	98.84		97.88–99.44

To further evaluate the assay, the BTA Reader MX values, which included the reruns on the second reader, were used to generate a Receiver Operating Characteristic (ROC) curve. Even though the reader values are not quantitative, the values can be used to further evaluate the accuracy of a detection test to discriminate the test-positive samples from those that are test negative using ROC analysis. The sensitivity and specificity are calculated for every possible cutoff point selected to discriminate between the positive and negative populations. This curve is created by plotting the true-positive rate as a function of the false-negative rate for every possible cut-off point. Figure 3 shows the ROC curve for the Tularemia BTA evaluation, and the area under the curve is 0.990. Interactive Dot Plot in Figure 4 provides a summary of all testing performed grouped into positive and negative results, with the cutoff line separating false positives from true negatives and false negatives from true positives.

Figure 3.

Receiver Operator Characteristic (ROC) curve provides a visual representation of the sensitivity and specificity of this assay. Each point on the curve is a possible cut-off value, and its place on the curve is determined by its specificity and sensitivity. The calculated assay sensitivity at the cutoff of 200 is 100.00%, and the specificity is 98.08%.

Figure 4.

Interactive Dot Plot provides a summary of all testing performed grouped into positive and negative results, with the cutoff line separating false positives from true negatives and false negatives from true positives.

Discussion

F. tularensis is a biological agent that can pose a tremendous public health risk because of its potential to be used in bioterrorism attacks. To have an effective response, it is important for there to be rapid, specific, sensitive, and robust tests that are portable and easy to use by first responders. Lateral flow immunochromatographic assays were first commercially introduced for pregnancy testing in 1988.³⁴ LFA assays require minimum samples and no specialized equipment³⁵ and could be used by first responders and law enforcement officers to test suspicious materials in field settings. Berdal et al¹⁶ used a lateral flow immunoassay, which employed a monoclonal antibody specific for F. tularensis lipopolysaccharide, to investigate an outbreak of water-borne tularemia. They were able to detect F. tularensis in both lemming carcasses and the well water in which the carcasses were found; however, this assay was less sensitive than PCR. Rapid BTA assays have previously been evaluated for the detection of biothreat agents including orthopoxviruses,³⁶ ricin,³⁷ abrin,³⁸ Bacillus anthracis,^32,39 and Yersinia pestis.⁴⁰ Limited evaluations have also been conducted with assays for the detection of Yersinia pestis,⁴¹ botulinum neurotoxins,^42,43 and staphylococcal enterotoxins.⁴⁴

The Tetracore Tularemia BTA test for F. tularensis is available for screening of suspicious powders and/or materials in the field to support necessary public safety actions. It is a rapid qualitative lateral flow test that can be used for the detection of F. tularensis using a combination of a polyclonal capture antibody and a monoclonal detection antibody. The purpose of this study was to evaluate the sensitivity, specificity, reproducibility, and robustness of this assay for its intended use in the field with environmental samples. When used in conjunction with the BTA Reader MX and using the cutoff value of 200, the LOD was found to be 10⁷ cfu/mL to 10⁸ cfu/mL. This LOD is supported by the testing performed where the LOD was 10⁸ cfu/mL through using only visual results.⁴¹ Using the BTA Reader MX in conjunction with the strips can potentially enable detection of faint lines that are not easily perceived through visual reading, but this also increases the likelihood of calling visually negative tests as false positives due to potential streaking effects. When comparing BTA LFAs to other commercially available tests for F. tularensis detection, this lateral flow has limited sensitivity, while more time-consuming tests such as the larger volume immune-filtration ABICAP tests came with the benefit of greater specificity.⁴¹ The LOD determined here is also lower than reported in an earlier study in which Zasada et al demonstrated an LOD of 10⁸ cfu/mL for F. tularensis using the Tularemia BTA assay.⁴¹ The difference in LOD may be because, in the previous study, F. tularensis organisms were inactivated by heating at 60°C for 22 hours prior to testing.

In this validation study, to assess the ability of the test to detect F. tularensis, suspensions prepared from 13 strains of F. tularensis (Table 1) were tested at a final concentration of 10⁸ cfu/mL to 10⁹ cfu/mL (1 log above LOD). For 4 strains, 1 of 5 replicates was negative when read on the BTA Reader MX. These strips were subsequently read on a second reader and were positive. To verify the specificity of this test, 8 near neighbor strains were tested at 3 logs above LOD, and 61 environmental background organisms were tested at 2 logs above LOD. The near neighbors gave negative results both visually and with the BTA Reader MX with the following exceptions. One F. philomiragia–like strain demonstrated a streaking effect on the lateral flow test strip (1 of the 5 replicates), resulting in a visual positive but BTA reader negative result. Repeat testing of another 5 replicates tested negative both visually and with the BTA reader. For 3 strains, 1 or 2 of the 5 replicates were visually negative and BTA Reader positive, but these same strips were re-read in a second reader and found to have negative values. Finally, 1 strain had 1 replicate testing positive in 2 BTA Reader MXs despite being visually negative, and it was noted by the operator that there was a streaking effect, which likely resulted in the false-positive call. When this strain was tested at a 1 log lower concentration of 10⁹ to 10¹⁰ cfu/mL, all 5 replicates had no streaking effect and tested negative visually and on the BTA reader. Of the 61 environmental background strains tested, 59 yielded negative results both visually and with the BTA Reader MX. False-positive results may in some cases be expected when testing bacteria containing Protein A, as the antibodies used in this lateral flow assay were purified on a Protein A column.

Limitations of this test include a relatively high LOD as compared to laboratory-based technologies such as real-time PCR and ABICAP, and any results obtained in the field should be verified by further analysis in a laboratory setting. In addition, the BTA readers were found to yield results that were not consistent with visual readings. These findings highlight the importance of these assays being performed by trained and experienced users with an understanding of the limitations of sample testing and result interpretation.

It should be noted that the screening of white powders was evaluated using 5 mg of powders. This test was evaluated only for suspicious materials, such as white powders, and has not been evaluated for other environmental specimens, such as soil, vectors, and the like. However, Berdal et al demonstrated that a rapid immunochromatography test similar to the BTA could be used with environmental samples like well water without any further processing.¹⁶ Benefits of the smaller footprint in its handheld format as well as the ability to test various sample materials made it the ideal field test at the time.

In conclusion, the results presented here demonstrate a sensitivity (100%), specificity (98.10%), and limit of detection (10⁷ cfu/mL to 10⁸/ cfu/mL) for the Tularemia BTA LFA. These performance data are important for accurate interpretation of qualitative results arising from testing suspicious white powders and aerosol samples in the field. The rapid 15-minute time frame between sample addition and result make this type of rapid diagnostic test suitable for first responders and law enforcement officers, especially when dealing with suspicious samples and, possibly, environmental samples. Highly suspicious samples should be tested by other methods in a reference laboratory. It is recommended that follow-up laboratory testing be performed after lateral flow result is obtained for an appropriate public health response.