Validation of the OptiSafe™ Eye Irritation Test

Abstract

Purpose:

OptiSafe is an in chemico test method that identifies potential eye irritants based on macromolecular damage following test chemical exposure. The OptiSafe protocol includes a pre-screen assessment that identifies test chemicals that are outside the applicability domain of the test method and thus determines the optimal procedure. We assessed the usefulness and limitations of the OptiSafe test method for identifying chemicals not requiring classification for ocular irritation (i.e., bottom-up testing strategy).

Materials and Methods:

Seventeen chemicals were selected by the lead laboratory and tested as an independent study. Ninety-five unique coded chemicals were selected by a validation management team to assess the intra- and inter-laboratory reproducibility and accuracy of OptiSafe in a multi-laboratory, three-phased validation study. Three laboratories (lead laboratory and two naive laboratories) evaluated 35 chemicals, with the remaining 60 chemicals evaluated by the lead laboratory only. Test method performance was assessed by comparing classifications based on OptiSafe results to classifications based on available retrospective in vivo data, using both the EPA and GHS eye irritation hazard classification systems. No prospective in vivo testing was conducted.

Results:

Phase I testing of five chemicals showed that the method could be transferred to naive laboratories; within-lab reproducibility ranged from 93% to 100% for both classification systems. Thirty coded chemicals were evaluated in Phase II of the validation study to demonstrate both intra- and inter-laboratory reproducibility. Intralaboratory reproducibility for both EPA and GHS classification systems for Phase II of the validation study ranged from 93% to 99%, while interlaboratory reproducibility was 91% for both systems. Test method accuracy for the EPA and GHS classification systems based on results from individual laboratories ranged from 82% to 88% and from 78% to 88%, respectively, among the three laboratories; false negative rates ranged from 0% to 7% (EPA) and 0% to 15% (GHS). When results across all three laboratories were combined based on the majority classification, test method accuracy and false negative rates were 89% and 0%, respectively, for both classification systems, while false positive rates were 25% and 23% for the EPA and GHS classification systems, respectively. Validation study Phase III evaluation of an additional 60 chemicals by the lead laboratory provided a comprehensive assessment of test method accuracy and defined the applicability domain of the method. Based on chemicals tested in Phases II and III by the lead laboratory, test method accuracy was 83% and 79% for the EPA and GHS classification systems, respectively; false negative rates were 4% (EPA) and 0% (GHS); and false positive rates were 40% (EPA) and 42% (GHS). Potential causes of false positives in certain chemical (e.g., ethers and alcohols) or hazard classes are being further investigated.

Conclusion:

The OptiSafe test method is useful for identifying non-surfactant substances not requiring classification for ocular irritancy. OptiSafe represents a new tool for the in vitro assessment of ocular toxicity in a tiered-testing strategy where chemicals can be initially tested and identified as not requiring hazard classification.

Introduction

For more than 75 years, the in vivo Draize rabbit eye test has been used to assess the irritation potential of chemicals that may come into contact with the eye [1]. Previous studies have suggested that the responses observed in animal studies are not always relevant to the responses observed in humans [2]. Additionally, animal welfare concerns and implementation of international regulations banning animal testing of chemicals, cosmetic formulations and ingredients have led to an increase in the development and evaluation of methods that may reduce or replace animal testing [3, 4].

Several in vitro and ex vivo methods have been validated in the last 20 years for the identification of severe eye irritants and corrosives, and chemicals identified as “not classified”. Some of these have been adopted as Organisation for Economic Cooperation and Development (OECD) test guidelines, which can assist in acceptance of data across several countries and reduce repeated testing [5, 6, 7, 8, 9, 10]. Despite these advances, there is currently no single alternative eye irritation test that is accepted as a complete replacement for the Draize eye test.

OptiSafe is an in chemico method that uses a set of multiplexed biochemical tests to assess eye irritation potential. OptiSafe is provided as a kit that includes all the reagents and most of the consumables required to perform the tests. When stored as directed, the kits have a shelf life of one year. The multiplex design allows the identification of chemicals within 24 hours.

To conduct the test, the test chemical is applied to macromolecules, and spectrophotometry is used to assess its effects. The optical density values from a spectrophotometer provide estimates of the chemical’s potential to cause eye injury by several different mechanisms, including: (1) denaturation of water-insoluble polymers that model the phospholipid bilayer of corneal epithelial and conjunctival cells; (2) direct denaturation of macromolecules that model ordered collagen present in the corneal stroma; (3) indirect denaturation via osmotic effects; (4) excessive oxidation and reactivity that could damage epithelium, stroma, conjunctiva, and iris tissues; and (5) extreme buffering that could damage epithelium, stroma, conjunctiva, and iris tissues [11]. OptiSafe includes a mandatory pre-screening step. This step ensures that the optimal test method procedure is used to evaluate the irritancy and corrosivity potential of the test chemical.

Decision criteria focused on the Globally Harmonized System (GHS) and U.S. Environmental Protection Agency (EPA) hazard classification and labelling systems (Table 1), used to convey the toxicity and irritation potential of a substance, have been developed for OptiSafe [12, 13] [Table 1 near here]. Differences between the two systems in the criteria for a positive response may cause a substance to be classified differently (e.g., a chemical may be classified as EPA Category III irritant, GHS Not Classified (NC) due to the observed response). The OptiSafe test method identifies chemicals that do not require classification in the GHS or EPA classification systems (GHS NC and EPA Category IV, respectively).

Table 1.

EPA and GHS Ocular Irritation Classification Systems

EPA Classification			GHS Classification
Category	Positive Response	Classification	Category	Positive Response	Classification
I	● Corneal opacity or iritis ≥1 ● Conjunctival redness or chemosis ≥2 in a single animal at any observed time point up to 21 days after substance administration	Corrosive (irreversible destruction of ocular tissue), or corneal involvement or irritation lasting for more than 21 days after administration of substance	1	At least 2 animals with mean response (over Days 1, 2, and 3) of ● Comeal opacity ≥3 ● Iritis ≥1.5 OR At least 1 animal with a score >0 on observation day 21	Effects on the cornea, iris, or conjunctiva that are not expected to reverse or do not fully reverse within 21 days
II		Comeal involvement or irritation clearing in 8 to 21 days after administration of substance	2A	At least 2 animals with mean response (over Days 1, 2, and 3) of ● Corneal opacity or iritis ≥1 ● Conjunctival redness or opacity ≥1	Effects on the cornea, iris, or conjunctiva that fully reverse within 21 days
III		Corneal involvement or irritation clearing in ≤7 days after administration of substance	2B		Effects on the cornea, iris, or conjunctiva that fully reverse within 7 days
IV		Irritation clearing in <24 hours after administration of substance	NC		No effects are produced, or minimal effects observed that do not lead to classification

Abbreviations: NC = not classified.

The National Toxicology Program Interagency Center for the Evaluation of Alternative Toxicological Methods (NICEATM) reviewed an initial study conducted by the OptiSafe test method developer and producer, Lebrun Labs, LLC, and concluded that the performance of OptiSafe compared similarly to other non-animal ocular toxicity testing methods (e.g., bovine corneal opacity and permeability (BCOP) test [5], the isolated chicken eye (ICE) test [6]. EpiOcular [7]. Ocular Irritection [14], the short time exposure (STE) test [8], and Vitrigel EIT [10]). The method’s portability, shelf-life, and ease of use, as well as its ability to assess multiple mechanisms of ocular irritation, suggest that it could be a useful addition to currently available non-animal methods for this purpose.

To extend and expand the initial evaluation, NICEATM coordinated a three-laboratory validation study of OptiSafe to fully assess its transferability, reproducibility, usefulness, and limitations for eye irritation hazard classification, as described in OECD Guidance Document 34 [15]. The purpose of this validation study was to evaluate whether the OptiSafe test method could be considered as a component of a tiered-testing strategy to replace or reduce the use of animals for ocular irritation testing. Specifically, the method was assessed for its usefulness as an initial step in a bottom-up testing strategy approach (i.e., identification of ocular non-irritants vs. ocular irritants/corrosives) [16].

Materials and Methods

Background

OptiSafe uses a set of multiplexed biochemical tests that are provided as a single kit to the user, to assess the eye irritation potential of a chemical. The test chemical is applied to proprietary macromolecules (e.g., whole cell extract and synthetic polymers), and spectrophotometry is used to assess its effects. The method is based on the theory that an ocular irritation response to chemical exposure is initiated by disruption of the molecular integrity of the ocular tissue, and that this disruption can be modelled in chemico. Three measurable target events are examined in OptiSafe: denaturation or fixation of water-soluble macromolecules, denaturation or fixation of water-insoluble macromolecules, and disruption of pH by strong buffering. OptiSafe evaluates 1) water-soluble and -insoluble materials, 2) controls for and includes pH effects (models excessive oxidation and reactivity, and extreme buffering mechanisms of eye irritation), and 3) models both fixation without denaturation and denaturation processes (models direct and indirect denaturation mechanisms of eye irritation).

OptiSafe Test Method Protocol

The OptiSafe test method consists of a pre-screening procedure followed by a main biochemical procedure (Figure 1). [Figure 1 near here]. The pre-screen ensures that the appropriate procedure is selected for the main biochemical test based on the physical and chemical properties of the test chemical. Predictions of EPA and GHS eye irritation hazard classifications are based on the calculated irritancy scores and the irritant/non-irritant thresholds determined in the main procedure(s) selected for testing. Additional information on the protocols and method can be reviewed in the procedures provided with the testing kit. The full protocol can be obtained upon request to info@lebrunlabs.com.

Figure 1.

OptiSafe Test Method Protocol Flow Chart

Pre-screening procedure

The first step of the pre-screen procedure identifies surfactant substances. Dilutions of the test chemical (1% and 10% in OptiSafe Blanking Buffer) are prepared at room temperature and mixed vigorously. A foaming check is performed within 5-10 minutes. A metric ruler is used to measure from the meniscus to the top of the foam. If the foam extends greater than 0.2 cm above meniscus, the substance is classified as a “surfactant” using this method. If the test chemical is identified as a surfactant, no further testing is conducted as these chemicals are outside the predefined applicability domain of the test method.

If foam is not present and the test chemical is a solid, a solubility check is performed by measuring the optical density at 400 nm of a 10% dilution. Solutions with an optical density below 0.350 are considered insoluble and are tested according to a specific procedure for insoluble chemicals.

The final step of the pre-screening procedure is the calculation of an H-buffering score, which is used to determine the appropriate main procedure(s) for the test chemical. This step applies to all non-surfactant substances (regardless of solubility). The H-buffering score is based on pH and is a measure of the buffering capacity of the unknown test chemical.

The results of the pre-screening procedure are used to determine which of the main procedures (Alpha, Delta, or Eta) should be used to assess chemical ocular irritancy potential (Figure 1).

Main procedure

The first main procedure, Procedure Alpha (α), models collagen denaturation due to direct or osmotic effects, as well as oxidative damage and excessive reactivity. Depending on the pre-screen solubility and H-buffering scores, one of the following α assays may be selected:

1. The Membrane Assay (MA) predicts the ocular response for non-surfactant substances within the MA applicability domain (little buffering capacity; H-buffering score <5.0).

2. The Eta Membrane Assay (HMA) predicts the ocular response for chemicals with a buffering score outside of the MA applicability domain (some buffering capacity; H-buffering score <100 and ≥5).

3. The Completely Insoluble Assay (Ci) predicts the ocular response of solids with little detectable solubility (i.e., optical density at 400 nm <0.350). The Ci assay is further differentiated based on the density of the chemical.

4. For intensely colored test chemicals, the Intensely Colored sub-protocol modification of the a procedure (conducted after the assay is completed; blank values >0.850) uses dilution or alternative wavelengths to overcome assay interference from pigmentation.

Procedure Delta (Δ) measures denaturation of a water-insoluble polymer and excessive reactivity, including reactive oxidation chemistry. This procedure models ocular damage to membranes and determines whether the test chemical is an oxidant or excessive reactant.

The final main procedure, Procedure Eta (H), is performed for chemicals with extreme buffering capacity (i.e., El-buffering score ≥100). This procedure determines whether the test chemical will cause a significant shift in ocular pH, since extremes in ocular pH are inconsistent with tissue function.

Independent Study

Prior to the commencement of the validation study, the lead laboratory initiated an independent study to ensure that newly updated test parameters related to changing the incubation temperature to 31°C allowed the assay to be conducted using incubators that the two naive laboratories already had in-house. Furthermore, the independent study confirmed that 1) parameters were appropriately set to achieve optimal performance; 2) the test kit manufacturing process did not introduce any unexpected results; and 3) the test kit reagents were stable. The lead laboratory selected 17 chemicals fortesting (see Supplemental Tables). Chemical selection was based on test chemical performance in previous validation studies for other in vitro eye irritation methods and availability of in vivo reference data. Additionally, the set of chemicals was selected to represent a balanced distribution of EPA and GHS ocular irritant classifications that correspond to in vivo responses ranging from non-irritant to severe irritant/corrosive. As time permitted, repeat tests were run for some chemicals to generate additional data points.

Interlaboratory Validation Study

Study management

A validation management team (VMT) was assembled to oversee the conduct of the validation study. The VMT was composed of members of the Interagency Coordinating Committee on the Validation of Alternative Methods’ Ocular and Dermal Irritation Workgroup, a committee of representatives with expertise in eye and/or skin irritation testing within the U.S. Federal government. To ensure that all decisions remained independent of the individuals performing the testing, no persons affiliated with the participating laboratories were included on the VMT.

Prior to commencement of the testing phases of the validation study, the VMT reviewed and approved the study design and work plan, test method protocol, study timeline, and deliverables. The VMT selected the chemicals to be tested in the validation study, which were coded and shipped to the participating laboratories by the National Institute of Environmental Health Sciences’ Chemistry and Absorption, Distribution, Metabolism and Excretion Resources Group. The VMT agreed the false negative rate could be no greater than 10% and there could be no false negative results for EPA Category I or GHS Category 1 chemicals. These are generally agreed upon criteria for a bottom-up test (i.e., identifying chemicals not requiring classification for ocular irritation).

At the conclusion of each test phase, the lead laboratory prepared final results and study reports and presented these to the VMT. The VMT reviewed the results, approved the data evaluations, and accepted the final study outcomes.

Participating laboratories

The developer of the OptiSafe test method, Lebrun Labs, LLC (Anaheim, California), was the lead laboratory for the study. This laboratory prepared the protocols and provided equipment (e.g., spectrophotometers) and OptiSafe kits to the naive laboratories. The naive laboratories, Cyprotex US, LLC (Kalamazoo, Michigan) and MB Research Laboratories (Spinnerstown, Pennsylvania), were experienced with alternative toxicity testing and have participated in previous interlaboratory validation studies but did not have specific experience with the OptiSafe test method. A representative from the lead laboratory provided on-site, in-person training at both naive laboratories prior to initiating the validation study. The training involved demonstration of the pre-screen method, excluding the surfactant check, using chemicals that the labs already had in stock. Examples of complete assays using a selection of chemicals to demonstrate each sub-protocol were also provided. At the conclusion of the training, it was determined by the trainer that both naive laboratories were capable of adequately conducting the assays.

Designated NICEATM staff members coordinated the validation study. The participating laboratories were permitted to communicate with one another without restriction throughout the training phase. However, once testing began, communications among the laboratories were transmitted in concert with NICEATM staff. During testing, each laboratory provided raw data sheets and test result summaries to NICEATM weekly.

Test chemical selection

The VMT selected chemicals for testing that represented a wide range of chemical classes and physicochemical properties including a percentage of chemicals that have been included in previous validation studies. Availability of EPA and/or GHS ocular irritancy classification and associated historical in vivo data was a criterion for chemical selection. Selected chemicals represented a range of in vivo ocular responses from non-irritant to severe irritant/corrosive, thereby representing a balanced distribution of both the GHS and EPA classification categories [17]. Kojima, personal communication; unreferenced, Barroso, personal communication; unreferenced]. Ultimately, a total of 95 unique coded chemicals (Table 2) [Table 2 near here] were evaluated in three runs over the duration of this study (35 chemicals evaluated by all three laboratories and an additional 60 chemicals evaluated by the lead laboratory only) to assess intra- and inter-laboratory reproducibility and accuracy compared to the in vivo classification.

Table 2.

Chemicals tested in OptiSafe interlaboratory validation study

No.	Phase	Chemical name	CASRN	In vivo EPA classification	In vivo GHS classification	Physical state	Purity (%)	Supplier
1	I	1,9-Decadiene	1647-16-1	Category IV	Not Classified	liquid	98.2	TCI America
2	I	2,4-Pentanediol	625-69-4	Category IV	Not Classified	liquid	99.9	Sigma-Aldrich
3	I	2-Ethyl-1-hexanol	104-76-7	Category II	Category 2A	liquid	99.8	Sigma-Aldrich
4	I	4-tert-Butylcatechol	98-29-3	Category I	Category 1	solid	99.3	Sigma-Aldrich
5	I	Isobutyraldehyde	78-84-2	Category III	Category 2B	liquid	99.6	Sigma-Aldrich
6	II	1,3-Di-iso-propylbenzene	99-62-7	Category IV	Not Classified	liquid	97.6	Sigma-Aldrich
7	II	1,6-Dibromohexane	629-03-8	Category IV	Not Classified	liquid	98.7	Sigma-Aldrich
8	II	1,9-Decadiene	1647-16-1	Category IV	Not Classified	liquid	98.2	TCI America
9	II	1-Bromo-4-chlorobutane	6940-78-9	Category IV	Not Classified	liquid	99.9	Sigma-Aldrich
10	II	1-Propoxy-2-propanol (Propasol solvent P)	1569-01-3	Category II	Category 2A	liquid	99.8	Sigma-Aldrich
11	II	2,2-Dimethyl-3-pentanol	3970-62-5	Category III	Not Classified	liquid	99.6	Sigma-Aldrich
12	II	2,4-Pentanediol	625-69-4	Category IV	Not Classified	liquid	99.9	Sigma-Aldrich
13	II	2-Methyl-1-pentanol	105-30-6	Category III	Category 2B	liquid	99.2	Sigma-Aldrich
14	II	3,3-Dithiodipropionic acid	1119-62-6	Category II	Category 2B	solid	99.9	Sigma-Aldrich
15	II	3,4-Dichlorophenyl isocyanate	102-36-3	Category I	Category 1	solid	99.9	TCI America
16	II	4,4-Methylene bis-(2,6-ditert-butyl)phenol	118-82-1	Category IV	Not Classified	solid	98.9	Sigma-Aldrich
17	II	Ammonium nitrate	6484-52-2	Category III	Category 2A	solid	99.6	Sigma-Aldrich
18	II	Benzalkonium chloride (5% solution in DMSO)	63449-41-2	Category I	Category 1	liquid	91.3	Sigma-Aldrich
19	II	Camphene	79-92-5	Category III	Category 2B	solid	98.6	Sigma-Aldrich
20	II	Cetyl pyridinium bromide (0.1% solution in DMSO)	140-72-7	Category III	Not Classified	liquid	93.0	Spectrum Chemical Mfg. Group
21	II	Dibenzyl phosphate	1623-08-1	Category II	Category 2A	solid	98.8	Sigma-Aldrich
22	II	Di-iso-butyl ketone	108-83-8	Category IV	Not Classified	liquid	99.2	Sigma-Aldrich
23	II	Glycerol	56-81-5	Category IV	Not Classified	liquid	100	Sigma-Aldrich
24	II	Isobutanol	78-83-1	Category II	Category 2A	liquid	99.9	Sigma-Aldrich
25	II	Isobutyraldehyde	78-84-2	Category III	Category 2B	liquid	99.6	Sigma-Aldrich
26	II	iso-Octyl acrylate	29590-429	Category IV	Not Classified	liquid	NP	Sigma-Aldrich
27	II	Methyl cyanoacetate	105-34-0	Category II	Category 2A	liquid	99.6	Sigma-Aldrich
28	II	n-Butanol	71-36-3	Category II	Category 1/Category 2A	liquid	99.9	Sigma-Aldrich
29	II	n-Hexyl bromide	111-25-1	Category IV	Not Classified	liquid	99.0	Sigma-Aldrich
30	II	n-Octyl bromide	111-83-1	Category IV	Not Classified	liquid	99.6	Sigma-Aldrich
31	II	Potassium tetrafluoroborate	14075-537	Category IV	Not Classified	solid	~99.4	Sigma-Aldrich
32	II	Propylene glycol	57-55-6	Category IV	Not Classified	liquid	99.5	Sigma-Aldrich
33	II	p-Tert-butylphenol	98-54-4	Category I	Category 1	solid	99.8	Sigma-Aldrich
34	II	Sodium chloroacetate	3926-62-3	Category III	Category 2B	solid	99.4	Sigma-Aldrich
35	II	Sodium lauryl sulfate (3% solution in DMSO)	151-21-3	Category III	Not Classified	liquid	99.7	Sigma-Aldrich
36	III	1,2,6-Hexanetriol	106-69-4	Category IV	Not Classified	liquid	97.6	Sigma-Aldrich
37	III	1,4-Dibromobutane	110-52-1	Category III	Not Classified	liquid	99.6	Sigma-Aldrich
38	III	1,5-Hexadiene	592-42-7	Category III	Not Classified	liquid	99.3	Sigma-Aldrich
39	III	2-(2-Ethoxy ethoxy) ethanol	111-90-0	Category III	Not Classified	liquid	99.82	Sigma-Aldrich
40	III	2,4-Pentanedione	123-54-6	Category III	Not Classified	liquid	99.8	Sigma-Aldrich
41	III	2,5 -Dimethylhexanediol	110-03-2	Category I	Category 1	solid	99.9	Sigma-Aldrich
42	III	2,6-Dichlorobenzoyl chloride	4659-45-4	Category II	Category 2A	liquid	99.8	TCI America
43	III	2-Amino-3-pyridinol	16867-031	Category III	Category 2A	solid	99.7	Sigma-Aldrich
44	III	2-Ethoxy ethyl methacrylate	2370-63-0	Category IV	Not Classified	liquid	99.9	Sigma-Aldrich
45	III	2-Ethylhexyl thioglycolate	7659-86-1	Category IV	Not Classified	liquid	99.6	Sigma-Aldrich
46	III	3-Chloropropionitrile	542-76-7	Category III	Category 2B	liquid	99.7	Sigma-Aldrich
47	III	3-Methoxy-1,2-propanediol	623-39-2	Category IV	Not Classified	liquid	98.3	TCI America
48	III	3-Phenoxy benzyl alcohol	13826-35-2	Category III	Not Classified	liquid	97.7	Sigma-Aldrich
49	III	4-(1,1-Dimethylethyl)-α-methyl-benzenepropanal (Protectol PP)	80-54-6	Category I	Category 1	liquid	99.5	TCI America
50	III	6-Methyl purine	2004-03-7	Category I	Category 2B	solid	98.98	Chem-Impex International, Inc. via Fisher Scientific
51	III	Acetone	67-64-1	Category II	Category 2A	liquid	99.75	Sigma-Aldrich
52	III	Allyl alcohol	107-18-6	Category III	Category 2A	liquid	99.4	Sigma-Aldrich
53	III	Benzalkonium chloride (1% solution in DMSO)	63449-41-2	Category I	Category 1	liquid	91.3	Sigma-Aldrich
54	III	Butanedioic acid, sulfo-,1,4-bis(2-ethylhexyl) ester, sodium salt	577-11-7	Category I	Category 1	solid	98.8	Sigma-Aldrich
55	III	Cyclohexanol	108-93-0	Category I	Category 1	liquid	99.3	Sigma-Aldrich
56	III	Cyclopentanol	96-41-3	Category II	Category 2A	liquid	99.7	Sigma-Aldrich
57	III	Cyclopentasiloxane	541-02-6	---	Not Classified	liquid	97.6	Sigma-Aldrich
58	III	Diethylaminopropionitrile	5351-04-2	Category II	Category 1	liquid	99.5	Chem Service Inc.
59	III	Dodecane	112-40-3	Category III	Not Classified	liquid	99.6	Sigma-Aldrich
60	III	Ethyl acetate	141-78-6	Category III	Not Classified	liquid	99.90	Sigma-Aldrich
61	III	Ethyl-2-methyl acetoacetate	609-14-3	Category III	Category 2B	liquid	96.3	TCI America
62	III	Ethylene glycol diethyl ether	629-14-1	Category IV	Not Classified	liquid	99.6	Sigma-Aldrich
63	III	gamma-Butyrolactone	96-48-0	Category II	Category 2A	liquid	99.9	Sigma-Aldrich
64	III	Hexamethyldisiloxane	107-46-0	Category IV	Not Classified	liquid	99.5	Sigma-Aldrich
65	III	Hexane	110-54-3	Category IV	Not Classified	liquid	99.3	Sigma-Aldrich
66	III	Hexyl cinnamic aldehyde	101-86-0	Category IV	Not Classified	liquid	99.2 with 0.122% BHT as stabilizer	Sigma-Aldrich
67	III	Imidazole	288-32-4	Category I	Category 1	solid	100	Sigma-Aldrich
68	III	iso-Octylthioglycolate	25103-09-7	Category IV	Not Classified	liquid	94.9	TCI America
69	III	Isopropanol	67-63-0	Category III	Category 2A	liquid	99.97	Sigma-Aldrich
70	III	Isopropyl acetoacetate	542-08-5	Category III	Category 2B	liquid	98.6	TCI America
71	III	iso-Propyl bromide	75-26-3	Category IV	Not Classified	liquid	99.7	Sigma-Aldrich
72	III	Lactic Acid	50-21-5	Category I	Category 1	liquid	88.6	Spectrum Chemical Mfg. Corp.
73	III	Lauric acid	143-07-7	Category I	Category 1	solid	99.6	Sigma-Aldrich
74	III	Maneb (solid)	12427-38-2	Category III	Category 2B	solid	>90	Toronto Research Chemicals, Inc.
75	III	Methyl acetate	79-20-9	Category II	Category 2A	liquid	99.96	Sigma-Aldrich
76	III	Methylthioglycolate	2365-48-2	Category II	Category 1	liquid	99.4	Sigma-Aldrich
77	III	N,N-Diethyl-m-toluamide	134-62-3	Category III	Category 2B	liquid	98.7	Sigma-Aldrich
78	III	N,N-Dimethylguanidine sulfate	598-65-2	Category III	Not Classified	solid	99.8	TCI America
79	III	n-Butanal	123-72-8	Category III	Category 2B	liquid	99.82	Sigma-Aldrich
80	III	n-Hexanol	111-27-3	Category II	Category 2A	liquid	98.6	Sigma-Aldrich
81	III	n-Octanol	111-87-5	Category II	Category 2A	liquid	99.2	Sigma-Aldrich
82	III	p-Methyl thiobenzaldehyde	3446-89-7	Category IV	Not Classified	liquid	98.9	TCI America
83	III	Polyoxyethylene hydrogenated castor oil (60E.O.)	61788-85-0	Category IV	Not Classified	solid	NP	Spectrum Chemical Mfg. Corp.
84	III	Sodium deoxycholate (10% solution in DMSO)	302-95-4	Category II	Category 2A	liquid	~95	Sigma-Aldrich
85	III	Sodium lauroyl sarcosinate (10% solution in water)	137-16-6	Category III	Category 2A	liquid	98	Sigma-Aldrich
86	III	Sodium perborate tetrahydrate	10486-00-7	Category I	Category 1	solid	97.5	TCI America
87	III	Styrene	100-42-5	Category III	Not Classified	liquid	99.9	Sigma-Aldrich
88	III	Triclocarban	101-20-2	Category IV	Not Classified	solid	98.9	TCI America
89	III	Triethylene glycol	112-27-6	Category IV	Not Classified	liquid	99.8	Sigma-Aldrich
90	III	Triphenyl phosphite	101-02-0	Category IV	Not Classified	liquid	96.7	Sigma-Aldrich
91	III	Triton X-100 (1% solution in DMSO)	9002-93-1	Category II	Not Classified	liquid	NP	Sigma-Aldrich
92	III	Triton X-100 (5% solution in DMSO)	9002-93-1	Category I	Category 2A	liquid	NP	Sigma-Aldrich
93	III	Tween 20	9005-64-5	Category III	Not Classified	liquid	NP	Sigma-Aldrich
94	III	Tween 80	9005-65-6	Category IV	Not Classified	liquid	NP	Sigma-Aldrich
95	III	Xylene	1330-20-7	Category II	Not Classified	liquid	99.0	Sigma-Aldrich

Sources: [17, Kojima, personal communication; unreferenced, Barroso, personal communication; unreferenced]

Abbreviations: DMSO = dimethylsulfoxide; E.O. = ethylene oxide; N/A = not applicable; NP = not provided.

The National Institute of Environmental Health Sciences Chemistry and Absorption, Distribution, Metabolism and Excretion Resources Group supplied all chemicals to each participating laboratory and provided the VMT with all descriptive information for each chemical (e.g., purity, supplier, Chemical Abstracts Service Registry Number® [CASRN], etc.). Coded test chemicals were packaged and shipped according to established regulatory procedures. Participating laboratory personnel were instructed to handle all test chemicals as very hazardous and potentially carcinogenic. Health and safety information was provided to each facility in a sealed package, which provided chemical hazard information and emergency instructions.

Data collection

The participating laboratories completed three independent OptiSafe runs for each chemical tested. Assay results, calculations, information about the performance of the positive and negative controls, and OptiSafe predictions of EPA and GHS irritation categories were recorded for each run on data collection worksheets specific to the main procedure used. The laboratories were instructed to send copies of the completed OptiSafe data collection and score calculation worksheets to NICEATM representatives weekly during testing. Each participating laboratory also provided NICEATM with regular updates of a test log spreadsheet. The spreadsheets contained the irritancy scores and corresponding predictions extracted from the worksheets and indicated whether each experiment passed or failed method acceptance criteria.

Data quality assurance

The participating laboratories conducted the validation study consistent with the principles of OECD Good Laboratory Practices [18]. Quality control checks were performed for each run in all laboratories. Following completion of each testing phase, quality assurance personnel from the lead laboratory audited the data collection worksheets from all participating laboratories.

Data analyses

Test chemicals were classified according to both the EPA and GHS classification systems: corrosives or irritants (EPA Category I, II, or III; or GHS Category 1 or 2) or chemicals not requiring classification and labelling (EPA Category IV or GHS NC). Chemicals that were not within the applicability domain of the method also were noted. Conditions for assigning an outside-of-applicability domain result are defined in the OptiSafe standard operating procedures. Laboratories noted when irritancy scores were within 10% of the irritant/non-irritant threshold values (Figure 1).

Intralaboratory and interlaboratory reproducibility and concordance were determined in each testing phase. Intralaboratory reproducibility was based on the ability of individual testing laboratories to obtain the same EPA or GHS classification from runs conducted independently. Test run classifications for each chemical were evaluated by each testing laboratory. The percentage of results with the same classification outcome was determined within and among testing laboratories.

A single classification for each chemical in each testing laboratory was determined based on the majority classification obtained from the individual test method runs. This classification was utilized to assess intralaboratory concordance. An overall regulatory classification for each test chemical was determined from the testing laboratories’ classifications based on the majority classification obtained by the three testing laboratories. This classification was utilized to assess interlaboratory concordance and the overall test method accuracy relative to classifications based on the existing in vivo data.

Results

Independent Study

Seventeen chemicals were evaluated by the lead laboratory in an independent study (see Supplemental Tables). A single run per chemical was conducted and the result was considered the final classification. However, runs were repeated for three chemicals (di-n-propyl disulphide (No. IND-6), dioctyl ether (No. IND-7), and tetraethylene glycol diacrylate (No. IND-16)) due to concern that a protocol deviation had affected the result. The repeat runs confirmed the classification predictions obtained in the initial runs for all three chemicals. Test method accuracy in the pilot study was 94% (16/17) for both the EPA and GHS classification systems. Only 1,5-dibromopentane (No. IND-2) was misclassified based on the EPA classification system, giving a false positive rate of 0% (0/4) and a false negative rate of 8% (1/13). Similarly, only 2,4,5,6-tetraaminopyrimidine sulfate salt (No. IND-3) was overpredicted based on the GHS classification system, giving a false positive rate of 17% (1/6) and a false negative rate of 0% (0/11).

Interlaboratory Validation Study

Chemical distribution

The 35 chemicals evaluated by the lead laboratory and each naive laboratory in Phases I and II are listed in Table 2. The five chemicals tested in Phase I encompassed a limited range of irritancy categories to ensure that each laboratory could accurately and reliably conduct the assay protocols. The 30 chemicals evaluated in Phase II encompassed a broader range of irritancy categories based on the EPA and GHS eye irritation classification systems and represented a range of physical states and organic functional groups.

Of the 30 Phase II chemicals, three (benzalkonium chloride [5%] (No. 18), sodium lauryl sulfate [3%] (No. 35), and cetyl pyridinium bromide [0.1%] (No. 20)) did not qualify for further analysis based on the results of the pre-screen protocol. These chemicals were identified during the pre-screen procedure as surfactants and therefore outside the applicability domain of the OptiSafe procedure (Figure 1).

In addition to the 30 chemicals evaluated in Phase II, the lead laboratory evaluated 60 chemicals in Phase III, representing a range of ocular irritancy categories, physical states, and organic functional groups. Eight of the chemicals evaluated in Phase III were identified as surfactants in the pre-screen protocol (Figure 1) and were not tested further. The excluded chemicals were benzalkonium chloride (1%) (No. 53), sodium deoxycholate (10%) (No. 84), sodium lauroyl sarcosinate (10%) (No. 85), Triton X-100 (1% and 5%) (Nos. 91 and 92, respectively), polyoxyethylene hydrogenated castor oil (60E.O.) (No. 83), Tween 20 (No. 93), and Tween 80 (No. 94).

Phase I

Transferability of the test method to naive testing labs was assessed using the five Phase I chemicals (Table 2). Negative and positive quality controls (QC1 [20% glycerol] and QC2 [98% ethanol], respectively) and standard solutions were run concurrently with each test run to identify any potential errors.

All testing laboratories reported results for the quality control chemicals and calculated scores that were within historical ranges for the MAα protocol. On the other hand, QC1 results for the HMAα protocol obtained by naive laboratory 2 were outside of historical ranges. Investigation into causes for the discrepant results indicated that protocol deviations (e.g., incubation temperatures and pH values of the quality control chemicals outside the recommended range) led to the out-of-range QC values.

Two standard solutions are included with the OptiSafe test kit for system calibration and data analyses and these are used for both the MAα and HMAα protocols. While all standards assessed with the MAα protocol were within historical ranges for all testing laboratories, some runs using the HMAα protocol were outside of historical ranges (data not shown). These deviations were caused by low pH levels (e.g., pH <5.0) of the standard solutions.

The lead laboratory proposed a protocol adjustment in which the test samples and quality controls would be placed in separate incubation containers to ensure that the pH was maintained at the required level. A second issue was noted when the sample incubation boxes were re-used for multiple assays. Concerns were raised that evaporation of the test chemical and reagents may occur if the boxes are not sealed appropriately. As a result, the boxes were used only once for future phases of the evaluation.

Based on the pre-screen results, all three testing laboratories tested three chemicals in the MAa method and one chemical in the HMAα method. Calculated scores for individual runs for these chemicals were similar within testing laboratories (data not shown). One chemical, isobutyraldehyde (No. 5), was tested using the MAa method by one testing laboratory and using the HMAa method by the other two testing laboratories. With three laboratories testing five chemicals (three runs each), a total of 15 runs per laboratory could be evaluated as an indicator of intralaboratory reproducibility to demonstrate successful protocol transfer. Phase I results in the lead laboratory indicated that all three runs for each of the five chemicals agreed with both the EPA and GHS classification systems (see Supplemental Tables). For the two naive laboratories, no more than one run for a single chemical disagreed with either classification system. Collectively, these results indicated that the test method protocol was successfully transferred, and the VMT agreed proceeding to Phase II was appropriate.

An investigation into why two different methods were identified for use during the pre-screen of isobutyraldehyde (No. 5) was conducted. The incubation time for the buffering step during pre-screen ranged from 3-8 minutes. It was suggested that the variable time length led to different protocols being identified. Therefore, it was proposed the protocol be changed to specify a 5-minute incubation time. Additionally, temperature changes were variable in the assay. It was proposed that the temperature be more precisely specified. These changes were implemented for Phase II testing.

Phase II

In the second phase of the validation effort, the three testing laboratories were provided 30 coded chemicals selected by the VMT. Three of the coded chemicals were surfactants. These chemicals were included specifically to evaluate the ability of the naive testing laboratories to accurately use the pre-screening procedure to identify chemicals outside the OptiSafe applicability domain. After all the results were provided to NICEATM for analysis, it was noted that only the lead testing laboratory correctly identified the three surfactants as outside the OptiSafe applicability domain. Investigation by NICEATM determined that the naive laboratories had not been fully trained on the surfactant pre-screen procedure. Upon receiving this training, the naive laboratories demonstrated that they would have successfully identified the out-of-domain chemicals. The data and irritancy classifications obtained for these chemicals by all three testing laboratories were excluded from the test method reproducibility and accuracy analyses.

Intralaboratory reproducibility of the remaining coded test chemicals was similar to the intralaboratory reproducibility observed in Phase I. The intralaboratory reproducibility ranged from 93% (74/80) to 99% (77/78) for the EPA and GHS classification systems (see Supplemental Tables).

Test method accuracy was evaluated for each laboratory based on classification results obtained for the EPA and GHS systems (Table 3). [Table 3 near here] Although 27 chemicals were tested in Phase II, the accuracy analysis for the lead laboratory was based on data for 25 chemicals. Lead lab results for 3,3-dithiodipropionic acid (No. 14) were excluded because the chemical was incompatible with the available methods and a novel protocol was developed using a combination of the H and Ci protocols. Since this combination protocol was not included in Phase I, the results for 3,3-dithiodipropionic acid (No. 14) from the lead laboratory were excluded. Results for 4,4-methylene bis-(2,6-ditert-butyl)phenol) (No. 16) from the lead laboratory were also excluded because this chemical produced a response that exceeded the upper range of the spectrophotometer, using the test method protocol, and therefore was outside of the applicability domain of the test method.

Table 3.

Phase II OptiSafe test method accuracy within each laboratory: EPA and GHS classification systems

		Lead Laboratory – OptiSafe classification			naïve Laboratory 1 – OptiSafe classification			naïve Laboratory 2 – OptiSafe classification
		+	−	Total	+	−	Total	+	−	Total
EPA in vivo classification	+	13	0	13	14	0	14	13	1	14
	−	3	9	12	5	8	13	3	10	13
	Total	16	9	25a	19	8	27b	16	11	27b
		Accuracy = 88%			Accuracy = 82%			Accuracy = 85%
		Negative predictivity = 100%			Negative predictivity = 100%			Negative predictivity = 91%
		Positive predictivity = 81%			Positive predictivity = 74%			Positive predictivity = 81%
		Prevalence = 52%			Prevalence = 52%			Prevalence = 52%
		Sensitivity = 100%			Sensitivity = 100%			Sensitivity = 93%
		Specificity = 75%			Specificity = 62%			Specificity = 77%
		False Positive Rate = 25%			False Positive Rate = 39%			False Positive Rate = 23%
		False Negative Rate = 0%			False Negative Rate = 0%			False Negative Rate = 7%
		Lead Laboratory – OptiSafe classification			naïve Laboratory 1 – OptiSafe classification			naïve Laboratory 2 – OptiSafe classification
		+	−	Total	+	−	Total	+	-	Total
GHS In vivo classification	+	12	0	12	13	0	13	11	2	13
	−	3	10	13	5	9	14	4	10	14
	Total	15	10	25a	18	9	27b	15	12	27b
		Accuracy = 88%			Accuracy = 82%			Accuracy = 78%
		Negative predictivity = 100%			Negative predictivity = 100%			Negative predictivity = 83%
		Positive predictivity = 80%			Positive predictivity = 72%			Positive predictivity = 73%
		Prevalence = 48%			Prevalence = 48%			Prevalence = 48%
		Sensitivity = 100%			Sensitivity = 100%			Sensitivity = 85%
		Specificity = 77%			Specificity = 64%			Specificity = 71%
		False Positive Rate = 23%			False Positive Rate = 36%			False Positive Rate = 29%
		False Negative Rate = 0%			False Negative Rate = 0%			False Negative Rate = 15%

+ = chemical identified as an EPA or GHS ocular corrosive or irritant (EPA Categories I, II or III; GHS Categories 1 or 2); − = chemical not identified as an EPA or GHS ocular corrosive or irritant (EPA Category IV; GHS Category NC)

^aCalculations for 25 chemicals. Results were excluded for 4,4-Methylene bis-(2,6-di-tert-butyl)phenol because it produced a response exceeding the upper range of the spectrophotometer and 3,3-dithiodipropionic acid because it was evaluated with a protocol not included in the validation study design. Benzalkonium chloride (5%), sodium lauryl sulfate (3%), and cetyl pyridinium bromide (0.1%) were not tested in the main protocol because they were identified as foaming agents in the pre-screen.

^bCalculations for 27 chemicals. Benzalkonium chloride (5%), sodium lauryl sulfate (3%), and cetyl pyridinium bromide (0.1%) were not tested in the main protocol because they were identified as foaming agents in the pre-screen.

The intralaboratory accuracy rates for the chemicals included in the analysis ranged from 78% (21/27) to 88% (22/25) for the GHS classification system and 82% (22/27) to 88% (22/25) for the EPA classification system. The false negative rate compared to in vivo classifications was 0% for two of the testing laboratories and 7% (1/14) to 15% (2/13) for the third laboratory, which underpredicted both the EPA and GHS classification for camphene (No. 19) and the GHS classification for ammonium nitrate (No. 17). The false positive rate ranged from 23% (3/13) to 39% (5/13) when classifications for EPA or GHS systems were compared to classifications based on in vivo results. All three testing laboratories misclassified both the EPA and GHS classifications of 1,3-di-iso-propylbenzene (No. 6) and 1,9-decadiene (No. 8), and the GHS classification of 2,2-dimethyl-3-pentanol (No. 11).

When results across laboratories were combined into a single call for 26 Phase II chemicals for which there were calls in at least two of the three testing laboratories, accuracy was 89% (23/26) for both the EPA and GHS classification systems (Table 4). [Table 4 near here]. The false negative rate was 0% for both classification systems. The false positive rates were 25% (3/12) and 23% (3/13) for the EPA and GHS classification systems, respectively. Two chemicals, 1,3-di-iso-propylbenzene (No. 6) and 1,9-decadiene (No. 8), were overpredicted in both classification schemes. Additionally, 2,4-pentanediol (No. 12) was overpredicted based on the EPA classification system while 2,2-dimethyl-3-pentanol (No. 11) was overpredicted based on the GHS classification system.

Table 4.

Phase II test method accuracy: results for all laboratories combined

	EPA	GHS
Accuracy	89% (23/26a)	89% (23/26 a)
Sensitivity	100% (14/14)	100% (13/13)
Specificity	75% (9/12)	77% (10/13)
False Positive	25% (3/12)	23% (3/13)
False Negative	0% (0/14)	0% (0/13)
Negative Predictivity	100% (9/9)	100% (10/10)
Positive Predictivity	82% (14/17)	81% (13/16)

^aFour chemicals were excluded from the analysis: three chemicals were identified as foaming agents in the pre-screen evaluation, and no overall call could be made for the fourth chemical.

Interlaboratory reproducibility, as assessed by agreement of in vitro hazard classification, was 91% (73/80) for both the EPA and GHS classification systems (see Supplemental Tables).. None of the testing laboratories produced the same classification for 4,4-methylene bis-(2,6-ditert-butyl)phenol (No. 16).

Phase III

During the third phase of the validation study, the lead laboratory tested an additional 60 coded chemicals to further evaluate the predictive capacity and applicability domain of the method. Eight of the 60 chemicals were identified as surfactants during pre-screen and thus were excluded from further testing. Five other chemicals were excluded from Phase III accuracy analyses because they were identified as non-surfactants that were outside of the applicability domain of the method, as defined in the OptiSafe standard operating procedures. Cyclopentasiloxane (No. 57), was excluded from only the EPA analyses because no in vivo reference classification based on retrospective animal studies could be located. A definitive in vitro classification for 1,5-hexadiene (No. 38) could not be determined because two runs of the chemical did not produce the same result (i.e., a majority classification was not possible since one run classified the chemical as an irritant, one run classified the chemical as not classified, and one run was identified as outside the applicability domain of the method). Therefore, the chemical was excluded from the GHS analyses. Thus, usable data were obtained from Phase III testing for 46 chemicals for either the EPA or GHS analyses. These results were then combined with Phase II results for the lead laboratory (25 chemicals tested) to assess the overall performance of the OptiSafe test method for the complete set of 71 tested chemicals.

The ability of OptiSafe to predict EPA or GHS ocular classifications was similar for both classification systems. Overall accuracy was 83% (59/71) and 79% (56/71) for the EPA and GHS classification systems, respectively. EPA and GHS false negative rates were 4% (2/46) and 0% (0/36), respectively. Overall false positive rates were 40% (10/25) and 42% (15/36) for EPA and GHS, respectively.

To investigate commonalities among false positive and false negative chemicals, structural features were identified and evaluated. The Organic Functional Group profiler in the OECD Toolbox (v 4.1), a quantitative structure-activity relationship program, was used to identify structural fragments in the tested chemicals [19] (see Supplemental Tables). It is noted that a single chemical may be assigned to more than one structural feature group. Isopropyl was the only structural feature group with a 100% (2/2) EPA false positive rate based on more than one chemical (see Supplemental Tables). Other structural feature groups having EPA false positive rates of at least 50% and at least two chemicals in the chemical class included aldehyde (1/2), alkane branched with a tertiary carbon (1/2), alkene (1/2), allyl (1/2), aryl (2/4), dihydroxyl derivatives (1/2), thiol (1/2), carboxylic acid ester (2/3), and ether (3/4). A similar pattern was observed for the GHS false positives (see Supplemental Tables).

Evaluation of false positives and negatives in Phases II and III

False positives and negatives in Phases II and III were evaluated separately to identify any common features. As noted above, false negative and false positive rates for the EPA classification system were 4% (2/46) and 40% (10/25). False negative and false positive rates for the GHS classification system were 0% (0/36) and 42% (15/36). The mis-identified chemicals are shown in Table 5 (Table 5 near here).

Table 5.

Phase II and III False Positives and False Negatives

No.	Phase	Chemical Name	In Vivo EPA	In Vitro EPA	In Vivo GHS	In Vitro GHS
6	II	1,3-Di-iso-propylbenzene	IV	I, II, or III	NC	1 or 2
8	II	1,9-Decadiene	IV	I, II, or III	NC	1 or 2
11	II	2,2-dimethyl-3-pentanol	-	-	NC	1 or 2
12	II	2,4-Pentanediol	IV	I, II, or III	-	-
37	III	1,4-Dibromobutane	III	IV	-	-
39	III	2-(2-Ethoxyethoxy)ethanol	-	-	NC	1 or 2
40	III	2,4-pentanedione	-	-	NC	1 or 2
44	III	2-Ethoxyethyl methacrylate	IV	I, II, or III	NC	1 or 2
48	III	3-Phenoxybenzyl alcohol	-	-	NC	1 or 2
59	III	Dodecane	III	IV	-	-
60	III	Ethyl acetate	-	-	NC	1 or 2
62	III	Ethylene glycol diethyl ether	IV	I, II, or III	NC	1 or 2
68	III	iso-Octylthioglycolate	IV	I, II, or III	NC	1 or 2
71	III	iso-Propyl bromide	IV	I, II, or III
78	III	n,n-Dimethylguanidine sulfate	-	-	NC	1 or 2
82	III	p-Methyl thiobenzaldehyde	IV	I, II, or III	NC	1 or 2
87	III	Styrene	-	-	NC	1 or 2
89	III	Triethylene glycol	IV	I, II, or III	NC	1 or 2
90	III	Triphenyl phosphite	IV	I, II, or III	NC	1 or 2

No EPA Category I or GHS Category 1 chemicals were underpredicted by OptiSafe (Table 5). The two chemicals that were underpredicted, 1,4-dibromobutane (No. 37) and dodecane (No. 59) were classified as Category III chemical by the EPA classification system based on in vivo data. Review of the in vivo data showed that classification for both chemicals as an EPA Category III irritant was based on a response in the conjunctival tissue (conjunctival redness score was 2 in a single animal) which cleared by day 2. Differences between the EPA and GHS classification systems (Table 1) lead these chemicals to be classified as Category III in the EPA system and NC in the GHS system. Therefore, these chemicals were underpredicted by OptiSafe when used for classification by the EPA but were correctly identified as Not Classified for the GHS classification system.

Due to the prediction model used by OptiSafe for this evaluation, overpredicted chemicals were all classified as Category IV by the EPA system or NC by the GHS system (Table 5). A review of the in vivo data for these chemicals indicated a majority either produced no response in any evaluated tissue or minimal responses in conjunctival tissues which cleared within 1 day.

Discussion

The current validation study showed that OptiSafe is transferable to naive laboratories and is reproducible within and between laboratories for the identification of non-surfactants as chemicals not requiring eye irritation hazard labelling. When used for this purpose, the intralaboratory reproducibility of OptiSafe in this study was greater than 92% for both the EPA and GHS classification systems. Within laboratory accuracy rates ranged from 78% (21/27) to 88% (22/25) for the GHS classification system and 82% (22/27) to 88% (22/25) for the EPA classification system. Interlaboratory reproducibility and accuracy rates were 91% (73/80) and 89% (23/26), respectively, for both eye irritation classification systems.

While the Phase I false negative rates for the lead laboratory and naive laboratory 1 were each 0%, the false negative rate for naïve laboratory 2 was 7% (1/14) for the EPA classification system and 15% (2/13) for the GHS classification system. Camphene (No. 19) was underpredicted relative to the in vivo data for the EPA and GHS classification systems, whereas ammonium nitrate (No. 17) was underpredicted relative to in vivo data for the GHS classification system. Additional studies are needed to determine the reason for the underprediction of these two chemicals by laboratory 2.

The Phase II false positive rates ranged from 23% (3/13) to 39% (5/13) and from 23% (3/13) to 36% (5/14) when in vitro classifications were compared to classifications based on retrospective in vivo data using the EPA or GHS eye irritation classification systems, respectively. All three testing laboratories misidentified 1,3-di-iso-propylbenzene (No. 6) and 1,9-decadiene (No. 8) as irritants in the EPA and GHS classification systems, although neither of these chemicals requires labelling as an eye irritation hazard for the EPA or GHS system. Additionally, all three testing laboratories misclassified 2,2-dimethyl-3-pentanol (No. 11) as an eye irritant, although it does not require labelling in the GHS classification system. The Organic Functional Group profiler in OECD Toolbox (v. 4.1) [19] was used to assess whether a common structural feature was associated with the observed overclassifications, but only limited structural similarity was observed among the three chemicals.

In Phase III, two chemicals were identified as false negatives when using the EPA classification system. Dodecane (No. 59) and 1,4-dibromobutane (No. 37) were both classified as Category III eye irritants based on in vivo data, but as Category IV based on OptiSafe results. A review of the supporting in vivo data showed that both chemicals produced positive conjunctival scores in only one animal at 24 hours, but these responses cleared by day 2 for each chemical [20].

No chemicals classified as corrosives (i.e., GHS Category 1 or EPA Category I) were mis-identified by OptiSafe. These results suggest that the method can accurately classify chemicals that produce severe effects either through a persistent response (e.g., an effect lasts >21 days) or a severe response.

Due to the prediction model used for this evaluation, the only chemicals that were identified as false positives were EPA Category IV and/or GHS NC chemicals. A review of the in vivo data indicated a majority of the chemicals produced minimal effects on the conjunctival tissue. However, due to the limited effects observed additional analyses are needed to further evaluate these chemicals.

OptiSafe accuracy was compared to accuracy for various alternative eye irritation test methods. Comparisons were conducted for chemicals that were evaluated by OptiSafe and alternative eye irritation test methods. When the same chemicals are compared, the accuracy and false negative rates for OptiSafe were as good as or better than each of the other test methods.

As is the case with any validation study, the uncertainty and variability of the in vivo Draize rabbit eye test should be considered when assessing the predictive ability of OptiSafe. Several studies have previously evaluated the response variability in the in vivo Draize rabbit eye test [21, 22, 23, 24]. Earl and colleagues showed that for a set of nine chemicals the coefficient of variation between 24 testing laboratories ranged from 42% to 59% [21]. Luechtefeld and colleagues evaluated the reproducibility of Draize results using conditional probabilities for different GHS hazard categories. Analyses were focused on the 491 chemicals in the ECHA online dossier database with more than one Draize eye test result reported. The highest reliability value was obtained for chemicals that were classified as a GHS NC (94% probability that a future outcome would also be GHS not classified). Comparatively, the analysis showed that there was a 74% probability that a future outcome for chemicals that were previously classified as GHS Category 1 would also be classified as GHS Category 1 in a subsequent study (i.e., 26% probability that it would be classified as a less severe irritant) [22]. To assess the within-test variability, Adriaens and colleagues resampled an in vivo rabbit eye irritation database from a variety of sources to assess over- and under-classification probabilities. The analyses showed there was at least an 11% probability that chemicals classified as GHS Category 1 could be classified as Category 2. Additionally, there was a 12% probability that Category 2 chemicals could be identified as not classified. Comparatively, the overclassification error rate was <1% [23]. Therefore, discordant results in OptiSafe relative to the reference in vivo data can only be made in context of the variability of the in vitro OptiSafe method and the in vivo Draize rabbit eye test.

Given that two different hazard classification and labelling schemes were evaluated in the validation studies, irritation classification differences were noted. While the EPA classification system is based on the most severe observed response in the tested animals, GHS uses the mean effect over two of three tested animals. These differences, combined with the subjective nature of scoring in the Draize eye test method, led to different classifications for some chemicals. This variability also should be noted when assessing the OptiSafe performance.

Another factor that should be considered in evaluating the predictive capacity of OptiSafe is that the assessment was compared to in vivo study results available at study initiation. Since no prospective animal testing was conducted for this study, the in vivo result classifications were obtained from a variety of sources. In some cases, differences in the purity of chemicals tested in vivo and in vitro (or a lack of purity information for the in vivo studies) may have impacted the predictive performance that was calculated for OptiSafe.

Phase III of the validation study allowed for a more comprehensive evaluation of the OptiSafe applicability domain in combination with Phase II results. Specifically, a group of 60 chemicals tested by the lead laboratory included various chemical classes, physical states, and irritancy severity categories. The Phase III structural classes were assigned using the Organic Functional Group profiler in the OECD Toolbox (v. 4.1) [19]. While many of the OECD Toolbox-assigned structural classes contained three or fewer chemicals, the analyses suggest that specific functional groups may be less suited to the method. For example, the OptiSafe false positive rate for ether-containing chemicals was 75% (3/4) using the EPA classification system and 83% (5/6) using the GHS classification system. Higher false positive rates also were noted for compounds that contained an alcohol (EPA 33% [1/3], GHS 60% [3/5]) or carboxylic acid ester (EPA 67% [2/3], GHS 75% [3/4]) group. Based on the results of the structural analysis for Phase III chemicals, the structural features present in overpredicted Phase II chemicals were evaluated. Of the EPA and GHS false positives, two chemicals contained an alcohol group: 2,2-dimethyl-3-pentanol (No. 11; GHS false positive) and 2,4-pentanediol (No. 12; EPA false positive). Several additional chemicals that contained an alcohol moiety were correctly identified. It is unclear if the presence of an alcohol moiety is associated with the observed misclassifications. However, additional studies are needed to further assess the applicability domain of the test method.

In the presence of light and air, ethers can be converted into peroxides [25]. Because strong oxidizing agents, including peroxides, are ocular irritants, the lead laboratory conducted preliminary studies using XploSens peroxide detection strips [26] to assess whether the presence of peroxides was associated with the overprediction of ether-containing chemicals. Peroxides were identified in ethylene glycol diethyl ether but not in the other ether-containing chemicals (data not shown).

In conclusion, OptiSafe is a transferable, reproducible, and accurate method for the identification of chemicals not requiring eye irritation hazard classification according to the EPA or GHS classification systems. This method could be used in the bottom-up approach outlined by Scott and colleagues [16] for the ocular hazard evaluation of non-surfactants. Additional advantages of OptiSafe include the convenience of all reagents and supplies being provided in a shelf-stable kit, the lack of requirements for specialized equipment, and the ability to obtain results within a single day. We envision that all these features will make OptiSafe a useful addition to available non-animal ocular irritation tests.