Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2022 Sep 5.
Published in final edited form as: Neurotoxicology. 2021 Jan 27;83:137–145. doi: 10.1016/j.neuro.2021.01.003

Evaluation of Chemical Compounds that Inhibit Neurite Outgrowth Using GFP-labeled iPSC-derived Human Neurons

Shuaizhang Li 1, Li Zhang 1, Ruili Huang 1, Tuan Xu 1, Frederick Parham 2, Mamta Behl 2,*, Menghang Xia 1,*
PMCID: PMC9444042  NIHMSID: NIHMS1807244  PMID: 33508353

Abstract

Due to the increasing number of drugs and untested environmental compounds introduced into commercial use, there is recognition for a need to develop reliable and efficient screening methods to identify compounds that may adversely impact the nervous system. One process that has been implicated in neurodevelopment is neurite outgrowth; the disruption of which can result in adverse outcomes that persist later in life. Here, we developed a green fluorescent protein (GFP) labeled neurite outgrowth assay in a high-content high-throughput format using induced pluripotent stem cell (iPSC) derived human spinal motor neurons and cortical glutamatergic neurons. The assay was optimized for use in a 1536-well plate format. Then, we used this assay to screen a set of 84 unique compounds that have previously been screened in other neurite outgrowth assays. This library consists of known developmental neurotoxicants, environmental compounds with unknown toxicity, and negative controls. Neurons were cultured for 40 hours and then treated with compounds at 11 concentrations ranging from 1.56 nM to 92 μM for 24 and 48 hours. Effects of compounds on neurite outgrowth were evaluated by quantifying total neurite length, number of segments and maximum neurite length per cell. Among the 84 tested compounds, neurite outgrowth in cortical neurons and motor neurons were selectively inhibited by 36 and 31 compounds, respectively. Colchicine, rotenone, and methyl mercuric (II) chloride inhibited neurite outgrowth in both cortical and motor neurons. It is interesting to note that some compounds like parathion and bisphenol AF had inhibitory effects on neurite outgrowth specifically in the cortical neurons, while other compounds, such as 2,2’,4,4’-tetrabromodiphenyl ether and caffeine, inhibited neurite outgrowth in motor neurons. The data gathered from these studies show that GFP-labeled iPSC-derived human neurons are a promising tool for identifying and prioritizing compounds with developmental neurotoxicity potential for further hazard characterization.

Keywords: Neurite outgrowth, developmental neurotoxicity, high-throughput screening, high-content imaging

Introduction

The prevalence of diagnosed neurodevelopmental disorders is increasing worldwide fueling public concern regarding lack of developmental neurotoxicology (DNT) data for large numbers of untested environmental chemicals. DNT can be induced by exposure to heavy metals (Wang, X. et al., 2010), certain foods and food additives (Lau et al., 2006), and other environmental compounds (Aschner et al., 2017). With a large number of chemical compounds introduced into our environment and the lack of potential developmental neurotoxicity hazard testing, there have been several global efforts to develop a set of high-throughput, high-content assays to screen compounds for possible DNT potential ( Bal-Price et al., 2015; Bal-Price et al., 2018; Behl et al., 2019; Brown et al., 2016; Coecke et al., 2007; Frank et al., 2017; Harrill et al., 2018; Smirnova et al., 2014). Developmental neurotoxicity can result from the perturbation of several key developmental processes including but not limited to neural proliferation, differentiation, migration, synaptogenesis and network formation ( Bal-Price et al., 2015; Mundy et al., 2015). Several in vitro systems have been developed to examine the effects of chemical exposure on the above mentioned neurodevelopmental processes (Balmer et al., 2012; Baumann et al., 2016; Frimat et al., 2010; Fritsche et al., 2017; Harrill et al., 2011; Krug et al., 2013; Radio and Mundy, 2008; Schmuck et al., 2017; Zimmer et al., 2012). The neurite outgrowth assay was included in a proposed battery of in vitro screens for DNT due to its relevance as a critical process in nervous system development in which neurons extend their neurites to form a complete neural network (Bal-Price et al., 2018; Harrill et al., 2013; Mundy et al., 2010; Radio and Mundy, 2008; Sanes et al., 2006). Disruption of this process can lead to adverse effects in humans and rodents, and several studies suggest that immature, developing, and mature neurites are targets of chemical toxicity (Krug et al., 2013; Radio and Mundy, 2008).

To date, most of the assays using Calcein AM staining or immunostainings for measuring neurite outgrowth required multiple sample processing steps during conduct of the experiments, which may introduce well-to-well variation in the assays and limit throughput. Furthermore, due to the use of staining, these assays can be evaluated only at a single time-point. To address some of these limitations, and to improve throughput, we developed a neurite outgrowth assay using GFP-labeled human iPSC-derived spinal motor neurons and cortical glutamatergic neurons in a 1536 -well plate format for the first time. The neurons labeled with GFP enable live and time-lapse imaging. The cells develop neurites rapidly and consistently as quantified using automated high-content imaging. The primary advantages of this assay are that it eliminates intermediate steps involved in washing and changing media, thereby having the potential to reduce experimental variability, save time, and improve throughput. Since these cells are GFP- labelled, it also enables for a continual evaluation of chemical effect over time.

Following assay development, we then screened a set of 84 compounds with previously published literature on several assays in the DNT battery including neurite outgrowth (Behl et al., 2019; Ryan et al., 2016) at concentrations ranging from 1.56 nM to 92 μM. Multiple endpoints were measured in this assay including total neurite length, number of segments, maximum neurite length and number of objects in 1536-well plates. The performance was evaluated, and compounds were ranked by comparing concentration-response curves. We compared results on the DNT reference compounds from our current study with those that have been previously been shown to alter neurite outgrowth in the literature (Krug et al., 2013; Radio and Mundy, 2008). We also compared the performance of this assay to previously conducted neurite outgrowth assays that used the same compound library (Behl et al., 2019; Delp et al., 2018; Ryan et al., 2016). There was a good concordance in the DNT reference compounds for neurite outgrowth between the current study and those previously tested in the literature. Additional DNT reference compounds from the library that were shown to affect other neurodevelopmental processes (not limited to neurite outgrowth) were also identified as active in the current study, along with some compounds with unknown DNT potential. Together, these data demonstrate that human iPSC-derived neurons are amenable for high-throughput assays of neurite outgrowth using high-content imaging and show promise as a reliable screening tool with increased throughput.

Materials and Methods

Materials

Cryopreserved human spinal motor neurons (Lot# 160916-0101A) and human cortical glutamatergic neurons (Lot# 170822-0301B) were purchased from BrainXell, Inc. (Waunakee, WI, USA). These iPSC-derived cells (WC-30) were labeled with GFP using the method described in the previous report (Qian et al., 2014) by BrainXell. Then iPSC cells were differentiated into motor and cortical neurons based on the protocols described previously (Du et al., 2015; Liu et al., 2013) by BrainXell. DMEM/F12 medium, neurobasal medium, B27 supplement, N2 supplement, GlutaMAX, and Geltrex™ LDEV-Free Reduced Growth Factor Basement Membrane Matrix were from Thermo Fisher Scientific, Inc. (Grand Island, NY, USA). Recombinant human/murine/rat BDNF (Brain-Derived Neurotrophic Factor), recombinant human GDNF (Glial cell-derived neurotrophic factor) and recombinant human TGF-β1 were from Peprotech (Rocky Hill, NJ, USA). Neuron supplement was provided by BrainXell, Inc. Rotenone and dimethylsulfoxide (DMSO) were obtained from Sigma-Aldrich Co. (St. Louis, MO, USA). The compounds that were used to screen this model were provided by the National Toxicology Program (NTP). The compound library consisted of a combination of 29 reference compounds with evidence of DNT as classified in the literature (Aschner et al., 2017; Mundy et al., 2015; Radio and Mundy, 2008), 50 compounds with unknown DNT potential (industrial chemicals, drugs, flame retardants, polycyclic aromatic compounds), and 4 negative controls. The complete list of chemicals tested can be found in Supplemental Table 1.

Thawing and seeding iPSC derived GFP-labeled human neurons

The neuron seeding medium was made according to manufacturer’s protocol. All the components are listed in Supplemental Table 2. The Geltrex was prepared by 1:10 dilution using cold DMEM/F12. The medium was equilibrated at room temperature for 15 minutes. The cryopreserved neurons were removed from the liquid nitrogen and placed in a 37°C water bath. The vials were removed from the water bath as soon as the last of the ice melted. Then, 800 μL of seeding medium was slowly added to the vials. All contents (1mL total) were gently transferred from the vial to a new sterile 50 mL conical tube. To collect any residual cells, another 1 mL of medium was added to the vial. Then the medium was transferred to the conical tube. Finally, an additional 3 mL of seeding medium was added slowly to the conical tube. All the work was done in a biological safety hood for cell culture. Then the neurons were counted using a hemocytometer and re-suspended for plating.

Neurite outgrowth assay

GFP-expressing human spinal motor neurons, or cortical glutamatergic neurons, are used to detect chemical-mediated inhibition of neurite outgrowth. Each assay was screened against the compound library in triplicate. Motor neurons were plated at 8000 cells/100 μL/well and 2000 cells/40 μL/well into Poly-D-lysine (PDL)-coated 96-well and 384-well plates, respectively, using a multi-channel pipettor. For a 1536-well plate protocol, motor or cortical neurons were dispensed at 800 cells/5 μL/well into low-base PDL-coated 1536-well black-wall/clear-bottom assay plates (Aurora Microplates, Whitefish, MT, USA) by BioRAPTR (Beckman Coulter, Inc., Brea, CA, USA). The plates were placed in the hood for 15 minutes then incubated at 37 °C for 40 hours to allow cell attachment and growth, and followed by the addition of 23 nL compounds via a Wako Pintool station (Wako Automation, San Diego, CA, USA). The final 11 concentrations of the compounds ranged from 1.56 nM to 92 μM (1:3 dilution from the top concentrations). Rotenone, a known neurotoxic compound, was used as a positive control and DMSO (0.46%) was used as a negative control in the screening. The control plate map is shown in Supplemental Fig. 1. The assay plates were then incubated at 37°C for 24 and 48 hours. These time-points were selected based on a pilot study that was run which showed that the linear growth phase for these cells was from 22 to 90 hours (Supplemental Fig. 2). The assay plates were sealed with Microporous Sealing Film (USA Scientific, Orlando, FL, USA) for imaging. To evaluate cell viability in the same plate, we used number of objects. Additionally, we ran parallel plates to evaluate cytotoxicity at 24 and 48 hours using Calcein Red™ AM (AAT Bioquest, Sunnyvale, CA, USA). Stock solution (2 mM Calcein Red™ AM) was prepared in DMSO. Then 2X working solution containing 10μM Calcein Red™ AM and 1.5mM Trypan Purple™ in HHBS (Hank’s Balanced Salt Solution with 20 mM Hepes buffer) was added into the desired wells already containing 5μL of culture medium. The plates were incubated at 37°C, 5% CO2 for 30 min and then fluorescence was read using Ex/Em=540/590 nm using EnVision.

High-content imaging

The fluorescence intensities (488 nm excitation, 530 nm emission for GFP) were measured using the Operetta High-Content Imaging System (PerkinElmer, Waltham, Massachusetts, USA) with a 20× Plan Fluor objective (Nikon). Images were acquired for four sites (four fields in a 1536-well plate) in each well and analyzed with the Cell Analysis software that included various image application modules for neurite outgrowth. Three algorithmic outputs were used for quantitative image analysis of neurite outgrowth: total neurite length, number of segments, and maximum neurite length. Number of objects, another parameter, was used to identify and count the number of cells in each well. Total neurite length is the sum of the length of each neurite segment assigned to each neuron, number of segments means the number of linear structures between branching points of the population assigned to each neuron, and maximum neurite length means the length (in pixels) of the longest path from a neuron body to an extreme segment (Wang, D. et al., 2010).

Data analysis

Three parameters, total neurite length, number of segments, and maximum neurite length were used to evaluate the inhibitory effects of compounds on neurite outgrowth. Number of objects was used to indicate cell viability. Analysis of compound concentration-response data from each parameter was performed as previously described (Huang, 2016; Inglese et al., 2006). Briefly, raw plate measurements for each titration point were first normalized relative to the positive control compound (46 μM rotenone for cell based assays; −100%) and DMSO-only wells (0%) as follows: Activity (%) = ([Vcompound – VDMSO]/[VDMSO –Vpos]) × 100, where Vcompound denotes the compound well values, Vpos denotes the median value of the positive control (46 μM rotenone) wells, and VDMSO denotes the median values of the DMSO-only wells. The data set was then corrected by applying an in-house pattern correction algorithm (Wang and Huang, 2016). The pattern correction algorithm is applied to remove background patterns in the microplates such as edge effects caused by solvent evaporation and subtle abnormalities such as tip effects or blotting from cell dispenses. The half-maximum inhibitory values (IC50) for each compound and maximum response (efficacy) values were obtained by fitting the concentration-response curves of each compound to a four-parameter Hill equation (Wang, Y. et al., 2010). Compounds with an IC50 and with efficacy ≤-20%, active in any of three endpoints, were identified as neurite outgrowth inhibitors. Selective neurite outgrowth inhibitors are defined as compounds that are active in total neurite length (IC50 value determined), but inactive in number of object (no IC50 value determined). A compound is also considered a selective neurite outgrowth inhibitor if there is a statistically significant difference between the IC50 values of total neurite length and number of object (t-test, p<0.05). We also ran a comparison between IC50 values of total neurite length and cytotoxicity as determined by Calcein red in parallel plates (Supplemental Table 3 and 4). For the remaining part of this manuscript, we will report differences in IC50s between neurite outgrowth and number of objects since it provides a more accurate comparison of selectivity due to concurrent evaluation in the same plate. The concordance was calculated as the ratio of number of active and inactive compounds to the total number of compounds.

Three parameters were used to evaluate the assay performance, including signal-to-background (S/B) ratio, coefficient of variation (CV) value, and Z’ factor, and these parameters were defined and calculated based on the previous report (Zhang et al., 1999). Compound data were analyzed and depicted using GraphPad Prism 5 (GraphPad Software, Inc., La Jolla, CA, USA).

Results

Development and optimization of high-content neurite outgrowth assay

To identify compounds with potential for DNT rapidly, a neurite outgrowth assay using GFP-labeled neurons was developed in a quantitative high-throughput and high-content format. The iPSC derived human spinal motor neurons and human cortical glutamatergic neurons expressing GFP allowed for real time visualization of neurites in 1536-well plates (Fig. 1). To optimize this assay, the motor and cortical neurons were first plated in 96-well and 384-well PDL-coated plates. Neurons form networks by growing out neurites that contact to other neurons after plating. As shown in Supplemental Fig. 3, rotenone, a known assay positive (Krug et al., 2013), inhibited neurite outgrowth formation in a concentration-dependent manner in 96-well and 384-well format, respectively. Then the assay was miniaturized and optimized into a 1536-well plate. Since these are GFP-labeled cells, they offer the advantage of measuring neurite outgrowth continually over a period of the time. Therefore, we evaluated the growth of neurites from 22 to 92 hours as noted in Supplemental Fig. 2. Continual linear growth to 90 hours was observed. Hence, for this initial study, we selected 24 and 48 hours as two timepoints to evaluate effects of chemical exposure following 40 hours in culture. The neurons were initially treated with rotenone for 24 and 48 hours as a DNT reference compound (Krug et al., 2013). As shown in Fig. 2, rotenone inhibited neurite outgrowth in a concentration dependent manner in both motor neurons and cortical neurons (Fig. 2A), respectively. The neuron images displayed rotenone-induced neurite inhibition in Fig. 2B. After rotenone treatment for 24 and 48 hours, the IC50 values were 7.7 and 1.48 μM in motor neurons, while the values were 2.0 and 1.5 μM in cortical neurons, respectively.

Fig.1.

Fig.1.

Open in a new tab

High-content assay development on iPSC-derived motor and cortical neurons. The cells extend neurites rapidly after plating in a 1536-well plate for 24 hours. Cell number and neurite morphology was also measured using automated image analysis. Input images was separated into component color channels to identify cell body and number of segments. Scale bar: 100 μm.

Fig. 2.

Fig. 2.

Open in a new tab

Concentration-response curves and representative images of the neurons in response to rotenone treatment. Motor neurons and cortical neurons labeled with GFP were cultured in 1536-well format and treated with varying concentrations of rotenone for 24 and 48 hours, respectively. There were dose curves in 1536-well using both motor neurons and cortical neurons (A). Blue circles (24 hours) and squares (48 hours) in the curves represent the number of objects listed on the right axis. Black circles (24 hours) and squares (48 hours) represent the total neurite length listed on the left axis for reference of neurite outgrowth. The unit of neurite length is expressed as pixel. The neurite growth was inhibited using rotenone treatment (0.36, 2.88 and 5.76 μM) for 24 and 48 hours, and DMSO was used as the negative control (B). The images were acquired using Operetta High-Content Imaging System with a 20× Plan Fluor objective (Nikon). Excitation: 488 nm; emission: 530 nm. Each value represents the mean ± SD of three independent experiments. Scale bar: 100 μm.

Assay performance from screening environmental chemicals and comparisons with DNT reference compounds for neurite outgrowth

Following assay optimization, we then screened a set of 84 unique compounds provided by the National Toxicology Program (NTP) (Ryan et al., 2016). This library consisted of a total of 29 DNT reference compounds, 10 of which have previously been shown to inhibit neurite outgrowth in the literature (Krug et al., 2013; Radio and Mundy, 2008) and 19 that have shown to affect the nervous system by other mechanisms (Aschner et al., 2017; Mundy et al., 2015). There were three algorithmic outputs used in the screening: total neurite length, number of segments, and maximum neurite length, for quantitative image analysis of neurite outgrowth. The correlation of IC50s calculated from these three outputs in cortical neurons at 48-hour treatment was evaluated with Pearson correlation. Correlation coefficient values (R) were 0.92 (number of segments versus total neurite length), 0.95 (maximum neurite length versus total neurite length), and 0.93 (number of segments versus maximum neurite length), respectively (Fig. 3). The correlations of IC50s calculated from these three outputs in cortical neurons (24-hour treatment) and motor neurons (24- and 48-hour treatment) were also determined and showed good correlations among these three outputs (Supplemental Fig. 4) The screening performance was evaluated using total neurite length as an algorithmic output. The DMSO control wells were used for CV calculation. For screening using motor neurons, the average signal-to-background (S/B) ratio, coefficient of variation (CV) value, and Z’ factor were 11.61 ± 0.68, 4.60 ± 0.97 and 0.42 ± 0.03 after 24-hour treatment, while the values were 10.30 ± 2.61, 16.04 ± 1.36 and 0.43 ± 0.01 after 48-hour treatment (Table 1). In the assay using cortical neurons, the average signal-to-background (S/B) ratio, coefficient of variation (CV) value, and Z’ factor were 3.81 ± 0.15, 12.75 ± 0.68 and 0.19 ± 0.14 after 24-hour treatment, while the values were 17.17 ± 0.41, 12.95 ± 1.17 and 0.35 ± 0.02 after 48-hour treatment.

Fig. 3.

Fig. 3.

Open in a new tab

Comparison of endpoints in the neurite outgrowth assay. Scatter plots showed IC50S for three neurite outgrowth endpoints for the 84 compounds using cortical neurons after 48-hour treatment. Pearson correlation values are shown between (A) Total neurite length and Number of segments; (B) Total neurite length and Maximum neurite length; (C) Number of segments and Maximum neurite length.

Table 1.

Assay performance using total neurite length algorithmic output in motor and cortical neurons.

Statistical parameters Motor Neurons Cortical Neurons
24 hours 48 hours 24 hours 48 hours
CV (%) 11.61 ± 0.68 16.04 ± 1.36 12.75 ± 0.68 17.17 ± 0.41
S/B 4.60 ± 0.97 10.30 ± 2.61 3.81 ± 0.15 12.95 ± 1.17
Z’ factor 0.42 ± 0.03 0.43 ± 0.01 0.19 ± 0.14 0.35 ± 0.02
Open in a new tab

All values represent the mean ± SD of 3 independent experiments. CV was calculated by DMSO control.

Assay performance was compared with 10 (out of 29) DNT reference compounds in the literature that have previously been shown to inhibit neurite outgrowth in various models (Krug et al., 2013; Mundy et al., 2010; Radio and Mundy, 2008; Ryan et al., 2016). A compound was considered active if it inhibited neurite outgrowth at either 24 or 48 hours. As shown in Supplemental Fig. 5, the current assays captured 9/10 reference DNT compounds that have previously been shown to affect neurite outgrowth in the literature (7/9 in the cortical assay and 8/9 in motor assay). The only compound that was not captured either following 24 or 48 hours of exposure was manganese tricarbonyl. Our current assays captured 6 additional DNT reference compounds in the cortical assay and 5 additional compounds in the motor assay that have previously either not been tested or have shown to be negative in the neurite outgrowth assay.

Identification of compounds that inhibited neurite outgrowth in cortical neurons

For cortical neurons, 44 out of the 84 tested compounds showed neurite outgrowth inhibition in total neurite length (Supplemental Table 3). Among these 44 actives, 16 compounds have previously been shown to either inhibit neurite outgrowth in the literature or classified as a DNT reference compound by other mechanisms (Aschner et al., 2017; Krug et al., 2013; Mundy et al., 2015; Mundy et al., 2010;Ryan et al., 2016) as noted in the section above. There were 36 out of 44 compounds considered selective inhibitors for neurite outgrowth based on the t-test between the IC50 values of total neurite length and concurrent cytotoxicity as calculated by number of objects (p≤0.05). Although we collapsed the actives across both the time-points (24 h and 48 h) for purposes of overall reporting, there were some time-dependent effects noted. For example, bisphenol A, deltamethrin, and tebuconazole were not active after 24 hours of treatment, but active after 48 hours of treatment (Supplemental Table 3). Other compounds such as ethylhexyl diphenyl phosphate and benzo (b) fluoranthene were shown to be active at 24 hours but not 48 hours (Supplemental Table 3).

Identification of compounds that inhibited neurite outgrowth in motor neurons

In motor neuron assay, 39 out of 84 compounds showed inhibitory effects on total neurite length. Among these 39 active compounds, 15 compounds were DNT reference compounds for neurite outgrowth or exhibited effects by other mechanisms (Aschner et al., 2017; Mundy et al., 2015). There were 31 out of 39 compounds considered selective for neurite outgrowth (Supplemental Table 4). The sensitivity (number of actives/number of known DNT compounds) was comparable between the cortical neurons (16/29 = 55.2%) and motor neurons (15/29 = 51.7%). There was 79.8% concordance (number of compounds with same activity in both neuron types/total number of compounds) between cortical and motor neurons. However, it was interesting to find that 3,3’,5,5’-tetrabromobisphenol A, and bisphenol AF only inhibited neurite outgrowth in cortical neurons, but not motor neurons, and some other compounds including 2,2′,4,4′-tetrabromodiphenyl ether, caffeine, and anthracene, only affected neurite outgrowth in motor neurons (Fig. 4). Colchicine and methyl mercuric (II) chloride inhibited neurite outgrowth in both cortical and motor neurons (Fig. 4, 5). Colchicine was the most potent compound with IC50 of 21.84 nM and 55.81 nM for cortical and motor neurons after 48 hours of treatment (Fig. 5).

Fig. 4.

Fig. 4.

Open in a new tab

Heat map of the 49 compounds in the neurite outgrowth assays. The motor and cortical neurons labeled with GFP were incubated with 11 concentration of compounds for 24 and 48 hours, respectively. The values were c total neurite length. The compound activity is colored according to potency (IC50) and efficacy. Inactive compounds are colored grey.

Fig. 5.

Fig. 5.

Open in a new tab

Concentration response curves for several selective neurite outgrowth inhibitors in motor and cortical neurons. Total neurite length (TNL) and viable cells (VC) were determined at 11 concentrations to identify hits. A compound is considered a selective neurite inhibitor if there is a statistically significant difference between the IC50 values of total neurite length and cell viability (t-test, p<0.05). Colchicine, and methyl mercuric (II) chloride were selective neurite outgrowth inhibitors both in motor neurons (A) and cortical neurons (B). Each value represents the mean ± SD of three independent experiments.

Discussion

There is a global recognition for a need to efficiently identify environmental compounds with unknown DNT potential (Bal-Price et al., 2015; Bal-Price et al., 2018; Bal-Price et al., 2010; Bjorling-Poulsen et al., 2008; Crofton et al., 2011; Fritsche et al., 2017; Mundy et al., 2010). The neurite outgrowth assay is considered a key component of a battery of assays representing cellular events critical to nervous system development (Aschner et al., 2017; Bal-Price et al., 2018; Mundy et al., 2015). The utility of this assay can be further improved by increasing the throughput, reducing plate-to-plate variability by minimizing the number of washing steps involved during the assay. To our knowledge, this is the first time that the neurite outgrowth assay has been developed in a 1536-well plate format using GFP-labeled human spinal motor neurons and cortical glutamatergic neurons. This eliminates washing steps thereby greatly increasing the screening throughput and decreasing cell disturbance due to the washing steps. Importantly, it enables a live and time-lapse imaging of neurite outgrowth, thereby allowing for observing chemical effects over time within the same experiment.

Based on guidance for the development of novel DNT screening assays described previously (Crofton et al., 2011), this study covered some important considerations. 1) It identifies a key event critical to nervous system development (i.e. neurite outgrowth), 2) provides a description of end-points measured, 3) provides some information with respect to assay validation and initial training/test set of chemicals, 4) initial characterization on the dynamic range of the assay, and 5) provides information on advantages with respect to its throughput and relevance to other neurite outgrowth assays.

We first optimized the assay using rotenone as an assay specific positive control. The assay performance from the screening in a 1536-well plate format was acceptable with Z’ factors ranging from 0.19 to 0.43. Although Z’ ≥ 0.5 is generally regarded as being robust for high-throughput screenings, 0 < Z′≤ 0.5 has shown to be acceptable for HCS assays according to previously published guidelines for image-based high content screening and analysis. This is because HCS assays are typically more complex and involve evaluating multiple functional endpoints that tend to have more variability compared to traditional receptor-based HTS assays (Bray and Carpenter, 2004). Next, in comparing actives in the current assays with previous DNT reference compounds in the literature that have been shown to inhibit neurite outgrowth, we noted over 90% concordance in both the cortical and motor neurons. Finally, we tested these compounds in an 84-chemical library comprising a combination of known DNT reference compounds, unknowns and compounds previously shown to be negative for DNT (Supplemental Table 1).

Our findings showed that there were 44 (52%) and 39 (46%) compounds that inhibited neurite outgrowth in the current cortical and motor neuron assays (Fig 6; Supplemental Tables 3, 4). The concordance was 79.8% between cortical and motor neurons. Of these, 36/44 (82%) and 31/39 (79%) compounds selectively inhibited neurite outgrowth in cortical and motor neurons, respectively (i.e. inhibited outgrowth at a statistically lower concentration that observed for concurrent cytotoxicity). Among compounds that were previously classified as “known DNTs” in the literature, approximately 50% were active in these assays. This is not unexpected since the library was designed to include compounds that exert effects on varying key neurodevelopmental processes, not limited to neurite outgrowth. Some of these compounds have been shown to alter neuronal firing (e.g., deltamethrin, diazepam), neuro-behavior in zebrafish (e.g., valproic acid, chlorpyrifos), and/or were based on other in vivo findings (e.g., phenobarbital, n-hexane). N-hexane may not have been active due to its volatility, and phenobarbital may not be active due to the fact that 92 μM may be much lower than concentrations required in vivo for therapeutical activity. Acrylamide has been shown to affect neurite initiation, but not neurite length (Nordin-Andersson et al., 2003). However, manganese tricarbonyl, a known DNT, did not show the inhibition of the neurite outgrowth after either 24 or 48 hours of exposure in this study. It may have been due to shorter time exposure for testing compounds compared to previous finding in the literature (Ryan et al., 2016).

Fig. 6.

Fig. 6.

Open in a new tab

Cross comparison of test data for the NTP collection. Neurite outgrowth inhibition data using GFP-labeled cortical neurons and motor neurons are shown in parallel with other two published data sets (Delp et al., 2018; Ryan et al., 2016). In our study, the positive compounds were defined by either active in 24- or 48-hour treatment. Selective neurite outgrowth inhibitors are defined as compounds where there is a statistically significant difference between the IC50 values of total neurite length and cell viability (t-test, p<0.05). Selective inhibitors were defined as ones having a high ratio (≥ 3.16) between BMC s (benchmark concentrations) for viability and neurite area (Ryan et al., 2016). Selective inhibitors were defined by an IC50 (viability/neurite area) ratio for NeuriTox test (≥4) and PeriTox test (≥3) (Delp et al., 2018). The effect of the compound is indicated as specific neurotoxicity (green), cytotoxic effect (red) or no effect (white). Compounds with * are known DNT as classified in the previous studies (Aschner et al., 2017; Mundy et al., 2015).

We also noted some differences in profiles of active compounds in response across 24- and 48-hour time points. For example, bisphenol A, deltamethrin, and tebuconazole only showed inhibitory effects on neurite outgrowth after 48-hour exposure. Chemicals such as bisphenol A and deltamethrin are less efficacious in the neurite outgrowth inhibition. Reproducibility of binary hit calls tends to be less when dealing with non-efficacious chemicals as opposed to highly efficacious chemicals. Other compounds, such as 2-ethylhexyl diphenyl phosphate and benzo (b) fluoranthene, were shown to be active at 24 hours but not 48 hours (Supplemental Table 3). Although we are currently unsure about implications as to why these compounds were inactive at 48 hour treatment, a similar pattern has been previously noted for cardiotoxicity for these compounds (Sirenko et al., 2017). In the current study, we evaluated only two time-points (24 and 48 hours) and summarized the results as overall activity, future studies allow for the evaluation of multiple time-points with a possibility to determine tipping points for neurite outgrowth similar to that shown recently for neuronal network development (Frank et al., 2018).

In comparing differences in compound activity in cortical and motor neurons, some compounds, such as bisphenol AF, and parathion only inhibited neurite outgrowth of cortical neurons, while other compounds, such as caffeine only affected motor neurons. There were 11/84 compounds including 2-ethylhexyl-2, 3, 4, 5-tetrabromobenzoate (TBB), auramine O, bisphenol AF, caffeine, estradiol, firemaster 550, and lead (II) acetate trihydrate that were shown to inhibit neurite outgrowth in this study for the first time (Figure 6). Out of the four negative controls tested in the study, one compound, L-ascorbic acid, was also identified as an inhibitor of neurite outgrowth. L-ascorbic acid has been reported to be cytotoxic to neurons (Delp et al., 2018; Pei et al., 2016), and was observed to be cytotoxic in our study using motor neurons. The other three negative controls were not found to be inhibitors, so the specificity for both cortical and motor neurons was 75%.

Finally, we compared findings from the current study with two other studies in the literature that used the same NTP compound collection (74/84 overlapping compounds) to contrast performance across various neurite outgrowth models using the same set of compounds (Fig. 6). These assays included the neurite outgrowth assay using iPSC-derived cortical neurons (Ryan et al., 2016), LUHMES cell-based developmental neurotoxicity (NeuriTox) and peripheral nervous system toxicity assays (PeriTox) (Delp et al., 2018) (Fig 6). For cortical neurons, we noted a 48% concordance between our neurite outgrowth assay and the study conducted by Ryan et al. (Ryan et al., 2016), and a 53% concordance with NeuriTOX. Diethylstilbestrol and rotenone showed selective effects on neurite outgrowth in all these studies. For motor neurons, there was a 58% concordance between our assay and PeriTox. Compounds including colchicine, diethylstilbestrol, methyl mercuric (II) chloride and rotenone were shown to selectively inhibit neurite outgrowth in motor neurons. 2,2′,4,4′-Tetrabromodiphenyl ether and 2,2′,4,4′,5-Pentabromodiphenyl ether were cytotoxic to both cortical and motor neurons in the three studies. Valinomycin was shown to have selective effects on neurite outgrowth in our study, and NeuriTox and PeriTox assays, which was cytotoxic to neurons in the study by Ryan et al. (Ryan et al., 2016). Some reasons for differences in concordance across the current assay with the other two assays may stem from the method of categorization of actives, differences in cell models (iPSC-derived neurons versus immortalized LUHMES), cell-types (primarily glutamatergic versus LUHMES primarily dopaminergic), compound concentrations, and other unknown factors. Some future studies may involve expanding the chemical library to better understand how this model performs with a more diverse set of chemicals over time to determine tipping points for neurite outgrowth similar to that done for neural network formation (Frank et al., 2018).

In conclusion, this is the first time that a GFP-labeled neurite outgrowth model has been developed in a high throughput platform which offers the advantages of 1) evaluating the neurite outgrowth in a 1536-well format for cortical and motor neurons, 2) eliminating the need for intermediate wash steps thereby increasing speed and decreasing experimental variability due to minimal processing, and 3) allowing for a continual evaluation of the effects of chemical exposure on neurons over time (up to 48 hours), thereby providing a potential to evaluate tipping points.

Supplementary Material

1

Highlights.

  • Neurite outgrowth is extensively used to evaluate compound with DNT potential

  • Neurons labeled with GFP were derived from human iPSC

  • Neurite outgrowth assay using GFP-labeled neurons was developed in a HCS platform

  • Many selective neurite outgrowth inhibitors have been identified

  • This HCS assay can be used to evaluate compound inhibition on neurite outgrowth

Acknowledgements

We thank Deborah Ngan for editing the manuscript. This study was supported in part by the Intramural Research Program of the National Center for Advancing Translational Sciences (NCATS) and Interagency Agreement IAA No. NTR12003 from the National Institute of Environmental Health Sciences/Division of the National Toxicology Program to the NCATS, National Institutes of Health.

Footnotes

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

Reference

  1. Aschner M, Ceccatelli S, Daneshian M, Fritsche E, Hasiwa N, Hartung T, Hogberg HT, Leist M, Li A, Mundi WR, Padilla S, Piersma AH, Bal-Price A, Seiler A, Westerink RH, Zimmer B, Lein PJ, 2017. Reference compounds for alternative test methods to indicate developmental neurotoxicity (DNT) potential of chemicals: example lists and criteria for their selection and use. ALTEX 34(1), 49–74. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Bal-Price A, Crofton KM, Leist M, Allen S, Arand M, Buetler T, Delrue N, FitzGerald RE, Hartung T, Heinonen T, Hogberg H, Bennekou SH, Lichtensteiger W, Oggier D, Paparella M, Axelstad M, Piersma A, Rached E, Schilter B, Schmuck G, Stoppini L, Tongiorgi E, Tiramani M, Monnet-Tschudi F, Wilks MF, Ylikomi T, Fritsche E, 2015. International STakeholder NETwork (ISTNET): creating a developmental neurotoxicity (DNT) testing road map for regulatory purposes. Arch Toxicol 89(2), 269–287. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Bal-Price A, Hogberg HT, Crofton KM, Daneshian M, FitzGerald RE, Fritsche E, Heinonen T, Bennekou SH, Klima S, Piersma AH, Sachana M, Shafer TJ, Terron A, Monnet-Tschudi F, Viviani B, Waldmann T, Westerink RHS, Wilks MF, Witters H, Zurich MG, Leist M, 2018. Recommendation on Test Readiness Criteria for New Approach Methods in Toxicology: Exemplified for Developmental Neurotoxicity. Altex-Alternatives to Animal Experimentation 35(3), 306–352. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Bal-Price AK, Hogberg HT, Buzanska L, Lenas P, van Vliet E, Hartung T, 2010. In vitro developmental neurotoxicity (DNT) testing: relevant models and endpoints. Neurotoxicology 31(5), 545–554. [DOI] [PubMed] [Google Scholar]
  5. Balmer NV, Weng MK, Zimmer B, Ivanova VN, Chambers SM, Nikolaeva E, Jagtap S, Sachinidis A, Hescheler J, Waldmann T, Leist M, 2012. Epigenetic changes and disturbed neural development in a human embryonic stem cell-based model relating to the fetal valproate syndrome. Hum Mol Genet 21(18), 4104–4114. [DOI] [PubMed] [Google Scholar]
  6. Baumann J, Gassmann K, Masjosthusmann S, DeBoer D, Bendt F, Giersiefer S, Fritsche E, 2016. Comparative human and rat neurospheres reveal species differences in chemical effects on neurodevelopmental key events. Arch Toxicol 90(6), 1415–1427. [DOI] [PubMed] [Google Scholar]
  7. Behl M, Ryan K, Hsieh JH, Parham F, Shapiro AJ, Collins BJ, Sipes NS, Birnbaum LS, Bucher JR, Foster PMD, Walker NJ, Paules RS, Tice RR, 2019. Screening for Developmental Neurotoxicity at the National Toxicology Program: The Future Is Here (vol 167, pg 6, 2019). Toxicological Sciences 168(2), 644–644. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Bjorling-Poulsen M, Andersen HR, Grandjean P, 2008. Potential developmental neurotoxicity of pesticides used in Europe. Environ Health 7, 50. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Bray MA, Carpenter A, 2004. Advanced Assay Development Guidelines for Image-Based High Content Screening and Analysis, in: Sittampalam GS, Coussens NP, Brimacombe K, Grossman A, Arkin M, Auld D, Austin C, Baell J, Bejcek B, Caaveiro JMM, Chung TDY, Dahlin JL, Devanaryan V, Foley TL, Glicksman M, Hall MD, Haas JV, Inglese J, Iversen PW, Kahl SD, Kales SC, Lal-Nag M, Li Z, McGee J, McManus O, Riss T, Trask OJ Jr., Weidner JR, Wildey MJ, Xia M, Xu X (Eds.), Assay Guidance Manual. Bethesda (MD). [PubMed] [Google Scholar]
  10. Brown JP, Hall D, Frank CL, Wallace K, Mundy WR, Shafer TJ, 2016. Editor’s Highlight: Evaluation of a Microelectrode Array-Based Assay for Neural Network Ontogeny Using Training Set Chemicals. Toxicol Sci 154(1), 126–139. [DOI] [PubMed] [Google Scholar]
  11. Coecke S, Goldberg AM, Allen S, Buzanska L, Calamandrei G, Crofton K, Hareng L, Hartung T, Knaut H, Honegger P, Jacobs M, Lein P, Li A, Mundy W, Owen D, Schneider S, Silbergeld E, Reum T, Trnovec T, Monnet-Tschudi F, Bal-Price A, 2007. Workgroup report: incorporating in vitro alternative methods for developmental neurotoxicity into international hazard and risk assessment strategies. Environ Health Perspect 115(6), 924–931. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Crofton KM, Mundy WR, Lein PJ, Bal-Price A, Coecke S, Seiler AEM, Knaut H, Buzanska L, Goldberg A, 2011. Developmental Neurotoxicity Testing: Recommendations for Developing Alternative Methods for the Screening and Prioritization of Chemicals. Altex-Alternatives to Animal Experimentation 28(1), 9–15. [PubMed] [Google Scholar]
  13. Delp J, Gutbier S, Klima S, Hoelting L, Pinto-Gil K, Hsieh JH, Aichem M, Klein K, Schreiber F, Tice RR, Pastor M, Behl M, Leist M, 2018. A high-throughput approach to identify specific neurotoxicants/ developmental toxicants in human neuronal cell function assays. ALTEX. [DOI] [PubMed] [Google Scholar]
  14. Du ZW, Chen H, Liu H, Lu J, Qian K, Huang CL, Zhong X, Fan F, Zhang SC, 2015. Generation and expansion of highly pure motor neuron progenitors from human pluripotent stem cells. Nat Commun 6, 6626. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Frank CL, Brown JP, Wallace K, Mundy WR, Shafer TJ, 2017. From the Cover: Developmental Neurotoxicants Disrupt Activity in Cortical Networks on Microelectrode Arrays: Results of Screening 86 Compounds During Neural Network Formation. Toxicol Sci 160(1), 121–135. [DOI] [PubMed] [Google Scholar]
  16. Frank CL, Brown JP, Wallace K, Wambaugh JF, Shah I, Shafer TJ, 2018. Defining toxicological tipping points in neuronal network development. Toxicol Appl Pharmacol 354, 81–93. [DOI] [PubMed] [Google Scholar]
  17. Frimat JP, Sisnaiske J, Subbiah S, Menne H, Godoy P, Lampen P, Leist M, Franzke J, Hengstler JG, van Thriel C, West J, 2010. The network formation assay: a spatially standardized neurite outgrowth analytical display for neurotoxicity screening. Lab Chip 10(6), 701–709. [DOI] [PubMed] [Google Scholar]
  18. Fritsche E, Crofton KM, Hernandez AF, Hougaard Bennekou S, Leist M, Bal-Price A, Reaves E, Wilks MF, Terron A, Solecki R, Sachana M, Gourmelon A, 2017. OECD/EFSA workshop on developmental neurotoxicity (DNT): The use of non-animal test methods for regulatory purposes. ALTEX 34(2), 311–315. [DOI] [PubMed] [Google Scholar]
  19. Harrill JA, Freudenrich T, Wallace K, Ball K, Shafer TJ, Mundy WR, 2018. Testing for developmental neurotoxicity using a battery of in vitro assays for key cellular events in neurodevelopment. Toxicol Appl Pharmacol 354, 24–39. [DOI] [PubMed] [Google Scholar]
  20. Harrill JA, Robinette BL, Freudenrich T, Mundy WR, 2013. Use of high content image analyses to detect chemical-mediated effects on neurite sub-populations in primary rat cortical neurons. Neurotoxicology 34, 61–73. [DOI] [PubMed] [Google Scholar]
  21. Harrill JA, Robinette BL, Mundy WR, 2011. Use of high content image analysis to detect chemicalinduced changes in synaptogenesis in vitro. Toxicology in Vitro 25(1), 368–387. [DOI] [PubMed] [Google Scholar]
  22. Huang R, 2016. A Quantitative High-Throughput Screening Data Analysis Pipeline for Activity Profiling. Methods Mol Biol 1473, 111–122. [DOI] [PubMed] [Google Scholar]
  23. Inglese J, Auld DS, Jadhav A, Johnson RL, Simeonov A, Yasgar A, Zheng W, Austin CP, 2006. Quantitative high-throughput screening: a titration-based approach that efficiently identifies biological activities in large chemical libraries. Proc Natl Acad Sci U S A 103(31), 11473–11478. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Krug AK, Balmer NV, Matt F, Schonenberger F, Merhof D, Leist M, 2013. Evaluation of a human neurite growth assay as specific screen for developmental neurotoxicants. Archives of Toxicology 87(12), 2215–2231. [DOI] [PubMed] [Google Scholar]
  25. Lau K, McLean WG, Williams DP, Howard CV, 2006. Synergistic interactions between commonly used food additives in a developmental neurotoxicity test. Toxicological Sciences 90(1), 178–187. [DOI] [PubMed] [Google Scholar]
  26. Liu Y, Liu H, Sauvey C, Yao L, Zarnowska ED, Zhang SC, 2013. Directed differentiation of forebrain GABA interneurons from human pluripotent stem cells. Nat Protoc 8(9), 1670–1679. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Mundy WR, Padilla S, Breier JM, Crofton KM, Gilbert ME, Herr DW, Jensen KF, Radio NM, Raffaele KC, Schumacher K, Shafer TJ, Cowden J, 2015. Expanding the test set: Chemicals with potential to disrupt mammalian brain development. Neurotoxicol Teratol 52(Pt A), 25–35. [DOI] [PubMed] [Google Scholar]
  28. Mundy WR, Radio NM, Freudenrich TM, 2010. Neuronal models for evaluation of proliferation in vitro using high content screening. Toxicology 270(2–3), 121–130. [DOI] [PubMed] [Google Scholar]
  29. Nordin-Andersson M, Walum E, Kjellstrand P, Forsby A, 2003. Acrylamide-induced effects on general and neurospecific cellular functions during exposure and recovery. Cell Biol Toxicol 19(1), 43–51. [DOI] [PubMed] [Google Scholar]
  30. Pei Y, Peng J, Behl M, Sipes NS, Shockley KR, Rao MS, Tice RR, Zeng X, 2016. Comparative neurotoxicity screening in human iPSC-derived neural stem cells, neurons and astrocytes. Brain Res 1638(Pt A), 57–73. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Qian K, Huang CT, Chen H, Blackbourn L.W.t., Chen Y, Cao J, Yao L, Sauvey C, Du Z, Zhang SC, 2014. A simple and efficient system for regulating gene expression in human pluripotent stem cells and derivatives. Stem Cells 32(5), 1230–1238. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Radio NM, Mundy WR, 2008. Developmental neurotoxicity testing in vitro: models for assessing chemical effects on neurite outgrowth. Neurotoxicology 29(3), 361–376. [DOI] [PubMed] [Google Scholar]
  33. Ryan KR, Sirenko O, Parham F, Hsieh JH, Cromwell EF, Tice RR, Behl M, 2016. Neurite outgrowth in human induced pluripotent stem cell-derived neurons as a high-throughput screen for developmental neurotoxicity or neurotoxicity. Neurotoxicology 53, 271–281. [DOI] [PubMed] [Google Scholar]
  34. Sanes DH, Reh TA, Harris WA, 2006. Development of the nervous system, 2nd ed. Elsevier, Amsterdam ; Boston. [Google Scholar]
  35. Schmuck MR, Temme T, Dach K, de Boer D, Barenys M, Bendt F, Mosig A, Fritsche E, 2017. Omnisphero: a high-content image analysis (HCA) approach for phenotypic developmental neurotoxicity (DNT) screenings of organoid neurosphere cultures in vitro. Archives of Toxicology 91(4), 2017–2028. [DOI] [PubMed] [Google Scholar]
  36. Sirenko O, Grimm FA, Ryan KR, Iwata Y, Chiu WA, Parham F, Wignall JA, Anson B, Cromwell EF, Behl M, Rusyn I, Tice RR, 2017. In vitro cardiotoxicity assessment of environmental chemicals using an organotypic human induced pluripotent stem cell-derived model. Toxicol Appl Pharmacol 322, 60–74. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Smirnova L, Hogberg HT, Leist M, Hartung T, 2014. Developmental neurotoxicity - challenges in the 21st century and in vitro opportunities. ALTEX 31(2), 129–156. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Valdivia P, Martin M, LeFew WR, Ross J, Houck KA, Shafer TJ, 2014. Multi-well microelectrode array recordings detect neuroactivity of ToxCast compounds. Neurotoxicology 44, 204–217. [DOI] [PubMed] [Google Scholar]
  39. Wang D, Lagerstrom R, Sun C, Bishof L, Valotton P, Gotte M, 2010. HCA-vision: Automated neurite outgrowth analysis. J Biomol Screen 15(9), 1165–1170. [DOI] [PubMed] [Google Scholar]
  40. Wang X, Meng D, Chang Q, Pan J, Zhang Z, Chen G, Ke Z, Luo J, Shi X, 2010. Arsenic inhibits neurite outgrowth by inhibiting the LKB1-AMPK signaling pathway. Environ Health Perspect 118(5), 627–634. [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Wang Y, Huang R, 2016. Correction of Microplate Data from High-Throughput Screening. Methods Mol Biol 1473, 123–134. [DOI] [PubMed] [Google Scholar]
  42. Wang Y, Jadhav A, Southal N, Huang R, Nguyen DT, 2010. A grid algorithm for high throughput fitting of dose-response curve data. Curr Chem Genomics 4, 57–66. [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Zhang JH, Chung TDY, Oldenburg KR, 1999. A simple statistical parameter for use in evaluation and validation of high throughput screening assays. Journal of Biomolecular Screening 4(2), 67–73. [DOI] [PubMed] [Google Scholar]
  44. Zimmer B, Lee G, Balmer NV, Meganathan K, Sachinidis A, Studer L, Leist M, 2012. Evaluation of developmental toxicants and signaling pathways in a functional test based on the migration of human neural crest cells. Environ Health Perspect 120(8), 1116–1122. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

1
NIHMS1807244-supplement-1.docx (436.1KB, docx)

RESOURCES