Addressing variability in the acoustic startle reflex for accurate gap detection assessment

Abstract

The acoustic startle reflex (ASR) is subject to substantial variability. This inherent variability consequently shapes the conclusions drawn from gap-induced prepulse inhibition of the acoustic startle reflex (GPIAS) assessments. Recent studies have cast doubt as to the efficacy of this methodology as it pertains to tinnitus assessment, partially, due to variability in and between data sets. The goal of this study was to examine the variance associated with several common data collection variables and data analyses with the aim to improve GPIAS reliability. To study this the GPIAS tests were conducted in adult male and female CBA/CaJ mice. Factors such as inter-trial interval, circadian rhythm, sex differences, and sensory adaptation were each evaluated. We then examined various data analysis factors which influence GPIAS assessment. Gap-induced facilitation, data processing options, and assessments of tinnitus were studied. We found that the startle reflex is highly variable in CBA/CaJ mice, but that this can be minimized by certain data collection factors. We also found that careful consideration of temporal fluctuations of the ASR and controlling for facilitation can lead to drastically divergent GPIAS results. Finally, we show that tinnitus decreases gap-induced inhibition of the acoustic startle reflex, but only for short gap durations. This study provides a guide for reducing variance in the GPIAS methodology – thereby improving the diagnostic power of the test.

1. Introduction

A reliable animal model of tinnitus is a prerequisite for tinnitus related therapies. Due to its relatively minor time commitment gap-induced prepulse inhibition of the acoustic startle reflex (GPIAS) has quickly become one of the predominant behavioral assessments for tinnitus (Turner et al., 2006) in several animal models (see Eggermont, 2013; Hayes et al., 2014; Galazyuk and Hébert, 2015). Many groups have used GPIAS to behaviorally assess tinnitus in rats (Turner et al., 2006; Lobarinas et al., 2013; Singer et al., 2013; Ropp et al., 2014), mice (Longenecker and Galazyuk, 2011, Middleton et al., 2011; Hickox & Liberman, 2014; Lowe and Walton, 2015; Yu et al., 2016), guinea pigs (Dehmel et al., 2012; Berger et al., 2013), and hamsters (Salloum et al., 2016). However, some uncertainty of this method is rooted in a lack of consistency of methodologies and data assessment strategies across labs (Galazyuk and Hébert, 2015). While the acoustic startle reflex (Landis and Hunt, 1939; Fleshler, 1965), prepulse inhibition (Hoffman and Searle, 1965; Ison and Hammond, 1971; Carlson and Willott, 1996; Swerdlow et al., 2001), and gap-induced prepulse inhibition (Ison, 1982; Walton et al., 1997) have been studied for decades, many of the specifics of stimuli, hardware, technical, and analytic information related to tinnitus detection have not yet been solidified. For this reason, we have previously addressed some hardware and stimulus presentation issues (Longenecker and Galazyuk, 2012), as well an in-depth analysis of the startle response (Grimsley et al., 2015). Although these initial steps have improved the confidence of GPIAS assessments, the finer details concerning data collection and data analysis, as they specifically relate to tinnitus assessment, need further attention.

GPIAS studies have largely neglected to provide details on data collection when assessing tinnitus in laboratory animals. However, many aspects of these approaches can dramatically affect conclusions of GPIAS experiments. This is especially true for animals that have high startle reflex variability (Berger et al., 2013; Longenecker and Galazyuk, 2016; Salloum et al., 2016). Limiting ASR variability is critical in a repeated measure designs which compare an animal’s performance before and after a tinnitus-inducing experimental treatment. Several factors influencing the ASR should be considered in order to control the inherent variability. These include issues that pertain to inter-trial intervals (ITI) (Ison and Hammond, 1971; Leitner et al., 1993; Willott and Carlson, 1995; Plappert et al., 2004), circadian rhythm (Chabot and Taylor, 1992a; Chabot and Taylor, 1992b), sex differences (Plappert et al., 2005; Koch, 1998), and sensory adaptation. A change to any of these factors can alter the startle response magnitude and startle response variability. These issues could be magnified when assessing the gap detection abilities in noise exposed animals, due to a suppressive effect of acoustic over-exposure on startle magnitude (Longenecker and Galazyuk, 2011; Lobarinas et al., 2013). Standardizing data collection efforts could decrease GPIAS data variability between animals, experiments, and lab groups.

Detailed descriptions of GPIAS tinnitus analyses are also under-described in the literature. One of the major confounds in GPIAS data analysis is gap-induced facilitation. A gap in a continuous background noise usually inhibits or reduces the startle response magnitude (Stitt et al., 1973; Ison, 1982). However, this is not always the case. Depending on stimulus conditions, a prepulse, gap, or alterations of the background noise can alternatively act as a facilitator of the startle reflex in mice (Plappert et al., 2004; Willott and Carlson, 1995), rats (Stitt et al., 1974; Ison et al., 1997), and humans (Aasen et al., 2005). While it is not fully understood why this dichotomy exists, both responses represent real sensory gating phenomenon but likely have separate complimentary biological circuit (Schmajuk and Larrauri, 2005). Thus, GPIAS assessments should be cognizant of this issue when examining “how well” an animal can detect a gap. If not appropriately addressed during data analysis, facilitation can limit the effectiveness of GPIAS as a tool to assess gap detection performance because of unnecessary variability. The exact details of data processing can also significantly affect GPIAS data interpretation.

Unfortunately, this part of data analysis is typically neglected in method sections of the relevant papers. Nevertheless, details such as which data points are included/excluded, how many testing sessions/days were used in the control and experimental conditions, how ratios were calculated and/or averaged together, are critical to draw defendable conclusions based on GPIAS data.

Our data indicate that ITI, circadian cycle, sex, and sensory adaptation all play roles in the degree of variance present in GPIAS experiments. Our data also suggest that particular data analyses are critical to minimize the effect of variance in GPIAS data. Taken together, these findings suggest that the GPIAS method is more complicated than previously described, but can provide accurate assessments of gap detection performance.

2. Materials and Methods

2.1. Subjects

A total of 62 CBA/CaJ mice were used in this study. Group A contained fourteen male mice which were used in the majority of experiments. Group B contained an additional 48 mice (24 males and 24 females) which were used in the sex-based variation experiments. In group A, mice were divided into two groups of seven and were housed in separate rooms: seven of these mice were housed in a regular light dark cycle (lights on 10 a.m. to 10 p.m.) (inactive mice) and other seven were kept in a reverse light dark cycle (lights off 10 a.m. to 10 p.m.) (active mice). Group B mice were housed in a regular 12-h light–dark cycle. Mice were obtained from Jackson Laboratories and were approximately 10 weeks old. Mice were housed in pairs within a colony room at 25°C. Procedures used in this study were approved by the Institutional Animal Care and Use Committee at the Northeast Ohio Medical University.

2.2. Study design

Group A underwent baseline GPIAS testing for 21 days (one session per day) with 1-2 days between sessions. We tested seven inactive Group A mice using long ITIs during three sessions (ITI details are described in section 3.2). Following baseline assessment, all animals were sound exposed to induce tinnitus, as described below. Three and five months post exposure, behavioral evidence of tinnitus was assessed. Mice in Group B were not sound exposed and were tested for 10 sessions in 21days for sex differences.

2.3. Acoustic trauma

Mice were at least five months old at the time of sound exposure. Mice were anesthetized with an intramuscular injection of a ketamine/xylazine mixture (100/10 mg/kg). An additional injection (50% of the initial dose) was given intramuscularly 30 min after the initial injection. Mice were exposed to a one octave band noise centered at 12.5 kHz (~8-17 kHz) unilaterally for one hour. This noise was generated using a waveform generator (Wavetek model 395), amplified (Sherwood RX-4109) to 116 dB SPL, and played through a loudspeaker (Fostex FT17H). The output of the loudspeaker was calibrated with a 0.25-in. microphone (Brüel and Kjaer 4135) and found to be ±4 dB between 10 and 60 kHz. The left external ear canal was obstructed with a cotton plug and a Kwik-Sil silicone elastomer plug (World Precision Instruments), a manipulation which reduces sound levels by 30 to 50 dB SPL (Turner et al., 2006; Ropp et al., 2014).

2.4. Acoustic startle hardware/software

The equipment used to collect all acoustic startle reflex (ASR) data has been described in detail previously (Longenecker and Galazyuk, 2012). Briefly, commercial hardware/software equipment from Kinder Scientific, Inc. was used. Each behavioral testing station was lined with anechoic foam to prevent sound reflection and wave cancelling sound echoes (Sonex foam from Pinta Acoustics). Mice restrainers were open walled to allow for maximum sound penetration (Fig. 3 in Longenecker and Galazyuk, 2012). Background sound levels within each testing chamber were calibrated with a 0.25-in. microphone (Brüel and Kjaer 4135) attached to a measuring amplifier (Brüel and Kjaer 2525) and found to be less than 40 dB SPL between 4 and 40 kHz. Startle waveforms were recorded using load-cell platforms which measure actual force changes during an animal’s startle. Each load cell was calibrated with a 100g weight, corresponding approximately to 0.98 newton of force in standard gravity.

2.5. Startle waveform identification and measurement

All waveforms collected during testing sessions were analyzed offline using a recently developed automatic method of startle waveform identification via a template matching paradigm (Grimsley et al., 2015). In that study we used high-speed video recordings (1,000 frames/s) to visualize the animal’s startles in order to identify stereotyped waveforms associated with a startle. This allowed us to develop custom software which automatically separates data into either startles or non-startle-related movements. Based on this separation, we only included trials that resulted in successful startle responses in our data analysis. We also used a mathematical approach to convert the force generated on a load cell plate to center of mass displacement (in mm) (Grimsley et al., 2015).

2.6. GPIAS data collection

The ability of mice to detect a gap of silence preceding the startle stimulus was determined by comparing the startle magnitude in response to a startle stimulus presented alone (No-Gap) and a startle stimulus paired with a gap (Gap). Both trial types had continuous background noise. The gap had a 20 ms duration and 1 ms rise/fall time. Background for all these trials was presented as a narrow band (1/3 octave) noise centered at six different frequencies (8, 10, 12.5, 16, 20, and 25 kHz). This background noise level was constant (55 dB SPL) throughout each session. The startle stimulus was presented at 105 dB SPL (white noise, 1 ms rise/fall, 20 ms duration). The gap was presented 100 ms before (onset to onset) the startle stimulus. The testing session started with an acclimation period lasting 5 min. All testing sessions were conducted in a lighted behavioral box, regardless of which circadian cycle the animal was on, to avoid introducing light as an additional variable.

During a testing session, Gap and No-Gap trials for the same background frequency were presented in blocks. Each block contained 5 Gap trials and 5 No-Gap trials presented for a given frequency in pseudo-random fashion. After all six background frequencies were completed, this stimulus sequence was repeated eight more times. Thus, during the entire testing session 45 (9×5=45) Gap and 45 No-Gap trials were presented for each of six background frequencies.

For best performance analysis (see Results), the total number of trials and testing parameters were identical to the above description. A Segment was defined as 6 background frequency blocks repeated 3 times (18 blocks total). Thus, for the purpose of best performance data analysis, each session was subdivided into 3 Segments.

2.7. Inter-trial interval

Two inter-trial interval (ITI) arrangements were used in this study. Some experiments utilized ITIs that were randomized between 4, 5 and 6 seconds, whereas other used ITIs randomized between 7, 8, 9 …, and 15 seconds. The short and long ITI sessions had the same number of trials and lasted about one and three hours, respectively. See Results to determine which arrangement was used for what experimental design.

2.8. Estrus cycle assessment in female mice

To test whether the estrus cycle in female mice has an effect on their GPIAS performance we visually assessed their external genitalia before each experiment (Byers et al., 2012).

2.9. GPIAS data and statistical analyses

Data obtained in GPIAS testing sessions were analyzed offline using several methodological approaches (discussed below, along with the benefits and limitations of each method). All of our analyses concern different strategies to manage GPIAS data directly following individual startle identification and separation.

We used nonparametric Kolmogorov–Smirnov two-sample test (K–S test) to determine whether two different absolute startle magnitude distributions were statistically different or if they could be considered to be from the same distribution. The K–S test statistic D is calculated by finding the maximum absolute vertical distance between the two distribution functions. We choose the Fligner–Killeen test of homogeneity of variances because it does not depend on the shape of the data distribution. To account for multiple comparisons p-values were adjusted using the Benjamini and Hochberg’s false discovery rate (FDR) procedure (see Table 1). Unequal variance t-tests were applied to ranked ratios to evaluate the effect of noise exposure on gap detection at each background frequency. All data analyses and statistical tests were carried out in R (R Core Team, 2016).

Table 1.

Startle magnitude and startle magnitude ratio variances.

Figure #	Variable	Comparisons	Variances	p-value
Figure 1 E	Startle magnitude	A vs B A vs C A vs D B vs C B vs D C vs D	2.253 vs 3.493 2.252 vs 1.166 2.252 vs 0.997 3.493 vs 1.166 3.493 vs 0.997 1.166 vs 0.997	<0.001* <0.001* <0.001* <0.001* <0.001* <0.001*
Figure 2 B	Startle magnitude	Long vs Short	2.881 vs 2.004	<0.001*
Figure 3 B	Startle magnitude	Inactive vs Active	1.973 vs 2.908	<0.001*
Figure 3 C	Startle magnitude	1-5 inactive vs 6-10 inactive 1-5 inactive vs 11-15 inactive 1-5 inactive vs 16-21 inactive 6-10 inactive vs 11-15inactive 6-10 inactive vs 16-21 inactive 11-15 inactive vs 16-21 inactive 1-5 active vs 6-10 active 1-5 active vs 11-15 active 1-5 active vs 16-21 active 6-10 active vs 11-15 active 6-10 active vs 16-21 active 11-15 active vs 16-21 active 6-10 active vs 11-15 active 6-10 active vs 16-21 active 11-15 active vs 16-21 active	2.965 vs 1.725 2.965 vs 0.970 2.965 vs 1.340 1.725 vs 0.970 1.725 vs 1.340 0.970 vs 1.340 4.163 vs 2.267 4.163 vs 2.160 4.163 vs 2.098 2.267 vs 2.160 2.267 vs 2.098 2.160 vs 2.098 2.267 vs 2.160 2.267 vs 2.098 2.160 vs 2.098	<0.001* <0.001* <0.001* <0.001* <0.001* <0.001* <0.001* <0.001* <0.001* 0.001* 0.027* 0.213 0.001* 0.027* 0.213
Figure 4 B	Startle magnitude	Male vs Female	3.138 vs 5.116	<0.001*
Figure 4 C	Startle magnitude	P vs E P vs M P vs D E vs M E vs D M vs D	5.394 vs 3.755 5.394 vs 5.803 5.394 vs 4.406 3.755 vs 5.803 3.755 vs 4.406 5.803 vs 4.406	<0.001* <0.001* <0.001* <0.001* <0.001* <0.001*
Figure 7 E	Ratio	Standard ratio vs Modified ratio Standard ratio vs Best performance Standard ratio vs Grand ratio Modified ratio vs Best performance Modified ratio vs Grand ratio Best performance vs Grand ratio	0.269 vs 0.033 0.269 vs 0.149 0.269 vs 0.022 0.033 vs 0.149 0.033 vs 0.022 0.149 vs 0.022	<0.001* 0.549 0.040* 0.003* 0.303 0.040*
Figure 9 E (baseline)	Ratio	Standard ratio vs Modified ratio Standard ratio vs Best performance Standard ratio vs Grand ratio Modified ratio vs Best performance Modified ratio vs Grand ratio Best performance vs Grand ratio	0.507 vs 0.051 0.507 vs 0.209 0.507 vs 0.05 0.051 vs 0.209 0.051 vs 0.05 0.209 vs 0.05	<0.001* 0.002* <0.001* <0.001* 0.230 <0.001*

Pairwise variance comparisons were conducted with the Fligner–Killeen test of homogeneity of variances. Benjamini and Hochberg’s FDR procedure was used to adjust the p-values where multiple comparisons were made.

3. Results

Data collection

Mice are known to have a highly variable ASR. To minimize the impact of this variability on experimental results, it is beneficial to identify experimental manipulations which might contribute to this variability. Mice were tested over multiple sessions to assess the amount of ASR variability caused by: ITI, circadian rhythm, sex differences, and sensory adaptation.

3.1 Inherent startle magnitude variance in CBA/CaJ mice

To determine the level of ASR variability in mice we first examined the startle variability in 14 individual mice during 21 individual testing sessions (Fig. 1). Four distinct types of ASR behavior were observed: constant fluctuation (Fig. 1A), state change (Fig. 1B), habituation (Fig. 1C), and minimal change (Fig. 1D). The amount of ASR variance seen in each of these types was dramatically different ranging from 3.5 mm² in “state change” to 1 mm² in “minimal change” (Fig. 1E; Table 1). These patterns of behavior were observed in all mice over 21 days of testing but some were observed in far greater proportions. The constant fluctuation type represented 75% of all sessions, making this pattern the most common (Fig. 1F). The other three types were observed in 15% or fewer of sessions (Fig. 1F). All mice demonstrated a sampling of each type of ASR behaviors, which importantly demonstrates that these effects were not mouse specific. Typically an individual mouse exhibited different ASR behavior from one session to another. Interestingly, within-session habituation (Fig. 1C) is a highly conserved ASR behavior in rats (Davis, 1984), whereas it was only observed in about 5% of all sessions in CBA/CaJ mice in this study (Fig. 1F).

Figure 1. Within Session ASR variability.

A–D. Four representative examples of ASR variability plotted as absolute startle magnitude (mm) for Gap (grey) and No-gap (black) trials within one testing session. A. Constant Flux: rapid and continual ASR fluctuations. B. State Change: distinct epochs of elevated or depressed ASR. C. Habituation: ASR declines as a function of trial number. D. Minimal Change: minimal ASR fluctuations. E. Variance (mm²) of each type of ASR behavior as shown in A–D. F. Occurrence (% of all sessions analyzed; 294 sessions total from 14 Group A mice) of each type of ASR behavior.

3.2 Inter-trial interval variation

Here we examined how ITI affects ASR magnitude and probability in two groups of mice (7 animals in each group). Two-sample Kolmogorov–Smirnov test indicated statistically significant difference between startle magnitudes in short (4 to 6 sec) versus long (7 to 15 sec) ITI trials (D = 0.18, p < 0.001; Fig. 2A). Although longer ITIs induce a greater startle reflex magnitude, this also increases the dynamic range of startle and thus increases the variance compared to shorter ITIs (Fig. 2B; Table 1). The probability that a valid startle was recorded was significantly higher in the long ITI condition compared to the short ITI condition (χ² = 281.88, p < 0.001; Fig. 2C).

Figure 2. ASR changes resulting from inter-trial interval.

A. Absolute startle magnitude (mm) for long and short ITI GPIAS trials (seven mice in each group) across three GPIAS sessions. Error bars represent SEM. B. Startle magnitude variance (mm²) for long and short ITI GPIAS trials. C. Startle probability (% total trials) for long and short ITI GPIAS trials. ** represents statistical significance at p ≤ 0.001.

3.3 Circadian rhythm variation

The effect of circadian cycles on ASR magnitude was assessed for 14 mice across 21 testing sessions (Fig. 3). Mice housed in a reverse light dark cycle (active mice, N=7) demonstrated higher magnitude ASR than regular light dark cycle mice (inactive mice, N=7) (Fig. 3A). Both Gap and No-Gap startle magnitudes were significantly different between inactive and active mice (Gap trials: D = 0.12, p < 0.001; No-gap trials: D = 0.11, p < 0.001). The startle magnitude variance was also greater in the active mice, (Fig. 3B; Table 1) with clear fluctuations in startle reflex magnitude across sessions (Fig. 3A). Interestingly, both groups had nearly identical initial ASR levels, but quickly differentiated on the second session of testing with the inactive mice having dramatically lower ASR compared to active mice. The magnitude and associated variance for both groups was subject to long term habituation. For both groups the most dramatic changes were observed during the first 5 sessions of testing (Fig. 3C; Table 1).

Figure 3. ASR changes resulting from circadian cycle.

A. Absolute startle magnitude (mm) of both Gap (dashed lines) and No-gap- (solid lines) of active (grey) and inactive (black) mice (7 mice in each group) across 21 GPIAS sessions. Error bars represent SEM. B. Startle magnitude variance (mm²) for inactive and active mice. C. Startle magnitude variance (mm²) for binned sessions across the 21 session experiment for both inactive and active mice.

3.4 Variation due to sex differences

Previous studies have shown that startle magnitudes can fluctuate throughout the estrous cycle in female mice (Plappert et al., 2005) and rats (Koch, 1998). Using long ITIs, ASRs were assessed across 10 sessions in 24 male and 24 female mice (Fig 4A). A two-sample Kolmogorov–Smirnov test indicated that both Gap and No-Gap startle magnitudes were significantly different between male and female mice (Gap trials: D = 0.11, p < 0.001; No-Gap trials: D = 0.12, p < 0.001). There was a significant difference in weight between males (M = 34.05, SD = 3.64) and females (M = 24.00, SD = 4.03); t(433.89) = −31.46, p < 0.001. A one-way Analysis of Covariance (ANCOVA) was applied to compare male and female startle magnitudes, controlling for weight. Both weight and sex had a significant effect on startle magnitudes (p < 0.001); startle magnitude decreased with increasing weight. After adjusting for the effect of weight, post-hoc tests showed significant differences in startle magnitudes between males (M = 2.68, SE = 0.11) and females (M = 2.07, SE = 0.11); estimated adjusted mean difference 0.61 mm, p = 0.001. Using the Grand ratio method (Fig. 7) we found that females mice showed slightly but significantly better GPIAS compared to males: males (M = 0.94, SD = 0.34), females (M = 0.87, SD = 0.29); t(2792) = −5.42, p < 0.001.

Figure 4. ASR changes relating to sex variation.

A. Absolute startle magnitude (mm) of both Gap (dashed lines) and No-gap (solid lines) of male (black) and female (grey) mice (24 mice in each group) across ten GPIAS sessions. Error bars represent SEM. B. Startle magnitude variance (mm²) for male and female mice. C. Startle magnitude variance (mm²) for various stages of the estrus cycle for female mice.

Figure 7. Five different analyses for GPIAS dataset.

Data from a single GPIAS session obtained from an individual mouse, analyzed with five different methods. A–D. Results of four different methods of data analysis utilizing ratios. Dotted black horizontal line represents ratios of one where the gap detection was not evident. Error bars represent SEM. E. Corresponding variance of ratios shown in A–D. F. Cumulative probability of absolute startle magnitudes (mm) in response to Gap (green) and No-gap (pink) trials. Separate panels show data from six background frequencies (8, 10, 12.5, 16, 20, and 25 kHz).

Females had far more startle magnitude variance than males (Fig. 4B; Table 1), which in part can be explained by the shifting variance between parts of the estrus cycle, which was assessed by visual observation of the external genitalia before each experiment (Byers et al., 2012). The startle magnitude variance was relatively higher in the metestrus and proestrus stages (Fig. 4C; Table 1). However, the female mice in this study did not spend equal time in each cycle, but far more time in diestrus (supplementary Fig. 1), which had lower variance.

3.5 Sensory adaptation

Data collection under the GPIAS paradigm can be confounded by sensory adaptation, a sensory learning process (Crofton et al., 1990). To test whether sensory adaptation occurs in CBA/CaJ mice, we tested 14 mice over 21 sessions in a span of 50 days. Both the grand ratio and modified ratio methods (see section 3.7 and 3.8 for method description) revealed little changes in ratios between sessions (Fig. 5) indicating that gap detection performance in CBA/CaJ mice was not a subject of sensory adaptation. The GPIAS ratios only slightly fluctuated between sessions from 0.86 to 0.93. Solutions to the potential confound in GPIAS data caused by sensory adaptation are presented in the data analysis section below.

Figure 5. Gap-induced inhibition changes related to sensory adaptation.

GPIAS ratio (Grand ratio and Modified ratio analysis) assessed across 21 sessions in 14 mice. Error bars represent SEM.

Data analysis

Here we address issues of data analysis as they relate specifically to variance in gap detection assessments. First, we demonstrate that gap-induced facilitation of the acoustic startle reflex is a noteworthy phenomenon in CBA/CaJ mice, and that this facilitation can significantly change the results of the study. Second, we analyze a single dataset from a normal mouse with several different data processing strategies. Third, we show that sample size is an important factor for the reliability of GPIAS data. Finally, we show that the various data processing strategies result in different degrees of gap detection which can impact evaluations of gap detection deficits associated with tinnitus.

3.6 Gap-induced Inhibition and facilitation

Throughout each testing session we observed blocks of inhibition, in which a mean of Gap trials was smaller than a mean of No-Gap trials, and blocks of facilitation, where a mean of Gap trials was higher than a mean of No-Gap trials (Fig. 6A). In CBA/CaJ mice, the occurrence of gap-induced facilitation is not a rare occurrence. To test for a relative contribution of inhibition and facilitation a mean was calculated from all Gap values and a mean from all No-Gap values at every background frequency for each Group A mouse (N=14) in each 21 baseline testing session. Then, daily ratios (for each background frequency, for each mouse) were obtained by dividing those means (Gap/No-Gap); x-axis in Figure 6B. Each cumulative probability curve shows a separate baseline testing session (21 curves). About 70% of Gap/No-Gap ratios resulted in gap-induced inhibition (ratios < 1) and about 30% of ratios resulted in gap-induced facilitation (ratios > 1). Similar proportions of facilitation and inhibition at each background frequency were observed before and after sound exposure (supplementary Fig. 2). Neither circadian cycle nor sex affected these proportions (supplementary Fig. 3).

Figure 6. Contributions of gap-induced inhibition and gap-induced facilitation.

A. Absolute startle magnitude (mm) of Gap (orange) and No-gap trials (black) within a segment GPIAS session. Blue box highlights the blocks with inhibition (mean of Gap trials < mean No-Gap trials) and the orange box highlights the blocks with facilitation (mean of Gap trials > mean No-Gap trials). B. Cumulative probability of daily GPIAS ratios (Gap/No-gap) across all 21 sessions from 14 mice. Solid vertical line indicates the ratio one where the gap detection was not evident. Values below one represent gap-induced inhibition, whereas values above one gap-induced facilitation. Vertical black lines indicate borders of blocks used for data collection.

3.7 Options for GPIAS analysis

Here we present five unique methods for GPIAS data analyses, which include: standard ratio, modified ratio, best performance, grand ratio, and cumulative probability (Fig. 7; Fig. 9). Each of these methods processes GPIAS data differently. Typically scientists collect GPIAS data by blocks where several Gap and No-Gap trials are presented randomly or pseudo-randomly at a given background frequency (Galazyuk and Hébert, 2015). Therefore the first three methods take advantage of block data collection for data analysis. Other two methods do not consider blocks and analyze all the data from each testing session as a whole. We demonstrate how each analysis results in different outcomes (Fig. 7) when using the same dataset shown in Figure 1A (one mouse; one session). In the discussion we suggest some benefits and limitations of each method which should be considered when planning a GPIAS experiment (Table 2).

Figure 9. GPIAS data analyses for tinnitus assessment.

GPIAS data combined from ten baseline sessions and 5 months after sound exposure in a representative mouse exhibiting a gap detection deficit at 25 kHz. A–D. GPIAS baseline ratios (colored) and after (black) sound exposure obtained with four different ratio-based data analyses. Stars indicate the frequencies where a significant gap detection deficit was identified (p ≤ 0.05, unadjusted for multiple comparisons). E. Variances of ratios for different methods shown in A–D (color matches A–D analyses). F. Changes in gap detection performance caused by exposure assessed with the cumulative probability approach. Gap detection performance was reduced at 25 kHz following sound exposure. The Gap and No-gap distributions were significantly different at baseline (p < 0.001) but not after sound exposure (p = 0.75) indicating gap detection deficit at 25 kHz (outlined by a rectangle).

Table 2.

Major benefits and limitations of five different methods of GPIAS data analysis.

Analysis Type	Benefits	Limitations
Standard Ratio (blocks)	May reduce an impact of ASR fluctuations	Increased ratio due to averaging of blocks with inhibition and facilitation
Modified Ratio (blocks)	May reduce an impact of ASR fluctuations, considers both inhibition and facilitation as gap detection	Overestimates gap-induced inhibition. Little is understood about mechanism of facilitation
Best Performance Ratio (blocks)	May reduce an impact of ASR fluctuations, reduces the effect of facilitation	Smaller data set
Grand Ratio	Does not depend on data collection paradigm, all raw data are used	Increased ratio due to facilitation inclusion
Cumulative Probability	Includes all raw data, minimal data processing	Does not account for temporal ASR fluctuations or facilitation

Standard ratio (blocks)

Startle responses considered valid from the automated classifier were counted in each block. Blocks were considered valid if they contained at least 1 valid Gap and 1 valid No-Gap from the maximum of 5 Gaps and 5 No-Gaps. Then, we calculated a mean of all available Gap values and a mean of all No-Gap values. A single ratio (mean of Gap/mean of No-Gap) was calculated for each valid block. For each background frequency all available ratios (45 max) were averaged. Standard ratio calculations included and averaged both the blocks showing inhibition (ratios <1) and those showing facilitation (ratios >1). The contribution of facilitation led to variable and markedly high ratio values ranging from ~0.8 to ~1.5 (Fig. 7A). Unsurprisingly, the associated variance was 0.269 (Fig. 7E), a value much higher than other types of analyses (Table 1).

Modified ratio (blocks)

This type of GPIAS data analysis is a modification of the standard ratio. The major difference between these two methods lays in the way of how gap-induced facilitation is treated. This method considers both gap-induced inhibition and gap-induced facilitation as an indication of gap detection. Both are expressed using the same scale (ranging from 0 to 1) and averaged together. To accomplish this, we divided the Gap by No-Gap for the blocks showing inhibition (mean of Gap < mean of No-Gap) and No-Gap by Gap for the blocks showing facilitation (mean of Gap > mean of No-Gap). The resulting ratios for this type of analysis ranged between ~0.6 and ~0.8 across different background frequencies with relatively small standard errors (Fig. 7B). This analysis provides a very low variance of 0.032 (Fig. 7E).

Best performance ratio (blocks)

This is another variation of the block analysis. As we described above, our testing sessions were organized in blocks. These blocks were repeated nine times for a given frequency, which can be further subdivided into three identical Segments. Thus every background frequency was tested with 3 identical blocks per segment and every testing session contained three identical segments. Such subdivision of the testing session allowed assessment of gap detection performance in each of these three segments independently. The best performance method compared the gap detection performance in an individual mouse across three segments. The lowest ratio value (mean of Gap/mean of No-Gap) or best performance was chosen among the three segments (Fig. 7C). This approach approximates those used in sporting contests in which the best performance out of several attempts is recorded. Similar to the modified ratio, this method minimizes the contribution of facilitation to the overall gap detection performance assessment. The values obtained in this analysis can have similar ranges of ratios as standard (Fig. 7A) or grand ratio (Fig. 7D) within a day (Fig. 7C). For example, in Figure 7, ratios ranged from ~0.7 to ~1.3. Variance for this example was 0.148 (Fig. 7E).

Grand ratio

This method does not take blocks into consideration. Instead, the mean of all values for Gap trials and the mean of No-Gap trials were calculated across an entire session for each background frequency. Then, one ratio was determined by dividing the mean of the Gap trials by the mean of the No-Gap trials for each background frequency tested. No standard errors can be calculated for Grand ratio because there is a single datum for an entire session. Additionally, by collapsing Gap and No-Gap trials into a single (mean) value, any differences in variability among the trial types (i.e., heteroscedasticity) is effectively ignored. Ratios in this dataset range from ~.8 to ~1.25 (Fig. 7D). The variance for this analysis was 0.022, which is low compared to other methods (Fig. 7E). However, this is not an accurate way of representing variance as it will be discussed below.

Cumulative probability analysis

This method avoids the use of both blocks and ratios. All trials resulting in a valid startle (Gap or No-Gap) were included into this analysis. Absolute startle magnitudes were plotted as cumulative probability functions for each background frequency. The Gap and No-Gap functions were then statistically compared with two-sample Kolmogorov–Smirnov tests. The associated D value determines the degree of difference between these two functions with the p values indicating the significance of this difference (Fig. 7F).

3.8 GPIAS sample size considerations

The GPIAS method was introduced to the field as a tool to assess tinnitus in laboratory animals (Turner et al., 2006). It was assumed that gap detection performance in the animals with tinnitus should be significantly reduced compared to baseline. Comparison of the performance before and after tinnitus induction in mice is challenging due to high variability from one session to another collected from the same mouse. Figure 8 A–D shows that the gap detection performance varies in a normal mouse (with no induction of tinnitus) across different testing sessions. These four sessions were selected from 21 sessions conducted to assess animal’s baseline gap detection. Such variability in gap detection was revealed when either the Modified ratio or Grand ratio methods were used for analysis. It is possible that this variability in gap detection performance is due to a low sample size collected during a single session. To test for this possibility we tested gap detection performance in this mouse when the sample size was increased by increasing the number of testing sessions (Fig. 8E). The number of valid blocks for each background frequency increased with the number of sessions combined for analysis. Modified and Grand ratio approaches were used here (Fig. 7). The first data set (one session) used in Figure 8E was the same as shown in Figure 8 A. Including an increasing number of sessions reduced the variance in ratios across different frequencies and also reduced standard error values. The combined data set from ten testing sessions made the resulting gap detection ratios more consistent across frequencies and reduced the standard errors (compare single sessions shown in Fig. 8 A-D with the combined session shown in F).

Figure 8. Increasing size of GPIAS data set improves accuracy of gap detection assessment.

GPIAS data analyzed with the Modified ratio and Grand ratio approaches, collected from a representative mouse in multiple testing sessions and analyzed individually (A–D; Modified ratio - black, Grand ratio - orange) or combined across multiple testing sessions (E). F for Modified ratio and (G). H for Grand ratio. For both methods of analysis the combining data from ten sessions reduces GPIAS ratio variability across different background frequencies and reduces SEM values.

3.9 GPIAS analyses for tinnitus assessment

As we showed above many factors can alter accurate GPIAS assessment which is crucial for identification of tinnitus-positive animals. For example, GPIAS performance in mice can vary due to ASR habituation (Fig. 3). To minimize this effect, we excluded the first five testing sessions from the baseline assessment analysis. We compared the results from ten combined GPIAS baseline sessions (sessions 6 - 15 were used with the results from five combined sessions after sound exposure. The same data set collected from a representative mouse was used for all five GPIAS data analysis methods shown in Figure 9. A frequency-specific gap detection deficit was evident by all five methods following sound exposure (Fig. 9). Ranked unequal variance t-tests were applied to assess the effect of acoustic trauma at each background frequency; we report p-values unadjusted for multiple comparisons. Standard ratios were significantly different at both 12.5 kHz (t (82.46) = −2.12, p = 0.04) and 25 kHz (t (91.39) = 2.27, p = 0.03; Figure 9A). Significant difference at 25 kHz was also found with modified (t (91.91) = 2.67, p = 0.009), best performance (t (26.62) = 2.04, p = 0.05) and grand ratios (t (12.66) = 2.25, p = 0.04; Figure 9B–D). This deficit at 25 kHz was also seen for the cumulative probability analysis: before noise exposure, Kolmogorov–Smirnov D-value was 0.18 (p < 0.001), changing to 0.09 (p = 0.75) after exposure – signifying that the Gap vs No-Gap separation was greater before exposure than after exposure. Such change was only observed at 25 kHz. Overall, these data suggest that gap detection deficits can be detected with several unique analyses.

4. Discussion

4.1 Current state of GPIAS for tinnitus assessment

Despite its popularity as a methodology for tinnitus assessment (Eggermont, 2013; Hayes et al., 2014; Galazyuk and Hebert, 2015), the GPIAS method is not well developed for several reasons. First, scientists who routinely use this method for tinnitus assessment rarely describe specific details of data collection and data analysis. Second, the major stimulus parameters used for GPIAS are not well justified. Criteria and statistical verification of the presence of tinnitus-like behavior, even those used for the same animal tinnitus model by different research groups, vary dramatically. Many of these details are critical and largely determine whether GPIAS can assess gap detection deficits in laboratory animals or in humans. Therefore, it is not surprising that some research groups have found that this methodology is able to assess tinnitus whereas other groups have questioned the validity of GPIAS for tinnitus assessment in animal models (Hickox and Liberman, 2014; Radziwon et al., 2015) and humans (Campolo et al., 2013; Fournier and Hebert, 2013). Our recent work attempted to shed light on some issues related to hardware and stimulus presentation issues (Longenecker and Galazyuk, 2012), as well as startle identification and startle magnitude measurement (Grimsley et al., 2015). While these initial steps increased our understanding of ASR and GPIAS complexity, the present study provides further in-depth analyses of GPIAS-related issues to help avoid obtaining inconclusive or even misleading results.

4.2 Considerations of data collection

The magnitude of the acoustic startle reflex fluctuates, and this fluctuation is especially pronounced in mice (Valsamis and Schmid, 2011). In order to explore and minimize the effect of these variations on GPIAS testing, we tested several parameters which have been known to modulate the startle reflex (Koch, 1999; Swerdlow et al., 2001). Here we show that inter-trial interval (Fig. 2), circadian stage (Fig. 3), sex (Fig. 4), and sensory adaptation (Fig. 5) contribute to ASR variability and affect GPIAS results. These results should be considered as descriptive, rather than hardline suggestions for how to conduct GPIAS experiments, especially when considering species differences.

4.2.1 Inherent variance of CBA/CaJ mice

To examine the degree of ASR variability, we tested mouse ASRs about three hundred of sessions and classified four types of behavior (Fig. 1). The most prevalent behavioral classification was the “constant flux.” This behavior was marked by continuously changing startle amplitudes for both No-Gap and Gap trials (Fig. 1A). While continual fluctuations of startle magnitudes would be expected to have high variance (Fig. 1E), interestingly “state change” showed even greater ASR variability. The “state change” classification, characterized by 10-20 minute periods of consistently elevated or decreased ASR, seems to represent distinct physiological or attentional state deviations within a testing session (Fig. 1B). These well-defined states should be further studied, perhaps with camera monitoring, corticosterone assessments, and possibly cortically implanted electrodes. It is likely that each animal’s emotional and/or attentional responses to the startle paradigm provide a basis for these “state changes.” It is well established that these changes, stemming from neural input from the amygdala and cortical centers, can modify the ASR magnitude in many species (Koch and Schnitzler, 1997; Grillon and Baas, 2003). Furthermore, attentional declines have been directly linked to restraint stress (Pérez et al., 2013), which is a necessary component of ASR testing in most rodents. Attentional and stress factors are likely contributors to large intra- and inter-session ASR variation. Regardless of the cause, highly variable ASR behavior (constant flux and state change; Fig. 1F) constitutes roughly 90% of all data collected in Group A mice, which implies that this phenomenon should not be ignored during GPIAS data collection, analysis, and interpretations.

Another finding from detailed analysis of startle behavior across a large data set was that within-session habituation, which occurs in nearly all mammalian species (Davis, 1984), does not seem to be a significant factor in CBA/CaJ mice (Longenecker et al., 2016). Clear habituation, as noted by a decrease in absolute startle magnitude from the beginning to the end of a testing session, was observed in only about 5% of all trials (Fig. 1C). This has positive implications for this model because it reduces concerns that the animal is responding with progressively weaker startle responses by the end of the testing session. Although within-session habituation seems to be uniquely underrepresented in this model (Fig. 1C), across-session habituation seems to be a universal phenomenon (Fig. 3; Fig. 4) (further discussed below). To determine whether ASR data are effected by habituation with a given animal model, a large number of trials should be collected within a single testing session. This will allow to assessment of whether the behavioral state of the animal changes over time (Fig.1). After such a careful review of the behavioral data, it would be easier to determine typical variations in ASR data for a given animal model and also to identify and if it is necessary to exclude “atypical” sessions or animals from further data analysis.

4.2.2 Inter-trial interval variation

Inter-trial interval, or the amount of time between startle eliciting trials, is an important factor for planning ASR or GPIAS experiments. This should not be confused with inter-stimulus interval (a.k.a. inter-pulse interval), which represents the time between a gap of silence and the startle stimulus (Ison and Hammond, 1971; Leitner et al., 1993; Willott and Carlson, 1995; Plappert et al., 2004). Here we assessed long vs. short ITIs to examine whether it affects ASR during GPIAS testing. We found that the GPIAS experiments could be conducted in CBA/CaJ mice using both long and short ITIs (Fig. 2). Resulting startle magnitudes were higher for the long ITIs compared to short ITIs (Fig. 2A). This has also been shown for rats (Davis, 1970). Reductions of the startle magnitude at shorter ITIs makes startle response identification and evaluation more challenging (Grimsley et al., 2015). Mice in our study show little habituation to the short ITIs (4-6 seconds) even during a session containing several hundred trials, which is consistent with previous reports (Bullock et al., 1997). Both the startle magnitude and startle probability decrease with shorter ITIs (Fig. 2C), which has been reported for mice (Longenecker et al., 2016) and humans (Prokasy and Ebel, 1967; Blumenthal and Berg, 1986; Blumenthal, 1996). A decreased ASR due to short ITIs could be explained by basal, non-neural factors, such as muscle fatigue or muscle refractory periods (Valsamis and Schmid, 2011). The reduction in startle response magnitude and startle response probability is important because it allows researchers to plan experiments based on predictable quantities of valid ASR trials. The apparent tradeoff in study design is between relative length of testing and the amount of valid startles. A quicker testing session with short ITIs reduces startle variability and provides more startle trials in a given amount of time, but reduces the number of valid startle responses. Perhaps the most important criterion in determining an appropriate ITI is the specific animal model. While we describe that CBA/CaJ mice startle rather consistently with very short ITIs, this might not be the case with other animal models. Therefore any startle-related work with a new animal model should probably begin with identification of optimal ITIs to minimize a possible effect of both the short-term and/or long-term habituation on data collection and interpretation.

4.2.3 Circadian variation

It is not surprising that ASR would be dramatically altered by testing at various epochs of the circadian cycle because nearly all modulatory neurotransmitters and hormones are subject to circadian modulation (Saper et al., 2005). Previous data in rats has shown that ASR magnitude is sensitive to a testing epoch (Chabot and Taylor, 1992a; Chabot and Taylor, 1992b). While the exact neural substrate of this modulation is still uncertain (Frankland and Ralph, 1995), an animal’s state of arousal is likely a major factor in this ASR modulation (Samuels et al., 2007). Our data confirm previous findings in rats (Horlington, 1970; Davis and Sollberger, 1971), by showing that mice tested during their active time (dark) demonstrate a greater ASR (Fig. 3A). Three distinct points highlight the importance of this data set. First, in both the initial testing (first 5 days) and later steady state phase, the active mice had a greater startle response (Fig. 3A). Second, the variance was also higher for the active mice compared to inactive mice across sessions (Fig. 3B, C). This suggests that using the inactive animals might provide more consistent results over the long term, even if some percentage of maximum startle magnitudes need to be sacrificed. Third, the amount of prepulse inhibition seems to be similar between groups. Because gap detection is thought to be reliant on higher level circuitry, including the cortex (Ison et al., 1991), this suggests that the neural substrates influencing circadian states are acting at levels at or below the caudal pontine reticular nucleus (Frankland and Ralph, 1995), the obligatory central output of the acoustic startle reflex (Koch, 1999). When considering this dataset, the most important suggestion for running ASR experiments is to test at a consistent time period each day, otherwise increases in startle variability would be expected. A more in depth study should be done to assess whether circadian cycles specifically impact prepulse inhibition.

4.2.4 Sex variation

It is well documented that female mice (Plappert et al., 2005), rats (Koch, 1998) and humans show estrus cycle-dependent fluctuations of startle magnitude. Here we observe that female mice have higher startle magnitudes compered to males, at least through the first several days of testing (Fig. 4A). This is largely due to lower body mass compared to their male cohorts. Towards the end of the experiment (10 testing sessions in total), however, both groups of animals reached a habituated ASR state and seemed to have similar startle magnitudes. This might imply that to compare sexes more precisely, the ASR should be habituated first. However, gap detection performance between sexes was significantly different but small, so using a ratio format for analyzing GPIAS testing would eliminate habituation concerns. Female mice also showed an increased variance (Fig. 4B), as might be expected with increased startle response magnitudes. However, increases in variability cannot be simply attributed to body mass differences. A similar phenomenon is also seen in humans, where the variability of the eye blink component of the startle response (indexed by recording EMG activity of the orbicularis oculi muscle; Aasen et al., 2005) was found to be more variable in females than males. With careful monitoring of the estrus cycle, we found that ASR variance differed across stages (Fig. 4C). These results did not match a previous study on mice in which ASR was tracked across the estrus cycle, however, that study used C57BL/6J and C3H mice (Plappert et al., 2005). In that study, the two strains differed in ASR-estrus modulation and thus inter-species differences should not be unexpected. It should be noted that mice in this study spent the majority of time in the diestrus phase (supplementary Fig. 1), which has also been shown previously (Byers et al., 2012). Fortunately, for ASR experiments, this part of the cycle seems to have the least variance. Importantly, even with the diestrus phase being the most common and associated with the least variance, male mice always had less ASR variance. This suggests that GPIAS studies should be planned with this variance difference in mind (Fig. 1). Estrogen levels can impact prepulse inhibition in mice (Charitidi et al., 2012), rats (Koch, 1998), and humans (Swerdlow et al., 1997; Van den Busse and Eikelis, 2001; Jovanovic et al., 2004). Thus, the estrus cycle should be taken into account during GPIAS experiments. While an equal representation between sexes is important for most disease models, excluding female animals, if possible, can lower session-to-session ASR variance. If both sexes need to be included, then data from female mice should be individualized, as the averaging between sexes or even between females could alter data due to fluctuations in estrous cycles.

4.2.5 Habituation and sensory adaptation

Long-term habituation of the ASR is a universal phenomenon across species and testing conditions (Davis, 1970; Davis and Gendelman, 1977; Pilz and Schnitzler, 1996; Pilz et al., 2014). We also found that CBA/CaJ mice, like other strains of mice (Bullock et al., 2007), exhibit long-term habituation, which was mainly evident during the first 4 to 5 testing sessions (Fig. 4; Fig. 5). This habituation should be taken into account during GPIAS studies for tinnitus assessment due to several reasons. First, for a repeated measures design in which the same animal is tested over several days, the ASR will decrease over sessions. If an animal is tested prior to an experimental manipulation (i.e. acoustic overexposure or salicylate treatment), it would be important to be certain that an animal’s ASR habituation has reached a steady state (Fig. 4; 5) (Jordan and Poore, 1998; Blaszczyk, 2003; Plappert and Pilz, 2005). If not, the conclusions of the experiment could be misinterpreted. Some recent publications have shown that after acoustic over-exposure the ASR magnitude drops (Longenecker and Galazyuk, 2011; Longenecker et al., 2014; Lobarinas et al., 2013; Salloum et al., 2016). However, this effect might partially be due to the habituation process rather than noise-induced hearing loss. Second, even if different groups of mice are used to compare a treatment vs. baseline effect via ASR assessments, the ASR habituation of these animals might differ between groups as a result of the treatment. Lastly, inter-animal, inter-strain, and inter-species differences of startle magnitude, startle habituation, and prepulse inhibition have often been reported (Bullock et al., 1997; Willott et al., 2003; Yee et al., 2005; Csomor et al., 2008). Thus, each animal, if possible, should be tested as an individual in a repeated measures design. This would ensure that all baseline and post treatment gap detection performances can be accurately assessed.

Another phenomenon that should be considered in GPIAS studies is sensory adaptation, a sensory learning process which can enhance prepulse inhibition over time (Crofton et al., 1990; Friedman et al., 2004; Plappert et al., 2006). This adaptation is plastic (Pietropaolo and Crusio, 2009) and developmentally regulated (Willott et al., 1994; Green et al., 2016). Our data indicate that mice (at least CBA/CaJ) show little signs of adaptation as revealed by both the Grand and Modified ratios (Fig. 5).

4.3 Data analysis considerations

4.3.1 Addressing facilitation in GPIAS research

Prepulse inhibition of the acoustic startle reflex is a well characterized phenomenon for many species (Hoffman and Ison, 1980; Koch, 1999). Prepulse facilitation is a far less studied phenomenon, but has been reported for mice (Willott and Carlson, 1995; Plappert et al., 2004), rats (Davis, 1974; Reijmers and Peeters, 1994; Ison et al., 1997), and humans (Blumenthal and Levey, 1989; Ludewig et al., 2003). Although sometimes contradictory among studies, several stimulus parameters that might trigger prepulse facilitation have been repeatedly reported. Inter stimulus intervals to study PPI (prepulse stimulus to startle stimulus) of 10 ms or below have been shown to cause prepulse facilitation (Willott and Carlson, 1995; Hoffman and Ison, 1980; Ison and Hammond, 1971). Intense background noise (70 dB SPL and above) was also shown to contribute to the gap-induced facilitation (Hoffman and Searle, 1965; Davis, 1974; Gerrard and Ison, 1990). Neither short inter stimulus intervals nor loud background noise could explain the facilitation in the present study because we used much longer intervals (100 ms) and very mild (55 dB SPL) background narrowband noise.

Perhaps the most reasonable explanation for the occurrence of facilitation in our study (Fig. 6) is that the background noise varied in frequency every few minutes within the construct of the block format. Some previous works have suggested that changes in background frequency can lead to facilitation of the startle (Stitt et al., 1974; Gerrard and Ison, 1990). Indeed, some data collected in this study seemed to have gap-induced inhibitory or facilitatory periodicity (Fig. 1; Fig. 6A), although no specific background frequency was more likely to produce facilitatory GPIAS responses (supplementary Fig. 2). Another reasonable explanation, is that the facilitation may occur as a product of stochastic fluctuations in startle magnitude. These fluctuations when it compared to the control startle value may result in inhibition or facilitation.

A few points could explain why our data set consisted of roughly 25% facilitation (Fig. 6B). The vast majority of ASR studies have been conducted on rats. While there are not many direct ASR comparisons between mice and rats, it seems that under most stimulus parameters, rats demonstrate robust inhibition (see Davis, 1984), whereas mice show a mixture of inhibition and facilitation (Plappert et al., 2004; Kraus et al., 2011; Hickox and Liberman, 2014). The issue is further complicated in mice by inter-strain variability (Plappert et al., 2004). Mice don’t habituate to an acoustic startle within a session very often (Bullock et al., 1997; Fig. 2C). This necessarily implies that the ASR in mice has some fundamental differences in its underlying neural circuity, and it would not be surprising for different circuitry to produce more facilitation compared to other laboratory animals. Lastly, and perhaps most importantly, data presented here is the first to show gap-induced facilitation as raw data on a block to block basis (Fig. 1; Fig. 6A), rather than an average over a testing session (e.g. Willott and Carlson, 1995; Plappert et al., 2004). Decades of study have shown that inhibition and facilitation are two independent processes (Hoffman and Wible, 1969; Stitt et al., 1974; Plappert et al., 2004; Schmajuk and Larrauri, 2005). However, more research should be done to understand the significance and the underlying neuronal mechanism of facilitation.

The presence of facilitation can create a significant problem for GPIAS research. If facilitation was an isolated phenomenon, its removal could be easily justified. However, facilitation is quite common (Fig. 6B), and if not addressed appropriately with data analysis, facilitation can mask the effects of inhibition. Until optimal testing parameters are discovered which reduce the occurrence of gap-induced facilitation in mice, GPIAS assessments which aim to identify tinnitus should objectively address this issue through analytical means that are discussed below.

4.3.2 Block analysis

We clearly demonstrated that the ASR is inherently variable (Fig. 1), and this variability extends to fluctuations between gap-induced inhibition and facilitation (Fig. 6A). To minimize the effect of these fluctuations on GPIAS assessment we propose grouping trials into frequency blocks. Block data collection and analysis allows for assessment of inhibition and facilitation in a temporally sensitive manner. The block analysis would benefit from when the ASR and the gap-induced inhibition are stable throughout a session. Thus, the need for using this method would depend on the particular animal model. The present study used blocks containing 5 Gap and 5 No-Gap trials at each background frequency. Although block approach in data analysis might be beneficial, it is still unclear whether this arrangement is optimal. Further research is necessary to address this point.

4.3.3 GPIAS sample size

We showed that combining several testing sessions for gap detection assessment has an obvious benefit for GPIAS studies because it reduces data variability and standard errors (Fig. 8). Such approach increases GPIAS sensitivity or statistical power to identify even small changes in gap detection performance which often are critical for detecting changes associated with tinnitus (Fig. 9). The appropriate number of combined sessions would largely depend on the variability in the gap detection performance in the same animal across different testing sessions. Additionally, this variation may change before and after treatment. For species in which the variability is minimal and the effect of a treatment is considerable, data sets from fewer sessions may to be combined.

An alternative approach to increase the data set for GPIAS studies would be to increase the duration of a single testing session. Unfortunately, this option has obvious restrictions, as mice demonstrate a reduction in gap detection performance when a testing session lasts more than one hour (Longenecker and Galazyuk, 2012). This would also be true for an animal model such as rats, which demonstrate substantial habituation of the ASR (Koch, 1999).

4.3.4 From data processing to tinnitus assessments

Precise descriptions of GPIAS data analysis are underrepresented in the literature. In this study we proposed, tested and demonstrated benefits and limitations for 5 different methods. The majority of these methods utilize the traditional ration of raw gap ASR divided by the raw no-gap ASR data. These gap ratios are resistant to startle magnitude fluctuations which allows intra- and inter-animal normalization. Another recently published method used A^’ assessments to accomplish the same task while utilizing the entire data set (Allen and Luebke, 2017), a limitation of some of the ratio methods discussed below. Similarly to the A^’ method, the cumulative probability function is a novel way to represent both ASR and GPIAS data. It utilizes all raw data and thus differs from ratio methods which plot rations of varying degrees of raw datasets. While we are not declaring that one method is superior to another, the major benefits and limitations of each of these five approaches are listed in Table 2 while details are discussed below.

4.3.4.1 Standard ratio (blocks)

The standard ratio is the analysis (Fig. 7A) that incorporates the concept of blocks in the data set. This minimizes the effect of ASR fluctuations when Gap and No-Gap trials are changing together within a session (Fig. 1B). The ratios are calculated in individual blocks which encompass short epochs within a testing session. This method also requires very little data processing other than the definition of a valid block as discussed above. The limitation of this analysis is that it will have a much higher variance (Fig. 7E) due to the averaging of blocks exhibiting gap-induced facilitation and gap-induced inhibition.

4.3.4.2 Modified ratio (blocks)

While the modified ratio analysis (Fig. 7B) might not be the most traditional option, it does introduce a creative solution to assess gap detection performance using both gap-induced inhibition and facilitation (Fig. 6). The major assumption behind this method is that both inhibition and facilitation represent gap detection. It is difficult to use both these measures for gap assessment due to the fact that the scale for inhibition and facilitation ratios are fundamentally different. The ratio values for inhibition are ranging from 0 to 1, whereas for facilitation from 1 to ∞ (practically, not more than 10). When the GPIAS data using these two scales are combined, the gap-induced inhibition and gap detection are underestimated because inhibition and facilitation are cancelling each other. To address this problem we suggest expressing facilitation as a ratio of no-gap/gap. In this case both inhibition and facilitation values are ranging from 0 to 1. Thus, this method measures gap detection (both facilitation and inhibition) rather than gap-induced inhibition only. This analysis does not generate a false positive result, because if an animal shows poor gap detection, both the inhibition and facilitation ratios have values closer to 1 (Fig. 9). Additional assurance that this is a valid alternative method comes from the fact that the other four methods of GPIAS data analysis detect the same frequency-specific deficit in this example animal (Fig. 9).

4.3.4.3 Best performance ratio (blocks)

Best performance analysis (Fig. 7C) is also an untraditional block analysis approach. This method naturally minimizes the effect of facilitation by only selecting the highest amount of inhibition (the lowest ratio values) throughout a testing session. Thus, it also removes a great deal of variance from the data (Fig. 7E). On the other hand, this analysis has been applied to performance based behaviors as an indicator of the maximum sensitivity an individual can achieve (Rosen et al., 2012; Sarro and Sanes, 2014; Gay et al., 2014). Despite some benefits of this analysis, it does suffer from higher variance. Although all data from each session is used to assess the best performance, this method is considered only a subset of data the segment with the greatest inhibition, is used for statistical analysis, which leads to higher variance. This method can also be subject to facilitation if all portions of the session demonstrate gap-induced facilitation. Even so, this suggests that facilitation can be a stable psychoacoustic reaction to the background noise and/or gap.

4.3.4.4 Grand ratio

The grand ratio is the most straightforward approach for assessing GPIAS data by excluding both block analysis and any adjustments for facilitation (Fig. 7D). It is calculated by dividing the mean of all startle only No-Gap trials by the mean of all the Gap trials at a given frequency. This approach provides one ratio per frequency tested per session. The major downside of this method is that gap-induced facilitation shift the ratios to higher values which are often referred as an absence of gap detection. Additionally, by collapsing all Gap or No-Gap data from a session into a single (mean) value,) this method effectively ignores all of the potential sources of startle magnitude variability discussed above. The benefit of this method is that it uses the entire data set of valid startles. Depending on the animal model, if ASR fluctuations and gap-induced facilitation are minimal, this method could be a good choice to analyze GPIAS data.

4.3.4.5 Cumulative probability

The cumulative probability assessment (Fig. 7F) is an alternative approach for assessing gap-induced changes in the ASR. The benefits of this method are as follows: 1. All data are used and considered in the analysis. 2. No extra data processing is required. 3. This method would be highly comparable between animal models and different labs. However, the negatives of these analyses are that they do not consider the temporal fluctuations of the ASR (Fig. 1) or gap-induced facilitation. Both of these factors might alter the measurements of the actual gap detection performance. For animal models without large ASR fluctuations such as “constant flux” and “state change” categories seen in this study (Fig. 1), this method could be considered as the most appropriate.

Supplementary Material

1. Supplementary Figure 1. Estrus cycle stages.

Amount of sessions (% of total sessions; 240 total sessions) of all 24 female mice spent in each stage of the estrus cycle.

2. Supplementary Figure 2. Contribution of facilitation/inhibition before and after sound exposure.

A–B. Cumulative probability for GPIAS ratios (Gap/No-gap) across ten sessions from 14 mice before (A) and five sessions after (B) sound exposure. Each curve represents a different background frequency tested (8, 10, 12.5, 16, 20, and 25 kHz). Values below one represent gap-induced inhibition, whereas values above one gap-induced facilitation.

3. Supplementary Figure 3. Contribution of facilitation/inhibition for sex and circadian factors.

A. Cumulative probability for GPIAS ratios (Gap/No-gap) across 21 baseline sessions for mice in either the active (grey) or inactive (black) part of their circadian cycle. B. Cumulative probability for GPIAS ratios (Gap/No-gap) across 10 baseline sessions for female (grey) or male (black) mice. Each curve represents a different mouse Values below one represent gap-induced inhibition, whereas values above one gap-induced facilitation.

Highlights.

• The ASR is variable, and this variation can contribute to “messy” GPIAS data.

• Careful consideration of ITI, sex, circadian, and sensory adaptation factors can reduce ASR variation.

• Preceding gaps can inhibit and facilitate the startle response which needs to be appropriately addressed in data analysis.

• The benefits and limitations of several GPIAS analyses are considered for tinnitus assessment.

Abbreviations

PPI

prepulse inhibition

ASR

acoustic startle reflex

ITI

inter trail interval