Nonlinear temporal receptive fields of neurons in the dorsal cochlear nucleus

Abstract

Studies of the dorsal cochlear nucleus (DCN) have focused on spectral processing because of the complex spectral receptive fields of the DCN. However, temporal fluctuations in natural signals convey important information, including information about moving sound sources or movements of the external ear in animals like cats. Here, we investigate the temporal filtering properties of DCN principal neurons through the use of temporal weighting functions that allow flexible analysis of nonlinearities and time variation in temporal response properties. First-order temporal receptive fields derived from the neurons are sufficient to characterize their response properties to low-contrast (3-dB standard deviation) stimuli. Larger contrasts require the second-order terms. Allowing temporal variation of the parameters of the first-order model or adding a component representing refractoriness improves predictions by the model by relatively small amounts. The importance of second-order components of the model is shown through simulations of nonlinear envelope synchronization behavior across sound level. The temporal model can be combined with a spectral model to predict tuning to the speed and direction of moving sounds.

sensitivity to the amplitude modulation (or envelope) of sounds is important in the recognition and identification of natural sounds (Chi et al. 1999; Singh and Theunissen 2003). A well-studied class of sounds is speech, in which modulation is an essential component of the signal, corresponding to the phonemes and syllables (Drullman 1995; Rosen 1992). Modifying or smearing the modulation envelope leads to losses in speech intelligibility.

The neural representation of the amplitude envelope of sound has been studied extensively using modulation transfer functions (MTFs; reviewed by Joris et al. 2004). The MTF shows the gain of the neural response, meaning the amplitude of the modulation of discharge rate divided by the stimulus modulation, as a function of modulation frequency (Backoff and Clopton 1991; Joris and Smith 1998; Neuert et al. 2005; Rhode and Greenberg 1994; Zhao and Liang 1995). Although the MTF would be a complete description of linear neural responses, it is clear that modulation responses are not linear in auditory neurons such as those of the dorsal cochlear nucleus (DCN; Joris and Smith 1998; Zhao and Liang 1995) and inferior colliculus (IC; Krishna and Semple 2000). Thus the MTF does not provide a full description of modulation responses (Krebs et al. 2008; Zheng and Escabi 2008).

In DCN, some aspects of the nonlinear responses can be attributed to the recruitment of inhibitory interneurons at higher sound levels (Joris and Smith 1998; Nelken et al. 1997; Nelken and Young 1994; Zhou et al. 2012). However, there are two additional nonlinear aspects of brain-stem modulation responses. The first is the conversion from temporal-only representation of modulation in the CN to a mixed rate and temporal representation in the IC (Joris et al. 2004). The rate transformation can be explained by rectification followed by low-pass filtering at the chain of synapses between CN and IC (Nelson and Carney 2004), a model that also accounts for the generation of complex intermodulation responses to mistuned harmonics in IC neurons (Sinex 2008; Sinex et al. 2002). The second nonlinearity is in the degree of separability of responses to the temporal and spectral modulations in the stimulus. Separable responses can be written as a product of a function of time and a function of frequency, suggesting independent processing along the two dimensions. Spectrotemporal receptive field maps (STRFs) are separable in auditory nerve fibers (Kim and Young 1994) and CN neurons (Nelken et al. 1997) but increasingly inseparable in more central nuclei, for example in neurons that are differentially sensitive to an upsweeping or a downsweeping frequency component (Andoni and Pollak 2011; Qiu et al. 2003). The mechanisms of the transition from separable to inseparable are unknown.

Here, we show results using a nonlinear model of temporal responses to envelope modulation for neurons in DCN. The model is similar to reverse correlation (de Boer and de Jongh 1978; Møller 1973), with the stimulus envelope instead of the stimulus waveform serving as the input to the model. The stimulus is a broad-band noise with a frequency spectrum that decreases as 1/f, similar to many natural sounds. The response is modeled with weighting functions, similar to the model used previously for spectral sensitivity of auditory neurons (Bandyopadhyay et al. 2007; Reiss et al. 2007; Young and Calhoun 2005; Yu and Young 2000). This method has advantages over the use of sinusoidally amplitude-modulated (SAM) stimuli and MTFs in that it is possible to investigate low-order (2nd-order) nonlinearities in the response as well as the possibility that the modulation sensitivity of neurons is time-varying following stimulus onset. With this model, we show how a neuron with physiologically separate spectral and temporal sensitivities could gain an inseparable response, namely selectivity for the direction of movement of a spectral feature in the stimulus.

METHODS

Surgical procedures.

Experiments were conducted on 14 adult cats (3–4 kg) with infection-free ears and clear tympanic membranes. Animal-use protocols were approved by the Johns Hopkins Animal Care and Use Committee. Cats were tranquilized with xylazine (2 mg im) and anesthetized with ketamine (40 mg/kg im, supplemental dose: 15 mg/kg im). Atropine (0.1 mg im) was given to control mucous secretion. A tracheal tube was inserted. Cats were decerebrated by aspirating between the superior colliculus and thalamus, after which anesthesia was discontinued. Core body temperature was maintained at ∼38.5°C using a regulated heating blanket, and lactated Ringer solution was given intravenously to maintain fluid volume.

The DCN was exposed by opening the skull and dura above the cerebellum and aspirating the part of the cerebellum overlying the DCN. Recording electrodes were advanced into the DCN under visual control. Single neurons were isolated and recorded extracellularly using platinum-iridium microelectrodes. Action potentials from single neurons were detected with a Schmitt trigger, and spike times were recorded with a precision of 10 μs.

Experimental protocol.

Recordings were made in a sound-attenuating chamber. Acoustic stimuli were delivered to the ipsilateral ear via an electrostatic speaker coupled to a hollow ear bar. The speaker was calibrated in situ using a probe tube placed ∼2 mm from the eardrum. The calibration is essentially flat with fluctuations of <10 dB from 0.5 to 30 kHz.

Once a neuron was isolated, the best frequency (BF) was determined manually as the tone frequency to which a response was obtained at the lowest sound level. Rate vs. level functions were collected for BF tones and broad-band noise (0–50 kHz) by presenting 200-ms stimulus bursts (10-ms rise/fall times) once per second over an 80- to 100-dB range of sound levels in 1-dB steps. Type IV neurons were classified as having moderate spontaneous rates and BF tone rate-level functions with excitation at low sound levels and inhibition at high sound levels (Shofner and Young 1985). Tone response maps showed a large inhibitory area usually centered below BF but encroaching on BF and dominating responses to BF tones except near threshold. Type III neurons were classified as having excitatory (but possibly nonmonotonic) responses to tones and noise at all levels and inhibition only at frequencies away from BF. Neurons were considered to be in DCN if they were located along the electrode track before a jump or reversal in BF gradient, which indicates a transition from DCN to ventral cochlear nucleus. This paper describes data only from DCN type III and type IV neurons, which are the principal cells of the DCN (Rhode et al. 1983; Young 1980).

Stimuli.

Over the course of the experiments, three sets of random temporal shape (RTS) stimuli were used. Two of the three sets were broad-band stimuli, with a noise carrier, and the other was narrow band, having a BF-tone carrier. They were presented at 10–30 dB referenced to the RTS threshold.

The broad-band stimuli consisted of sums of tones spaced 1/64th octave apart over 6.125 octaves centered at 6 kHz. All of the tones had the same amplitude so that the underlying stimulus had a flat spectrum on a log-frequency scale, meaning an equal amount of power in filters for which bandwidths are a fixed fraction of the center frequency. This distribution of energy gives the stimuli a 1/f spectrum on a linear frequency scale. The unmodulated stimulus sounds like noise but is easily distinguished from white noise, presumably because of the spectral difference. The raw stimulus waveform, s_r(t) (Fig. 1A), was generated as the sum of tones described above with length T (0.042 or 0.05 s) and sampling rate F_s (100 kHz). The stimuli were created by modulating the raw waveform with a signal like that shown in Fig. 1B having a time resolution of δ = 1 ms (some data were taken with δ = 3 ms, but these are not discussed here). The modulating amplitudes in decibels (dB) were drawn from an approximately Gaussian distribution with varying standard deviation (SD) or contrast (SD = 3, 6, or 12 dB). The dB amplitudes in the series of modulator time bins can be represented as the stimulus vector s (Fig. 1B) with length M_T, equal to the number of time bins T/δ. One hundred and twenty-five such modulation waveforms were generated, and each was used to modulate the raw waveform. The stimulus set consisted of the 125 modulated stimuli and 5 unmodulated raw waveforms (equivalent to s = 0). To prevent clicks at stimulus onset, the 1st and the last 10 bins (10 ms) were onset and offset ramps and were unmodulated. The central 30 time bins were modulated. Each stimulus was generated from a different random carrier.

Fig. 1.

Stimulus generation. A: a raw unmodulated noise waveform [s_r(t)] constructed by adding equal amplitude (Ampl.) tones spaced 1/64th octave apart centered on 6 kHz spread over 6.125 octaves, each tone started at random phase. B: a sample temporal modulation (Mod.) waveform (solid line). The decibel amplitudes of the modulating waveform in each bin [of width time resolution (δ) = 1 ms] are drawn from an approximately Gaussian distribution with a certain SD (6 dB in this example) that defines the contrast. Transitions between time bins were smoothed with a Gaussian window (dashed line, SD = 0.1 ms). s_k, Stimulus vector; a_k(t), modulating waveform. C shows the waveform s_k(t) generated after imposing the random temporal modulation shape on the unmodulated waveform as in Eq. 1. D: the actual modulating signal s_k^′ derived from the waveform in C, as described in the text. Note that the amplitudes in the actual modulating shape are only slightly different from the originals in B.

The modulation vectors s_k were converted to modulating waveforms a_k(t) by upsampling to the sampling frequency (F_s = 100 kHz) of the raw waveform. Time-bin transitions were smoothed with a Gaussian function (SD = 0.1 ms) to control spectral splatter. The smoothed version of a_k(t) is plotted as a dashed line in Fig. 1B. The final kth stimulus, shown in Fig. 1C, was obtained by modulating s_r(t) with a_k(t) as in Eq. 1:

$$s_k(t)={10}^{a_k(t)/20}\times s_r(t).$$ (1)

There is an additional scaling factor, not included in Eq. 1, that was used to set the global maximum sample value across the whole stimulus to 1.

The original stimulus vector s_k does not necessarily accurately represent the modulation amplitudes because of spectral interactions and the presence of some modulations in the raw waveform itself. Thus s_k was replaced by s_k^′, which is estimated from the envelopes of the generated stimuli as follows. First, ξ_k(t) and ξ_r(t) were computed; these are the analytic signals of s_k(t) and the raw noise signal s_r(t), respectively. s_k^′(t) Was then computed from the ratio of the average envelopes of these analytic signals $\langle {\xi }_k\rangle /\langle {\xi }_r\rangle$, where the averages are taken over the modulation intervals δ. Figure 1D shows s_k^′(t), which is the upsampled version of the estimated amplitude modulation vector s_k^′. The corrected envelope amplitudes were used in the later analyses.

In a second set of stimuli, the procedure was the same, except the carrier s_r(t) was a tone at the BF of the neuron. In each of the stimuli, the carrier tone was started at a random phase.

Temporal processing models.

The first-order model for responses to the RTS stimuli is as follows:

$$r(t)=r_0(t)+{\displaystyle \sum _{j=0}^{t_{\max }}{w}_T(j)·s(t-j).}$$ (2)

r₀(t) Is the response to the all-0-dB (unmodulated) stimulus vector s = 0. This term captures rate adaptation and other time-varying effects not dependent on the stimulus modulation. w_T is the first-order temporal kernel, with units spikes per second per dB, estimated as described below. It describes the current response of the neuron to the past stimulus envelope. Note that this model ignores responses to spectral shape, which does not vary, and assumes that the response of the neuron is linearly related to the logarithm of stimulus energy. This formulation does not properly represent the situation of no stimulus, for which the dB value of s is undefined. Thus t_max must be limited in Eq. 2 to include only bins during the stimulus.

Equation 2 can be extended to form a second-order time-invariant model as in Eq. 3. M_T contains the interactions between the stimulus temporal fluctuations at different times.

$$r(t)=r_0(t)+{\displaystyle \sum _{j=0}^{t_{\max }}{w}_T(j)·s(t-j)+{\displaystyle \sum _{k=0}^{t_{\max }}{\displaystyle \sum _{j=0}^{t_{\max }}{M}_T(j,k)·s(t-j)·s(t-k).}}}$$ (3)

In Eqs. 2 and 3, the kernel w_T is the same for all response time bins t. A more general first-order model is obtained by allowing the temporal weighting function w_T to change over time:

$$r=r_0+Ls.$$ (4)

r Is a vector containing the peristimulus time histogram (PSTH) in response to a stimulus vector s containing the dB amplitudes in successive time bins, and r₀ is the PSTH in response to the all-0-dB (unmodulated) stimulus s = 0. L is a matrix of weights with each row containing the first-order weights in successive time bins. Stated another way, each row in L is a first-order temporal weight function like w_T for a different time bin. From causality, L must have nonzero elements only on or below the diagonal (lower triangular). This is the general linear case of a time-varying temporal weighting function; the second-order time-varying model was not studied because of the large number of parameters required.

Estimation of the temporal models.

Spike times in response to each of the 125 RTS stimuli in a set were recorded with multiple repetitions (14–80, mean ≈ 40) to obtain good estimates of PSTHs r(t) in 1-ms time bins. The PSTHs were smoothed with a Gaussian filter with SD = 0.2 ms. An estimate of the PSTH r₀(t) was obtained by averaging the responses to the all-0-dB stimuli. For each of the models (Eqs. 2, 3, or 4), the PSTHs for the 125 stimuli can be arranged into a set of simultaneous linear equations with the model parameters w_T(t), M_T(t,k), and L as unknowns. These sets of equations are overdetermined, given the data available, and can be solved for the parameters by the method of normal equations (Press et al. 2007).

A correction for the latency of the neuron was done by shifting the PSTHs in relation to the stimuli by an amount Δ, which was varied in steps of 0.1 ms over 0–6 ms; PSTHs were recomputed in 0.1-ms steps to accomplish this. The Δ that gave the maximum prediction performance for the second-order time-varying model (Eq. 3) was used (the prediction method is explained in the next section). Note that Δ is not a measure of latency per se but only a means of aligning the stimuli and the weight functions so that the stimulus bins at short delays where the weight functions are zero are not included in the calculation. Δ had a mean value of 3.2 ± 1.3 ms. This is shorter than the usual first-spike latency value for DCN neurons in this preparation (median 4.5 ms; M. R. Sydorenko and E. D. Young, unpublished observations) for the reasons discussed previously (Møller 1975).

In all of the figures, the weight functions are plotted with the delay estimate Δ added back to the time axis. Thus the first few bins at times less than Δ are assumed to be zero.

Evaluating the model through prediction.

The models were evaluated by estimating model parameters from a randomly chosen group of the stimuli (110 stimuli and the 5 all-0-dB stimuli) and then using those parameters to predict the responses to the remaining 10 stimuli. The prediction performance was quantified as fraction of variance (fv) defined in Eq. 5, where d_ij is the measured rate change for the ith test stimulus in the jth time bin [r_i(t_j) − r₀(t_j)], ${\widehat{\text{d}}}_{ij}$ is the rate change from the model, and d̄ is the mean of the rate changes.

$$fv=1-\frac{{\displaystyle \sum _{i,j}{({\text{d}}_{ij}-{\widehat{\text{d}}}_{ij})}^2}}{{\displaystyle \sum _{i,j}{({\text{d}}_{ij}-\overline{\text{d}})}^2}}\mathrm{.}$$ (5)

Prediction performance was used as the criterion to minimize overfitting and increase generalizability. fv Is used instead of correlation coefficient R because fv is a more conservative measure. In particular, fv is sensitive to a DC (constant) error in the model, whereas R is not. Empirically, fv ≤ R² with approximate equality for good fits (roughly fv > 0.5).

RESULTS

Responses to modulated stimuli (Fig. 1) were obtained in 2 sets of experiments from 9 type III and 33 type IV neurons in the DCN. A total of 69 response sets were collected sometimes with different stimulus sets (different intensities, tone or broad-band carrier, different contrasts) from the same neuron. Weight-function models (Eqs. 2–4) were computed from the responses and used to characterize the neurons as to modulation sensitivity and degree of nonlinearity. This analysis characterizes responses up to a modulation frequency of 500 Hz, somewhat above the usual modulation corner frequency of principal neurons in DCN (261 Hz, Rhode and Greenberg 1994; ∼200 Hz, Joris and Smith 1998).

First-order temporal processing models.

Examples of weight functions for three neurons of the time-varying (Eq. 4) and time-invariant (Eq. 2) first-order models are shown in Fig. 2. The first-order time-invariant weights w_T are shown on the middle column; these show the weighting of energy as a function of time preceding spikes and are essentially reverse correlation filters for the stimulus envelope (de Boer and de Jongh 1978). In the left column, the time-varying weight matrices L (Eq. 4) are shown. The diagonal of the matrix corresponds to current time, and each row from the diagonal to the left shows the weighting of the stimulus in previous time bins. The weight amplitudes are plotted according to the color bars.

Fig. 2.

Sample 1st-order models. Shown are models and prediction performance for 3 sample dorsal cochlear nucleus (DCN) principal neurons with best frequencies (BFs) = 1.5 kHz (A, D, and G), 12 kHz (B, E, and H), and 8.8 kHz (C, F, and I). A–C: 1st-order time-varying models (the matrix L, in Eq. 4). The lags Δ between the stimulus and the response used in deriving the models were 4.3, 2.7, and 3.5 ms, respectively. D–F: 1st-order time invariant receptive fields [1st-order temporal kernel w_T(t) in Eqs. 2 and 3]. They correspond to the spike-triggered average of the temporal envelope. Weights are 0 for preceding times less than the lag Δ and are not drawn in this and subsequent figures. sp/s/dB, spikes per second per decibel. G–I: each row shows the actual (blue) and predicted (red) peristimulus time histograms (PSTHs) for the 1st 3 stimuli of the prediction set.

Broadly, two types of weighting functions were seen in the population. The first type has a large excitatory weight preceded by a small inhibitory or suppressive weight (Fig. 2, A and D). The second type has a large inhibitory weight flanked by small or sometimes large excitatory weights (Fig. 2, B and E). Figure 2, C and F, shows an example of a weighting function that did not fall into either category. The two types of weighting functions did not correlate with types of neurons or prediction performance (data not shown).

Figure 2, G–I, shows three examples each of PSTH predictions by the time-varying model (Eq. 4) for these neurons. The prediction performances as measured by fv were 0.49 (Fig. 2G), 0.65 (Fig. 2H), and 0.25 (Fig. 2I). The third example shown in Fig. 2I is a case of mediocre prediction performance for this data set.

Prediction performance of the models.

The time variation of the 1st-order weights contributed significantly to the prediction performance of the model. In the population, the time-varying model gave better performance than the time-invariant model in 30/35 cases (Fig. 3), and there was a small yet significant increase in the median prediction performance measured by fv, from 0.23 to 0.29. There was no difference between different principal cell types or different stimulus contrasts (Fig. 3).

Fig. 3.

Comparison of prediction performance (as fraction of variance, fv) between the 1st-order time-varying and time-invariant models. Different neuron types are identified in the legend. The unfilled symbols show results for stimuli with 3-dB contrast; the other symbols show 6- and 12-dB contrast. The median performances are marked (m) on the axes (significantly different, P < 10⁻⁴ Wilcoxon signed rank test).

One explanation for the improved prediction performance of the time-varying model is that there is some spiking-activity-dependent change in the properties of the neuron that is captured by the time-varying but not the time-invariant model. For example, it might be that a refractory effect of previous spikes changes the firing rate of the neuron transiently and that this effect cumulates through the response (Pillow et al. 2005). This possibility was tested by adding a term to Eq. 2 that explicitly changes the spike rate for a short time following spikes according to a refractoriness function u(t). When the model was fit with such a term, it was found to give a small improvement in the prediction performance (median increase in fv of 0.037, range 0–0.1). When the spiking-activity effect was significant, it took the form of refractoriness as discussed above, reducing the response for approximately 3–4 ms. The effect was larger at higher discharge rates. Because the improvement was small and because of the added complexity of estimating u(t), the refractory term is not included in the results shown here.

Second-order models of temporal processing.

Second-order models of temporal processing were estimated. Because the number of parameters to be estimated increases greatly, a time-varying second-order model was not feasible. Hence, we took all of the rates in 1-ms bins from 12 to 30 ms (after the onset ramp) and estimated an average time-invariant second-order model (Eq. 3). Examples of second-order models for two DCN type IV neurons are shown in Fig. 4.

Fig. 4.

Second-order temporal models for 2 DCN type IV neurons (BFs = 20.1 kHz, A and B, and 15.6 kHz, C and D). Broad-band random temporal shape (RTS) stimulus sets with δ = 1 ms and 3-dB contrast were used. The left shows the 1st-order weighting functions, and the right shows the eigenvectors corresponding to the largest positive (pos.) and largest negative (neg.) eigenvalues of the 2nd-order interaction matrix M_T. The error bars show ±1 SD computed by bootstrap. The eigenvectors act as filters on the amplitude waveforms, for which outputs are squared and multiplied by the corresponding eigenvalue to compute the neural response. The 1st-order weights (red) are repeated from A and C to compare the shapes. The 1st-order weights are scaled for comparison of shapes, and the 1 in D is inverted. Inversion is allowed because the sign of the 2nd-order eigenvectors does not matter; the outputs of those filters are squared, and excitation or inhibition is determined by the sign of the eigenvalue.

The effects of the second-order matrix M_T are best represented using its eigenvectors (Reiss et al. 2007; Yamada and Lewis 1999; Young et al. 2005). The second-order term in Eq. 3 is equivalent to weighting the stimulus with each of the eigenvectors and multiplying the square of that result by the corresponding eigenvalue. The sum of those values is the second-order term in Eq. 3. The eigenvectors that make the largest contribution are those with the largest absolute eigenvalues. In Fig. 4, B and D, the eigenvectors corresponding to the largest negative (inhibitory) eigenvalue and the largest positive (excitatory) eigenvalue are plotted. The largest positive eigenvector usually has a shape similar to the first-order weights, shown in Fig. 4, B and D, as the red plots copied from Fig. 4, A and C. For all of the neurons, the cross-correlation of the first-order weights with each of the eigenvectors was computed, and the maximum correlation was found; this was the eigenvector with the largest positive eigenvalue in over half of the cases and was the second largest positive eigenvalue in the remainder. The median over the population of the maximum correlation coefficient was 0.71, similar to the correlation for spectral weighting models (Miller et al. 2002; Reiss et al. 2007). This result suggests that one of the effects of the second-order term is to make the response to the first-order weights quadratic (Schwartz et al. 2006; Touryan et al. 2002).

Second-order effects over a period >12 ms were not examined due to the increase in the size of the parameter set. However, it is possible that there are effects beyond the length of the first-order filter as is seen in the largest positive eigenvector in Fig. 4B, where the eigenvector does not settle down to zero in the last time bin and could have significant components further back in time.

The 2nd-order model provides a small but significant increase in performance of the time-invariant model, shown in Fig. 5. The different symbols show the different neuronal types and stimulus contrasts. The 2nd-order model gives better results in 24/35 cases. Note that the largest improvement in prediction by the 2nd-order model occurs for the few cases with larger temporal fluctuations (6 and 12 dB). This improvement is significantly greater than that for 3-dB contrasts (P = 0.0018 by Wilcoxon rank sum).

Fig. 5.

Comparison of the prediction performance between the 1st- and 2nd-order time-invariant models. Different neuron types are identified in the legend as in Fig. 3. The unfilled symbols show results for 3-dB contrast, and the filled symbols show 6- and 12-dB contrasts. The median performances of the models are marked on the axes (medians differ significantly, P = 0.0014 Wilcoxon signed rank test).

Models with BF tone carriers.

The major nonlinearities in spectral processing in DCN neurons have to do with changes in sound level (Bandyopadhyay et al. 2007; Nelken et al. 1997). Some insight into the nonlinear behavior of these neurons is provided by using tone instead of noise carriers in modulation experiments. The predominant inhibitory input to DCN type IV neurons, the vertical cell or type II neuron, gives strong excitatory responses to narrow-band stimuli (tones) but weak responses to noise (Spirou et al. 1999; Young and Davis 2002). Thus a difference in responses to RTS with noise or tone carriers is expected. In three experiments, data were collected with tone-carrier as well as broad-band noise carrier RTS stimuli.

Figure 6, D–F, shows the first-order time-invariant weight functions for three cases with tone carriers. Plots of discharge rate vs. sound level are shown in the top row; the vertical dotted lines show the reference level of the tone carrier (when the modulation stimulus was 0 dB). The first and second columns are from the same neuron at different carrier sound levels. When the tone carrier was on the rising slope of the tone rate-level function (Fig. 6A), the inhibitory inputs were not active, and the first-order weight had an excitatory shape (Fig. 6D). At the higher carrier level, with stronger inhibitory inputs (Fig. 6B), the weights are purely inhibitory (Fig. 6E). For another neuron studied near the peak of the tone rate-level function (Fig. 6C), the weighting function was a combination of inhibitory and excitatory effects (Fig. 6F).

Fig. 6.

Variation of weights with carrier sound level for tone carriers. A–C: the tone rate-level functions (solid line) and spontaneous rates (dashed line) for 2 DCN type IV neurons (A and B are from the same neuron, BFs = 6.2 kHz; C has BF 4.4 kHz). The rate-level functions were smoothed by convolving them with a Gaussian window with SD = 1 dB. The levels at which the tone RTS carriers were presented are marked by dotted vertical lines. SPL, sound-pressure level. D–F: the corresponding 1st-order time-invariant temporal weighting functions for 3-dB contrast stimuli. The 1st (D) shows a primarily excitatory weighting function, which changes to a primarily inhibitory 1 (E) with an increase in the stimulus level of 10 dB. The 2nd neuron shows a weighting function that has a long inhibitory tail (F).

Data with a tone carrier were collected from 15 DCN neurons (10 type IV and 5 type III) for a total of 21 different stimulus levels in different neurons. In 12 of those (7 type IV and 5 type III), data were also collected with wide-band stimuli. The 1st- and 2nd-order weighting functions derived with tone carriers showed similar characteristics as those with broad-band carriers except that 2nd-order weights were often larger for tone carriers (data not shown). Inclusion of 2nd-order models improved prediction performance in cases of tone carriers (2nd-order: median fv 0.38; 1st-order: median fv 0.25, P ≈ 0.0004) as well as noise carriers (Fig. 5). The improvement in prediction performance was significantly more with the inclusion of 2nd-order terms in the case of BF-tone carriers (median improvement in fv 23% for tone vs. 8% for noise carriers, P < 0.02), suggesting that DCN temporal envelope processing is more nonlinear for narrow-band stimuli.

Effects of change in stimulus contrast.

For spectral weighting functions analyzed previously (Reiss et al. 2007), the weights increase significantly in amplitude and so does prediction performance as spectral contrast decreases. These effects are smaller in temporal weighting functions; in fact, temporal weighting functions show the opposite change in prediction performance. Figure 7, A and B, shows examples of first- and second-order weights derived from two type IV neurons. Figure 7A contains spectral weights for unmodulated stimuli computed from sounds with random spectral shape variations as in Reiss et al. (2007). Temporal weights from a different type IV neuron are shown in Fig. 7B. The spectral weights increase by a factor of 1.5 between 6- and 3-dB contrast, but the temporal weights are similar in magnitude. Figure 7C shows histograms of the relative amplitudes of temporal and spectral first-order weights, shown as the ratio of the norms of the first-order weights for 3- and 6-dB contrast. The median ratio for temporal models is near 1 (n = 11 cases), whereas the median spectral weight ratio is ∼1.5 (n = 15 cases; different P ≈ 0.0026, Wilcoxon rank sum). The effects of contrast are larger at 12 dB, although only two neurons were studied at this contrast, including the one shown in Fig. 7A. For these, the ratios of 3-to-12-dB norms were 2.1 and 1.9 for temporal weights. In previous studies, spectral weights increased by (median) ratios of 2.9 (Bandyopadhyay et al. 2007) and 2.7 (Reiss et al. 2007) between 12- and 3-dB spectral contrast.

Fig. 7.

Changes in stimulus contrast produce larger effects in the spectral vs. the temporal domain. A: spectral weight-function models (1st-order, left, and 2nd-order, right, in color) derived from a type IV neuron (BF = 5.2 kHz) for 2 different contrasts (3 and 6 dB). Second-order weights are shown here as the weight matrix (M in Eq. 3) and not as eigenvectors. wts, Weights; re, referenced to. B: temporal weight functions (1st-order, left, and 2nd-order, right, in color) at the same stimulus contrasts for another neuron (BF = 2.4 kHz). C: histogram of the ratio of the amplitude (norm) of the 1st-order weight vectors for 3- and 6-dB spectral (gray bars) and temporal (black bars) contrast. Medians are indicated by the arrows at the top of the plot. Data on temporal weights are from this paper, and spectral weights are from Bandyopadhyay et al. (2007).

The prediction performance of weight-function models differs between spectral and temporal models. For the second-order model (Eq. 3), which is the standard model for analysis of spectral weighting functions, median fv values increased from 0.5 for 12-dB contrast to 0.9 for 3-dB contrast (Reiss et al. 2007). For the second-order temporal model, the change was in the opposite direction, medians of 0.29 (3-dB contrast) and 0.46 (6- and 12-dB contrast together) for the data in Fig. 5.

Predicting MTFs.

The relationship between the weight-function model and the MTF was tested by computing MTFs from the weight model in one of two ways. First, the Fourier transform of the first-order weights yields a function, which would be the same as the MTF if the response to modulation were linear. Examples of MTFs computed this way are shown in Fig. 8A. It is evident that the sampling frequency was not high enough to avoid some aliasing at high frequencies, but the general nature of the functions, including their best modulation frequencies (BMFs), is still clear. Second, MTFs can be computed directly from the model by computing its responses to sinusoidally modulated noise. The MTFs were computed for first- and second-order models (Fig. 8). As the examples show, the SAM calculations are somewhat noisy, especially for the second-order models, but their general shapes and BMFs were approximately the same as for the Fourier transform results. In particular, the second-order terms in the model usually did not change the BMF by much. Figure 8B shows a comparison of the BMFs computed from first- and second-order models. For most neurons, the data cluster near the line of equality with some deviant cases (correlation coefficient 0.65, P < 1e−8). The computed MTFs were similar to those reported previously for DCN neurons (Rhode and Greenberg 1994; Joris and Smith 1998); most were band-pass, as in the top example of Fig. 8A, with a few low-pass. The mean BMF of the model MTFs was 244 Hz, comparable with previous results (261 Hz, Rhode and Greenburg 1994; ∼200 Hz, Joris and Smith 1998).

Fig. 8.

Modulation transfer functions predicted by the temporal weights model. A: 2 examples of modulation transfer functions (MTFs) computed as the magnitude of the Fourier transform (FT) of the 1st-order weight functions (black circles) or by simulating responses to sinusoidally amplitude-modulated (SAM) noise using the 1st-order (1st ord; black squares) or 2nd-order (red circles) models. The simulations were done at 20 and 15 dB re threshold, respectively. B: best modulation frequencies (BMFs) for SAM noise responses of 1st-order models plotted against BMFs for 2nd-order models. BMF was computed as the modulation frequency at which the response was maximum. Points were dithered with SD 3 Hz to avoid overlapping.

Changes in modulation responses as sound level changes.

As discussed above, neurons in the DCN show substantial nonlinearity as the stimulus level is changed (e.g., the data in the left 2 columns of Fig. 6; Bandyopadhyay et al. 2007; Joris and Smith 1998; Zhao and Liang 1995). To investigate how large an effect is produced by the second-order nonlinearities in fitted weight models, responses to SAM were computed from the three weight models (Eqs. 2–4) over a 20-dB range of carrier sound levels centered on the level used to construct the temporal weight-function model. Figure 9 shows a comparison of the behavior of the first- and second-order models over this range of levels for two DCN type IV neurons. Figure 9, A and B, shows the synchronization amplitude of SAM responses computed from the models plotted against stimulus level. Synchronization to a 100-Hz modulation is plotted.

Fig. 9.

A and B: examples of the synchronization predicted by the temporal weight models calculated for 2 neurons for responses to SAM tones with BF-tone carriers (with modulation depth 0.7). The model parameters are those determined at 0 dB, and we assume that the weights and the response to the all-0-dB (unmodulated) stimulus vector s = 0 [r₀(t)] do not change with level. Synchronization is the component of the response modulation at the stimulus modulation frequency divided by the average response rate. Results are shown across a 20-dB range of carrier levels, where 0 dB (“ref”) is the level of the carrier used to derive the weight model. Responses of 1st-order (black) and 2nd-order (red) models are shown. C and D: phases of the data in A and B relative to the stimulus. BFs = 13.7 kHz in A and C and 6.2 kHz in B and D, the same neuron as in Fig. 6, A and B.

The first-order models give monotonic or near monotonically increasing or decreasing synchronization-level functions (Fig. 9). This is expected for a linear system that weights the input by a constant vector. The second-order model gives substantially nonmonotonic outputs, reflecting the quadratic nature of the model. Figure 9, C and D, shows the phase of the output modulation; it shows little change with level in the first-order models. However, there are significant transitions for the second-order model reflecting the interaction of excitatory and inhibitory terms (corresponding to the positive and negative eigenvalue terms in Fig. 4, B and D).

Hence, although the second-order terms improve predictions of responses to RTS stimuli only slightly, they potentially have substantial effects in causing nonmonotonic envelope synchronization behavior across sound level. The results in Fig. 9 are computed for tone carriers, but similar results are obtained when using models based on broad-band carriers. The nonlinear effects shown in Fig. 9 are qualitatively similar to effects of sound level shown by Joris and Smith (1998; Fig. 3) for DCN type IV neurons.

Predictions of responses to simulated moving stimuli and direction dependence.

In six neurons, data on spectral as well as temporal responses were measured. These cases allow the interaction of spectral and temporal selectivity to be modeled in a cascade that produces directionally selective responses (Fig. 10). A convenient stimulus for this analysis is a moving spectral notch. A hypothesized role of the DCN is in representing spectral notches generated by the acoustics of the head and external ear (Bandyopadhyay et al. 2007; May 2000; Reiss and Young 2005; Young et al. 1992). An unknown dimension of this response is the effect of pinna and head movements as well as moving sound sources, which cause the spectral notches in the stimulus at the eardrum to move according to the acoustics of the external ear (Rice et al. 1992). The response to the moving notch would be shaped by both the spectral and temporal filtering of a DCN neuron. We assume, for the argument, that the spectral and temporal filtering are functionally separate in the neuron, as would happen if the spectral filtering is determined by the presynaptic neural elements, the weights on their inputs to the neuron, and the integration of inputs in the dendritic tree; the temporal filtering is then a property of the general low-pass filtering of dendritic integration including effects of K⁺ channels (e.g., Friedland et al. 2007; Kanold and Manis 2001) and inhibition and the final integration steps involved in producing the spike train of the neuron. We assume that the spectral and temporal weighting functions of the neuron, determined in separate stimulus presentations, adequately define the two processes for steady (nonmoving) stimuli.

Fig. 10.

A model for spectrotemporal interaction in responses to complex stimuli. A shows the rate response computed from a spectral model of a DCN type IV neuron (BF = 21.6 kHz; as in Bandyopadhyay et al. 2007) to a 1/4-octave-wide notch with the spectrum shown at the top of the figure. The plot shows the discharge rate when the upper edge frequency of the notch is positioned at various frequencies relative to BF and is not moving. B: presumed temporal waveforms of the responses to the notch when moved from high to low (green) or low to high (red). These waveforms are just the rate function in A with time scaled on the abscissa for a movement speed of 1/12 octaves (oct)/ms. The inset at the top of the figure shows the time-reversed 1st-order temporal weighting function of the same DCN neuron aligned to compute the response to the stimuli at t = 30 ms (dashed line). C: rate responses of the model neuron computed by filtering the temporal patterns plotted in B with the 1st-order temporal weighting function in B. Notice the asymmetry in the peak rate. D: the peak discharge rate plotted vs. notch movement speed for this model, showing larger response to downward-moving notches. Note the change in the rate scale between C and D. Oscillations at the 2 ends of the plots in B and C caused by end effects in the calculation have been removed.

Figure 10A shows the spectrum of a broad-band noise with a quarter octave notch at the top of the plot. The plot shows the response rate predicted by the spectral weighting function model of a neuron (as in Bandyopadhyay et al. 2007) when the notch is presented as a static (nonmoving) stimulus at a range of notch positions. The response is excitatory when the rising edge of the notch is at BF (as in the spectrum shown) but inhibitory with the notch centered on BF and above BF (as in Reiss and Young 2005). Assuming that this rate plot reflects the net spectral filtering of the inputs to the cell, Fig. 10B shows the inputs to the temporal filtering step for the same notch stimulus moving (at 1/12 octaves/ms) from low to high and from high to low frequency before the effects of temporal filtering are applied. These rate profiles were obtained by extending the responses in Fig. 10A by the equivalent of 10 ms to the right and left. The 1st-order temporal weights of this neuron are shown by the dashed plot at the top of Fig. 10B. The results of applying this temporal weighting to the 2 waveforms in Fig. 10B are shown in Fig. 10C. The shape of the temporal filter waveform matches the downward-moving notch better than the upward notch, thus producing a larger peak rate in response to the downward waveform. Figure 10D shows the peak discharge rate of the responses of the model to 1/4th octave notches moving at various speeds. The response is tuned to a speed of 1/12 octaves/ms and is strongly asymmetric, more strongly driven by downward-moving notches, over a range of speeds. This direction sensitivity derives from the serial interaction of the separate spectral and temporal characteristics of the neuron.

Sufficient data were obtained from six DCN neurons to allow both spectral and temporal models to be developed. Five of the models showed directionally sensitive responses to notches with some degree of tuning centered near 0.1 octave/ms.

DISCUSSION

Relationship to the MTF.

Previous studies of temporal responses of DCN neurons have used MTFs as the basis for the analysis (Joris and Smith 1998; Kim et al. 1990; Rhode and Greenberg 1994; Zhao and Liang 1995). The MTF is a complete description if the system is linear. However, because of the nonlinearities in temporal responses in the DCN (Figs. 4–6 and 9), it is necessary to consider the strength of nonlinear effects as well as when they are important. The weight-function models derived here provide a way of doing so, at least for the second-order terms. MTFs and weight functions can be connected by computing the Fourier transform of the first-order weights or by computing responses to SAM stimuli from the weight-function models including first-order or both first- and second-order terms. Here, the properties of the computed MTFs were similar in either case to those described in the literature for DCN neurons (Fig. 8). Moreover, the BMFs are similar for first- and second-order models (Fig. 8B). This result and the relatively small increase in prediction performance with nonlinear terms (Fig. 5) suggest that the nonlinear effects are small, typically <10% of the variance. However, the effects of nonlinearities are much larger when responses are examined over a range of sound levels (Fig. 9). This result is consistent with previous studies in DCN (Joris and Smith 1998) and IC (Krishna and Semple 2000) showing significant MTF changes with sound level.

Adaptation of weight functions.

There are a number of reasons to expect temporal weight functions to change with time. Adaptation of spike responses to a high-contrast or a high-sound-level stimulus is observed throughout the auditory system (Costalupes et al. 1984; Dean et al. 2005; Gibson et al. 1985; Kvale and Schreiner 2004; Nagel and Doupe 2006; Wen et al. 2009) as well as in other sensory systems (Nowak et al. 2005). These changes occur over time scales of tens to hundreds of milliseconds and occur following the onset of a stimulus (as here) or following a sudden change in the stimulus mean or variance (as in Kvale and Schreiner 2004; Nagel and Doupe 2006; Wen et al. 2009). On shorter time scales, refractoriness and interactions of excitatory and inhibitory effects in the membrane (Kanold and Manis 2001) could produce additional changes in the receptive fields of neurons. The temporal weight-function model allows such adaptation to be analyzed either as a change in the temporal weights with time (the time-varying model, Eq. 4) or as a significant effect of previous spike discharges in a suitable augmentation of the model (as with the model of refractoriness mentioned in results). Here, a clear pattern of temporal variation was not observed with either analysis. Only small improvements in prediction performance were observed for the time-varying model (Fig. 3), and the form of the adaptation was not consistent across neurons. Thus our results suggest that such effects are small in the DCN. This is a useful result because auditory neurons are often studied with methods that use stationary long-duration stimuli, as in STRF studies; our results support the unspoken assumption in those studies that short-term temporal nonstationarities can be ignored.

Note that our analysis is consistent with those of previous studies of adaptation to stimulus transients, mentioned above, because of the definition of our analysis model. The effects of the adaptation on the mean discharge rate of the neuron is contained in the r₀(t) or r terms in Eqs. 2–4. That is, the weight model used here analyzes the representation of the changes in response rate produced by changes in stimulus level and not the adaptation of average rate. Our results thus suggest that adaptation at stimulus onset affects the baseline rate of the neuron and less so the response to stimulus transients.

Nonlinear responses.

The weight-function model allows some aspects of the nonlinearity of the neural responses to be studied, namely the lowest (2nd-order) nonlinear term. The nonlinear terms model the quadratic deviations from linearity and extend the range over which a computation like the weight-function model can accurately represent neural responses (e.g., Yu and Young 2013). Nonlinearities in the DCN are most easily seen as changes in the properties of the system as sound level changes (Bandyopadhyay et al. 2007; Nelken and Young 1997); possibly related results are obtained in the IC (Lesica and Grothe 2008; Yu and Young 2013) and auditory cortex (Ahrens et al. 2008).

Previous analysis of the DCN circuit has led to the hypothesis that the nonlinearity of the DCN is primarily associated with the activation the inhibitory (type II) circuit (Nelken and Young 1997; Zhou et al. 2012). The inversion of the first-order temporal weighting function between the two cases shown in Fig. 6, A and B, provides an example of this nonlinear effect, which is easy to understand in terms of the underlying neural responses. In one case (Fig. 6, A and D), the temporal modulation is around an operating point on the positive slope of the input/output function of the neuron, where the inhibitor is not active, whereas in the other (Fig. 6, B and E), the level is well above threshold for the inhibitor, and the signs are reversed. This example shows clearly that weighting-function models are only locally applicable, in a range of intensities near the reference level of the stimuli used to derive the model.

Despite the comments in the previous paragraph, the second-order model improved prediction performance over the first-order model only to a small degree (Fig. 5). Nevertheless, the nonlinearities are important. First, the second-order terms can produce nonmonotonic behavior of envelope synchronization with changing sound intensity (Fig. 9), giving results similar to those observed in DCN neurons (Joris and Smith 1998). Second, the eigenvector decomposition of the second-order matrix M_T allows two nonlinear effects, a positive one and a negative one, to be separated (Lewis and van Dijk 2004; Schwartz et al. 2006; Touryan et al. 2002). The fact that the eigenvectors corresponding to the largest or second largest positive eigenvalues are similar to the first-order weights (Fig. 4, B and D) means that the neuron responds to changes in sound intensity with a quadratic input/output function, containing a linear response given by the 1st-order weights and a squared response that is part of the 2nd-order weights. The quadratic nature of the response is responsible for the nonmonotonic responses in Fig. 9, A and B. Second, the eigenvectors with negative eigenvalues are inhibitory inputs to the neuron. These usually have different temporal weighting from the excitatory inputs (as represented by the 1st-order weights) and thus represent additional interactions that might be important in complex stimuli.

Effects of stimulus contrast.

The above discussion applies to the overall sound level of the stimulus. Also important is the stimulus contrast, defined here as the dB amplitude of sound intensity increases and decreases from the mean or reference level. Following results in the visual system (Baccus and Meister 2002; Bonin et al. 2006; Chander and Chichilnisky 2001; Mante et al. 2005; Shapley and Victor 1978; Zaghloul et al. 2005), contrast effects have been observed in auditory neurons, up to now primarily in spectral response measures (Barbour and Wang 2003a,b; Escabi et al. 2003; Nagel and Doupe 2006; Reiss et al. 2007). The common result obtained is that the gain functions do not change shape with increasing contrast; however, their magnitudes decrease, reflected either in shallower slopes in the output nonlinearity (Nagel and Doupe 2006) or simply smaller first-order weights and smaller eigenvalues of the second-order matrix (Reiss et al. 2007). At the same time, prediction performance of the weighting function model improves at lower spectral contrast.

For the temporal weighting functions discussed here, the results are somewhat different. A decrease in gain was not observed between 3- and 6-dB contrast stimuli (Fig. 7C), and only small decreases were seen at 12 dB in two cases studied. More important, prediction performance changed in the opposite direction, with better prediction for the second-order model at higher stimulus contrast, both 6 and 12 dB (Fig. 5, ordinate values). Thus there are differences between the spectral nonlinearities, reflecting simultaneous interactions of sound energy at different frequencies, and the temporal nonlinearities, reflecting sequential interactions of energy through time. The above factors could reflect some sluggish aspect of the operation of inhibitory circuits like the type II neurons in DCN. These neurons have relatively long latencies (Joris and Smith 1998), which could delay inhibitory effects or reduce their amplitude in responding to the stimulus transients. Thus the strong inhibitory effects that contribute to nonlinearities in spectral processing would be blunted and reduced for temporal processing. There are also differences in the slopes of the input/output functions of excitatory and inhibitory inputs to principal cells (Zhou et al. 2012), which would differentially affect the growth of modulation responses as contrast changes.

Predictions of direction selectivity and its implications.

The temporal model was used to predict tuned and direction-selective responses to moving spectral notches (Fig. 10, C and D). This result has wide implications for processing of moving sounds and for dealing with the acoustic effects of pinna and head movements. The predictions were made only with first-order models, but the effects are actually amplified (data not shown) with second-order models.

One question not addressed in Fig. 10 is the robustness of this model as sound level changes. Certainly, the data for tone carriers in Fig. 6, A and B, suggest that the properties of the proposed system could change across sound level, as the sign of the temporal kernel changes. However, for broad-band stimuli, like the noise bands used in this paper, such effects should be small. For the spectral kernel of Fig. 10A, detailed study has shown that there is little change with sound level (Reiss and Young 2005). Because rate variation across sound level is generally close to monotonic in cat DCN for noise stimuli (although not in the gerbil; see Davis et al. 1996a), it seems likely that the temporal kernel (Fig. 10B) would also not change dramatically with sound level. However, data were not taken across a range of sound levels in this paper, so this question requires further study. Even if there are changes with sound level, the mechanism discussed in Fig. 10 could apply to a variety of spectrotemporal integration phenomena in hearing.

The DCN is a site for integration of somatosensory inputs in the auditory pathway (Davis et al. 1996b; Young et al. 1995) and is hypothesized to be involved in processing pinna or head movements, which also cause changes in head and ear acoustics (Davis et al. 1996b; Kanold and Young 2001; Young et al. 1995, 1996). Most studies have looked at the problem of sound localization from the perspective of a stationary sound source. However, inputs to the auditory system are usually dynamic in nature, and both the sound source and the ear position with respect to the sound source often move, producing changes in the sound at the eardrum that are modeled by the notch sweeps used in the simulation. The DCN gets proprioceptive information on ear position from the pinna, head, and neck muscles through the parallel fibers in the superficial DCN, which synapse on the apical dendrites of the principal neurons (type IV). This gives rise to the possibility of association of the auditory and somatosensory inputs in the DCN. The exact nature of this interaction is not yet clear (Kanold et al. 2011; Shore 2005) so that a hypothesis as to how the two may be combined cannot be formulated. However, such selectivity to moving notches combined with the somatosensory inputs may serve as the basis for an animal to learn and later use the acoustic notches produced by its own external ear for monaural sound localization.

GRANTS

This study was supported by National Institute of Deafness and Other Communications Disorders Grants DC-00115 and DC-005211.

DISCLOSURES

No conflicts of interest, financial or otherwise, are declared by the author(s).

AUTHOR CONTRIBUTIONS

S.B. and E.D.Y. conception and design of research; S.B. performed experiments; S.B. and E.D.Y. analyzed data; S.B. and E.D.Y. interpreted results of experiments; S.B. and E.D.Y. prepared figures; S.B. drafted manuscript; S.B. and E.D.Y. edited and revised manuscript; S.B. and E.D.Y. approved final version of manuscript.