Spike Count Analysis for MultiPlexing Inference (SCAMPI)

Abstract

Understanding how neurons encode multiple simultaneous stimuli is a fundamental question in neuroscience. We have previously introduced a novel theory of stochastic encoding patterns wherein a neuron’s spiking activity dynamically switches among its constituent single-stimulus activity patterns when presented with multiple stimuli (Groh et al., 2024). Here, we present an enhanced, comprehensive statistical testing framework for such “multiplexing”. As before, our approach evaluates whether dual-stimulus responses can be accounted for as mixtures of Poissons related to single-stimulus benchmarks. Our enhanced framework improves upon previous methods in two key ways. First, it introduces a stronger set of foils for multiplexing, including an “overreaching” category that captures overdispersed activity patterns unrelated to the single-stimulus benchmarks, reducing false detection of multiplexing. Second, it detects continuous mixtures, potentially indicating faster fluctuations - i.e. at sub-trial timescales - that would have been overlooked before. We utilize a Bayesian inference framework, considering the hypothesis with the highest posterior probability as the winner, and employ the predictive recursion marginal likelihood method for non-parametric estimation of the latent mixing distributions.

Reanalysis of previous findings confirms the general observation of fluctuating activity and indicates that fluctuations may well occur on faster timescales than previously suggested. We further confirm that multiplexing is more prevalent for (a) combinations of face stimuli than for faces and non-face objects in the inferotemporal face patch system; and (b) distinct vs fused objects in the primary visual cortex.

1. Introduction

A central postulate in computational neuroscience holds that neurons encode sensory signals in a repeatable way. The activity elicited by a given stimulus is assumed to follow a consistent profile across time and across repeated presentations of that stimulus, with any remaining variation falling within a given unimodal distribution (Kass et al., 2005; Amarasingham et al., 2006; Maimon and Assad, 2009; Pillow and Scott, 2012; Stevenson, 2016; Charles et al., 2018); see also Goris et al. (2014); Czanner et al. (2008). Such repeatability may indeed hold when only one stimulus is present, e.g. in a typical, highly controlled laboratory setting with a single visual image or sound at a time. But is this central postulate true when multiple stimuli are presented together?

We have recently documented evidence that when two stimuli are present, a subset of neurons in a population show additional stochastic variations in their activity profile. Spiking activity appears to dynamically switch between the activity patterns typically observed when each stimulus is presented alone, even when the activity is aggregated over relatively long time windows (100s of ms) (Caruso et al., 2018; Mohl et al., 2020; Glynn et al., 2021; Jun et al., 2022; Schmehl et al., 2024; Groh et al., 2024). Such stochastic juggling, also termed “multiplexing”, is an exciting possibility because it offers a novel theory for how information may be preserved in multi-stimuli environments. It suggests that by taking advantage of the temporal dimension and perhaps by allowing different neurons to “switch” separately of one another, code switching may underlie our ability to perceive the individual elements of a crowded sensory scene.

This discovery was made possible mainly by novel statistical tools that probed the spiking activity of individual neurons aggregated by the “trial” – one uninterrupted presentation of a sensory scene lasting several hundred milliseconds – with the spike train recorded within each trial summarized into a single trialwise spike count. This statistical analysis framework, first presented in Caruso et al. (2018), evaluated an across-trials multiplexing hypothesis: when two stimuli A and B are presented together, an individual neuron may encode stimulus A in some trials and stimulus B in the remaining trials – switching randomly between these options. When this hypothesis is true, the double-stimuli trialwise spike count distribution must resemble a weighted mixture of the corresponding single-stimuli distributions. This resemblance should be statistically testable with spike count data collected from three types of trials: stimulus A presented alone, stimulus B presented alone, or stimuli A and B presented together.

This existing statistical tool for detecting multiplexing has gaps that may contribute to both false positive and false negative discoveries. On the one hand, in trying to detect multiplexing only through fluctuations across trials, one fails to capture the notion that a neuron may switch its activity level one or more times during the course of a trial. If such sub-trial switching happens with any regularity, the corresponding trialwise spike count distribution could concentrate within an interval sandwiched between the A and B benchmarks, rather than splitting and spreading itself across the two (Figure 1). Accordingly, testing for across-trials fluctuations does not help in discovering the existence of multiplexing when it happens at a faster timescale.

Fig. 1.

Conceptual framework of multiplexing and corresponding distributions of trial aggregated spike count. We consider two scenarios of neural switching patterns when exposed to multiple stimuli. Slow juggling, or fluctuation across trials, indicates switching across trials, leading to a mixture of benchmarked distributions. Fast juggling, or fluctuation at sub-trial timescales depicts switching within trials, where the corresponding distribution concentrates within an interval sandwiched between benchmarked distributions.

On the other hand, the statistical test we developed in Caruso et al. (2018) is susceptible to concluding in favor of multiplexing should the double-stimuli spike count distribution show patterns of across-trials fluctuation between broadly separated modes even if those modes do not resemble the two benchmark distributions. For example, if the spike count distributions under A and B are, respectively, Poisson(50) and Poisson(80), but the spike count distribution under AB is a mix¹ between Poisson(65) and Poisson(95), the test would conclude in favor of multiplexing on average four out of five times², even though the AB distribution is not an exact match to the predicted mix of the A and B benchmark distributions. This type of pattern is consistent with fluctuating activity, but the failure to coincide with the benchmarks makes it unclear how information about the individual component stimuli could be discerned from the signal.

Here, we fill in these gaps by refining the statistical testing framework to evaluate switching at either the whole-trial or the potential sub-trial timescale from trialwise spike count data. Furthermore, we expand the framework and incorporate as a control against multiplexing a generalized notion of “overdispersion”– extra trial-to-trial variability of trialwise spike counts above and beyond what one should observe under the benchmarked-fluctuation assumption. Our new statistical evaluation method, which we call Spike Count Analysis for MultiPlexing Inference (SCAMPI), retains the Poisson benchmark assumption for the two single-stimulus trialwise spike count distributions. For double-stimuli exposures, four distinct hypotheses are considered for the spike count distribution, two of which adhere to the theory that multiplexing is a computational strategy where information is itemized over time (see Figure 2), and the other two scenarios can be seen as “controls” against this theory.

Fig. 2.

Conceptual approach of the SCAMPI model in comparison to the earlier trialwise spike count model (Caruso et al., 2018; Mohl et al., 2020; Jun et al., 2022; Schmehl et al., 2024; Groh et al., 2024). In both frameworks, we use the number of spikes occurring in response to each stimulus presentation as the metric of activity, obtaining a spike count for each trial. Next, we establish response benchmarks on the basis of the distribution of spike counts observed on A-alone and B-alone trials. In our previous model, we then categorized responses on AB trials into four subgroups, defined in relation to those benchmarks. mixture was defined as the case in which responses appear to be drawn stochastically from either the A-alone or B-alone distributions, varying across trials. The remaining categories were all assumed to be drawn from single Poisson distributions as illustrated. In the new approach, we refine this framework to include additional potential classifications. slow-juggling is nominally identical to our previous mixture category, but the inclusion of a fast-juggling option permits the dissociation of truly across-trial fluctuations from fluctuations that could occur more rapidly – producing an overdispersed distribution that is nevertheless still bounded by the A-alone and B-alone benchmarks. We also include a non-bounded overreach category, i.e. a response pattern that is broader than a Poisson but does not appear to be constrained by the benchmarks. Finally, there remains a fixed-Poisson category, which can be subdivided into various labels depending on the mean firing rate in relation to the benchmarks. See Table 1 for a summary of the relationship between the old and the new categories.

The first hypothesis matches Caruso et al. (2018) in positing the AB distribution as a discrete mixture of the two benchmark Poisson distributions, representative of slow across-trials multiplexing (slow-juggling). The second hypothesis formalizes the AB distribution as a continuously weighted mixture of multiple Poisson distributions each with a mean firing rate sandwiched between benchmark A and B mean rates. This hypothesis is compatible with fast sub-trial multiplexing (fast-juggling), though it may not be the only possible explanation of a continuous mixture. The third hypothesis offers the simplest possible control against the above two multiplexing hypotheses. It states that the AB distribution is a fixed Poisson distribution (fixed), regardless of what its mean rate is relative to the benchmark rates – either matching one or the other (a winner-take-all like scenario), lying between them, or falling outside the range of the benchmarks (as if there were an additive or suppressive interaction). The fourth hypothesis offers a more complex control and postulates that the AB distribution is a continuous mixture of Poisson distributions but some component mean firing rates are outside the range defined by the two benchmarks (overreach). Although this situation involves switching (and such switching is potentially interesting), we avoid labeling it as multiplexing because the existence of a mixture component with mean outside the A and B range cannot be explained by juggling between the benchmarked states.

Our new method SCAMPI adopts a Bayesian inference framework to test benchmarked discrete or continuous mixtures against the two controls, all based on trialwise spike count data collected from A, B, and AB exposure trials. The use of a formal Bayesian testing framework, based on Bayes factors, is useful for comparing four competing hypotheses without having to label any one of them as a “null” hypothesis. It is also useful in combining single cell evidence into a combined population level inference on the prevalence of potential multiplexing. In our evidence calculation, the Poisson mixture models under both fast-juggling and overreach hypotheses are materialized by introduction of continuous mixing distributions which are estimated nonparametrically. The introduction of nonparametric distributions complicates the calculation of Bayes factors. We overcome this issue by adopting the predictive recursion marginal likelihood method of Martin and Tokdar (2011). Our analysis here pertains only to two competing stimuli, but the theory and methods could be generalized in principle to situations with more than two stimuli.

SCAMPI could also be seen as a reinforcement of our previous attempt at evaluating sub-trial switching in which we modeled AB spike trains as a dynamic admixture of point processes (DAPP; Caruso et al. (2018); Glynn et al. (2021)) and carried out statistical estimation by dividing each spike train into smaller bin counts (25–50 ms). While the DAPP analysis can quantify the propensity of a neuron to exhibit whole-trial or sub-trial level switching, or no switching at all, it does not offer a rigorous statistical testing framework to evaluate formal hypotheses about switching. It also faces considerable statistical difficulty in providing precise estimates because of the scarcity of data in short time bins. SCAMPI, which operates with whole-trial level spike count data, can offer much higher precision in identifying continuous mixtures consistent with sub-trial switching under the fast juggling hypothesis.

2. Statistical Analysis Framework

2.1. Multiplexing and Alternative Models

Let $\left(Y_j^e:j\in \left\{1,\dots ,n_e\right\},e\in \{\text{A},\text{B},\text{AB}\}\right)$ denote trialwise spike count data recorded under three signal settings: exposure to a stimulus A presented in isolation (labeled $e=\text{A}$), exposure to a distinct stimulus B presented in isolation $(e=\text{B})$, and exposure to stimuli A and B presented together $(e=\text{AB})$. The corresponding trialwise spike count distributions are labeled $P^{\text{A}},P^{\text{B}}$ and $P^{\text{AB}}$. Single stimulus spike counts are assumed Poisson distributed:

$$P^{\text{A}}=\text{Poi}\left({\mu }^{\text{A}}\right),P^{\text{B}}=\text{Poi}\left({\mu }^{\text{B}}\right),$$ (1)

with unknown average firing rates ${\mu }^{\text{A}},{\mu }^{\text{B}}>0$. The Poisson shape assumption is partly a matter of mathematical convenience. The well-known additivity property of Poissons allows one to explain continuously weighted mixtures as sub-trial level switching between different steady states each of which is represented by a Poisson distribution. We note here that modeling trialwise spike counts to be Poisson distributed is a much weaker assumption than modeling the spike train as a Poisson process. When aggregating over 100s of milliseconds, the former model could give a good approximation to reality even when the latter fails to account for fine timescale patterns, such as the existence of a refractory period after each spike. However, since our testing framework mathematically depends on the Poisson benchmark assumption, we will take care to validate this assumption for each application. In theory, our multiplexing classification could be sensitive to deviations from the “no overdispersion” property of the Poisson distribution, which has its variance equal to its mean. Since overdispersion (variance > mean) typically arises from trial-to-trial variation of the mean firing rate, it is crucial for us to exclude scenarios where the A or the B distribution shows too much overdispersion, rendering the Poisson model unreliable to capture the benchmark steady states. Specifically, we will screen out treatment conditions where either the A or the B distribution has a Fano factor larger than 3, although our experiments reveal that the results are not very sensitive to the exact choice of this Fano threshold. More comments on this issue are provided in Section 4.2.

As mentioned earlier, SCAMPI considers four different hypotheses for $P^{\text{AB}}$, two of which are representative of across-trials or sub-trial multiplexing, and the remaining two provide controls against the first two. Below we provide clear mathematical details and some interpretations of these hypotheses.

slow-juggling:

This hypothesis postulates the AB distribution as a discrete mixture

$$P^{\text{AB}}=\rho \text{Poi}\left({\mu }^{\text{A}}\right)+(1-\rho )\text{Poi}\left({\mu }^{\text{B}}\right),\rho \in \left(\mathrm{0,1}\right),$$ (2)

representing across-trials fluctuations between the two benchmark rates ${\mu }^{\text{A}}$ and ${\mu }^B$. The mixing weight $\rho$ gives the (unknown) relative prevalence of stimulus A within the ensemble: a $\rho$ fraction of the trials can be classified as A-like and remaining $1-\rho$ fraction can be classified as B-like.

fast-juggling:

This hypothesis postulates the AB distribution as a continuously weighted mixture of multiple Poisson distributions whose mean rates are bounded between ${\mu }^{\text{A}}$ and ${\mu }^{\text{B}}$:

$$P^{\text{AB}}=\int \text{Poi}\left(r{\mu }^{\text{A}}+\left(1-r\right){\mu }^{\text{B}}\right)f\left(r\right)\mathit{\text{dr}},f\text{is}\text{a}\text{density}\text{on}\left(\mathrm{0,1}\right),$$ (3)

What (3) says is that every individual AB trial produces a total spike count according to a weighted averaged firing rate $r{\mu }^{\text{A}}+(1-r){\mu }^{\text{B}}$ subject to Poisson variability. Moreover, the averaging weight $r$ varies stochastically from one trial to the next according to some probability density $f\left(r\right)$. An important question then arises: what causes the averaging and what explains the trial-to-trial variation in the averaging weight? We contend that a compelling answer to both questions can be given by considering a theory of sub-trial multiplexing, although it may be impossible to establish such a theory beyond doubt simply on the basis of spike count data analysis.

Sub-trial level fluctuation may arise when a random fraction $r$ of the trial duration is spent encoding for stimulus A; the remaining $1-r$ fraction is devoted to stimulus B. To appreciate this interpretation consider a neural circuit where the neuron under study receives its inputs from two channels regulated by one gate, where one channel carries signal from stimulus A and the other carries information from stimulus B. When only stimulus A is presented, signal flows in only from the first channel and evokes $W^{\text{A}}~\text{Poi}\left({\mu }^{\text{A}}\right)$ many spikes. Similarly, in presence of only stimulus B, signal flows in only from the second channel and evokes $W^{\text{B}}~\text{Poi}\left({\mu }^{\text{B}}\right)$ many spikes. When both signals are present simultaneously, a gate regulates the information flow which alternates between the two channels over time within the course of the trial. Assuming the first channel is open for only a fraction $r$ of the trial duration, the signal flowing from it would evoke an $X^{\text{A}}~\text{Bin}\left(W^{\text{A}},r\right)$ many spikes with $W^{\text{A}}~\text{Poi}\left({\mu }^{\text{A}}\right)$. The binomial thinning property of the Poisson distribution then implies $X^{\text{A}}~\text{Poi}\left(r{\mu }^{\text{A}}\right)$. Similarly, the second channel, which is open for the remaining $1-r$ fraction of the trial duration, would evoke $X^{\text{B}}~\text{Poi}\left((1-r){\mu }^{\text{B}}\right)$ many spikes. Therefore, the total spikes evoked would be $Y^{\text{AB}}=X^{\text{A}}+X^{\text{B}}~\text{Poi}\left(r{\mu }^{\text{A}}+(1-r){\mu }^{\text{B}}\right)$ by the additivity property of the Poisson distribution. The fraction $r$ can be clearly identified as the sub-trial level (relative) temporal dominance of information flow from stimulus A within the ensemble. Because such dynamic flows can be hard to reproduce exactly across trials, a trial-to-trial stochastic variation of $r$, according to an unknown density $f\left(r\right)$, appears an entirely natural assumption. Therefore, in absence of a natural alternative explanation, we will identify (3) with the hypothesis of fast-juggling, with the caveat that detailed spike train analysis will be necessary to carry out more precise testing of this hypothesis.

Importantly, the distributional shapes of such continuous mixtures can be statistically detected and formally tested from trialwise spike count data. But an intrinsic caveat to note is that it is not possible to discern the actual dynamics of such sub-trial switching, i.e., the exact timing of the switches. Analysis of trialwise data can only indicate that such within-trial switches are likely to have occurred, and may shed additional light on the distribution $f\left(r\right)$ of the fraction of time devoted to coding for A versus B during the course of a typical trial.

For either of the above two scenarios, $P^{\text{AB}}$ is relatively overdispersed compared to $P^{\text{A}}$ or $P^{\text{B}}$ (both have mean = variance) and also bracketed between $P^{\text{A}}$ and $P^{\text{B}}$ in terms of stochastic ordering³, as one would expect under the multiplexing theory. Accordingly, we formalize alternatives to multiplexing as distributions that violate either overdispersion or stochastic bracketing, or both.

fixed:

This hypothesis postulates the AB distribution to be a fixed Poisson distribution representing some sort of a steady state itself and without any fluctuating activity:

$$P^{\text{AB}}=\text{Poi}\left(\mu \right),\mu \in (0,\infty ).$$ (4)

Such behavior may arise when the AB signal is encoded by the neuron as a single fused stimulus of comparable complexity to that of the A or B stimulus. Based on the relative location of $\mu$ compared to ${\mu }^{\text{A}}$ and ${\mu }^{\text{B}}$, we identify four sub-types: (1) fixed-preferred, the case where the neuron always encodes its preferred signal from the ensemble, ignoring the other $\left(\mu ={\mu }^{\text{A}}\vee {\mu }^{\text{B}}\right)$; (2) fixed-non-preferred, the case where the neuron always encodes its non-preferred signal $(\mu ={\mu }^{\text{A}}\wedge {\mu }^{\text{B}})$; (3) fixed-middle, where the neuron encodes a signal that elicits a firing rate between those of benchmarked signals $\left(\mu \in \left({\mu }^{\text{A}}\wedge {\mu }^{\text{B}},{\mu }^{\text{A}}\vee {\mu }^{\text{B}}\right)\right)$; and (4) fixed-outside, where the neuron encodes a signal that elicits firing rates outside the range of both benchmarked signals $\left(\mu \in \left(0,{\mu }^{\text{A}}\wedge {\mu }^{\text{B}}\right)\cup \left({\mu }^{\text{A}}\vee {\mu }^{\text{B}},\infty \right)\right)$.

overreach:

This hypothesis postulates the AB distribution to be again a continuously weighted mixture, but requires that some of the component mean rates must be outside the range determined by ${\mu }^{\text{A}}$ and ${\mu }^{\text{B}}$:

$$P^{\text{AB}}=\int \text{Poi}\left(m\right)g\left(m\right)\mathit{\text{dm}},g\text{is}\text{a}\text{density}\text{on}\left(0,\infty \right),$$ (5)

One could take the overreach hypothesis, which considers only non-benchmarked and overdispersed $P^{\text{AB}}$, as a catch-all alternative to the first three hypotheses. Under overreach, the neuron’s response to AB is more complex or ambiguous than its responses to either A or B, and its activity may not be related to one another in an information-preservative manner. Instead, the AB response could simply present as a different type of signal than either component of the ensemble (Festa et al., 2021; Semedo et al., 2019). Interpreted in a strictly mathematical sense, fast-juggling would appear a special case of overreach. But our testing framework relies on estimating $g\left(m\right)$ under the overreach assumption that the support of $g$ is much wider than the range determined by ${\mu }^A$ and ${\mu }^{\text{B}}$. Additionally, our test is formally carried out by using Bayes factors which have the Occam’s razor property of favoring more parsimonious explanation of the data (Berger and Pericchi, 1996). In our context, fast-juggling is more parsimonious than overreach, precisely because the mixing density $f$ in fast-juggling is required to have a much narrower range than the mixing density $g$ in overreach.

Notice that our SCAMPI model encompasses all hypotheses in the model proposed by Caruso et al. (2018), with some nuance (Figure 2). We refer to the model by Caruso et al. (2018) as the original model. Conceptually, slow-juggling would appear to correspond to the mixture category in the original model (Caruso et al., 2018; Mohl et al., 2020; Jun et al., 2022; Schmehl et al., 2024; Groh et al., 2024), but because the assignment to any given category is impacted by the competition from other categories, the match will be affected by the introduction of the new categories. Specifically, some mixture may now fall into either the fast-juggling or non-benchmarked overreach categories (Figure 2, Table 1). Conditions that previously fell into the former intermediate category may now be assigned either to fast-juggling or fixed-middle. single is labeled here either fixed-preferred or fixed-non-preferred, depending on which single stimulus “wins”. And the former outside category is now likely to subdivide into fixed-outside and overreach. In total, the additional categories provide enhanced competition for the old mixture classification, strengthening the robustness of the testing framework in two ways: the odds that overdispersed responses that are not truly related to the individual stimulus distributions are erroneously assigned to the mixture category is reduced, and the possibility of detecting faster fluctuations that would have been overlooked before is improved. This enhancement will be revisited through real analysis, as illustrated in Figure 7.

Table 1.

Potential relationship between the whole trial spike count model categories of the original model (columns) and the categories/subcategories of the SCAMPI model (rows). New potential re-assignments that are of particular interest from a neuroscience perspective are indicated with an uppercase X. For example, the previous mixture category could have included what we now attempt to identify as slow-juggling and fast-juggling, and may also have contained cases better classified as overreach (overdispersed but unrelated to the benchmarks established from the A-alone and B-alone trials). Similarly, the intermediate category and outside category might encompass multiple distinct categories in the SCAMPI model.

			Original
			Mixture	Single	Intermediate	Outside
SCAMPI	Slow juggling		X
	Fast juggling		X		X
	Overreaching		X			X
	Fixed-	Preferred		x
		Non-preferred		x
		Middle			X
		Outside				x

Fig. 7.

Results from both the original model and the SCAMPI model for the IC dataset. Each stratum presents different categories under distinct frameworks, with the height indicating the frequency of triplets classified into each category. Each stream represents a triplet, flowing from the classification of the original model to that of the SCAMPI model. Classifications under the SCAMPI model was largely consistent with the original model. A significant proportion mixture triplets was classified as overreach, suggesting that the detected fluctuations were unrelated to encoding benchmarked stimuli. Moreover, more mixture triplets were classified as fast-juggling than slow-juggling, indicating a higher prevalence of fast-juggling in the IC dataset. Additionally, a non-negligible proportion of intermediate triplets was classified as fast-juggling, suggesting that triplets previously assumed to average between two benchmark stimuli may actually switch between them at a rapid rate. The SCAMPI model unravels previous ambiguous cases to a more refined scale.

2.2. Nonparametric Mixture Models

All four hypotheses specify the AB spike count distribution as $P^{\text{AB}}=\int \text{Poi}\left({\mu }^{\text{AB}}\right)\mathit{\text{dF}}\left({\mu }^{\text{AB}}\right)$ with an unknown rate mixing distribution $F$ on $(0,\infty )$, with distinctive requirements on the support of $F$ relative to the benchmark rates ${\mu }^{\text{A}}$ and ${\mu }^{\text{B}}$. For slow-juggling, $F$ is a discrete distribution with support $\left\{{\mu }^{\text{A}},{\mu }^{\text{B}}\right\}$, which can be succinctly expressed as $F\ll {\delta }_{{\mu }^{\text{A}}}+{\delta }_{{\mu }^{\text{B}}}$, where ${\delta }_x$ denotes the Dirac measure at a point $x$ and ≪ refers to absolute continuity of measures. For fast-juggling, $F\ll {\lambda }_{\left({\mu }^{\text{A}}\wedge {\mu }^{\text{B}},{\mu }^{\text{A}}\vee {\mu }^{\text{B}}\right)}$ where ${\lambda }_I$ denotes the Lebesgue measure restricted to an interval $I\subset \mathbb{R},u\wedge v=\text{min}(u,v)$ and $u\vee v=\text{max}(u,v)$. For overreach, $F\ll {\lambda }_{\left({\mu }_{\text{L}},{\mu }_{\text{U}}\right)}$, for some prefixed lower and upper bounds $0\le {\mu }_{\text{L}}<{\mu }_{\text{U}}<\infty$ on the overall spiking rates. For fixed, we have a degenerate $F={\delta }_{\mu }$ for some unknown $\mu >0$.

A marginal likelihood score for each hypothesis could be obtained by integrating out $F$ and $\theta =\left({\mu }^A,{\mu }^{\text{B}}\right)$. Because of the identity $p\left(Y^{\text{A}},Y^{\text{B}},Y^{\text{AB}}\right)=p\left(Y^{\text{A}}\right)p\left(Y^{\text{B}}\right)\int p\left(Y^{\text{AB}}\mid \theta \right)p\left(\theta \mid Y^{\text{A}},Y^{\text{B}}\right)d\theta$, and the fact that we have the same model for $\left(Y^{\text{A}},Y^{\text{B}}\right)$ across all scenarios, it suffices to concentrate only on $Y^{\text{AB}}$ as the observed data. The information in $\left(Y^{\text{A}},Y^{\text{B}}\right)$ can be completely captured through a second-stage prior $\pi \left(\theta \right)$ on $\theta$ obtained as its posterior distribution given $(Y^{\text{A}},Y^{\text{B}})$ under a first-stage prior ${\pi }_0$. We choose ${\pi }_0$ to be the Jeffreys’ prior, which is non-informative and invariant under reparametrization. We take ${\pi }_0\left({\mu }^{\text{A}},{\mu }^{\text{B}}\right)=1/\sqrt{{\mu }^A{\mu }^{\text{B}}}$, which produces a product gamma distribution in $\pi \left({\mu }^{\text{A}},{\mu }^{\text{B}}\right)=\text{Gam}\left({\mu }^{\text{A}}\mid 0.5+\sum _{j=1}^{n_{\text{A}}}{Y}_j^{\text{A}},n_{\text{A}}\right)\times \text{Gam}\left({\mu }^{\text{B}}\mid \right.\left.0.5+\sum _{j=1}^{n_{\text{B}}}{Y}_j^{\text{B}},n_{\text{B}}\right)$.

For the two multiplexing hypotheses, i.e., slow-juggling and fast-juggling, we adopt the predictive recursion marginal likelihood (PRML) method of Martin and Tokdar (2011) to integrate out $F$ and augment it with a Laplace approximation to integrate out $\theta$. These methods are described in the next subsection. In order to apply this framework, one first needs to use a change of variable to express the dependence of $P^{\text{AB}}$ on $\theta$ through a kernel ${\kappa }_{\theta }$ rather than the support and continuity of $F$. For the slow-juggling case, we can write $P^{\text{AB}}\left(y\right)=\int {\kappa }_{\theta }(y\mid u)f\left(u\right)d\nu \left(u\right)$ with $\nu ={\delta }_0+{\delta }_1$, where ${\kappa }_{\theta }(y\mid u)=\text{Poi}\left(y\mid u{\mu }^{\text{A}}+(1-u){\mu }^{\text{B}}\right),u\in \left\{\mathrm{0,1}\right\}$. The same kernel works for the fast-juggling case, but now with $u\in \left(\mathrm{0,1}\right)$ and $\nu ={\lambda }_{\left(\mathrm{0,1}\right)}$.

In the case of overreach, the marginal likelihood involves an integration of $F$ alone. No integration over $\theta$ is needed since the support of $F$ is free of $\theta$. We approximate the integral over $F$ via PRML with the kernel $\kappa (y\mid u)=\text{Poi}\left(y\mid u{\mu }_{\text{L}}+(1-u){\mu }_{\text{U}}\right)$, each with $u\in \left(\mathrm{0,1}\right)$ and $\nu ={\lambda }_{\left(\mathrm{0,1}\right)}$. An initial guess $f_0$ of $f$ is needed to carry out PRML in each of these three cases. We take $f_0$ to be the appropriate uniform density with respect to the corresponding dominating measure $\nu$; see Table 2.

Table 2.

The SCAMPI model settings under different hypotheses on $P^{\text{AB}}$ in a reparametrized form. Notice that the fixed hypothesis is not listed here since we can calculate marginal likelihood by integrating out $\mu$.

Model	${\mathit{\kappa }}_{\mathit{\theta }}(\mathit{y}\mid \mathit{u})$	Support	${\mathit{f}}_0\left(\mathit{u}\right)$	$\mathit{\nu }$
slow-juggling	$\text{Poi}\left(y\mid u{\mu }^{\text{A}}+(1-u){\mu }^{\text{B}}\right)$	$u\in \left\{\mathrm{0,1}\right\}$	0.5	${\delta }_0+{\delta }_1$
fast-juggling	$\text{Poi}\left(y\mid u{\mu }^{\text{A}}+(1-u){\mu }^{\text{B}}\right)$	$u\in \left(\mathrm{0,1}\right)$	1	${\lambda }_{\left(\mathrm{0,1}\right)}$
overreach	$\text{Poi}\left(y\mid u{\mu }_{\text{L}}+(1-u)\left({\mu }_{\text{U}}\right)\right)$	$u\in \left(\mathrm{0,1}\right)$	1	${\lambda }_{\left(\mathrm{0,1}\right)}$

Finally, the marginal likelihood score for fixed could be calculated via direct integration over the scalar parameter $\mu$. Again, no integration over $\theta$ is necessary. The integration over $\mu$ must be carried out under a suitable prior specification. A reasonable default choice is the Jeffreys’ prior $\pi \left(\mu \right)=C{\mu }^{-1/2}$, defined up to an arbitrary constant $C>0$. This presents a technical difficulty for marginal likelihood calculation, because the integral is also defined up to the arbitrary multiplicative constant $C$. A potential remedy is to borrow from the intrinsic Bayes factor proposal of Berger and Pericchi (1996) where marginal likelihoods are replaced with intrinsic marginal likelihoods. Specifically, a subset of the data is sacrificed to update (train) the improper prior into a proper probability distribution, which is then used to calculate the marginal likelihood of the remaining data. Berger and Pericchi (1996) advocate carrying out the intrinsic adjustment to all hypotheses under consideration, and using the same training subset across models. We do not pursue this approach because the intrinsic adjustment to the other three models substantially adds to the computational cost. We experimented with a cost-effective modification of this proposal where the intrinsic adjustment was applied only to fixed, but the resulting tests were found to be heavily biased in favor of fixed in synthetic data experiments (Appendix Figure A4). A second remedy is to truncate Jeffreys’ prior to the compact interval $\left[{\mu }_{\text{L}},{\mu }_{\text{U}}\right]$ used for overreach. We do not pursue this either because our simulation studies showed that the resulting tests are heavily biased against fixed (Appendix Figure A5).

Instead, we focus on fixing $C$ to ensure that the probability of incorrectly rejecting fixed is controlled at a given threshold. In other words, if one fixated on our test as an assessment of fixed versus the other hypotheses, what $C$ do we need to ensure a given $\alpha$ level of significance? We established through a numerical study (Appendix Figure A3) that the choice of $C=1$ ensures an $\alpha =5\%$. All our numerical analyses are carried out with the choice of $C=1$.

2.3. Predictive Recursion Marginal Likelihood and Bayes Factor

Suppose data $Y_1,\dots ,Y_n$ are modeled as independent realizations from the mixture distribution

$$m_f\left(y\right)=\int \kappa (y\mid u)f\left(u\right)d\nu \left(u\right)$$ (6)

where $(y,u)\mapsto \kappa (y\mid u)$ is a known kernel on $𝒴\times 𝒰$ and $f$ is an unknown mixing density in $\mathbb{F}$: the set of probability densities with respect to a $\sigma$-finite Borel measure $\nu$ on $𝒰$. The predictive recursion algorithm (Newton et al., 1998) estimates $f$ by starting with an initial guess $f_0\in \mathbb{F}$ and then making a single pass through the observations to recursively update to a final estimate $f_n$. At stage $i\in \{1,\dots ,n\}$ of the recursion, only observation $Y_i$ is used to update the current estimate $f_{i-1}$ to a new estimate $f_i$ according to,

$$f_i\left(u\right)=\left(1-w_i\right)f_{i-1}\left(u\right)+w_i\frac{\kappa \left(Y_i\mid u\right)f_{i-1}\left(u\right)}{m_{i-1}\left(Y_i\right)},u\in 𝒰,$$ (7)

where $w_i\in \left(\mathrm{0,1}\right)$ are prefixed recursion weights and

$$m_{i-1}\left(y\right)=\int \kappa (y\mid u)f_{i-1}\left(u\right)d\nu \left(u\right),y\in 𝒴,$$ (8)

is the estimated predictive density of $Y_i$ given $Y_1,\dots ,Y_{i-1}$. Under certain regularity conditions on the kernel $\kappa (y\mid u)$, both $f_n$ and $m_n$ asymptotically converge to the true densities $f=f^{\star }$ and $m=m^{\star }≔m_{f^{\star }}$, respectively, provided ${\sum }_{i=1}^{\infty }{w}_i=\infty$ and ${\sum }_{i=1}^{\infty }{w}_i^2<\infty$ (Tokdar et al., 2009). The regularity conditions remain valid in our case, wherein we utilize Poisson kernels with mean parameters restricted within a compact space.

When the kernel $\kappa ={\kappa }_{\theta }$ involves an unknown parameter $\theta \in \Theta$, Martin and Tokdar (2011) recommend using the PRML score

$$L_n\left(\theta \right)≔\prod _{i=1}^n{m}_{i-1,\theta }\left(Y_i\right)$$

to evaluate a marginal likelihood score of $\theta$, which is easily computed as a byproduct of running the predictive recursion algorithm with $\theta$ fixed at the given value. This recommendation is justified on three accounts. First, since $m_{i-1,\theta }$ gives an estimate of the predictive density of $Y_i$ given $Y_1,\dots ,Y_{i-1}$ and $\theta$, the proposed likelihood score gives a corresponding estimate of the true marginal density evaluation $p\left(Y_1,\dots ,Y_n\mid \theta \right)=\prod _{i=1}^{n}p\left(Y_i\mid Y_1,\dots ,Y_{i-1},\theta \right)$. Second, under appropriate regularity conditions, the PRML score enjoys the asymptotic consistency property that $n^{-1}\text{log}\left\{L_n\left(\theta \right)/L_n\left({\theta }^{\star }\right)\right\}\to -{\text{inf}}_{f\in \mathbb{F}}{d}_{\text{KL}}\left(m^{\star },m_{f,\theta }\right)$. Third, for the special choice of $w_i=(1+a{)}^{-1}$, for some $a>0$, predictive recursion could be seen as an expectation filtration approximation to a fully Bayesian estimation of $f$ under a Dirichlet process prior with precision $a$ and base density $f_0$, and hence, $L_n\left(\theta \right)$ could be seen as the corresponding filtration approximation to the fully Bayesian marginal likelihood score of $\theta$. Martin and Tokdar (2011) provide extensive numerical evidence of the latter correspondence.

Now consider a finite set of competing mixture models $M_h=\left\{m(\cdot )=\int {\kappa }_{\theta }^h(\cdot \mid \right.\left.u\left)f\right(u\left)d{\nu }_h\right(u):\theta \in {\Theta }_h,f\in {\mathbb{F}}_h\right\},h\in H$. We propose to evaluate a Bayes factor $B_{i,j}$ between models $M_i$ and $M_j$ as $B_{12}=I\left(M_i\right)/I\left(M_j\right)$ where

$$I\left(M_h\right)≔{\int }_{{\Theta }_h}{L}_{n,h}\left({\theta }_h\right){\pi }_h\left({\theta }_h\right)d{\theta }_h,h\in H,$$

gives the integrated PRML score under an appropriate prior ${\pi }_h$ on ${\Theta }_h$. Again, our interpretation of this ratio as a Bayes factor relies on interpreting each $L_{n,h}\left(\theta \right)$ as an approximation to the marginal likelihood score under the corresponding Dirichlet process mixture idealization of $M_h$. Extending this analogy we also propose computing a posterior probability for model $M_h$ as:

$$p_h\left(Y\right)=\frac{I\left(M_h\right)p_{0,h}}{\sum _{h^{\prime }\in H}I\left(M_{h^{\prime }}\right)p_{0,h^{\prime }}}$$

where $p_{0,h},h\in H$ are prior weights assigned to the competing hypotheses. The hypothesis with the highest posterior probability is the winning model, where the posterior probability measures the confidence of making a right choice from these four hypotheses suggested by the data. We consider 0.25 – 0.5, 0.5 – 0.75, 0.75 – 1.0 as the weak, moderate and strong confidence on the model choice respectively.

Assuming that the kernel ${\kappa }_{\theta }^h$ is smooth in $\theta$ allows us to approximate $I\left(M_h\right)$ by a Laplace type approximation: $\hat{I}\left(M_h\right)=L_{n,h}\left({\hat{\theta }}_h\right)(2\pi {)}^{-d_h/2}{\left|{\Sigma }_h\right|}^{1/2}$, where ${\hat{\theta }}_h={\text{arg}\text{max}}_{\theta \in {\Theta }_h}{L}_{n,h}\left(\theta \right)$, and ${\Sigma }_h={\left\{-{\nabla }^2\text{log}{L}_{n,h}\left({\hat{\theta }}_h\right)\right\}}^{-1}$. The necessary optimization could be carried out by using the gradient based extension of PRML in Martin and Tokdar (2011) in which $\nabla \text{log}{L}_n\left(\theta \right)=\sum _{i=1}^n\nabla \text{log}{m}_{i-1,\theta }\left(Y_i\right)$ is calculated recursively during the same single pass through the data used for the original predictive recursion and PRML calculation. In our applications, we use the gradient-based optimzation algorithm due to Broyden-Fletcher-Goldfarb-Shanno (BFGS) to compute ${\hat{\theta }}_h$. Then, a numerical evaluation is done of ${\Sigma }_h$.

It is worth noting that the PRML score $L_n\left(\theta \right)$ depends on the order in which the observations enter the single-pass recursive algorithm. Such an order dependence is problematic from both theoretical and practical perspectives when data are believed to be exchangeable. We could mitigate this problem by considering a permutation version of the PRML score that averages over the $n!$ many scores obtained by running the single pass algorithm on every possible permutation of the data. In practice, however, it suffices to consider a Monte Carlo approximation, where averaging is done on a subset of randomly drawn permutations.

3. Performance Assessment

3.1. Experimental Design

To assess statistical accuracy of the SCAMPI model, we conducted experiments using synthetic data and focused on two key aspects: detecting fluctuating activity, and correctly identifying $P^{\text{AB}}$ as one of fixed, slow-juggling, fast-juggling, and overreach. We generated data from four scenarios: F, SJ, FJ, and O, which correspond to four hypotheses: fixed, slow-juggling, fast-juggling, overreach, respectively. These four scenarios have a tuning parameter controlling the complexity of the classification task. For each scenario, 100 experiment sets were created, each set comprising of synthetic A, B and AB trial data with $n_{\text{A}}=n_{\text{B}}=n_{\text{AB}}\in \left\{\mathrm{20,30,50}\right\}$. Trial sizes and average A and B firing rates were set to broadly match similar statistics obtained from IC data described in Section 4. For every experiment set, A and B spike counts were generated, respectively, from $P^{\text{A}}=\text{Poi}\left(50\right)$ and $P^{\text{B}}=\text{Poi}\left(80\right)$.

The cases differed from one another in the choice of $P^{\text{AB}}$ and the complexity parameter. The exact form of data generating models $P^{\text{AB}}$ are:

$${\text{Scenario}\text{F}:P}^{\text{AB}}=\text{Poi}\left(80m\right)$$

$${\text{Scenario}\text{SJ}:P}^{\text{AB}}=m\text{Poi}\left(50\right)+(1-m)\text{Poi}\left(80\right)$$

$${\text{Scenario}\text{FJ}:P}^{\text{AB}}={\int }_0^1\text{Poi}(r50+(1-r\left)80\right)\text{Beta}\left(r;m{r}_0,m\left(1-r_0\right)\right)\mathit{\text{dr}}$$

$${\text{Scenario}\text{O}:P}^{\text{AB}}=1/3\text{Poi}\left(50\right)+2/3\text{Poi}\left(80m\right).$$

A visual summary is given in Figure 3. For scenario F, complexity is coded as a multiplier $m$ to the mean. The multiplier $m$ is varied to encompass all sub-types of the fixed hypothesis: fixed-preferred, fixed-non-preferred, fixed-middle, fixed-outside. With $m=5/8,P^{\text{AB}}$ equals $P^{\text{A}}$. Similarly, with $m=1,P^{\text{AB}}=P^{\text{B}}$. Notice that $P^{\text{AB}}=P^{\text{A}}$ and $P^{\text{AB}}=P^{\text{B}}$ are boundary cases for each of the other three hypotheses. For example, setting $\rho =1$ in the mixture formulation (2) produces $P^{\text{AB}}=P^{\text{A}}$. Therefore $m=5/8$ or $m=1$ corresponds to ambiguity between fixed and other hypotheses, making the classification task more complex for $m$ values near these two points. In addition, if $m$ falls between these two points, where $m\in (5/\mathrm{8,1})$, the corresponding firing rate ${\mu }^{\text{AB}}$ will be bounded by the firing rates of constituent stimuli ${\mu }^{\text{A}}$ and ${\mu }^B$. As a result, the corresponding probability distribution $P^{\text{AB}}$ becomes a borderline case of fast-juggling, which also poses a great challenge for classification.

Fig. 3.

Choice of $\left(P^{\text{A}},P^{\text{B}},P^{\text{AB}}\right)$ for all groups in the experimental design. Grey graphs show examples of $P^{\text{AB}}$, generated under fixed, slow-juggling, fast-juggling, and overreach assumptions.

For scenario SJ, complexity increases as the skewness parameter $m$ gets close to either 0 or 1, making $P^{\text{AB}}$ close to the boundary cases, respectively, $P^{\text{A}}$ or $P^{\text{B}}$.

For scenario FJ, where a beta distribution with mean $r_0$ is used as the mixing density, complexity is controlled by varying the precision parameter $m$, which in turn controls the Fano factor of the resulting $P^{\text{AB}}$. We take $r_0=0.56$ and let $m$ vary from zero to infinity so that the Fano factor goes from a minimum possible value of 1 to a maximum possible value of 4.5. When the Fano factor is $1,P^{\text{AB}}=\text{Poi}\left(63.2\right)$ and hence can also be explained as fixed. When the Fano factor is at the maximum, $P^{\text{AB}}=0.56\text{Poi}\left(50\right)+0.44\text{Poi}\left(80\right)$, causing confusion with slow-juggling. So the complexity of the task is high at both extremes of the Fano factor values. See Appendix C for more explanation of the choice of $r_0$ and the range of Fano factor values one can obtain with a beta mixing density.

For scenario O, we consider bimodal mixtures of $\text{Poi}\left({\mu }^{\text{A}}\right)$ with another $\text{Poi}\left(m{\mu }^{\text{B}}\right)$, with the multiplier $m$ controlling complexity. As $m$ moves closer to $1,P^{\text{AB}}$ reduces to a mixture of $\text{Poi}\left({\mu }^{\text{A}}\right)$ and $\text{Poi}\left({\mu }^{\text{B}}\right)$ and hence can be explained by the slow-juggling hypothesis, making a correct classification of overreach difficult.

For each scenario, PRML based posterior probabilities of the four competing hypotheses were calculated. For each set, we recorded whether the correct label received the maximum posterior probability, i.e., whether a correct identification was made. Additionally, we recorded whether the test correctly detected multiplexing. It’s important to note the distinction between correct classification and correct detection of multiplexin. If the truth were slow-juggling, being classified as fast-juggling will still be counted as a correct detection of multiplexing even though it is a misclassification. Similarly, when a true fast-juggling case is misclassified as slow-juggling, it would still count toward a correct detection of multiplexing. In other words, when the focus is detection of multiplexing, a correct identification of the exact sub-type is viewed as of secondary importance.

3.2. Results

Accurate detection of multiplexing is the primary focus for our applications. We consider two key metrics: sensitivity, which is the proportion of multiplexing cases that are correctly identified as multiplexing, and specificity, which is the proportion of non-multiplexing cases that are correctly identified as not being multiplexing. Figure 4 demonstrates that our SCAMPI model performs well in terms of sensitivity and specificity. For data generated from scenario F, specificity is above 0.9, even for cases where the multiplier $m$ is close to either 1 or 5/8, or falls between these two points. For data generated from scenario O, even with a small sample size of 20, specificity can reach 0.75 if the multiplier $m$ is greater or equal to 1.15, where one of two modes is reasonably away from ${\mu }^{\text{B}}$. As the sample size increases, specificity improves significantly when the multiplier $m$ ranges between 1.1 to 1.15. The sample size has less impact on specificity as the multiplier $m$ approaches 1. For data generated from scenario SJ, sensitivity is above 0.8 if the skewness parameter is less than 0.85. Increasing the sample size to 50 improves sensitivity to be around 0.75 even for the complicated cases, where the skewness parameter is equal to 0.95. When the true generating process is scenario FJ, sensitivity is above 0.85, if the Fano factor is greater than or equal to 2.5. However, as the Fano factor approaches 1, more fast-juggling triplets are mis-classified as fixed, lowering sensitivity. Increasing the sample size to 50 mitigates this issue to some extent, ensuring sensitivity is at least 0.625 even for complex cases, where the Fano factor is equal to 1.5. It is worth noting that as the Fano factor approaches 4.5, more fast-juggling triplets are mis-classified as slow-juggling, which is still a sub-type of multiplexing, and thus may not be of much practical concern.

Fig. 4.

Classification accuracy and multiplexing detection accuracy versus complexity parameters across fixed, slow-juggling, fast-juggling, and overreach experimental cases (columns). Sample size 20, 30, 50 are color coded as amber, green and blue.

The most challenging cases are found at the limits of the model spaces, where they lie at the boundary of four hypotheses. These cases can be categorized into three main types. Firstly, sensitivity is low for scenario SJ with a high skewness or for scenario FJ with a low Fano factor, where it tends to be mis-classified as fixed. Secondly, specificity is low for scenario O with a low multiplier, where it tends to be mis-classified as slow-juggling. Thirdly, there is ambiguity between sub-types of multiplexing for scenario FJ with a high Fano factor, since it tends to be classified as slow-juggling. As explained earlier, the model spaces underlying the four hypotheses are nested in the order: fixed ⊂ slow-juggling ⊂ fast-juggling ⊂ overreach. For aforementioned complicated situations arising at the boundaries of the model spaces, a simpler model is always favored by the SCAMPI model due to Occam’s razor property of Bayes factor. Increasing the sample size can greatly improve the sensitivity for the first type of challenge, but has very limited impact on specificity for the second type of challenges, especially when cases are near the boundaries. Additionally, increasing the sample size does not help in identifying sub-types of multiplexing, the third type of challenge. As previously noted, distinguishing between two sub-types of multiplexing is not a primary concern in practical applications.

4. Applications

Caruso et al. (2018) first reported statistical evidence espousing the viewpoint that an individual neuron may display fluctuating activities when preserving dual stimuli in visual and auditory areas, such as the inferior colliculus (IC) area and the middle fundus (MF) face patch systems in the inferotemporal (IT) cortex. Subsequently, such fluctuating patterns were observed in other visual areas, including the primary visual cortex (V1), visual area V4, the middle temporal visual area (MT), and the anterior lateral (AL) face patch systems in the IT cortex (Jun et al., 2022; Schmehl et al., 2024). Moreover, fluctuations in V1 and MT have been demonstrated to relate to object identification (Jun et al., 2022; Schmehl et al., 2024). However, as we mentioned in Section 1 earlier, our original analysis method could not distinguish fast vs. slow juggling, and fluctuations that were not related to the single stimulus benchmarks could have been labeled as juggling when such a categorization may not be appropriate. Accordingly, we reanalyzed the IC, V1, and IT datasets from the earlier studies, to (1) confirm the presence of multiplexing; (2) determine what proportions may be fast vs. slow, (3) validate our earlier findings that multiplexing is more prevalent for combinations of faces than for face-object pairs in the face patch system (Schmehl et al., 2024); and is associated with object segregation (Jun et al., 2022) for individual neurons in the V1 area.

4.1. Datasets

IC dataset:

Caruso et al. (2018) reported a study where spiking activity of individual IC neurons were recorded while monkeys made saccades towards one or two sound locations (Figure 5(A)). In dual-sound trials, the two sounds were presented in different hemifields, and consisted of two distinct frequencies (differing in frequency by at least 22%). Every dual-sound condition was matched to two single-sound conditions corresponding to each of the two constituent location-frequency pairs. Stimulus conditions were shuffled within and across experiment sets. Here, the spike counting window was from 0 ms until 600 ms after stimulus onset (the original study used some longer time windows).

Fig. 5.

Stimulus conditions for the datasets re-analyzed in the current study. (A) For the IC dataset, bandpass sounds of different center frequencies were presented individually or simultaneously at different spatial locations, and monkeys made saccades to any and all sounds presented. No constraints were imposed on which sound the monkey looked at first. See Caruso et al. (2018) for details. (B) In the face-patch datasets, stimuli consisted of either faces or non-face objects presented at different locations. Monkeys performed a fixation task. See Ebihara (2015); Caruso et al. (2018); Schmehl et al. (2024) for details. (C) In the V1 superimposed gratings dataset, one or two gratings were presented at the same location while monkeys fixated. See Jun et al. (2022) and Ruff and Cohen (2016) for details. (D) In the V1 adjacent gratings dataset, the A and B conditions consisted of single gratings presented in the contralateral hemifield while monkeys attended to another grating presented in the ipsilateral hemifield. The AB condition involved both of those A and B gratings presented side-by-side while monkeys continued to attend to a third grating in the ipsilateral hemifield. See Jun et al. (2022); Ruff and Cohen (2016) for details.

IT face patch dataset:

In a study by Ebihara (2015), spike activities of individual cells in the MF and AL face patch areas were recorded when monkeys performed a fixation task (Figure 5(B)). The stimuli presented were classified into two categories: a preferred face which elicited a high firing rate, and a non-preferred stimulus (either a face or an object) which elicited little to no neural activity. During dual stimuli trials, a preferred face was always presented at the center of the receptive field, with the non-preferred stimulus presented adjacent to it simultaneously at varying locations. In corresponding single stimulus trials, one of the constituent stimuli was presented alone. Although different faces and objects were considered as different experimental conditions, the heterogeneity of the locations of non-preferred stimuli was ignored, following the analysis by Caruso et al. (2018). The spike counting window was set between 50 ms and 250 ms after stimulus onset (same as before).

V1 superimposed gratings dataset:

Ruff et al. (2016) utilized a multi-electrode array to record spike activities of multiple cells in the V1 area while monkeys passively viewed drifting gratings (Figure 5(C)). In dual stimuli trials, superimposed drifting gratings with orthogonal orientations and moderate contrast were presented, producing a “plaid” appearance. In corresponding single stimulus trials, the component gratings were presented individually. Distinct orientations of gratings defined different experimental conditions. Following the analysis of Jun et al. (2022), the spike counting window was between 30 ms and 230 ms after stimulus onset.

V1 adjacent gratings dataset:

Similar to the superimposed grating dataset, Ruff and Cohen (2016) also used multi-electrode recordings in V1 area, but their stimuli consisted of smaller drifting Gabor patches (Figure 5(D)). For the data included in the present study, either one grating or two gratings were presented in the contralateral hemifield, covering the set of receptive fields of the V1 neurons, while an additional grating was always presented in the ipsilateral hemifield. Monkeys performed a motion direction change detection task involving that ipsilateral stimulus. We treated trials with one contralateral grating as single stimulus trials and two contralateral gratings as dual stimulus trials. Following Jun et al. (2022), the spike counting window was taken to be 30 ms through 230 ms after stimulus onset.

4.2. Preprocessing

We reanalyzed the same data with the SCAMPI model with one additional difference in the preprocessing step. Caruso et al. (2018) tested the Poisson distribution assumption on A and B spike counts by using a chi-square goodness of fit test which calculated expected and observed counts by binning the data histogram. This test has the ability to detect both over- and under-dispersion. For the purposes of our analysis, overdispersion of $P^{\text{A}}$ and $P^{\text{B}}$ is a bigger issue than underdispersion. If trial-to-trial variability was caused by factors extrinsic to the chosen stimuli, one would expect overdispersed count distributions under all three conditions.

We replaced the chi-square goodness of fit test with thresholding Fano factor (variance-to-mean ratio) greater or equal to 3 to remove the triplets that had heavily over-dispersed distributed $P^{\text{A}}$ and $P^{\text{B}}$ (Schmehl et al., 2024). By filtering out triplets where either $P^{\text{A}}$ or $P^{\text{B}}$ is overdispersed, we secured a more conservative position in interpreting classification analysis results of the remaining triplets purely through the lens of stimulus-related switching. A sensitivity analysis (see Figure A10 in the Appendix) confirmed the robustness of both the classification results and the drawn conclusions across varying Fano factor thresholds. A threshold of 3 assured robust results while retaining an appropriate amount of data for each dataset. This choice preserved slightly more triplets compared to the screening based on Poisson variance test with p-value greater than 0.001.

Keeping with other preprocessing steps of previous studies (Caruso et al., 2018; Jun et al., 2022; Schmehl et al., 2024), we only included triplets having at least 5 trials for each condition (A, B, AB), and with well-separated⁴ single-stimulus distributions $P^{\text{A}}$ and $P^{\text{B}}$. Summary statistics for each dataset are shown in Appendix Figure A1. Data form example cells are shown in Figures A8 and A9 in the Appendix.

4.3. Results and Interpretation

We applied the SCAMPI model on the preprocessed triplets to first classify the triplets into four categories, namely fixed, fast-juggling, slow-juggling, and overreach. To permit comparisons with earlier findings (e.g. Caruso et al. (2018)), we also further sub-classified the fixed triplets $(P^{\text{AB}}=\text{Poi}(\mu ),\mu \in (0,\infty \left)\right)$ into the fixed-preferred, fixed-non-preferred, fixed-middle and fixed-outside categories (Figure 2, Table 1). Triplets were classified according to the highest posterior probability, which also quantifies the level of confidence in the classification. A full report of the number of triplets categorized in each category by dataset and confidence level is provided in Appendix Figure A2 and Table A2. To facilitate comparison across datasets, we calculated the percentage of occurrences for each “winning” category within a given dataset, subsequently comparing these percentages across datasets within the same winning category.

Results for category assignments are illustrated in Figure 6, focusing first on the two multiplexing categories, fast and slow (Figure 6(A)). These results reaffirm our earlier findings that multiplexing is a general coding scheme observed in the IC area, IT area, and V1 area (Figure 6(A)). In the IC, fast-juggling and slow-juggling together accounted for a non-negligible proportion of the classifiable – about 18.2% of such triplets. In contrast with our earlier evaluation of this dataset, the new analysis method suggests that fast-juggling is more common than slow-juggling (blue bar, ~ 12.5%, vs orange bar, ~ 5.7%).

Fig. 6.

Results of the SCAMPI model for the IC, IT face patch, and V1 datasets. Percentages shown reflect the percentage of conditions, out of the total number of conditions. (A) Prevalence of fast-juggling and slow-juggling across datasets. fast-juggling is seen in non-negligible amounts in all datasets except the V1 “superimposed grating” dataset. It is generally more common than slow-juggling. Juggling is more common for face-face than for face-object datasets in the facepatch system. (B) Prevalence of overreach. This was most common in AL. (C) Prevalence of fixed (fixed-preferred and fixed-non-preferred). Conditions were far more likely to be classified as fixed-preferred than fixed-non-preferred. (D) Prevalence of fixed-middle. Such a response pattern may indicate that neurons are averaging their inputs. (E) Prevalence of fixed-outside. This pattern was chiefly observed in the V1-superimposed grating dataset.

In IT cortex, fast-juggling was observed – at roughly equivalent levels – in both AL and MF when two faces were presented (21.3% and 19.5% respectively), and in both areas the prevalence of fast-juggling was higher for two-faces than for face-object combinations (AL: 11.9% and MF: 8.0% respectively)⁵. This supports the suggestion we previously made in Schmehl et al. (2024) that selectivity for the type of stimulus involved may contribute to the prevalence of switching patterns. In the case of the face patch system, non-face objects are not thought to evoke strong responses; rather, a distinct population of neurons outside these areas are likely to be encoding these stimuli and “juggling” may therefore not be needed within the face patches when only one of the two stimuli is a face.

In V1, 6.8% of the triplets were categorized as “juggling” when adjacent stimuli were presented. Among these, fast-juggling (3.9%) was slightly more common than slow-juggling (2.9%). Although “juggling” was seen at lower levels than in the IC and IT cortex regions, the relative rarity of such switching potentially relates to the much smaller size of the receptive fields in this structure. Small receptive fields mean different stimuli can potentially be encoded by separate subpopulations of neurons, reducing the need for “juggling” within neurons.

In contrast, “juggling” was much less prevalent in the V1 superimposed gratings experiment, where Gabor patches were presented as a fused object (a plaid, see Figure 5(C)): only 0.7% of triplets were labeled as multiplexing. This confirms our previous observation that “juggling” appears to be connected to perceptual object formation (Jun et al., 2022; Schmehl et al., 2024; Groh et al., 2024).

Across all the datasets and conditions, fast-juggling was more prevalent than slow-juggling. This supports our earlier suspicion that the previous mixture category, while nominally identifying switching activity across trials, included many cases of activity where some “switches” actually happened within trials (We will discuss in more detail later, as shown in Figure 7 for an example). Our current results suggest that such within trial fluctuation is indeed present.

The winning percentages of the other model categories varied across studies. overreach was relatively common among the studies that showed signs of juggling, but rare in the V1 superimposed dataset (2.3%) (Figure 6(B)). Recall that overreach is indicative of fluctuating activity under dual-stimuli exposure, just that the patterns of these fluctuation are not demarcated by the A and B benchmarks. It is entirely possible that some overreach patterns represent a form of multiplexing but one in which the spike count benchmarks observed on single-stimulus trials do not yield good predictions of those found on dual-stimulus trials. One possibility is that an attentional process comes into play on dual stimulus trials, adjusting the signal strength for each stimulus and changing the expected benchmark values. Another possibility is that a more subtle operation involving spike timing is at play, which analyses at the spike count level overlook; a finer timescale analysis of the underlying spike trains would be necessary. Such directions remain interesting avenues for future explorations.

Across all studies, a common occurrence was neurons responding to just one stimulus or another (fixed-preferred and fixed-non-preferred); see Figure 6(C). Such responses are information-preserving, since only one stimulus is encoded in a given neuron’s activity pattern. Whereas previously we did not distinguish between “winner-take-all”, i.e., responding at a level corresponding to the preferred stimulus, and “loser-take-all”, i.e., responding at a level corresponding to the non-preferred stimulus, we can now see that the majority of response patterns across datasets involve encoding the preferred stimulus (Figure 6(C) blue bars larger than orange bars).

Membership in the categories fixed-middle and fixed-outside varied considerably across datasets (Figure 6(D) vs Figure 6(E)). fixed-middle suggests that an averaging operation is at play, i.e., neurons may be responding at a value corresponding to the average of their A and B-like responses. This was more common in the IC and the MF face patch than in the other datasets. However, this pattern can also occur if there is switching between signaling of A and B that is both fast and regular. That is, if neurons switch rapidly between encoding A and encoding B within the spike-counting window, and if the amount of time spent encoding A and encoding B is consistent across trials, then the resulting spike count is likely to be well fit by a single Poisson with a mean rate that is in the middle between the A and B rates. Indeed, the DAPP model of Caruso et al. (2018) and Glynn et al. (2021) identified patterns of rapid fluctuation for these two datasets.

fixed-outside was the predominant outcome in the V1 superimposed datasets, accounting for 43.1% of the classifiable conditions. This observation suggests that neurons may treat the fused object as a new object unrelated to its constituent stimuli in multiple trials. In contrast, in the V1 adjacent gratings dataset, where Gabor patches were presented adjacent to each other (Figure 5(D)), only 2.1% of triplets are labeled as fixed-outside, while 6.8% of triplets are labeled as multiplexing, which is significantly different from the rate under superimposed conditions.

4.4. Comparison of classifications between the original Caruso et al. (2018) and SCAMPI models

Here we turn to a detailed comparison of the SCAMPI findings with those of the earlier analyses which were primarily focused on the slow-juggling hypothesis. We use the IC study as a representative example (refer to the alluvial plot Figure 7) to showcase the mapping between the two frameworks. Results for the remaining studies are provided in the Appendix Figure A7 and Figure A6.

In Figure 7, the left and right columns depict classifications under the original model and SCAMPI model, respectively. Each stratum displays different categories under different frameworks, with the height representing the frequency of triplets classified into a particular category. Each stream represents a triplet, colored according to its classification under the SCAMPI model. A stream flowing from mixture to fast-juggling indicates that the triplet was classified as mixture in the original model but as fast-juggling in SCAMPI model.

We note that ambiguity primarily occurred within mixture and occasionally in intermediate categories, consistent with Table 1. As intended, the mixture category of the original model now subdivides into several classifications in the SCAMPI model. About half of previously labeled mixture triplets retain a multiplexing classification, roughly split 60–40 between fast and slow juggling. This suggests that rapid switching within a trial was more prevalent than switching across trials. The remainder are relabeled overreach under which the switching behavior is not tightly tied to the A and B spike count benchmarks. This further underscores the competitive strength of overreach as an alternative, effectively addressing the gap in the previous framework. A minority of mixture triplets were classified as fixed(fixed-preferred, fixed-non-preferred, fixed-middle), suggesting that SCAMPI model exhibited greater tolerance to overdispersion for the fixed category.

Among intermediate triplets, a majority were categorized as fixed-middle, indicating consistency between the two frameworks. A small subset of intermediate triplets was classified as fast-juggling, implying that in some cases information-preserving fluctuation may have occurred at a faster rate, a detail previously hidden within the intermediate category in the original model.

The remaining classifications under the original model mostly aligned with the results from the SCAMPI model. The majority of triplets labeled as single were classified as either fixed-preferred or fixed-non-preferred (with some of each). Additionally, all triplets labeled as outside were classified as fixed-outside(with some re-categorized as overreach).

The disparity between the two frameworks offers evidence that the SCAMPI model enhances and refines the original model by enabling the detection of jugging within a trial (fast-juggling) and ruling out fluctuations unrelated to information preservation (overreach).

4.5. Assessment of false discovery rate

Finally, to guard against false discoveries, we evaluated Bayesian False Discovery Rates (BFDRs) for each dataset: 33.5% (IC), 45.8% (IT-AL), 41.3% (IT-MF), 37.8% (V1-imp), 38.3% (V1-adj). BFDR represents the expected proportion of multiplexing being false discoveries, calculated by averaging posterior probabilities of being non-multiplexing over the cases labeled as multiplexing (fast-juggling and slow-juggling). The relatively high values of BFDR suggest that a non-negligible proportion of triplets are hard to accurately classify, leaving open the door to more precise statistical evaluation.

5. Discussion

This paper proposes new statistical tools for evaluating evidence of multiplexing in neural activity under dual stimuli exposure relative to the activity patterns triggered by each constituent stimulus in isolation. Our findings suggest that although multiplexing may not be detectable in every neuron in a population, it manifests within a non-negligible sub-population. This manifestation, in spite of its partial nature, is noteworthy because we believe multiplexing could be a key computational tool for the brain to preserve multiple stimuli at little additional cost as the number of simultaneously presented stimuli increases. Let us elaborate.

On a first thought, it might seem unlikely that the brain should implement a flexible switching operation to preserve information about multiple stimuli. But the problem must be solved somehow and there are few compelling alternatives. It is unlikely that there is a lookup table with a unique value in the neural code for every possible stimulus combination that might exist. Such a computing strategy will be quickly defeated by the curse of dimensionality of the signal space. In contrast, the multiplexing paradigm is not plagued by the curse of dimensionality of the signal space, as it takes advantage of task splitting over the temporal dimension so that every single neuron can alternate between encoding only a small set of signals that are within its receptive field. Such compositional computing strategies are massively scalable and could be efficient methods for preserving information from complex sensory scenes.

Although our analysis offers scientific evidence supporting the existence of multiplexing in multiple brain areas, as well as its association to object identification, there is room for improvement. Indeed, our Bayesian false discovery rate analyses suggest that some of the triplet classifications are associated with low confidence. A partial explanation of this classification ambiguity is that spike counts mask a lot of the information about the neuronal activity dynamics that take place within the course of each trial. Consequently, trialwise aggregated spike count analysis could average over important features, diminishing the statistical separation between juggling and non-juggling activities. Future work will look into whether a deeper analysis of spike timing data could produce more precise statistical evaluation.

One potential extension is to incorporate over-dispersion in single-stimulus trials. Although neuronal response to a single stimulus is typically modeled as Poisson distributed around an expected count (Ventura et al., 2002; Kass et al., 2005), in some datasets many neurons fail to exhibit single-stimulus trials that conform to Poisson assumptions (e.g., Amarasingham et al., 2006; Maimon and Assad, 2009; Stevenson, 2016; Pillow and Scott, 2012). Specifically, about half of neurons in the IC suffer from this problem, whereas in general more than 90% of neurons in visual areas survive screening for Poisson-like response patterns (Figure A10). To address this, we could extend the Poisson model to a negative binomial model. However, such an extension is not straightforward to analyze statistically. Under the neural circuit described after (3), the distribution of AB spike counts under the intermediate hypothesis would be that of $Y^{\text{AB}}=X^{\text{A}}+X^{\text{B}}$ where $X^{\text{A}}~\text{Bin}\left(W^{\text{A}},r\right)$ and $X^{\text{B}}~\text{Bin}\left(W^{\text{B}},1-r\right)$ with $W^{\text{A}}~P^{\text{A}}$ and $W^{\text{B}}~P^{\text{B}}$. As we saw earlier, the distribution of $Y^{\text{AB}}$ could be neatly expressed by the formula in (3) provided both $P^{\text{A}}$ and $P^{\text{B}}$ are Poisson distributions thanks to the binomial thinning and additivity properties of the Poisson. Because Poisson is the only distribution with these properties, no such formula exists for any other count distributions! Therefore, statistical analysis of multiplexing with non-Poisson assumptions would require substantive extension of the method developed in this paper.

An additional avenue for extension could involve a more detailed analysis of non-benchmarked fluctuation. In the current framework, we introduce overreach as a control for multiplexing, encompassing notable cases where switches occur between mixture component means that fall outside the range defined by the benchmarks.

There are two potential classes of interpretation of this kind of pattern. One possibility is that the overdispersion is unrelated to the presence of the two stimuli per se, but rather has occurred because of other kinds of non-stationarity in the neural signals (Deitch et al., 2021; Montijn et al., 2016). For example, neural responses can depend on variables such as motivational state (e.g. Metzger et al., 2006), which will tend to fluctuate over the course of a recording session, thus potentially causing overdispersed signals. The quality of isolation of single unit recording can also degrade over time, potentially increasing the apparent variability of neural responses. The potential for such effects is a strong reason for including “overdispersion” as a category, to limit the possibility that such factors could lead to a neuron’s responses being erroneously categorized as showing switching between coding of A vs. B.

A more conceptually interesting possibility is that the “overdispersed” category could include cases where the activity is indeed fluctuating between two values, but the values do not correspond to the benchmarks established on the single stimulus trials. One possible explanation for this could involve attentional processes. During dual stimulus trials, attention might modulate the signal strength associated with each stimulus, potentially leading to shifts in the expected benchmark means. Another explanation could be the presence of subtle dynamics occurring on a finer timescale, which may be obscured by the trialwise aggregated spike counts. For example, each switch might incur a “cost” by briefly inhibiting neuronal firing. Although multiplexing occurs within a trial, it may still be classified as overreach because the mixture component means estimated from spike counts could deviate from the benchmark due to this inhibition. Finally, if the switching process is leaky or imperfectly timed, there could be temporal epochs in which input from both “A” and “B” sources reach the neuron under study, temporarily causing firing rates that are higher than expected from either input alone. Future research could focus on conducting a more granular analysis of the underlying spike trains to investigate these phenomena further.

A broader issue that we can address in our future research is the neural coordination for information preservation. The multiplexing patterns of a single neuron have paved the way for understanding how neural populations preserve information from multiple stimuli. According to a recent study by Jun et al. (2022), individual neurons exhibit fluctuating patterns that are positively and negatively related to other neurons with similar and different stimulus preferences, respectively. This finding demonstrates pairwise synchronous and asynchronous coordination among the neuron population. Moving forward, our future research will focus on detecting joint synchronous and asynchronous multiplexing, known as co-multiplexing, among multiple neurons. In particular, we hypothesize that subpopulations of neurons may fluctuate together, but asynchronously or in anticorrelated fashion with other subpopulations of neurons. How many such subgroups exist in a population may impose a limit on how many stimuli can be encoded by that neural population. This investigation is an important step towards unraveling the mystery of the brain’s ability to perceive and represent complex sensory scenes.

Another broader issue to be addressed by future research is how multiplexing works in the presence of more than two stimuli. Computational and statistical precision considerations aside, the SCAMPI framework introduced here most easily generalizes to the setting where the animal is exposed to either one stimulus from a set of $M$ distinct stimuli, or to all $M$ stimuli simultaneously. Then, slow-juggling would simply mean that the spike count distribution would be a mixture of the $M$ benchmark distributions, each assumed a Poisson. Analogously, for fast-juggling, Equation (3) would have to be upgraded to $\int \text{Poi}\left(\sum _j{r}_j{\mu }_j\right)f\left(r_1,\dots ,r_M\right)d{r}_1\cdots d{r}_M$. The SCAMPI framework could be extended to such $M$ dimensional cases provided $M$ is fairly small (3 or 4) because the computation cost of PRML increases exponentially in $M$. For higher dimensions, one would need to consider generalization of the predictive recursion algorithm, such those considered in Hahn et al. (2018), but it would take substantive work to develop a comprehensive analog of the SCAMPI analytical pipeline. Additionally, such extensions to $M$ stimuli raise new open questions, including the possibility that statistical precision will deteriorate with increasing $M$ and one might require many more trials per condition to derive sharp conclusions. Furthermore, from a neuroscience perspective, one could imagine other extensions of the experimental design, where one has additional data from exposures of a subset of the all stimuli, e.g., with $M=3$, there are total 2³ – 1 = 7 possible exposure conditions one could imagine. Such considerations bring up additional demands on designing a theoretical framework which allows seamless mathematical cross-connections between all such conditions.

It is also worth considering how multiplexed codes might be “read out” or interpreted by downstream neurons. Key questions include whether the readout algorithm needs to be informed about the time course of the juggling process (see e.g. Caruso et al., 2018, Figure 7) as well as how juggling is coordinated across the neural population (Moldakarimov et al., 2005). A very minimalist idea is that downstream neurons might be “listening” to the multiplexing code for evidence that particular stimuli are being encoded. A downstream neuron wired up to report stimulus “A” for example might respond if any subset of the A-preferring neurons in the multiplexing code were responding, and another neuron wired to report stimulus “B” could do likewise if any of the B-preferring multiplexing neurons were responding. Indeed, some of the neurons in the datasets that exhibit consistent A-like or B-like responses across every trial might lie at such an output stage of the multiplexed code. Of course, the reality may be more complicated but the general point is that by taking advantage of the temporal dimension, reading out a multiplexed code need not be substantially more challenging than reading out a non-multiplexed one.

At a mechanistic level, multiplexed codes likely involve selective routing of neural signals: the signals from potential input neurons must sometimes, but not always, affect the recipient neurons. There are numerous plausible mechanisms for this in the nervous system. Figure 8 illustrates two possibilities. In Figure 8a, a potential circuit that involves a gating signal is shown. Neuron X has an excitatory synapse onto Neuron Z, but a third neuron, Y, has an inhibitory axo-axonic connection onto Neuron X. If Neuron Y is active, then the activity of Neuron X will not reach Neuron Z (or it will be reduced in amplitude). Neuron Y could mediate a clock signal of some kind. Figure 8b shows an alternative mechanism, involving mutual inhibition. Here, Neurons X and Y can both excite Neuron Z, but they inhibit each other, which will ensure that Neuron Z tends to only receives input from one at a time. These are surely not the only ways that selective routing can occur in the brain, but provide plausible examples.

Fig. 8.

Two possible biologically plausible mechanisms for gating of signal flow in the nervous system. A. Neuron X has an excitatory synapse on Neuron Z, but this synapse is regulated by an inhibitory axo-axonic synapse from Neuron Y. Only when Y is not active will the signals from X reach Z. B. Neurons X and Y both have excitatory synapses on Neuron Z, but they also mutually inhibit each other. This can ensure that only one or the other of X and Y are contributing to Z’s activity at any given time.

Finally, we note parallels with previous studies involving binocular rivalry. In such paradigms, different stimuli are presented to each eye, and humans typically report perceiving only one stimulus at a time. Related studies in animals indicate that neurons in visual cortical areas alternate between responding to the two presented stimuli and that these fluctuations are correlated with contemporaneous behavioral reports (Leopold and Logothetis, 1996; Logothetis and Schall, 1989). The fluctuating activity patterns reported here suggest that this process occurs even when the stimuli in question are neither ambiguous nor multi-stable, but instead commonly found in natural sensory scenes.

Appendix A. Additional Figures and Tables

Fig. A1.

Summary statistics for each dataset. Each row (from top to bottom) represents the sample size, firing rate, Fano factors of each dataset under different conditions. Different colors represents different conditions, blue represents preferred single stimulus (the one with higher firing rate), orange represents non-preferred single stimulus (the one with lower firing rate), grey represents dual stimuli. The sample sizes under different conditions are similar, the medians ranging from 15 to 30. IC dataset has highest firing rate and IT dataset has lowest firing rate. The Fano factors under dual stimuli have more large values compare to those under single stimulus, which may indicate a overdispersion.

Table A1.

Number of triplets retained at each screening step.

	Different screening steps
	Total	5-5-5	SepBF	Fano Factor
IC	2250	1608	809	442
IT-AL	211	211	169	159
IT-MF	200	200	179	177
V1-imp	7318	7318	1745	1708
V1-adj	3371	3371	2130	1935

Table A2.

Total count for different classification and datasets by the SCAMPI model.

Classification by SCAMPI Model
	FastJug	SlowJug	OvReach	FxdPrf	FxdNon	FxdMid	FxdOut	Total
IC	55	25	55	79	45	155	28	442
IT-AL	26	0	48	65	2	18	0	159
IT-MF	24	0	21	79	0	52	1	177
V1-imp	9	3	39	869	4	48	736	1708
V1-adj	75	57	171	1372	4	216	40	1935

Fig. A2.

Summary of confidence levels for classification results for different datasets. Figures (a1)-(d1) present a selected experiment condition for a triplet defined in each of four datasets. Figures (a2)-(d2) visualize the corresponding classification results by the SCAMPI model, colored by posterior probability.

Fig. A3.

Performance evaluation of the SCAMPI model (frequentists’ two-stage framework). Classification accuracy and multiplexing detection accuracy versus complexity parameters across fixed, slow-juggling, fast-juggling, and overreach experimental cases (columns). Sample size 20, 30, 50 are color coded as amber, green and blue.

Fig. A4.

Performance evaluation of the SCAMPI model (intrinsic Bayes factor with Jeffreys’ prior for fixed hypothesis). Classification accuracy and multiplexing detection accuracy versus complexity parameters across fixed, slow-juggling, fast-juggling, and overreach experimental cases (columns). Sample size 20, 30, 50 are color coded as amber, green and blue.

Fig. A5.

Evaluation of the SCAMPI model (bounded uniform prior for fixed hypothesis). Classification accuracy and multiplexing detection accuracy versus difficulty tuning parameters across fixed, slow-juggling, fast-juggling, and overreach experimental cases (columns). Sample size 20, 30, 50 are color coded as amber, green and blue.

Fig. A6.

Results from both the original model and the SCAMPI model for the V1 dataset show the classifications under the SCAMPI model were largely consistent with the original model, especially when dual stimuli were superimposed gratings. For V1 adjacent gratings, disparities were most evident for mixture triplets, with equivalent proportions classified as fast-juggling, slow-juggling, and fixed-preferred. A slightly larger proportion of mixture triplets were classified as overreach. Additionally, a very small proportion of single triplets were classified as overreach.

Fig. A7.

Results from both the original model and the SCAMPI model for the face patch dataset. Classifications under the SCAMPI model was largely consistent with the original model. mixture triplets were primarily classified as overreach and fast-juggling. Additionally, a notable proportion of mixture triplets were classified as fixed-preferred. It is also noteworthy that a small proportion of intermediate triplets were classified as fast-juggling. However, neurons in the MF area with faces as distractors behaved differently compared to neurons in the AL area or when object were used as distractors. A larger proportion of intermediate triplets were classified as fast-juggling. The SCAMPI model provides a more refined classification of previously ambiguous cases.

Fig. A8.

Three example neurons from the IC dataset, one categorized as “slow juggling” (a), and two categorized as “fast juggling” (b). Red and blue dashed reference lines indicate the mean responses to the “A” and “B” alone stimulus conditions. Gray bars show distribution of spike counts on the combined “AB” conditions; the black trace shows a slightly smoothed version of the gray bars (smoothed with a [1/9 2/9 1/3 2/9 1/9] triangular filter).

Fig. A9.

Two example units from the V1 adjacent gratings dataset, one categorized as “slow juggling” (a) and one as “fast juggling” (b). Red and blue dashed reference lines indicate the mean responses to the “A” and “B” alone stimulus conditions. Gray bars show distribution of spike counts on the combined “AB” conditions; the black trace shows a spline fit to the gray bars.

Fig. A10.

Sensitivity analysis for Fano factor thresholds. (a) Distribution of average Fano factor of single stimulus trials across different dataset (with mean shown as text at bottom right). (b) Proportion of multiplexing triplets for different Fano factor thresholds across different dataset. (c) Comparison of classification analysis results across different Fano factor thresholds applied at the screening step. The classification results were robust across varying Fano factor thresholds. The relative distributions of each category remained similar, and the range of each category constrained to within 10%, except for AL dataset at low thresholds. Accordingly, the conclusion drawn in Section 4.3 were valid across threshold, with one exception. In the AL Face-Face dataset, when the Fano factor threshold was set to 1.5, the proportion of fast juggling was reduced by approximately half compared to other thresholds. In this case, the prevalence of fast-juggling was slightly lower for two-faces than for face-object combinations. We further observed a trend of more multiplexing and overreaching, fewer fixed Poisson as Fano factor threshold increased from 1.5 to 3, and then stablized beyond 3. This trend was particularly non-negligible for the AL dataset. We chose a Fano factor threshold of 3, since it delivered robust results while retaining an appropriate amount of data for each dataset.

Appendix B. Technical Details

B.1. Gaussian Quadrature

For the integral calculation, we approximate the integral ${\int }_{\Theta }\cdot d\theta$ with Gaussian quadrature. The basic idea is to approximate integral with a weighted sum of function values at specified points within the domain of integration. Here we set the points and weights according to Legendre polynomials proposed by Abramowitz and Stegun (1965).

$${\int }_{-1}^{1}f\left(x\right)\mathit{\text{dx}}=\sum _{i=1}^n{w}_{i}f\left(x_i\right).$$

And change the interval from [−1, 1] to $[a,b]$ according to

$${\int }_a^{b}f\left(x\right)\mathit{\text{dx}}=\frac{b-a}{2}{\int }_{-1}^{1}f\left(\frac{b-a}{2}x+\frac{a+b}{2}\right)\mathit{\text{dx}}.$$

which is

$${\int }_a^{b}f\left(x\right)\mathit{\text{dx}}\approx \frac{b-a}{2}\sum _{i=1}^n{w}_{i}f\left(\frac{b-a}{2}{x}_i+\frac{a+b}{2}\right).$$

For two-dimension Gaussian quadrature, the calculation is similar. Here we also need to tranform the interval from [−1, 1] × [−1, 1] to $[a,b]\times [c,d]$

$${\int }_{-1}^1{\int }_{-1}^{1}f(\xi ,\eta )d\xi d\eta \approx \sum _{i=1}^N\sum _{j=1}^N{w}_i{w}_{j}f\left({\xi }_i,{\eta }_j\right)$$

B.2. Laplace Approximation

According to Laplace approximation, approximate the posterior distribution with normal distribution.

$$\pi (\theta \mid y)=\frac{p(y\mid \theta )\pi \left(\theta \right)}{p\left(y\right)}\approx N(\theta \mid \hat{\theta },\hat{\Sigma })$$

where

$$\hat{\theta }=\underset{\theta }{\text{arg}\text{max}}\text{log}\pi (\theta \mid y)=\underset{\theta }{\text{arg}\text{max}}\text{log}p(y\mid \theta )\pi \left(\theta \right)$$

$$\hat{\Sigma }={\left\{-{\left.{\nabla }^2\text{log}\pi (\theta \mid y)\right|}_{\theta =\hat{\theta }}\right\}}^{-1}={\left\{-{\left.{\nabla }^2\text{log}p(y\mid \theta )\pi \left(\theta \right)\right|}_{\theta =\hat{\theta }}\right\}}^{-1}=\{-H{\}}^{-1}$$

So we have the estimation for marginal likelihood valued at $\hat{\theta }$:

$$p\left(y\right)\approx \left(2\pi {)}^{k/2}\right|\hat{\Sigma }{|}^{1/2}p(y\mid \theta =\hat{\theta })\pi (\theta =\hat{\theta })$$

where $k=\mathit{\text{dim}}\left(\theta \right),H$ is Hessian matrix. So approximation on marginal likelihood is transformed to an optimization problem:

$$p\left(y\right)\approx \left(2\pi {)}^{k/2}\right|H{|}^{-1/2}{e}^{l\left(\hat{\theta }\right)}$$

where $\hat{\theta }=\underset{\theta }{\mathit{\text{argmaxl}}}\left(\theta \right),H$ is corresponding Hessian matrix.

B.3. PRML-Gradient Algorithm

Algorithm 1 presents a pseudo-algorithm to calculate the gradient of the predictive recursion marginal likelihood by extending the original recursion algorithm of Newton (2002). These new calculations still require a single pass through the data, but additional bookkeeping is needed to derive and store information on gradient. We use the notation ∇ to denote the gradient operator which returns the vector of partial derivatives of a scalar function of vector input.

Appendix C. Fano Factor for intermediate

The data is denoted by $Y$, assumed to have a distribution $P$ for intermediate, where

$$P={\int }_0^1\mathit{\text{Poi}}\left(r{\mu }^{\text{A}}+(1-r){\mu }^B\right)\mathit{\text{Beta}}\left(r;m{r}_0,m\left(1-r_0\right)\right)\mathit{\text{dr}}.$$

This can be rewritten as

$$R~\mathit{\text{Beta}}\left(r;m{r}_0,m\left(1-r_0\right)\right),Y~\mathit{\text{Poi}}\left(R{\mu }^{\text{A}}+(1-R){\mu }^B\right).$$

Denote the Fano factor of $Y$ as $\mathit{\text{FF}}\left(Y\right)$, we have

$$\mathit{\text{FF}}\left(Y\right)=\frac{\mathit{\text{Var}}\left(Y\right)}{E\left(Y\right)}=\frac{E\left(\mathit{\text{Var}}\right(Y\mid R\left)\right)+\mathit{\text{Var}}\left(E\right(Y\mid R\left)\right)}{E\left(E\right(Y\mid R\left)\right)}=1+\frac{\mathit{\text{Var}}\left(E\right(Y\mid R\left)\right)}{E\left(E\right(Y\mid R\left)\right)}$$ (C6)

We can compute $E\left(E\right(Y\mid R\left)\right)$ and $\mathit{\text{Var}}\left(E\right(Y\mid R\left)\right)$ as follows:

$$E\left(E\right(Y\mid R\left)\right)={\mu }^A+\left({\mu }^B-{\mu }^A\right)(1-E(R\left)\right)={\mu }^A+\left({\mu }^B-{\mu }^A\right)\left(1-r_0\right)$$ (C7)

$$\mathit{\text{Var}}\left(E\right(Y\mid R\left)\right)={\left({\mu }^B-{\mu }^A\right)}^2\mathit{\text{Var}}\left(R\right)=\frac{{\left({\mu }^B-{\mu }^A\right)}^2{r}_0\left(1-r_0\right)}{m+1}$$ (C8)

Plug equation (C7) and (C8) into equation (C6), we have

$$\mathit{\text{FF}}\left(Y\right)=1+\frac{{\left({\mu }^B-{\mu }^A\right)}^2{r}_0\left(1-r_0\right)}{(m+1)\left({\mu }^A+\left({\mu }^B-{\mu }^A\right)\left(1-r_0\right)\right)}$$ (C9)

As shown in equation (C9), $\mathit{\text{FF}}\left(Y\right)$ monotonically decreases as $m$ increases. When $m$ approaches infinity, the Fano factor becomes 1. This corresponds to the case where fast-juggling degenerates into fixed, where $P=\text{Poi}\left(r_0{\mu }^A+\left(1-r_0\right){\mu }^B\right)$. In the scenario where $m=0$, fast-juggling degenerates into slow-juggling, where $P=r_0\text{Poi}\left({\mu }^A\right)+\left(1-r_0\right)\text{Poi}\left({\mu }^B\right)$. In this case, the Fano factor is a function of $r_0$, and it reaches its maximum value when $r_0=\frac{\sqrt{{\mu }^A{\mu }^B}-{\mu }^B}{{\mu }^A-{\mu }^B}$.

In Section 3, we conducted simulations with ${\mu }^A=50$ and ${\mu }^B=80$. The maximum Fano factor of around 4.5 was achieved when $r_0=0.56$ and $m=0$, corresponding to the case where fast-juggling degenerates to slow-juggling $P=0.56\text{Poi}\left(50\right)+0.44\text{Poi}\left(80\right)$. We kept $r_0$ fixed at 0.56, and varied the precision parameter $m$ from 0 to infinity, obtaining a wide range of Fano factors between 1 and 4.5. The minimum Fano factor of 1 was attained when $r_0=0.56$ and $m=0$, corresponding to the case where fast-juggling degenerates into fixed, with $P=\text{Poi}\left(63.2\right)$.

Data availability:

The data collected at Duke University is available upon request. We are working on building a user-friendly repository to host this data.

Code availability:

Coding for SCAMPI model is available in R package neuromplex. Related instructions on implementation are also provided.