Quantum superposition inspired spiking neural network

Summary

Despite advances in artificial intelligence models, neural networks still cannot achieve human performance, partly due to differences in how information is encoded and processed compared with human brain. Information in an artificial neural network (ANN) is represented using a statistical method and processed as a fitting function, enabling handling of structural patterns in image, text, and speech processing. However, substantial changes to the statistical characteristics of the data, for example, reversing the background of an image, dramatically reduce the performance. Here, we propose a quantum superposition spiking neural network (QS-SNN) inspired by quantum mechanisms and phenomena in the brain, which can handle reversal of image background color. The QS-SNN incorporates quantum theory with brain-inspired spiking neural network models from a computational perspective, resulting in more robust performance compared with traditional ANN models, especially when processing noisy inputs. The results presented here will inform future efforts to develop brain-inspired artificial intelligence.

Graphical abstract

Highlights

• •Quantum and neuroscience research inspire new methods for artificial intelligence
• •QS-SNN integrates characteristics of quantum superposition state and brain spikes
• •QS-SNN can handle the background reverse visual inputs well
• •QS-SNN model is robust to reverse pixels noise and Gaussian noise on visual inputs

Quantum theory; Artificial intelligence; Artificial intelligence theory

Introduction

Many machine learning methods using quantum algorithms have been developed to improve parallel computation. Quantum computers have also been shown to be more powerful than classical computers when running certain specialized algorithms, including Shor's quantum factoring algorithm (Shor, 1999), Grover's database search algorithm (Grover, 1996), and other quantum-inspired computational algorithms (Manju and Nigam, 2014).

Quantum computation can also be used to find eigenvalues and eigenvectors of large matrices. For example, the traditional principal components analysis (PCA) algorithm calculates eigenvalues by decomposition of the covariance matrix; however, the computational resource cost increases exponentially with increasing matrix dimensions. For an unknown low-rank density matrix, quantum-enhanced PCA can reveal the quantum eigenvectors associated with the large eigenvalues; this approach is exponentially faster than the traditional method (Lloyd et al., 2014).

K-means is a classic machine learning algorithm that classifies unlabeled data sets into k distinct clusters. A quantum-inspired genetic algorithm for K-means has been proposed, in which a qubit-based representation is employed for exploration and exploitation in discrete “0” and “1” hyperspace. This algorithm was shown to obtain the optimal number of clusters and the optimal cluster centroids (Xiao et al., 2010). Quantum algorithms have also been used to speed up the solving of subroutine problems and matrix inversion problems (Harrow et al., 2009); for example, Grover's algorithm (Grover, 1996) provides quadratic speedup of a search of unstructured databases.

The quantum perceptron and quantum neuron computational models combine quantum theory with neural networks (Schuld et al., 2014). Compared with the classical perceptron model, the quantum perceptron requires fewer resources and benefits from the advantages of parallel quantum computing (Schuld et al., 2015; Torrontegui and García-Ripoll, 2019). The quantum neuron model (Cao et al., 2017; Mangini et al., 2020) can also be used to realize classical neurons with sigmoid or step function activation by encoding inputs in quantum superposition, thereby processing the whole data set at once. Deep quantum neural networks (Beer et al., 2020) raise the prospect of deploying deep learning algorithms on quantum computers.

Spiking neural networks (SNNs) represent the third generation of neural network models (Maass, 1997) and are biologically plausible from neuron, synapse, network, and learning principles perspectives. Unlike the perceptron model, neurons in an SNN accept signals from pre-synaptic neurons, integrating the post-synaptic potential and firing a spike when the somatic voltage exceeds a threshold. After spiking, the neuron voltage is reset in preparation for the next integrate-and-fire process. SNNs are powerful tools for representation and processing of spatial-temporal information. Many types of SNNs have been proposed for different purposes. Examples include visual pathway-inspired classification models (Zenke et al., 2015; Zeng et al., 2017), basal ganglia-based decision-making models (Héricé et al., 2016; Cox and Witten, 2019; Zhao et al., 2017), and other shallow SNN (Khalil et al., 2017; Shrestha and Orchard, 2018). Different SNN may include different types of biologically plausible neurons, e.g., the leaky integrate-and-fire (LIF) model (Gerstner and Kistler, 2002), Hodgkin–Huxley model, Izhikevich model (Izhikevich, 2003), and spike response model (Gerstner, 2001). In addition, many different types of synaptic plasticity principles have been used for learning, including spike-timing-dependent plasticity (Dan and Poo, 2004; Frémaux and Gerstner, 2016), Hebbian learning (Song et al., 2000), and reward-based tuning plasticity (Héricé et al., 2016).

Quantum superposition SNN has theoretic basis in both biology (Vaziri and Plenio, 2010) and computational models (Kristensen et al., 2019). From one perspective, spiking neuron models, such as the LIF and Izhikevich models, can be reformed by quantum algorithms in order to accelerate their processing using a quantum computer. On the other hand, quantum effects such as entanglement and superposition are regarded as special information-interactive methods and can be used to modify the classical SNN framework to generate similar behavior to that of particles in the quantum domain. In this work, we follow the latter approach. More specifically, we use a quantum superposition mechanism to encode complementary information simultaneously and further transfer it to spike trains, which are suitable for SNN processing. In our proposed quantum superposition SNN (QS-SNN) model, quantum state representation is integrated with spatiotemporal spike trains in SNN. This characteristic is conducive to good model performance not only on standard image classification tasks but also when handling color-inverted images. QS-SNN encodes the original image and the color-inverted image in the format of quantum superposition; the changing background context demonstrated by the spiking phase and spiking rate contains the image pixels' identity information.

We combine quantum superposition information encoding with SNN for three reasons. First, the possible influence of quantum effects on biological processes and the related quantum brain hypothesis have been theoretically investigated (Vaziri and Plenio, 2010; Fisher, 2015; Weingarten et al., 2016). Second, quantum superposition states are represented by vectors in complex Hilbert space, in contrast to traditional artificial neural network (ANN), which operate in real space only; this is more representative of brain spikes, as the spiking rate and spiking phase spatiotemporal property also have complex number representation. In essence, SNN are more appropriate for quantum-inspired superposition information encoding. Third, current quantum machine learning methods, especially those used for quantum image processing, focus on encoding a classical image in the quantum state, with the image processing methods accelerated by quantum computing (Iyengar et al., 2020). There has been less exploration of the possibility of using a quantum superposition state coding mechanism for different pattern information-processing frameworks. More importantly, owing to the use of statistical methods and fitting functions, current ANN show a huge performance drop when required to recognize a background-inverted image. This inspired us to develop a new information representation method unlike that used in traditional models. The integration of characteristics from SNN and quantum theory is intended to achieve a better representation of multi-states and potentially enable easier solving of tasks that are challenging for traditional ANN and SNN models.

The subsequent sections describe how complementary superposition information is generated and transferred to spatiotemporal spike trains. A two-compartment SNN is used to process the spikes. The proposed model, combining complementary superposition information encoding with the SNN spatiotemporal property, can successfully recognize a background color-inverted image, which is hard for traditional ANN models.

Complementary superposition information encoding

Quantum image processing

Quantum image processing combines image processing methods with quantum information theory. There are many approaches to internal representation of an image in a quantum computer, including flexible representation of quantum images (FRQIs), NEQR, GQIR, MCQI, and QBIP (Iyengar et al., 2020; Mastriani, 2020), which transfer the image to appropriate quantum states for the next step of quantum computing. Our approach is inspired by the FRQI method (Le et al., 2011), as shown in Equations (1) and (2):

$$|I\left(\theta \right)⟩=\frac{1}{2^n}\sum _{i=0}^{2^{2n}-1}(sin\left({\theta }_i\right)|0⟩+cos\left({\theta }_i\right)|1⟩)|i⟩,$$ (Equation 1)

$${\theta }_i\in [0,\frac{\pi }{2}],i=\mathrm{1,2,3},\dots ,2^{2n}-1,$$ (Equation 2)

where $|I\left(\theta \right)⟩$ is the quantum image, qubit $|i⟩$ represents the position of a pixel in the image, and $\theta =\left({\theta }_0,{\theta }_1,\dots ,{\theta }_{2^{2n}-1}\right)$ encodes the color information of the pixels. FRQI satisfies the quantum state constraint in Equation (3):

$$\left|\left||I\left(\theta \right)⟩\right|\right|=\frac{1}{2^n}\sqrt{\sum _{i=0}^{2^{2n}-1}\left(co{s}^2{\theta }_i+si{n}^2{\theta }_i\right)}=1.$$ (Equation 3)

Complementary superposition spikes

We propose a complementary superposition information encoding method and establish a linkage between quantum image formation and spatiotemporal spike trains. The complement code is widely used in computer science to turn subtraction into addition. We encode the original information and complementary information into a superposition state; one example is shown in Equation (4), with the rightmost sign bit removed and taking the complement:

$$|I\left({\theta }_i\right)⟩=cos\left({\theta }_i\right)|{\left[0000001\right]}_b⟩+sin\left({\theta }_i\right)|{\left[1111110\right]}_b⟩.$$ (Equation 4)

Equation (4) is an illustration of how complementary superposition information encoding works, with no factual significance. In this work, we focus on quantum image superposition encoding, as in Equation (5). However, it should be noted that any form of information that has a complement format, not just an image, can be encoded as a superposition state. Images in complementary quantum superposition states are further transferred to spike trains, as depicted in Figure 1. An image in its complementary state has an inverted background.

$$|I\left(\theta \right)⟩=\frac{1}{2^n}\sum _{i=0}^{2^{2n}-1}(cos\left({\theta }_i\right)|x_i⟩+sin\left({\theta }_i\right)|{\overline{x}}_i⟩)\otimes |i⟩,$$ (Equation 5)

$${\theta }_i\in [0,\frac{\pi }{2}],i=\mathrm{1,2,3},\dots ,2^{2n}-1.$$ (Equation 6)

Figure 1.

Quantum complementary superposition information encoding

(A) The horizontal axis and longitudinal axis represent $|x⟩$ and $|\overline{x}⟩$, respectively. The parameter θ in Equation (5) measures the degree to which the image background is inverted, from $\theta =0$ (the original image) to the complementary state $\theta =\frac{\pi }{2}$ (totally inverted background).

(B) The top pictures show images inverted to different degrees, and the spikes to which they are encoded. The bottom axis corresponds to the value of θ. It should be noted that the pictures are intuitive demonstration instead of exact display.

The complementary quantum superposition encoding is shown in Equations (5) and (6), where the $|i⟩$ represent pixel positions. Unlike FRQI, which uses qubits only for color encoding, here we use complementary qubits for encoding both original image pixels $x_i$ and the color-inverted image ${\overline{x}}_i$ with ${\overline{x}}_i=1-x_i$, supposing the pixel $x_i$ domain ranges from 0 to 1.0. The parameter ${\theta }_i$ represents the degree of quantum image $|I⟩$, mixing the original state $|x⟩$ and reverse state $|\overline{x}⟩$.

We designed quantum circuit for the generation of quantum superposition image $|I\left({\theta }_i\right)⟩$ as shown in the Figure 2A, which is also discussed in (Le et al., 2011; Dendukuri and Luu, 2018). The quantum state $|x_i⟩$ is processed by Hadamard transform H and controlled $NOT$ gate to form the complementary state $|{\beta }_{i{x}_i}⟩$ with:

$$|{\beta }_{i{x}_i}⟩=\frac{|0,x_i⟩+{\left(-1\right)}^i|1,{\overline{x}}_i⟩}{\sqrt{2}},$$ (Equation 7)

Figure 2.

Quantum superposition spike trains

(A) The circuit to generate quantum image. Only one image pixel state is depicted for perspicuity.

(B) A schematic diagram shows the transformation of quantum superposition states to spike trains $S_i\left(t;{\varphi }_i\right)$. With parameter ${\theta }_i$ increasing, spike trains are shifted in time dimension. T is a simulation period in which spikes emerge within the $T_{sp}$ time window. Also, the relation of parameter ${\theta }_i$ and spiking phase ${\varphi }_i$ is intuitive example and not exact correspondence.

Then rotation matrices $R_i$ is used to encode phase information as

$$R_i=\left[\begin{array}{ll}cos\frac{{\theta }_i}{2}\hfill & -sin\frac{{\theta }_i}{2}\hfill \\ \hfill & \hfill \\ sin\frac{{\theta }_i}{2}\hfill & cos\frac{{\theta }_i}{2}\hfill \end{array}\right]$$ (Equation 8)

Finally, the superposition state $|I\left({\theta }_i\right)⟩$ is measured and two states are retrieved with probability $P_i$ and $Q_i$.

The complex information in quantum encoding is similar to signal processing in SNN. Neuron spikes can encode spatiotemporal information with specific spiking rates and spiking times, which can be used to represent quantum information.

Neuron spikes have the attribute of spatiotemporal dimension, which are identical in shape but differ significantly in frequency and phase, seeing Izhikevich neuron model (Izhikevich, 2003) in Figure S1, and are well-suited to the implementation of the vector form quantum image in Equation (5). We use spike trains with firing rate $r_i$ and firing phase ${\varphi }_i$ to represent quantum image state $|I\left({\theta }_i\right)⟩$. As shown in Figure 2A, the spike trains containing information of $|I\left({\theta }_i\right)⟩$ can be generated using Equation(9) and (10).

$$r_i=\frac{\left|\left||I\left(\theta \right)⟩\right|\right|-sin\left({\varphi }_i\right)}{cos\left({\varphi }_i\right)-sin\left({\varphi }_i\right)},$$ (Equation 9)

$${\varphi }_i=\mathcal{F}\left\{arctan\left(\frac{P_j}{Q_j}\right)\right|j=\mathrm{1,2},\dots ,N.\},$$ (Equation 10)

Notation $\mathcal{F}\left\{X_i\right\}$ is set operation, and is specific in different tasks. The superposition state encoding $|I\left({\theta }_i\right)⟩$ is transferred to spike trains $S_i\left(t;{\varphi }_i\right)$, which is generated from a Poisson spikes $S_i\left(t\right)$ with spike rate $r_i$ and extended phase ${\varphi }_i$ shown as Equation (11). Here, T is the time interval of neuron processing spikes received from pre-synaptic neurons, and $T_{sp}$ is the spiking time window in this period, as shown in Figure 2B:

$$S_i\left(t;{\varphi }_i\right)=S_i\left(t-t_0\right),$$ (Equation 11)

$$t_0=\frac{{\varphi }_i}{\pi /2}\text{∗}\left(T-T_{sp}\right).$$ (Equation 12)

Two-compartment SNN

Synapses with time-differential convolution

Synapses play an important part in the conversion of information from spikes in pre-synaptic neurons to membrane potential (or current) in post-synaptic neurons. In this work, the time-differential kernel (TCK) convolution synapse is used, as shown in Equations (13) and (14) and Figure S2. The spikes, $S_i$, from pre-synaptic neurons are convoluted with a kernel and then integrated with the dendrite membrane potential $V_b\left(t\right)$. This process can be considered as a stimulus-response convolution with the form of a Dirac function (Urbanczik and Senn, 2014):

$$\{\begin{array}{l}\kappa \left(t\right)=\zeta \left(t\right)-\zeta \left(-t\right)\hfill \\ \zeta \left(t\right)=\text{Θ}\left(t\right)\left(e^{-\frac{t}{\tau }}\right)\hfill \end{array},$$ (Equation 13)

$$V_j^b\left(t\right)=\sum _i{w}_{i,j}\Vert {\int }_{-T}^{+T}\kappa \left(\tau \right)S_i\left(\tau \right)d\tau \Vert .$$ (Equation 14)

Two-compartment neurons

Both the hidden layer and the output layer contain biologically plausible two-compartment neurons, which dynamically update the somatic membrane potential $V_i\left(t\right)$ with the dendrite membrane potential $V_i^b\left(t\right)$, as shown in Figure S2.

In the compartment neuron model, $V_i^h\left(t\right)$ is the membrane potential of neuron i in the hidden layer, which is updated with Equation (15); $g_B$, $g_L$, and ${\tau }_L$ are hyperparameters that represent synapse conductance, leaky conductance, and the integrated time constant, respectively; $V_j^{PSP}\left(t\right)$ is the synaptic input from neuron j; $V_i^{h,b}\left(t\right)$ is the dendrite potential with adjustable threshold $b_i^h$ in the hidden layer; and $w_{ij}^h$ is the synaptic weight between the input and hidden layers:

$$\{\begin{array}{l}{\tau }_L\frac{d{V}_i^h\left(t\right)}{dt}=-V_i^h\left(t\right)+\frac{g_B}{g_L}\left(V_i^{h,b}\left(t\right)-V_i^h\left(t\right)\right)\hfill \\ V_i^{h,b}\left(t\right)=\sum _j{w}_{ij}^h{V}_j^{PSP}\left(t\right)+b_i^h\hfill \\ V_j^{PSP}\left(t\right)=\Vert {\int }_{-T}^{+T}\kappa \left(\tau \right)S_j\left(\tau \right)d\tau \Vert .\hfill \end{array}$$ (Equation 15)

The somatic neuron model in the output layer contains 10 neurons corresponding to 10 classes of the MNIST data set. As shown in Equation (16), the hidden layer neurons deliver signals to the output layer with integrated spike rate $r_i$, which is differentiable; hence, it can be tuned with back-propagation. Here, $V_i^o\left(t\right)$ is the membrane potential in the output layer, $V_i^{o,b}\left(t\right)$ is the dendrite potential, and $r_{max}$ is the hyperparameter for rescaling of fire-rate signals:

$$\{\begin{array}{l}{\tau }_L\frac{d{V}_i^o\left(t\right)}{dt}=-V_i^o\left(t\right)+\frac{g_B}{g_L}\left(V_i^{o,b}-V_i^o\left(t\right)\right)\hfill \\ V_i^{o,b}\left(t\right)=\sum _j{w}_{ij}^o{r}_j\left(t\right)+b_i^o\hfill \\ r_j=r_{max}\sigma \left(V_j^h\right)\hfill \\ \sigma \left(x\right)=1/\left(1-exp\left(-x\right)\right).\hfill \end{array}$$ (Equation 16)

The shallow three-layered architecture is shown in Figure 3. The input layer receives quantum spikes with encoding of complementary qubits. The hidden layer of QS-SNN consists of a two-compartment model with time-differential convolution synapses. Neurons in the output layer are integrated spike rate neurons, which receive an integrated fire rate from pre-synaptic neurons, as well as the teaching signal $V_I$ as information about class labels.

Figure 3.

Quantum superposition SNN, three-layer architecture of QS-SNN with TCK synapses and two-compartment neurons

Images are transferred to spikes as network inputs. The hidden layer is composed of 500 two-compartment neurons with dendrite and soma. The output layer contains 10 two-compartment neurons corresponding to 10 classes. In the training period, only original images are fed to the network, whereas in the test period, the trained network is tested with inverted background images. Neurons with maximum spiking rates at the output layer are taken as the network prediction and output.

Computational experiments

We examined the performance of the QS-SNN framework on a classification task using background color-inverted images from the MNIST (LeCun et al., 2010) and Fashion-MNIST (Xiao et al., 2017) data sets. QS-SNN encodes the original image and its color-inverted mirror as complementary superposition states and transfers them to spiking trains as an input signal to the two-compartment SNN. The dendrite prediction and proximal gradient methods used to train this model can be found in STAR Methods.

For comparison, we also tested several deep learning models on the color-inverted data sets, including a fully connected ANN, a 10-layers CNN (Lecun et al., 1998), VGG (Simonyan and Zisserman, 2015), ResNet (He et al., 2016), and DeseNet (Huang et al., 2017). All models are trained with original image $x_i$ and then tested on the background reverse image $I\left({\theta }_i\right)$. The only difference is that, for QS-SNN, quantum superposition state image $|I\left({\theta }_i\right)⟩$ is transferred to spike trains which is compatible with spiking neural networks. In other words, our essential idea is that the spatiotemporal property of neuron spikes enables the brain to transform spatially variant information into time-differential information.

In this work, we formulated the spike trains transformation as the quantum superposition shown in Equation (5). And we have also demonstrated the numerical calculation of $I\left({\theta }_i\right)$ which is used for traditional ANN and CNN model testing in STAR Methods.

In addition, it is worth noting that the superposition state image $|I\left({\theta }_i\right)⟩$ is constructed from original information $|x_i⟩$ and reverse image $|{\overline{x}}_i⟩$. The original image and its complementary reverse information are maintained in the superposition state encoding $|I\left({\theta }_i\right)⟩$ at the same time. Because SNN is not processing pixel value directly, we transformed $|I\left({\theta }_i\right)⟩$ to spike trains, which can be regarded as different expression of superposition state encoding image in spatiotemporal dimension.

Standard and color-inverted images

The standard MNIST data set contains images of 10 classes of handwritten digits from 0 to 9; images are 28x28 pixels in size, with 60,000 and 10,000 training and test samples, respectively. Fashion-MNIST has the same image size and the same training and testing split as MNIST but contains grayscale images of different types of clothes and shoes.

The original MNIST and Fashion-MNIST images and their color-inverted versions, with different degrees of inversion as measured by parameter θ, are depicted in Figures 4A and 4B, respectively. To be specific, the spiking phase estimating operation $\mathcal{F}\left\{X_i\right\}$ in Equation (10) is set as piecewise selection function as

$${\varphi }_i=\{\begin{array}{ll}arctan\left(\frac{P_j}{Q_j}\right),\hfill & j=i,\\ 0,\hfill & j\ne i.\end{array}$$ (Equation 17)

Figure 4.

Classification of color-inverted images

(A) MNIST background color-inverted images. Parameter θ takes values from 0 to $\frac{\pi }{2}$, denoting the degree of color inversion.

(B) Fashion-MNIST background color-inverted images.

(C) Background color-inverted MNIST classification results. QS-SNN initially showed performance degeneration similar to that of the fully connected ANN and CNN, with θ values from 0 to $\frac{4\pi }{16}$. However, as the background-inversion degree further increased, QS-SNN gradually recovered its accuracy, whereas the other networks did not. When the background was totally inverted ($\theta =\frac{8\pi }{16}$), QS-SNN showed almost the same performance as when classifying original images, whereas the second-best network (VGG16) retained only half its original accuracy.

(D) Background color-inverted Fashion-MNIST results. Similar results as in the MNIST experiment were achieved, with QS-SNN showing an even greater advantage (right-hand side).

Robustness to reverse pixel noise and Gaussian noise

Besides the effects of changing the whole background, we were interested in the capability of QS-SNN to handle other types of destabilization of images. For this purpose, we added reverse spike pixels and Gaussian noise to the MNIST and Fashion-MNIST images, and further tested the performance of QS-SNN in comparison with that of ANN and CNN. Reverse spike noise is created by randomly flipping image pixels to their reverse color and can be described as $Reverse\left(image\left[i\right]\right)=1-image\left[i\right]$. The position i of the pixel to be flipped is randomly chosen, as shown in Figures 5A and 5B. The noisy images were encoded and processed in the same way as described in Algorithm 1. However, in the color-inverted experiment, all pixels of reverse degree ${\theta }_i$ were the same, resulting in the same change being applied to the whole image. By contrast, in the reverse pixel noise experiment, only a proportion of randomly chosen image pixels were changed; thus, every image pixel had a specific ${\theta }_i$ parameter. These image pixels are transferred to spike trains with heterogeneous phase ${\varphi }_i$. Specially in reverse pixel experiment we took the mean operation for $\mathcal{F}\{\cdot \}$ as the estimation phase:

$${\varphi }_i=\frac{1}{N}\sum _{j=1}^{N}arctan\left(\frac{P_j}{Q_j}\right)$$ (Equation 18)

Figure 5.

Reverse spike noise results

(A) Reverse spike noise MNIST. The possibility of pixel inversion is controlled by parameter r. When $r=0$, no noise is added, i.e., the image is original data. When $r=1.0$, all pixels are flipped.

(B) Reverse spike noise Fashion-MNIST.

(C) Classification of MNIST images with reverse noise. QS-SNN performed better compared with the inverted background experiment.

(D) Classification of Fashion-MNIST images with reverse noise.

Algorithm 1. The learning procedure of QS-SNN.

• 1.Initialize weights $W_{j,i}$ with random uniform distribution, membrane potential states $V_i$, and other related hyperparameters as in Table S1.
• 2.
  Start training procedure with only original images in training dataset, ${\theta }_i=0$:

  • 2.1
    Load training samples.
  • 2.2
    Construct quantum superposition state representations of images.
  • 2.3
    Input neuron spikes as Poisson process with spiking rate and phase time according to quantum superposition image.
  • 2.4
    Process time-differential convolution to obtain dynamical updating of membrane potential of post-synaptic neurons.
  • 2.5
    Update multi-layer membrane potential.
  • 2.6
    Train the QS-SNN with dendrite prediction and proximal gradient.
  • 2.7
    Select neurons in output layer with maximum spiking rate as the output class.
• 3.
  Start test procedure using color-inverse images with different degree of color inversion from the test dataset, ${\theta }_i=0,\frac{\pi }{16},\dots ,\frac{8\pi }{16}$.

  • 3.1
    Load the test samples and transfer to spike trains as in steps 2.2 and 2.3.
  • 3.2
    Test the performance of the trained QS-SNN on color-inverse images.
  • 3.3
    Output the test performance.

Additive white Gaussian noise (AWGN) is commonly used to test system robustness. We also examined the performance of the proposed QS-SNN on AWGN MNIST and Fashion-MNIST images, as shown in Figures 6A and 6B.

Figure 6.

Gaussian noise image classification

(A) Additive white Gaussian noise on MNIST. The mean of Gaussian random noise was set to zero, and different $std$ values were used.

(B) Additive white Gaussian noise on Fashion-MNIST.

(C) Classification of MNIST images with Gaussian noise. Compared with other networks, QS-SNN showed much slower degeneration.

(D) Classification of Fashion-MNIST images with Gaussian noise. QS-SNN performed even better compared with its results on MNIST.

In contrast to color-inverted noise, AWGN results in uncorrelated disturbances on the original image. We were interested in the robustness of our proposed method when faced with this challenging condition. The procedure used to process AWGN images was the same as that used in the reverse pixel noise experiment, except that the whole image phase was estimated using half the median operation:

$${\varphi }_i=M\left\{arctan\left(\frac{P_j}{Q_j}\right)\right|j=1,\dots ,N\}$$ (Equation 19)

Results

Standard and color-inverted data sets experiment

We constructed a three-layer QS-SNN with 500 hidden layer neurons and 10 output layer neurons. The structure of the experimental fully connected ANN was set to be the same for comparison. A simple CNN structure with three convolution layers and two pooling operations was used to determine the ability of different feature extraction methods to deal with inverted background images. We also tested VGG16, ResNet101, and DenseNet121 to investigate whether deeper structures could classify color-inverted images correctly. ANN, CNN, and QS-SNN were trained for 20 epochs with the Adam optimization method, and the learning rate was set to 0.001. VGG16, ResNet101, and DenseNet121 were trained for 400 epochs using stochastic gradient descent with learning rate 0.1, momentum 0.9, weight decay 5e-4, and learning rate decay 0.8 every 10 epochs. In the training phase, only the original image ($\theta =0$) was used; the testing phase used different color-inverted images (θ ranging from 0 to $\frac{\pi }{2}$). All results were obtained from the final epoch test step.

The results showed that the traditional fully connected ANN and convolution models struggled to handle huge changes in image properties such as background reversal, even when the spatial features of the image remained the same. Our proposed method showed much better performance than these traditional models (see Figures 4C and 4D and Tables S2 and S3 for details). Significant performance degradation occurred when processing color-inverted images with ANN and CNN, and even deeper networks such as VGG16, ResNet101, and Densenet121 experienced problems with color-inverted image classification. By contrast, QS-SNN, although affected by a similar performance drop when images were made blurry (θ from 0 to $\frac{4\pi }{16}$), regained its ability when the images' backgrounds were inverted, and the clarity was improved (θ from $\frac{4\pi }{16}$ to $\frac{8\pi }{16}$). When the image color was fully inverted ($\theta =\frac{8\pi }{16}$), QS-SNN retained the same accuracy as when classifying the original data ($\theta =0$) and correctly recognized color-inverted MNIST and Fashion-MNIST images.

Robustness to noise experiments

Compared with other state-of-the-art models, the performance of QS-SNN was closer to human vision capacity. As more flipped-pixel noise was added to the images ($r=0$ to 0.5), they became increasingly difficult to recognize, as indicated by the left side of the “U”-curve for QS-SNN in Figures 5C and 5D. However, as more noise was added to the pixels, the image features became clear again. When $r=1.0$, with all pixels reversed, there was no conflict with the features of the original image when $r=0$. QS-SNN can exploit these conditions owing to its image superposition encoding method (Equation 5), which is similar to the human vision system. As shown in Figures 5C and 5D and Tables S4 and S5, randomly inverting image pixels caused substantial performance degradation of ANN and CNN, as well as of the deep networks. On the contrary, the red “U”-shaped curve for QS-SNN indicated that it recovered its accuracy as the image's features became clear but the background was inverted ($r=0.6$ to 1.0).

Gaussian noise influenced all networks significantly, with all methods showing a performance drop as noise (standard deviation; $std$) increased, as shown in Figures 6C and 6D and Tables S6 and S7. QS-SNN behaved more stably on the AWGN image processing task, with accuracies of 90.2% and 82.3% on the MNIST and Fashion-MNIST data sets, respectively, for $std=0.4$; by contrast, the other methods achieved no more than 60% and 50%, respectively. Images with $std=0.4$ are not very difficult for human vision to distinguish. Thus, by combining a brain-inspired spiking network with a quantum mechanism, we obtain a more robust approach to images with noise disturbance, similar to the performance of human vision.

Discussion

This work aimed to integrate quantum theory with a biologically plausible SNN. Quantum image encoding and quantum superposition states were used for information representation, followed by processing with a spatial-temporal SNN. A time-convolution synapse was built to obtain neuron process phase information, and dendrite prediction with a proximal gradient method was used to train the QS-SNN. The proposed QS-SNN showed good performance on color-inverted image recognition tasks that were very challenging to other models. Compared with traditional ANN models, QS-SNN showed better generalization ability.

It is worth noting that the quantum brain hypothesis is quite controversial. Nevertheless, this paper does not aim to provide direct persuasive evidence for the quantum brain hypothesis but to explore novel information-processing methods inspired by quantum information theory and brain spiking signal transmission.

Limitations of the study

Our model was inspired by quantum image processing methods, in particular, quantum image superposition state presentation. The model and corresponding experiments were run on a classical computer and did not use any quantum hardware; thus, our work could not benefit from quantum computing. Artificial neurons can be reformed to run on quantum computers (Schuld et al., 2014; Cao et al., 2017; Mangini et al., 2020). Efforts to build a quantum spiking neuron are still at a preliminary stage (Kristensen et al., 2019). Simulating spiking neuronal networks on a classical computer is hindered by heavy resource consumption and slow processing.

Future work includes modifying both the quantum superposition encoding strategy and the SNN architecture to suit quantum computing better. In computational neuroscience research, neuronal spikes are typically generated with the Poisson process, which samples data from the binomial distribution. Quantum bits, also named qubits, are fundamental components in a quantum computer. A qubit can take the value of “0” or “1” with a certain probability, which is very similar to neuronal spikes. Thus, a set of qubits can encode all possible states of a spike train, as well as the quantum superposition images, in the quantum computer. Although it requires much effort to reconstruct spiking neural models suited for quantum computing, it is significant in neurology and artificial intelligence research to explore more quantum-inspired mechanisms to explain brain functions that traditional theories fail to.

STAR★Methods

Key resources table

REAGENT or RESOURCE	SOURCE	IDENTIFIER
Software and algorithms
CNN	Lecun et al.,1998	https://github.com/tensorflow/docs-l10n/blob/master/site/zh-cn/tutorials/images/cnn.ipynb
VGG16	Simonyan and Zisserman, 2015	https://github.com/pytorch/vision/blob/6db1569c89094cf23f3bc41f79275c45e9fcb3f3/torchvision/models/vgg.py
ResNet	He et al., 2016	https://github.com/pytorch/vision/blob/6db1569c89094cf23f3bc41f79275c45e9fcb3f3/torchvision/models/resnet.py
DensNet	Huang et al., 2017	https://github.com/pytorch/vision/blob/6db1569c89094cf23f3bc41f79275c45e9fcb3f3/torchvision/models/densenet.py
QS-SNN	This paper	https://github.com/Brain-Inspired-Cognitive-Engine/Q-SNN
Other
MNIST	LeCun et al., 2010	http://yann.lecun.com/exdb/mnist/
Fashion-MNIST	Xiao et al., 2017	https://github.com/zalandoresearch/fashion-mnist

Resource availability

Lead contact

Further information and requests for resources and reagents should be directed to and will be fulfilled by the lead contact, Yi Zeng (yi.zeng@ia.ac.cn).

Materials availability

This study did not generate new unique reagents.

Data and code availability

The Python scripts can be downloaded from the GitHub repository: https://github.com/Brain-Inspired-Cognitive-Engine/Q-SNN.

Method details

Generating background inverse image

Different degree of background color-inverse images $|I\left(\text{Θ}\right)⟩$ using for experiment are generated according to the quantum superposition encoding. Here we describe the numerical form value of quantum image used to test model. Suppose the original image is represented by X and the reversed image is $\overline{X}$. Parameter $\text{Θ}$ controls the proportion of original image and reverse image in background color-inverse images with

$$I\left(\text{Θ}\right)=Xcos\left(\text{Θ}\right)+\overline{X}sin\left(\text{Θ}\right)$$ (Equation 20)

Learning procedure of the QS-SNN algorithm

Dendrite prediction (Urbanczik and Senn, 2014) and proximal gradient methods are used for tuning of QS-SNN. In Equation (21), $I_i$ is the teaching current, which is the integration of correct labels in $g_{E_i}\left(E_E-U_1\right)$ and wrong labels in $g_{I_i}\left(E_I-U_1\right)$. $E_E$ (8 mV) and $E_I$ (-8 mV) are the excitatory and inhibitory standard membrane potentials, respectively. The teaching current is injected to the soma of neurons in the output layer, generating added potential $V_{I_i}$ with membrane resistance $r_B$, as shown in Equation (22):

$$\{\begin{array}{l}{I}_i^{ject}=g_{E_i}\left(E_E-U_1\right)+g_{I_i}\left(E_I-U_1\right)\hfill \\ g_{E_i}=\{\begin{array}{ll}1,i=label,\hfill & \hfill \\ 0,i\ne label.\hfill & \hfill \end{array}\hfill \\ g_{I_i}=\{\begin{array}{ll}0,i=label,\hfill & \hfill \\ 1,i\ne label.\hfill & \hfill \end{array}\hfill \\ V_{I_i}=r_B\cdot I_i^{ject},\hfill \end{array}$$ (Equation 21)

$${\tau }_L\frac{d{V}_i^o\left(t\right)}{dt}=-V_i^o\left(t\right)+\frac{g_B}{g_L}\left(V_i^{o,b}-V_i^o\left(t\right)\right)+V_{I_i}-V_i^o\left(t\right).$$ (Equation 22)

Setting the left side of Equation (22) to zero and $V_i^o=V_{I_i}$, we get the steady state of somatic potentials $V_i^o$ and $V_i^{o,b}$ with $V_i^{o\ast }=g_B/\left(g_B+g_L\right)V_i^{o,b}$. The dendrite prediction rule defines the soma-dendrite error as Equation (23):

$$L=\frac{1}{2}\sum _{i=0}^N{\Vert r_{max}\sigma \left(V_i^o\right)-r_{max}\sigma \left(V_i^{o\ast }\right)\Vert }^2.$$ (Equation 23)

Minimizing this error based on the differential chain rule, we obtain updated synaptic weights $w_{ij}^o$, as shown in Equations (24) and (25):

$$\frac{\partial L}{\partial w_{ij}^o}=\frac{\partial L}{\partial V_i^{o,b}}\frac{\partial V_i^{o,b}}{\partial w_{ij}^o}$$ (Equation 24)

$$=r_{max}\frac{g_B}{g_B+g_L}[\sigma \left(V_i^{o\ast }\right)-\sigma \left(V_i^o\right)]{\sigma }^{\text{′}}\left(V_i^{o\ast }\right)r_j$$

$$={\delta }_i^o{r}_j,$$

$$\frac{\partial L}{\partial b_o^1}=\frac{\partial L}{\partial V_i^{o,b}}\frac{\partial V_i^{o,b}}{\partial b_i^o}$$ (Equation 25)

$$=r_{max}\frac{g_B}{g_B+g_L}[\sigma \left(V_i^{o\ast }\right)-\sigma \left(V_i^o\right)]{\sigma }^{\text{′}}\left(V_i^{o\ast }\right)$$

$$={\delta }_i^o.$$

Equation (26) shows the iterative updating of synaptic weights $w_{ij}^y$ and bias $b_i^y$:

$$\{\begin{array}{l}{w}_{ij}^o\leftarrow w_{ij}^o-\eta \frac{\partial L}{\partial w_{ij}^o}\hfill \\ b_i^o\leftarrow b_i^o-\eta \frac{\partial L}{\partial b_i^o}.\hfill \end{array}$$ (Equation 26)

For the hidden layer, error signal ${\delta }_i$ is passed from the previous layer, and neuron synapses $w_{ij}^h$ are adapted using Equations (27) and (28):

$$\frac{\partial L}{\partial w_{ij}^h}=\sum _k\frac{\partial L}{\partial V_k^{o,b}}\frac{\partial V_k^{o,b}}{\partial r_i^h}\frac{\partial r_i^h}{\partial V_i^h}\frac{\partial V_i^h}{\partial V_i^{h,b}}\frac{\partial V_i^{h,b}}{\partial w_{ij}^h}=\sum _k{\delta }_k^o{w}_{ki}^o{r}_{max}\frac{g_B}{g_B+g_L}{\sigma }^{\text{′}}\left(V_i^h\right)V_j^{PSP}=\sum _k{\delta }_k^o{\delta }_i^h{w}_{ki}^o{V}_j^{PSP},$$ (Equation 27)

$$\frac{\partial L}{\partial b_i^h}=\sum _k\frac{\partial L}{\partial V_k^{o,b}}\frac{\partial V_k^{o,b}}{\partial r_i^h}\frac{\partial r_i^h}{\partial V_i^h}\frac{\partial V_i^h}{\partial V_i^{h,b}}\frac{\partial V_i^{h,b}}{\partial b_i^h}=\sum _k{\delta }_k^o{w}_{ki}^o{r}_{max}\frac{g_B}{g_B+g_L}{\sigma }^{\text{′}}\left(V_i^h\right)=\sum _k{\delta }_k^o{\delta }_i^h{w}_{ki}^o.$$ (Equation 28)

Equation (29) shows the iterative updating of synaptic weights $w_{ij}^h$ and bias $b_i^h$:

$$\{\begin{array}{l}{w}_{ij}^h\leftarrow w_{ij}^h-\eta \frac{\partial L}{\partial w_{ij}^h}\hfill \\ b_i^h\leftarrow b_i^h-\eta \frac{\partial L}{\partial b_i^h}.\hfill \end{array}$$ (Equation 29)

The training and test procedure for the QS-SNN model is shown in Algorithm 1.

Published: August 20, 2021