Neuromorphic photonic networks using silicon photonic weight banks

Abstract

Photonic systems for high-performance information processing have attracted renewed interest. Neuromorphic silicon photonics has the potential to integrate processing functions that vastly exceed the capabilities of electronics. We report first observations of a recurrent silicon photonic neural network, in which connections are configured by microring weight banks. A mathematical isomorphism between the silicon photonic circuit and a continuous neural network model is demonstrated through dynamical bifurcation analysis. Exploiting this isomorphism, a simulated 24-node silicon photonic neural network is programmed using “neural compiler” to solve a differential system emulation task. A 294-fold acceleration against a conventional benchmark is predicted. We also propose and derive power consumption analysis for modulator-class neurons that, as opposed to laser-class neurons, are compatible with silicon photonic platforms. At increased scale, Neuromorphic silicon photonics could access new regimes of ultrafast information processing for radio, control, and scientific computing.

Introduction

Light forms the global backbone of information transmission yet is rarely used for information transformation. Digital optical logic faces fundamental physical challenges¹. Many analog approaches have been researched^2–4, but analog optical co-processors have faced major economic challenges. Optical systems have never achieved competitive manufacturability, nor have they satisfied a sufficiently general processing demand better than digital electronic contemporaries. Incipient changes in the supply and demand for photonics have the potential to spark a resurgence in optical information processing.

A germinating silicon photonic integration industry promises to supply the manufacturing economomies normally reserved for microelectronics. While firmly rooted in demand for datacenter transceivers⁵, the industrialization of photonics would impact other application areas⁶. Industrial microfabrication ecosystems propel technology roadmapping⁷, library standardization^{8, 9}, and broadened accessibility¹⁰, all of which could open fundamentally new research directions into large-scale photonic systems. Large-scale beam steerers have been realized¹¹, and on-chip communication networks have been envisioned^12–14; however, opportunities for scalable silicon photonic information processing systems remain largely unexplored.

Concurrently, photonic devices have found analog signal processing niches where electronics can no longer satisfy demands for bandwidth and reconfigurability. This situation is exemplified by radio frequency (RF) processing, in which front-ends have come to be limited by RF electronics, analog-to-digital converters (ADCs), and digital signal processors (DSP)^{15, 16}. In response, RF photonics has offered respective solutions for tunable RF filters^{17, 18}, ADC itself¹⁹, and simple processing tasks that can be moved from DSP into the analog subsystem^20–22. RF photonic circuits that can be transcribed from fiber to silicon are likely to reap the economic benefits of silicon photonic integration. In a distinct vein, an unprecedented possibility for large-scale photonic system integration could enable systems beyond what can be considered in fiber. If scalable information processing with analog photonics is to be considered, new standards relating physics to processing must be developed and verified.

Standardized concepts that link physics to computational models are required to define essential quantitative engineering tools, namely metrics, algorithms, and benchmarks. For example, a conventional gate has simultaneous meaning as an abstract logical operation and as an arrangement of electronic semiconductors and thereby acts as a conduit between device engineering and computational performance. In another case, neuromorphic electronics adopt unconventional standards defining spiking neural networks as event-based packet networks^23–25. These architectures’ adherence to neural network models unlocks a wealth of metrics²⁶, algorithms^{27, 28}, tools^{29, 30}, and benchmarks³¹ developed specifically for neural networks. Likewise, scalable information processing with analog photonics would rely upon standards defining the relationship between photonic physics and a suitable processing model. Neural networks are among the most well-studied models for information processing with distributed analog elements. The fact that distributed interconnection and analog dynamics are performance strongsuits of photonic physics motivates the study of neuromorphic photonics.

“Broadcast-and-weight”³² was proposed as a standard protocol for implementing neuromorphic processors using integrated photonic devices. Broadcast-and-weight is potentially compatible with mainstream silicon photonic device platforms, unlike free-space optical neural networks^{4, 33}. It is capable of implementing generalized reconfigurable and recurrent neural network models. In the broadcast-and-weight protocol, shown in Fig. 1(a), each neuron’s output is assigned a unique wavelength carrier that is wavelength division multiplexed (WDM) and broadcast. Incoming WDM signals are weighted by reconfigurable, continuous-valued filters called photonic weight banks and then summed by total power detection. This electrical weighted sum then modulates the corresponding WDM carrier through a nonlinear or dynamical electro-optic process. Previous work on microring (MRR) weight banks have established a correspondence between weighted addition operations and integrated photonic filters. In reference to the operation, MRR weight bank scalability³⁴ and accuracy³⁵ metrics can be defined, but MRR weight banks have not been demonstrated within a network.

Figure 1.

Broadcast-and-weight protocol and experiment. (a) Concept of a broadcast-and-weight network with modulators used as neurons. MRR: microring resonator, BPD: balanced photodiode, LD: laser diode, MZM: Mach-Zehnder modulator, WDM: wavelength-division multiplexer. (b) Micrograph of 4-node recurrent broadcast-and-weight network with 16 tunable microring (MRR) weights and fiber-to-chip grating couplers. (c) Scanning electron micrograph of 1:4 splitter. (d) Experimental setup with two off-chip MZM neurons and one external input. Signals are wavelength-multiplexed in an arrayed waveguide grating (AWG) and coupled into a 2 × 3 subnetwork with MRR weights, w ₁₁, w ₁₂, etc. Neuron state is represented by voltages s ₁ and s ₂ across low-pass filtered transimpedance amplifiers, which receive inputs from the balanced photodetectors of each MRR weight bank.

In this manuscript, we demonstrate a broadcast-and-weight system configured by microring weight banks that is isomorphic to a continuous-time recurrent neural network (CTRNN) model. As opposed to “brain-inspired” and “neuro-mimetic”, “neuromorphic” is an unambiguous mathematical concept meaning that a physical system’s governing equations are isomorphic to those describing an abstract neural network model. Isomorphic dynamical systems share qualtitative changes in dynamics as a result of parameter variation. We adopt a strategy for proving neuromorphism experimentally by comparing these dynamical transitions (a.k.a. bifurcations) induced in an experimental device against those predicted of a CTRNN model. In particular, we observe single-node bistability across a cusp bifurcation and two-node oscillation across a Hopf bifurcation. While oscillatory dynamics in optoelectronic devices have long been studied^{36, 37}, this work relies on configuring an analog photonic network that can be scaled to more nodes in principle. This implies that CTRNN metrics, simulators, algorithms, and benchmarks can be applied to larger neuromorphic silicon photonic systems. To illustrate the significance of this implication, we simulate a 24-modulator silicon photonic CTRNN solving a differential equation problem. The system is programmed by appropriating an existing “neural compiler”²⁹ and benchmarked against a conventional CPU solving the same problem, predicting an acceleration factor of 294×.

Results

The CTRNN model is described by a set of ordinary differential equations coupled through a weight matrix.

$$\frac{d\mathbf{s}(t)}{dt}=\mathbf{Wy}(t)-\frac{\mathbf{s}(t)}{\tau }+{\mathbf{w}}_{in}u(t)$$ 1

$$\mathbf{y}(t)=\sigma \left[\mathbf{s}(t)\right]$$ 2

where s(t) are state variables with timeconstants τ, W is the recurrent weight matrix, y(t) are neuron outputs, w _in are input weights, and u(t) is an external input. σ is a saturating transfer function associated with each neuron. Figure 1(b,c) shows the integrated, reconfigurable analog network and experimental setup. Signals u and y are physically represented as the power envelope of different optical carrier wavelengths. The weight elements of W and w _in are implemented as transmission values through a network of reconfigurable MRR filters. The neuron transfer function, σ, is implemented by the sinusoidal electro-optic transfer function of a fiber Mach-Zehnder modulator (MZM). The neuron state, s, is the electrical voltage applied to the MZM, whose timeconstant, τ, is determined by an electronic low-pass filter. We aim to establish a correspondence between experimental bifurcations induced by varying MRR weights and the modeled bifurcations³⁸ derived in Supplementary Section 1.

Cusp Bifurcation

A cusp bifurcation characterizes the onset of bistability in a single node with self-feedback. To induce and observe a cusp, the feedback weight of node 1 is parameterized as w ₁₁ = W _F. The neurosynaptic signal, s ₁, is recorded as the feedback weight is swept through 500 points from 0.05–0.85, and the input is swept in the rising (blue surface) and falling (red surface) directions. The parameters τ, α, and κ in equation (S1.1) are fit to minimize root mean squared error between model and data surfaces. The best fit model has a cusp point at W _B = 0.54. Figure 2(a) shows a modeled cusp surface described by equation (S1.1) and fit to data from Fig. 2(b). In Fig. 2(b), the data surfaces are interpolated at particular planes and projected onto 2D axes as red/blue points. The corresponding slices of the model surface are similarly projected as green lines. The u = 0 slice projected on the s − W _F plane yields a pitchfork curve described by equation (S1.2). The W _F = 0.08 slice projected on the u − s plane yields the bistable curve described by equation (S1.3). Finally, the s = 0 slice projected on the u − W _F plane yields the cusp curve described by equation (S1.4).

Figure 2.

A cusp bifurcation in a single node with feedback weight, W _F, external input, u, and neurosynaptic state, s. (a) Theoretical model surface (gray) and bifurcation curves (green) plotted in 3D. Parameters of the model are fit to data. (b) Experimental data for increasing (blue surface) and decreasing (red surface) input. Theoretical bifurcation curves – with parameters identical to those in (a) – are projected onto 2D axes. The data surfaces are sliced at the planes: u = 0, s = 0, and W _F = 0.08, and similarly projected onto the axes (red and blue points) to illustrate the reproduction of pitchfork, cusp, and saddle-node bifurcations, respectively.

The experimental reproduction of pitchfork, bistable, and cusp bifurcations is demonstrative of an isomorphism between the single-node model and the device under test. An opening of an area between rising and falling data surfaces is characteristic of bistability. The transition boundary closely follows a cusp form. While the pitchfork and bistable slices reproduce the number and growth trends of fixed points, their fits have non-idealities. These non-idealities can be attributed to a hard saturation of the electrical transimpedance amplifier when the input voltage and feedback weight are high. Furthermore, the stability of cusp measurements serve as a control indicating the absence of time-delayed dynamics resulting from long fiber delays and causing spurious oscillations³⁹. Electrical low-pass filtering is used to eliminate these unmodeled dynamics in order to observe modeled bifurcations.

Hopf Bifurcation

Dynamical systems are capable of oscillating if there exists a closed orbit (a.k.a. limit cycle) in the state space, which must therefore exceed one dimension. The Hopf bifurcation occurs when a stable fixed-point becomes unstable while giving rise to a stable limit cycle. Hopf bifurcations are further characterized by oscillations that approach zero amplitude and nonzero frequency near the bifurcation point³⁸. We induce a Hopf bifurcation by configuring the MRR weight matrix to have asymmetric, fixed off-diagonals and equal, parameterized diagonals.

Figure 3 compares the observed and predicted oscillation onset, amplitude, and frequency. Figure 3(a–c) show the time traces for below, near, and above the oscillation threshold. Above threshold, oscillation occurs in the range of 1–5 kHz, as limited by electronic low-pass filters and feedback delay. Figure 3(d) shows the result of a fine sweep of self-feedback weights in the 2-node network, exhibiting the paraboloid shape of a Hopf bifurcation. The voltage of neuron 1 is plotted against that of neuron 2 with color corresponding to W _F parameter. The peak oscillation amplitude for each weight is then projected onto the W _F − y ₂ plane in black, and these amplitudes are fit using the model from equation (S1.8) (red). Bifurcation occurs at W _B = 0.48 in the fit model. Figure 3(e) plots the oscillation frequency above the Hopf point. Data are discarded for W _B < W _F < 0.53 because the oscillations are erratic in the sensitive transition region. Frequency data are then fit with the model of equation (S1.10). The frequency axis is scaled so that 1.0 corresponds to the model frequency at region boundary, which is 4.81 kHz. The Hopf bifurcation only occurs in systems of more than one dimension, thus confirming the observation of a small integrated photonic neural network.

Figure 3.

A Hopf bifurcation between stable and oscillating states. (a–c) Time traces below, near, and above the bifurcation. (d) Oscillation growth versus feedback weight strength. Color corresponds to feedback weight parameter, W _F, to improve visibility. Black shadow: average experimental amplitudes; solid red curve: corresponding fit model; dotted red line: unstable branch. (e) Frequency of oscillation above the Hopf bifurcation. The observed data (black points) are compared to the expected trend of equation (S1.10) (red curve). Frequencies are normalized to the threshold frequency of 4.81 kHz.

Significantly above the bifurcation point, experimental oscillation amplitude and frequency closely match model predictions, but discrepancies are apparent in the transition regime. Limit cycles with amplitudes comparable to noise amplitude can be destabilized by their proximity to the unstable fixed point at zero. This effect could explain the middle inset of Fig. 3, in which a small oscillation grows and then shrinks. Part of this discrepancy can be explained by weight inaccuracy due to inter-bank thermal cross-talk. The two MRR weight banks were calibrated independently accounting only for intra-bank thermal cross-talk. As seen in Fig. 1(c), the physical distance between w ₁₂ (nominally −1) and w ₂₂ (nominally W _F) is approximately 100 μm. While inter-bank cross-talk is not a major effect, w ₁₂ is very sensitive because weight −1 corresponds to on-resonance, and the dynamics are especially sensitive to the weight values near the bifurcation point.

Emulation Benchmark

A dynamical isomorphism between a silicon photonic system and the CTRNN model of equations (1 and 2) implies that larger, faster neuromorphic silicon photonic systems could utilize algorithms and tools developed for generic CTRNNs. Here, we apply a “neural compiler” called the Neural Engineering Framework (NEF)⁴⁰ to program a simulated photonic CTRNN to solve an ordinary differential equation (ODE). This simulation is benchmarked against a conventional central processing unit (CPU) solving the same task. The procedures for each approach are detailed in Methods. As opposed to implementation-specific metrics, benchmarks are task-oriented indicators suitable for comparing technologies that use disparate computing standards. Benchmarking approaches are therefore needed to evaluate the potentials of any unconventional processor in application domains currently served by conventional processors. The chosen benchmark problem consists of solving a well-known ODE called the Lorenz attractor, described by a system of three coupled ODEs with no external inputs:

$$\begin{array}{lll}\hfill \gamma {\dot{x}}_0& \hfill =\hfill & \upsilon ⁎(x_1-x_0)\hfill \\ \hfill \gamma {\dot{x}}_1& \hfill =\hfill & -x_0{x}_2-x_1\hfill \\ \hfill \gamma {\dot{x}}_2& \hfill =\hfill & x_0{x}_1-\beta (x_2+\rho )-\rho \hfill \end{array}$$ 3

where x are the simulation state variables, and γ is a time scaling factor. When parameters, υ, β, and ρ, are set to (υ, β, ρ) = (6.5, 8/3, 28), the solutions of the attractor are chaotic. The photonic CTRNN and CPU solutions are compared in x phase space in Fig. 4(a,b) and the time-domain in Fig. 4(c,d). Figure 4(e) plots the physical modulator voltages, s, linear combinations of which represent simulation variables, x, as discussed in Methods. Because the two simulators are implemented differently, they cannot be compared based on equivalent metrics; however, the time scaling factor, γ, links physical real-time to a virtual simulation time basis, in which a direct comparison can be made.

Figure 4.

Photonic CTRNN benchmarking against a CPU. (a,b) Phase diagrams of the Lorenz attractor simulated by a conventional CPU (a) and a photonic CTRNN (b). (c,d) Time traces of simulation variables for a conventional CPU (c) and a photonic CTRNN (d). The horizontal axes are labeled in physical real time, and cover equal intervals of virtual simulation time, as benchmarked by γ _CPU and γ _Pho. The ratio of real-time values of γ’s indicates a 294-fold acceleration. (e) Time traces of modulator voltages s _i (minor y-axis) for each modulator neuron i (major y-axis) in the photonic CTRNN. The simulation variables, x, in (d) are linear decodings of physical variables, s, in (e).

Figure 4(c,d) plots photonic CTRNN and CPU solutions in the real-time bases, scaled to cover equal simulation intervals. The discrete-time simulation is linked to physical real-time by the step calculation time, Δt = 24.5 ns, and its stability is limited by numerical divergence. We find that γ _CPU ≥ Δt × 150 is sufficient for <1% divergence probability, resulting in γ _CPU = 3.68 μs. The CTRNN simulation is linked to physical real-time by its feedback delay, t _fb = 47.8 ps, and its stability is limited by time-delayed dynamics. We find that γ _Pho ≥ t _fb × 260 is sufficient to avoid spurious dynamics, resulting in γ _Pho = 12.5 ns. The acceleration factor, γ _CPU/γ _Pho, is thus predicted to be 294×.

Implementing this network on a silicon photonic chip would require 24 laser wavelengths, 24 modulators, and 24 MRR weight banks, for a total of 576 MRR weights. The power used by the CTRNN would be dominated by static MRR tuning and pump lasers,as discussed in the Methods section. Minimum pump power is determined by the requirement that recurrently connected neurons are able to exhibit positive eigenvalues. Considering 24 lasers with a realistic wall-plug efficiency of 5%, minimum total system power is expected to be 106 mW. The area used by the photonic CTRNN is split evenly between MRR weight banks (576 × 25 μm × 25 μm = 0.36 mm²) and modulators⁴¹ (24 × 500 μm × 25 μm = 0.30 mm²). The fundamental limits of these performance metrics are compared with other neuromorphic approaches below.

While the qualitative Lorenz behavior is reproduced by CTRNN and CPU implementations, the chaotic nature of the attractor presents a challenge for benchmarking emulation accuracy. Non-chaotic partial differential equations (PDEs) exist to serve as accuracy benchmarks^{42, 43}; however, most non-trivial ODEs are chaotic. One exception is work on central pattern generators (CPGs) that are used to shape oscillations for driving mechanical locomotion⁴⁴. CPGs have been implemented with analog ASICs⁴⁵ and digital FPGAs⁴⁶. While work on CPG hardware has fallen in sub-kHz timescales, similar tasks could be developed for GHz timescales with possible application to adaptive RF waveform generation. Further work could develop CPG-like tasks to benchmark the accuracy of photonic CTRNN ODE emulators.

Accuracy can be assessed through metrics of weight accuracy. Previous work discussed the precision and accuracy to which MRR weight banks could be configured to a desired weight vector³⁵. Even in the presence of fabrication variation and thermal cross-talk, the dynamic weight accuracy was demonstrated to be 4.1 + 1(sign) bits and anticipated to increase with improved design. Since the weight is analog, its dynamic accuracy (range/command error) is not an integer; however, this metric corresponds to bit resolution in digital architectures. In the majority of modern-day neuromorphic hardware, this resolution is selected to be 4 + 1(sign) bits^{24, 47, 48} as a tradeoff between hardware resources and functionality. Significant study has been devoted to the effect of limited weight resolution on neural network function⁴⁹ and techniques for mitigating detrimental effects⁵⁰.

Neuromorphic photonic metrics

In addition to task-driven benchmark analyses, we can perform component-level metric comparisions with other neuromorphic systems in terms of area, signal bandwidth, and synaptic operation (SOP) energy. The 24-modulator photonic CTRNN used as an emulation benchmark was predicted to use a 4.4 mW/neuron using realistic pump lasers, and was limited to 1 GHz bandwidth to spoil spurious oscillations. This results in a computational efficiency of 180 fJ/SOP. The area of an MRR weight is approximately (l × w)/N ² = 25 × 25 = 625 μm²/synapse.

The coherent approach described by Shen et al.⁵¹ based on a matrix of Mach-Zehnder interferometers (MZIs) would exhibit similar fundamental energy and speed limitations, given similar assumptions about detection bandwidth and laser efficiency. The power requirements of this approach were limited by nonlinear threshold activation, rather than signal cascadability. Supposing a laser efficiency of 5%, this was estimated to be around 20 mW/neuron. A 24-neuron system limited to 1 GHz bandwidth would therefore achieve 830 fJ/SOP. While there is no one-to-one correspondence between MZIs and synapses, there is still a quadratic area scaling relationship: (l × w)/N ² = 200 × 100 = 20,000 μm²/synapse, as limited by thermal phase shifter dimension.

The superconducting optoelectronic approach described by Shainline et al.⁵² would be optimized for scalability and efficiency, instead of speed. This can be attributed to the extreme sensitivity of cryogenic photodetectors, but this difference also limits signal bandwidth to 20 MHz. For a 700-neuron interconnect, the wall-plug efficiency is estimated to be around 20 fJ/SOP. Area was calculated to be (l × w)/N ² = 1.4 × 15 = 21 μm²/synapse.

A metric analysis can extend to include neuromorphic electronics, although metrics do not necessarily indicate signal processing merit. Electronic and photonic neuromorphics are designed to address complementary types of problems. Akopyan et al.²⁴ demonstrated a chip containing 256 million synapses dissipating 65 mW. The signal bandwidth, determined by the effective tick, or timestep, is 1.0 kHz, and the chip area is 4.3 cm². This results in an effective 240 fJ/SOP and effective area of 6.0 μm²/synapse. We note that TrueNorth is event-based, meaning explicit computation does not occur for every effective SOP, but only when the input to a synapse is nonzero.

Discussion

We have demonstrated an isomorphism between a silicon photonic broadcast-and-weight system and a reconfigurable CTRNN model through observations of predicted bifurcations. In addition to proof-of-concept, this repeatable method could be used to characterize the standard performance of single neurons and pairs of neurons within larger systems. Employing neuromorphic properties, we then illustrated a task-oriented programming approach and benchmark analysis. Similar analyses could assess the potentials of analog photonic processors against state-of-the-art conventional processors in many application domains. Here, we discuss the implications of these results in the broader context of information processing with photonic devices.

This work constitutes the first investigation of photonic neurons implemented by modulators, an important step towards silicon-compatible neuromorphic photonics. Interest in integrated lasers with neuron-like spiking behavior has flourished over the past several years^{53, 54}. Experimental work has so far focused on isolated neurons^55–57 and fixed, cascadable chains^{58, 59}. The shortage of research on networks of these lasers might be explained by the challenges of implementing low-loss, compact, and tunable filters in the active III/V platforms required for laser gain. In some cases where fan-in is sensitive to input optical phase, it is also unclear how networking would occur without global laser synchronization. In contrast to lasers, Mach-Zehnder, microring, and electroabsorption modulators are all silicon-compatible. Modulator-class neurons are therefore a final step towards making complete broadcast-and-weight systems entirely compatible with silicon foundry platforms. While laser-class neurons with spiking dynamics present richer processing opportunities, modulator-class neurons would still possess the formidable repertoire of CTRNN functions.

In parallel with work on individual laser neurons, recent research has also investigated systems with isomorphisms to neural network models. A fully integrated superconducting optoelectronic network was recently proposed⁵² to offer unmatched energy efficiency. While based on an exotic superconducting platform, this approach accomplishes fan-in using incoherent optical power detection in a way compatible with the broadcast-and-weight protocol. A programmable nanophotonic processor was recently studied in the context of deep learning⁵¹. Coherent optical interconnects exhibit a sensitivity to optical phase that must be re-synchronized after each layer. In the demonstration, optical nonlinearity and phase regeneration were performed digitally. Analog solutions for counteracting signal-dependent phase shifts induced by nonlinear materials⁶⁰ have not yet been proposed. Recurrent neural networks have been investigated in fiber⁶¹. While the current work employs fiber neurons, it is the first demonstration of a recurrent weight network that is integrated. Optical neural networks in free-space have also been investigated in the past⁴ and recently³³. Free-space systems occupy an extra dimension but can not necessarily use it for increased scalability. The volume between focal planes is used for diffractive evolution of the optical field and unused for network configuration. Spatial light modulators that configure the network are generally planar. Shainline et al. noted that integrated neuromorphic photonic systems could potentially be stacked to take advantage of a third dimension⁵².

Reservoir computing techniques that take inspiration from certain brain properties (e.g. analog, distributed) have received substantial recent attention from the photonics community^62–65. Reservoir techniques rely on supervised learning to discern a desired behavior from a large number of complex dynamics, instead of relying on establishing an isomorphism to a model. Neuromorphic and reservoir approaches differ fundamentally and possess complementary advantages. Both derive a broad repertoire of behaviors (often referred to as complexity) from a large number of physical degrees-of-freedom (e.g. optical intensities) coupled through interaction parameters (e.g. transmissions). Both offer means of selecting a specific, desired behavior from this repertoire using controllable parameters. In neuromorphic systems, network weights are both the interaction and controllable parameters, whereas, in reservoir computers, these two groups of parameters are separate. This distinction has two major implications. Firstly, the interaction parameters of a reservoir do not need to be observable or even repeatable from system to system. Reservoirs can thus derive complexity from physical processes that are difficult to model or reproduce, such as coupled amplifiers⁶⁶, coupled nonlinear MRRs⁶⁷, time-delayed dynamics in fibers⁶⁴, and fixed interferometric circuits⁶³. Furthermore, they do not require significant hardware to control the state of the reservoir. Neuromorphic hardware has a burden to correspond physical parameters (e.g. drive voltages) to model parameters (e.g. weights), as was shown in this paper. Secondly, reservoir computers can only be made to elicit a desired behavior through instance-specific supervised training, whereas neuromorphic computers can be programmed a priori using a known set of weights. Because neuromorphic behavior is determined only by controllable parameters, these parameters can be mapped directly between different system instances, different types of neuromorphic systems, and simulations. Neuromorphic hardware can leverage existing algorithms (e.g. NEF) and virtual training results. Particular behaviors, fully determined by the virtual/hardware weights, are guaranteed to occur. Photonic RCs can of course be simulated; however, they have no corresponding guarantee that a particular hardware instance will reproduce a simulated behavior or that training will be able to converge to this behavior.

At increased scale, neuromorphic silicon photonic systems could be applied to unaddressed computational areas in scientific computing and RF signal processing. A key benefit of neuromorphic engineering is that existing algorithms can be leveraged. A subset of CTRNNs, Hopfield networks⁶⁸, have been used extensively in mathematical programming and optimization problems²⁷. The ubiquity of PDE problems in scientific computing has motivated the development of analog electronic neural emulators⁴². Further work could explore the use of NEF to emulate discrete space points of PDEs. Neural algorithms for CTRNNs have been developed for real-time RF signal processing, including spectral mining⁶⁹, spread spectrum channel estimation⁷⁰, and arrayed antenna control⁷¹. There is insistent demand to implement these tasks at wider bandwidths using less power than possible with RF electronics. Additionally, methodologies developed for audio applications, such as noise mitigation²⁸, could conceivably be mapped to RF problems if implemented on ultrafast hardware. Unsupervised neural-inspired learning has been used with a single MRR weight bank for statistical analysis of multiple RF signals⁷².

We have demonstrated a reconfigurable analog neural network in a silicon photonic integrated circuit using modulators as neuron elements. Network-mediated cusp and Hopf bifurcations were observed as a proof-of-concept of an integrated broadcast-and-weight system³². Simulations of a 24 modulator neuron network performing an emulation task estimated a 294× speedup over a verified CPU benchmark. Neural network abstractions are powerful tools for bridging the gap between physical dynamics and useful application, and silicon photonic manufacturing introduces opportunities for large-scale photonic systems.

Methods

Experimental Setup

Samples shown in Fig. 1(b) were fabricated on silicon-on-insulator (SOI) wafers at the Washington Nanofabrication Facility through the SiEPIC Ebeam rapid prototyping group¹⁰. Silicon thickness is 220 nm, and buried oxide (BOX) thickness is 3 μm. 500 nm wide WGs were patterned by Ebeam lithography and fully etched through to the BOX⁷³. After a cladding oxide (3 μm) is deposited, Ti/W and Al layers are deposited. Ohmic heating in Ti/W filaments causes thermo-optic resonant wavelength shifts in the MRR weights. The sample is mounted on a temperature stabilized alignment stage and coupled to a 9-fiber array using focusing subwavelength grating couplers⁷⁴. The reconfigurable analog network consists of 2 MRR weight banks each with four MRR weights with 10 μm radii.

Each MRR weight bank is calibrated using a multi-channel protocol described in past work^{35, 75}: an offline measurement procedure is performed to identify models of thermo-optic cross-talk and MRR filter edge transmission. During this calibration phase, electrical feedback connections are disabled and the set of wavelength channels carry a set of linearly seperable training signals. After calibration, the user can specify a desired weight matrix, and the control model calculates and applies the corresponding electrical currents.

Weighted network outputs are detected off-chip, and the electrical weighted sums drive fiber Mach-Zehnder modulators (MZMs). Detected signals are low-pass filtered at 10 kHz, represented by capacitor symbols in Fig. 1(c). Low-pass filtering is used to spoil time-delayed dynamics that arise when feedback delay is much greater than the state time-constant³⁷. In this setup with on-chip network and off-chip modulator neurons, fiber delayed dynamics would interfere with CTRNN dynamical analysis³⁹. MZMs modulate distinct wavelengths λ ₁ = 1549.97 nm and λ ₂ = 1551.68 nm with neuron output signals y ₁(t) and y ₂(t), respectively. The MZM electro-optic transfer function serves as the nonlinear transfer function, y = σ(s), associated with the continuous-time neuron. A third wavelength, λ ₃ = 1553.46 nm, carries an external input signal, x(t), derived from a signal generator. Each laser diode source (ILX 7900B) outputs +13 dBm of power. All optical signals (u, y ₁, and y ₂) are wavelength multiplexed in an arrayed waveguide grating (AWG) and then coupled back into the on-chip broadcast STAR consisting of splitting Y-junctions⁷⁶ (Fig. 1(c)).

Photonic CTRNN Solver

Recently developed compilers, such as Neural ENGineering Objects (Nengo)⁷⁷, employ the Neural Engineering Framework (NEF)⁴⁰ to arrange networks of neurons to represent values and functions without relying on training. While originally developed to evaluate theories of cognition, the NEF has been appropriated to solve engineering problems⁷⁸ and has been used to program electronic neuromorphic hardware⁷⁹. Background on the NEF compilation procedure is provided in Supplementary Section 2. Simulation state variables, x, are encoded as linear combinations of real population states, s. Each neuron in a population has the same tuning curve shape, σ, but differ in gain g, input encoder vector e, and offset b. The input-output relation of neuron i – equivalent to equation (2), is thus s _i = σ (g _i e _i · x + b _i). In this formulation, arbitrary nonlinear functions of the simulation variables, f(x), are represented by linear combinations of the set of these tuning curves across the domain of values of x considered. Introducing recurrent connections in the population introduces the notion of state time-derivatives, as in equation (1). By applying the decoder transform to both sides of equation (1) and using the arbitrary function mapping technique to find W, the neural population emulates an effective dynamical system of the form $\dot{\mathbf{x}}=\mathbf{f}\left(\mathbf{x}\right)$. Given equations (3) stated in this form, Nengo performs the steps necessary to represent the variables, functions, and complete ODE.

Modifications were made to the standard Nengo procedure. Firstly, we specify the tuning curve shape as the sinusoidal electro-optic transfer characteristic of a MZM. Secondly, to reduce the number of MZMs required, we choose encoders to be the vertices of a unit-square {e} = [1, ±1, ±1], while they are typically chosen randomly. Thirdly, the MZM sinusoidal transfer function provides a natural relation to the Fourier basis. Gains are chosen to correspond to the first three Fourier frequencies of the domain: g ∈ s _π/2 · {1, 2, 3}, where s _π is the MZM half-period. Offsets were chosen to be b ∈ {0, s _π/2}, corresponding to sine and cosine components of each gain frequency. The total number of modulator neurons is therefore #e · #g · #b = 4 · 3 · 2 = 24. Figure 4(e) shows the MZM states, s(t), of which simulation variables, x(t), are linear combinations. From this plot, it appears that some neurons are barely used. Thus, further optimizations of number of neurons could be made by pruning those neurons after compilation of the weight matrix.

The operational speed of this network would be limited by time-of-flight feedback delay. In Fig. 1(a), the longest feedback path is via the drop port of the last (pink) MRR weight of the first (yellow) neuron’s bank. The path includes the perimeter of the square MRR weight matrix, plus a drop waveguide folded back along the bank. Supposing a minimum MRR pitch of 25 μm and MZM length of 500 μm, the feedback delay would then be (6 × 25 × 25 + 500) · n/c = 48 ps. We model this delayed feedback in the Nengo simulation and then adjust feedback strength to find the minimum stable simulation timescale. For γ _Pho/t _fb < 65, spurious time-delayed dynamics dominate. For γ _Pho/t _fb < 104, the butterfly phase diagram in Fig. 4(b) is not reproduced accurately. γ _Pho/t _fb ≥ 260 is chosen for robust reproduction of the expected dynamics.

Conventional CPU Solver

Conventional digital processors must use a discrete-time approximation to simulate continuous ODEs, the simplest of which is Euler continuation:

$$\mathbf{x}\left[(n+\mathrm{1)}\mathrm{\Delta }t\right]=\mathbf{x}\left[n\mathrm{\Delta }t\right]+\mathrm{\Delta }t\mathbf{f}\left(\mathbf{x}\left[n\mathrm{\Delta }t\right]\right)$$ 4

where Δt is the time step interval. To estimate the real-time value of Δt, we develop and validate a simple CPU model. For each time step, the CPU must compute f(x[nΔt]) as defined in equation (3), resulting in 9 floating-point operations (FLOPs), and 12 cache reads of the operands. The Euler update in equation (4) constitutes one multiply, one addition, and one read/write for each state variable, resulting in 6 FLOPs and 6 cache accesses. Supposing a FLOP latency of 1 clock cycle, Level 1 (L1) cache latency of 4 cycles, and 2.6 GHz clock, this model predicts a time step of Δt = 33 ns. This model is empirically validated using an Intel Core i5-4288U. The machine-optimized program randomly initializes and loops through 10⁶ Euler steps of the Lorenz system, over 100 trials. CPU time was measured to be Δt = 24.5 ± 1.5 ns. The minimum stable simulation timescale is limited by divergent errors stemming from time discretization. We performed a series of 100 trials over 100 values of Δt/γ _CPU, finding that <1% probability of divergence occurred for γ _CPU/Δt ≥ 150.

Minimum Power Calculations

Static thermal power must be applied to each weight in order to track MRRs to the on-resonance condition. Supposing a bank length set by an MRR pitch of 25 μm and count of 24, the MRR network would occupy a square with 600 μm sides. Within this length, resonances can be fabricated with repeatability within ±1.3 nm⁸⁰. Supposing a tuning efficiency of 0.25 nm/mW⁸¹, it would take an average of 5.2 mW/weight to track resonance, for a static power dissipation of 3.0 W. On the other hand, if depletion-based tuning can be used, there would be negligible static power dissipation in the weights.

The laser bias power must be set such that a modulator neuron can drive downstream neurons with sufficient strength. A neuron fed back to itself should be able to elicit an equivalent or greater small-signal response after one round-trip. This condition is referred to as signal cascadability and can be stated as g ≥ 1, where g is round-trip, small-signal gain. If the cascadability condition is not met, all signals will eventually attenuate out with time. In other words, in a recurrent network, the real part of system eigenvalues would not be able to exceed zero. Round-trip gain is expressed as

$$g=\frac{d{P}_{out}}{d{P}_{in}}$$ 5

For a modulator-based broadcast-and-weight system, this breaks down into receiver and modulator components. Assuming a voltage-mode modulator, such as reverse-biased MRR depletion modulator,

$${\frac{d{P}_{out}}{d{V}_{mod}}|}_{max}=\frac{\pi }{2V_{\pi }}{P}_{pump}$$ 6

$$\frac{d{V}_{mod}}{d{P}_{in}}=R_{PD}{R}_r$$ 7

where V _π is modulator π-voltage, P _pump is modulator pump power, and R _PD is detector responsivity. Because input power generates a photocurrent, yet a depletion modulator is voltage-driven, the receiver’s impedance, R _r, determines the conversion and can be set externally. As R _r increases, round-trip gain also increases, but bandwidth decreases according to f = (2πR _r C _mod)⁻¹, where C _mod is PN junction capacitance of the modulator. By setting the cascadability condition: g = 1 and combining the above equations, we find that

$$P_{pump}(R_r)=\frac{2V_{\pi }}{\pi R_{PD}{R}_r}$$ 8

$$P_{pump}(f)=\mathrm{4(}{V}_{\pi }{C}_{mod})R_{PD}^{-1}f$$ 9

The values of V _π, C _mod, and R _PD on a typical silicon photonic foundry platform have been published⁴¹. For an MRR depletion modulator, V _π = 1.5 V, C _mod = 35 fF. For a PD on the same platform, R _PD = 0.97 A/W. This means that the minimum pump power for a given signal bandwidth is 2.2 × 10⁻¹³ W/Hz.

In this paper, we study a 24-node CTRNN whose signal bandwidth is restricted to 1 GHz to avoid time-delay dynamics. This means that, for the cascadability condition to be met, modulator pumping must be at least 0.22 mW/neuron of optical power. Adding up 24 lasers and accounting for laser inefficiency, wall-plug system power would be 106 mW.

Author Contributions

A.N.T. conceived of the methods and experiments. A.N.T., E.Z., and A.X.W. conducted the experiments and analyzed the results. T.FdL developed methods and simulations of the photonic CTRNN emulator. All authors reviewed the manuscript.

Competing Interests

The authors declare that they have no competing interests.