Bioinspired mechano-photonic artificial synapse based on graphene/MoS₂ heterostructure

A bioinspired mechano-photonic artificial synapse with synergistic mechanical and optical plasticity is demonstrated.

Abstract

Developing multifunctional and diversified artificial neural systems to integrate multimodal plasticity, memory, and supervised learning functions is an important task toward the emulation of neuromorphic computation. Here, we present a bioinspired mechano-photonic artificial synapse with synergistic mechanical and optical plasticity. The artificial synapse is composed of an optoelectronic transistor based on graphene/MoS₂ heterostructure and an integrated triboelectric nanogenerator. By controlling the charge transfer/exchange in the heterostructure with triboelectric potential, the optoelectronic synaptic behaviors can be readily modulated, including postsynaptic photocurrents, persistent photoconductivity, and photosensitivity. The photonic synaptic plasticity is elaborately investigated under the synergistic effect of mechanical displacement and the light pulses embodying different spatiotemporal information. Furthermore, artificial neural networks are simulated to demonstrate the improved image recognition accuracy up to 92% assisted with mechanical plasticization. The mechano-photonic artificial synapse is highly promising for implementing mixed-modal interaction, emulating complex biological nervous system, and promoting the development of interactive artificial intelligence.

INTRODUCTION

Human brain implements the cognition, learning, and memory tasks through visual, auditory, olfactory, and somatosensory interactions (1). This process is critically difficult to be emulated by conventional von Neumann architectures, which additionally requires sophisticated functions of event-driven, adaptive, parallel/convolutional computations and multimodal plasticization (2). Brain-inspired neural network (3, 4) is commonly composed of various artificial synaptic devices [e.g., memristor (5, 6) and transistor (7, 8)] to conduct the information transmission and processing by updating the synaptic weight. Emerging photonic synapse combines the optical and electric neuromorphic modulation and computation, offering a favorable option with high bandwidth, fast speed, low cross-talk, and substantially reduced power consumption (9, 10). In particular, the persistent photoconduction in photonic synapse based on metal oxides (9, 11), perovskites (12), carbon nanotubes (13), and two-dimensional (2D) materials (14, 15) are favorable to emulate typical synaptic behaviors, such as spike timing–dependent plasticity, neural facilitation/depression, and conversion from short-term to long-term memory. To emulate a more practical nervous system, it is preferred to update the connection weight in multistep or multimodal plastic strategies, which convey more flexible and dexterous synaptic plasticity (16). Sequential (or superimposed) multimodal modulations in synaptic devices are also the fundamental of implementing complex neural behaviors and activities, which are still a significant challenge to conventional artificial synapses. Hence, new architectures of neuromorphic systems are urgent to diversify multimodal interaction/coupling toward interactive image recognition and perception-assisted instruction execution.

Biomechanical motions (e.g., touch, step, handshake, arm swing/waving, and eye blinking) are abundant and ubiquitous in daily life, which are potentially applicable as trigger/interactive signals for electronics operation or as mechanical strategy for artificial synapse plasticization (17). The photoreceptor cells existing in the retina of higher animals play critical roles in extracting visual signals through optical/photonic sensing, which has been successfully mimicked by photodetectors, phototransistors, optical memories, and optoelectronic synapses (12, 18–21). In particular, the optoelectronic neuromorphic devices have exhibited important applications in selective ultraviolet light detection (22), ultrafast machine vision sensors (23), on-chip photonic synapse (24), optical spiking afferent nerve (25), stretchable sensorimotor synapses (26), etc. The associative analysis of biomechanical and visual information is the basis of perception and cognition ability of human brain, which is of great significance for acquisition of somatosensory data and emulating artificial intelligence (16, 27). How to synergize mechanical and optical strategies to update the synaptic weight is essential to realize multimodal plasticity for preferential interactive neuromorphic computation. Triboelectric nanogenerator (TENG) (28), using Maxwell’s displacement current (i.e., a time-dependent electric field) (29) as the driving force, offers an effective manner to modulating charge transport in semiconductor devices (30–32), logic inverter (33), multifunctional sensors (34), neuromorphic devices (35–37), etc. Coupling triboelectric potential modulation with photonic sensitization in artificial synapses may offer an active and direct way to harness the synaptic plasticity and implement multimodal neuromorphic computation. Among the active materials for photonic synapses, binary or heterostructure constructed from 2D materials (27, 38–42) allows a broadband and ultrafast photoresponse with facile photocurrent generation and impressive quantum efficiency. The noninvasive van der Waals heterostructures (43) with trap-free interface are also ready to preserve high electron mobility/photoresponsivity of each layer with reinforced gate-tunable interlayer charge exchange ability. These superior optoelectronic characteristics paired with multimodal plastic strategy may promote the development of tunable and versatile neuromorphic devices.

Here, we present a mechano-photonic artificial synapse based on graphene/MoS₂ (Gr/MoS₂) heterostructure with synergistic mechanical and optical plasticity (i.e., mechanical displacement–assisted optoelectronic synaptic plasticity). The synaptic device comprises an optoelectronic transistor based on Gr/MoS₂ heterostructure and an integrated TENG in contact-separation mode. The triboelectric potential by TENG displacement can effectively drive the synaptic transistor. By controlling the charge transfer/exchange between the graphene and MoS₂ layer, the triboelectric potential can also readily modulate the optoelectronic synaptic behaviors, e.g., postsynaptic photocurrents, persistent photoconductivity, and photosensitivity. The photonic synaptic plasticity including long-term memory and neural facilitation is investigated under the synergistic effect of mechanical displacement (as a status parameter) and the light pulse embodying spatiotemporal information (e.g., intensity and illumination time). It is also facile to realize photonic programming and mechanical erasing process in the mechano-photonic artificial synapses. Beyond the device-level simulation of synaptic functions, artificial neural networks (ANNs) are further built to demonstrate the feasibility of improved image recognition assisted with mechanical plasticization. The mechano-photonic artificial synapses with sophisticated combination of synergistic mechanical and photonic plasticity have huge prospective for mixed-modal neuromorphic chip and unconventional convolutional neural networks toward the applications of interactive optoelectronic interfaces, synthetic retinas, intelligent robots, etc. The programmable persistent photonic conductivity and mechanical behavior–derived modulation/erasure/plasticization are highly promising for in-memory neuromorphic computation of various sensory data (e.g., optical, photonic, tactile, pressure, displacement, etc.) and dual-modal (or even higher level of multimodal) assisted imitation of memory behavior and associative learning.

RESULTS

Biological tactile/visual neurons and mechano-photonic artificial synapse

The perception and cognition ability of human brain assisted with associative biomechanical and visual sensations are critical for acquiring somatosensory and visual information (16). The brain encloses numerous neurons to receive the interactive signals in different modalities (e.g., mechanical and optical signals) and implements cross-modal neuromorphic computation in the multisensory association area. The synapses are the connection points of two adjacent neurons (44), which also play crucial roles in the neural information transmissions (Fig. 1A). Inspired by the brain and nervous system, we are trying to demonstrate a mechano-photonic artificial synapse with synergistic mechanical and optical plasticity. The schematic illustration of the mechano-photonic artificial synapse is shown in Fig. 1B. It includes an optoelectronic transistor based on Gr/MoS₂ heterostructure and an integrated TENG. In the optoelectronic transistor, the monolayer graphene by chemical vapor deposition (Raman spectrum in fig. S1A) is stacked on a multilayer MoS₂ flake (fig. S1B) on a Si/SiO₂ substrate [optical image in Fig. 1B (i)]. In the hybrid Gr/MoS₂ structure, the photogenerated charge carriers in MoS₂ can be transferred to graphene through electrostatic band bending (Fig. 1B, ii). The interface barrier prevents the fast recombination of photogenerated electron-hole pairs and leads to the persistent photoconductivity, which is beneficial to emulate the decay behavior of biologic synapse (39). The integrated TENG component is composed of Cu/polytetrafluoroethylene (PTFE)/Cu in contact-separation mode, with one friction layer (PTFE/Cu) connected to the transistor gate and another Cu electrode as the movable counter friction layer. The mechanical displacement (D) between two friction layers of TENG can induce triboelectric potential coupling to the transistor, which can directly affect the charge transfer/exchange in the transistor channel (i.e., Gr/MoS₂ heterostructure) and modulate the photocurrent of the optoelectronic synapse (Fig. 1B, iii). Thus, the mechano-photonic artificial synapse can realize the synergistic mechanical and optical modulation on the synaptic plasticity. The detailed fabrication process of mechano-photonic artificial synapse following the process of MoS₂ transfer, Gr/MoS₂ heterostructure construction, graphene patterning, source and drain electrode (Cr/Au) deposition, and TENG integration is illustrated in fig. S2 (see experimental section for details; the optical image of each step for transistor fabrication is in fig. S3).

Fig. 1. Biological tactile/visual neurons and mechano-photonic artificial synapse.

(A) Schematic illustrations of biological tactile/visual sensory system. (B) Schematic diagram of the mechano-photonic artificial synapse based on graphene/MoS₂ (Gr/MoS₂) heterostructure. (i) Top-view scanning electron microscope (SEM) image of the optoelectronic transistor; scale bar, 5 μm. The cyan area indicates the MoS₂ flake, while the white strip is graphene. (ii) Illustration of charge transfer/exchange for Gr/MoS₂ heterostructure. (iii) Output mechano-photonic signals from the artificial synapse for image recognition.

TENG and optoelectronic transistor based on Gr/MoS₂ heterostructure

To ensure the feasibility of triboelectric potential gating, the TENG output voltage (V_TENG, equivalent to the gate voltage) versus mechanical displacement is first characterized with a test circuit where the transistor dielectric capacitor (C_MIS) and test system capacitor (Keithley 6514) are connected with TENG in parallel (Fig. 2A). The integrated TENG can supply an equivalent gate voltage (V_G) ranging from 62 to −52 V with the displacement varied from −1.75 to 1.75 mm. An initial preset position of the movable layer is defined as D₀ = 0 with V_TENG = 0 (electrostatic equilibrium state). The separation (D+) and contact (D−) motions of TENG can induce negative and positive gate voltage coupling to the transistor, respectively (fig. S4). The intrinsic electrical performances of the synaptic phototransistor are systematically studied in fig. S5. It exhibits typical transfer characteristics of the transistor device based on Gr/MoS₂ heterostructure. The drain current (I_D) shows gradual increment with V_G in the region of V_G < V_T (V_T is the MoS₂ conduction threshold; fig. S5C). In contrast, it shows an invariable behavior (insensitive to V_G) in the region of V_G > V_T; the transfer characteristics are quite different from the transport behaviors compared with pristine graphene or MoS₂ device. This can be attributed to the fact that the underlying MoS₂ (n-type) starts to exhibit substantial transport behavior at V_G > V_T, which screens the back gating on graphene (39). Under light illumination [green light-emitting diode (LED): wavelength, 525 nm], I_D shows a decrement tendency with the increased light intensity in the region of V_G < V_T, while it has no obvious change in the region of V_G > V_T regardless of the light intensity due to the suppressing effect of MoS₂ on charge exchange in the heterostructure (fig. S5C). Corresponding photocurrents and photoresponsivity (γ) under applied V_G are discussed in fig. S6. Notably, because of the readily modulated Fermi level of graphene and electronic states in MoS₂ by electric field, γ exhibits an increment trend with decreased V_G, which means a tunable photoresponsivity of the Gr/MoS₂ transistor. The influence of MoS₂ thickness on the optoelectronic performances are also discussed in fig. S7. Optimization of MoS₂ thickness in the heterostructure is critical to the device photosensitivity and electrical performances, which can potentially influence the eventual device/system-level applications of Gr/MoS₂ heterostructure.

Fig. 2. Mechano-optoelectronic transistor based on Gr/MoS₂ heterostructure and corresponding working mechanism.

(A) The TENG output voltage (V_TENG) versus displacement (D). Inset: Equivalent circuit diagram for V_TENG characterization. (B) Transfer curves (I_D versus D) in the dark and under different green light power intensity (P_LED). (C) Working mechanism of the mechano-optoelectronic transistor based on Gr/MoS₂ heterostructure. Schematic illustrations of the working principles and the corresponding energy band diagram at (i) initial flat-band state, (ii) separation state (D+), and (iii) contact state (D−).

Mechano-optoelectronic transistor based on Gr/MoS₂ heterostructure

To characterize the mechano-optoelectronic transistor, the output performances are measured under the synergistic effects of TENG displacement and illumination with the green LED at different power intensities (P_LED^’s) in fig. S8. By extracting the I_D according to the applied TENG displacement under different illumination states (i.e., the transfer curves of I_D versus D at drain voltage V_D = 1 V, extracted from fig. S8), the I_D markedly decreases with the increased P_LED in the region of D > −0.1 mm (Fig. 2B). However, it has no obvious change in the region of D < −0.1 mm. The optoelectrical performances under mechanical displacement are consistent with the situation under applied V_G. The TENG displacement of −0.1 mm is equivalent to the MoS₂ conduction threshold of ~5 V according to Fig. 2A. The working mechanism of the mechano-optoelectronic transistor is illustrated in Fig. 2C. The initial preset position (D₀) is an electrostatic equilibrium state where charge transfer between Cu electrode and PTFE/Cu friction layer has already finished and the transferred charges are neutralized by grounding. In this state, no triboelectric potential is applied to the transistor and the Gr/MoS₂ heterostructure is in flat-band state without charge transfer/exchange (Fig. 2C, i). When the two friction layers separate (D+, the transferred electrons in TENG cannot be neutralized), an induced −V_TENG is coupled to the transistor gate, shifts the graphene Fermi level downward, and leads to the electrostatic doping of graphene with holes. Under light illumination and with TENG further separated until −V_TENG < V_T, the photogenerated electrons are injected from the conduction band of MoS₂ into the low energy level of hole-doped graphene (Fig. 2C, ii) (39). The additional injection of electrons increases the resistance of graphene and leads to the current decrement under light illumination, as shown in Fig. 2B. When the two friction layers approach each other (D−), +V_TENG is induced and coupled to the transistor gate because more electrons in TENG are transferred to ground with positive charges left. When the +V_TENG increases to be higher than V_T, the energy bands in MoS₂ will be flat because of its near-metallic characteristics at this stage (i.e., gating by +V_TENG > V_T). The flat band of MoS₂ suppresses the injection of electrons into graphene and equilibrates the carrier distribution in Gr/MoS₂ heterostructure (Fig. 2C, iii).

To further evaluate the tunable photoresponsivity of the mechano-photonic Gr/MoS₂ transistor, we investigate the influence of mechanical displacement on the photocurrent and photosensitivity in fig. S9. Under the light illumination at power intensity of 11.5 mW cm⁻², the photocurrent varies from −1.4 to −53.4 μA with D increased from 0.5 to 1.5 mm. The extracted photosensitivity shows a similar variation trend, increasing from 2.4 × 10⁴ to 9.3 × 10⁵ A/W. More positive displacement (D+) induces larger photocurrent and higher photoresponsivity based on the electric field–dependent Fermi level and electronic states in the heterostructure. The mechanical displacement–related tunable photoresponse is of great significance to enhance the device photosensitivity or modulate the photosensitivity at a desired level on demand.

Mechano-photonic artificial synapse based on Gr/MoS₂ heterostructure

The channel conductivity (i.e., synaptic weight) of the Gr/MoS₂ transistor can be readily modulated by both mechanical displacement and light illumination, which is the fundamental of multimodal plasticity in the mechano-photonic artificial synapse. The basal postsynaptic current (PSC; i.e., the I_D of the Gr/MoS₂ transistor under dark state) represents a clear variation trend from 260 to 308 μA with the displacement changed from 0 to 1.5 mm (Fig. 3A), matching well with the transfer characteristics in Fig. 2B. The PSCs can also be maintained stable at different levels under different displacement states, which is prerequisite to ensure reliable study on the synaptic photoresponse. By taking D = 1 mm as the example, when the device is illuminated with the green light pulse (P_LED = 3.5 mW cm⁻²; pulse width, 50 ms), the photo-activated postsynaptic current shows a marked negative increment (defined as −ΔPSC) by 20 μA and tends to be saturated at a lower level of ~18 μA after switching off the light (Fig. 3B). The decay time (τ) in this work is defined as the time of PSC decreasing from the peak current to the steady current level (to distinguish from the retention time), which is evaluated from fig. S10. Both mechanical displacement and light illumination can synergistically affect the decay time as shown in fig. S11. This electrical characteristic under single light pulse is similar to the long-term depression (LTD) behavior in biological synapse (a few seconds to minutes) (45). In the mechano-photonic artificial synapse, the displacement can affect the basal PSC level, while the light illumination can lead to dynamic variation of PSC. This means that the triboelectric potential–modulated electrical behaviors and the optoelectrical behaviors can synergistically affect the PSC. Corresponding −ΔPSCs under the synergistic modulation by displacements and light illumination are investigated in Fig. 3C. The visual signal is applied by the same green LED (P_LED = 3.5 mW cm⁻²; pulse width, 50 ms), while the mechanical displacements are applied as status parameters with four different values (0.75, 1, 1.25, and 1.5 mm), respectively. The peak value of −ΔPSC increases from 9 to 24 μA with D varied from 0.75 to 1.5 mm. The intrinsic retention time of the mechano-photonic artificial synapse is also evaluated to be capable of retaining for over 1 hour without obvious changes (fig. S12), which is also dependent on the displacement. These results are vigorous evidences to the implementation of the synergistic mechanical and optical modulation on long-term synaptic plasticity.

Fig. 3. Mechano-photonic artificial synapse based on Gr/MoS₂ heterostructure.

(A) Real-time evaluation of the initial PSC in the dark; D varies from 0 to 1.5 mm. (B) Photo-activated postsynaptic current (−ΔPSC) of the artificial synapse at D = 1 mm and P_LED = 3.5 mW cm⁻² with light pulse width at 50 ms. (C) −ΔPSC under the synergetic effect of light illumination and different D (from 0.75 to 1.5 mm) at V_D = 1 V. Top inset: The schematic diagram of mechanical and visual presynaptic signals. The P_LED is fixed at 3.5 mW cm⁻² with light pulse width at 50 ms. (D) Schematic illustrations of the density of states and carrier distribution in Gr/MoS₂ heterostructure under light illumination and after light illumination (persistent photocurrent region) at two different displacement status (D+ and D+′).

To explore the underlying mechanism, the decrement of PSC can be attributed to the weakened density of holes in graphene, while the persistent PSC is a synergetic result determined by the localized states in MoS₂ and the directional triboelectric field (i.e., the equivalent gate electric field). In detail, when the light is illuminated, photogenerated electron-hole pairs are induced in MoS₂. The negative triboelectric field (at D+) drives the electrons and holes to move toward graphene and the opposite direction, respectively. The photogenerated electrons are injected into hole-doped graphene, thereby decreasing the effective density of holes (increase graphene resistance) and inducing the decrement of PSC (or negative increment, −ΔPSC = 20 μA). When the light is turned off, a small portion of electron-hole pairs gradually enters the recombination process, resulting in the slight recovery of −ΔPSC to ~18 μA. However, the potential barrier at Gr/MoS₂ interface prevents the photogenerated electrons from totally diffusing back to MoS₂ (fig. S13). Hence, persistent regimes of photo-injected electrons and localized holes are formed and lead to the consequent persistent PSC (31). Notably, under smaller displacement status (D+′, e.g., D = 0.75 mm), the initial lower basal PSC leads to the smaller magnitude of −ΔPSC (Fig. 3C) due to the limited available free charge carriers for optical excitation compared with the larger D status. In this situation, the persistent regimes assigned to the density of photo-injected electrons and localized holes will be smaller (Fig. 3D, left). When the applied D+ is smaller than 0.5 mm (the equivalent V_G tends to increase and approach V_T, the MoS₂ conduction threshold), −ΔPSC triggered by light pulse is further decreased until no obvious PSC change is observed at D = 0 (fig. S14). This is attributed to the fact that MoS₂ starts to exhibit near-metallic characteristics beyond the conduction threshold and suppresses the injection of photogenerated electrons. The erasing process of persistent PSC can be readily realized by the TENG contact pulse (i.e., D−, equivalent to more positive V_G; fig. S15), which can instantaneously raise the Fermi energy over V_T and redistribute the charge carriers in Gr/MoS₂ heterostructure. Compared with previous researches of bioinspired synaptic devices with single-modal processing capacity [e.g., electrical signal (46–48), visual signal (9, 11), mechanical signal (2, 28, 49), etc.], the mechano-photonic artificial synapse can realize dual-modal plasticization by mechanical and visual signals simultaneously.

Synergistic effect of mechanical and visual signal for mechano-photonic artificial synapse

To further explore the synergistic effect of mechanical and optical modulation on the artificial synapse, we investigate more −ΔPSCs under the light pulse inputs embodying different spatiotemporal information (e.g., light intensity and illumination time). The schematic diagram of the synergistic effect of the mechanical and optical plasticization for the artificial synapse is shown in Fig. 4A. Under a fixed D at 1 mm, the peak value of −ΔPSC increases (negative increment) from 26 to 35 μA as P_LED increases from 0.73 to 13.5 mW cm⁻² (Fig. 4B). This is due to the fact that more photogenerated carriers are excited under stronger light, which contributes to the decrease in the conductivity of graphene and leads to the decreased PSC. When the light is turned off, all the currents exhibit a slight decrement and tend to maintain at a lower current level (Fig. 4C), exhibiting repeatable LTD synaptic behaviors. Different light illumination time reflects the temporal information of the light pulse, which determines the photogenerated charge carriers per unit time. The maximum −ΔPSC amplitude shows a notable increment trend from 4.67 to 32 μA as the illumination time increases from 1 to 500 ms (D = 1 mm, P_LED = 3.5 mW cm⁻²; Fig. 4D). Both the light intensity and illumination time can be used to realize optical modulation on the synaptic inhibitory behaviors with conditional TENG displacement. The influence of multiple consecutive light pulses on the mechano-photonic artificial synapse under different displacements is investigated in Fig. 4E. The −ΔPSC shows an overall increment trend with increased displacement from 0.5 to 1.5 mm. When 40 consecutive light pulses (P_LED = 3.5 mW cm⁻²; pulse width, 50 ms) are applied to the synapse device at D = 1 mm, the amplitude value of −ΔPSC gradually increases to 20 μA and tends to be stabilized at ~33 μA. The gain value of −ΔPSC (defined as A_n/A₁, where A is the amplitude of the −ΔPSC peak value) reaches 132% after the stimulation of consecutive light pulses forty times (fig. S16). Another important parameter of the conductance margins (G_max/G_min, defined as the ratio between the maximum and minimum conductance value) is also introduced to elaborate the influence of TENG displacements on the postsynaptic currents. As shown in Fig. 4F, under light illumination power of 3.5 mW cm⁻², the conductance margin G_max/G_min between the PSC at pulse number of 40 and the initial PSC increases from 590 to 857 as D varies from 0.5 to 1.5 mm. The multiple light pulse–enhanced synaptic plasticity under mechanical displacement has great potential in emulating the memory and supervised learning functions.

Fig. 4. Synergistic effect of mechanical and visual signal for mechano-photonic artificial synapse.

(A) Schematic diagram of the synergistic mechanical and optical signal for mechano-photonic artificial synapse. (B) −ΔPSCs under different P_LED^’s at a fixed D of 1 mm, V_D = 1 V, and light pulse width of 0.5 s. Inset: The peak current of ΔPSC versus P_LED. (C) −ΔPSCs under different P_LED^’s when the light is turned off. (D) −ΔPSCs at different light pulse width (P_LED = 3.5 mW cm⁻² and D = 1 mm). (E) The −ΔPSCs under 40 consecutive light pulses under different displacements (P_LED = 3.5 mW cm⁻²; pulse width, 50 ms; D = 0.5, 1, and 1.5 mm). (F) The conductance margins (G_max/G_min) as a function of displacement.

Simulation of ANN for image recognition

On the basis of the synergistic effect of mechanical and optical plasticity in the artificial synapse, a multilayer perception–based ANN is simulated by using typical synaptic characteristics [e.g., G_max/G_min from the depression/potentiation (D/P) curve; fig. S17] for supervised learning function with the Modified National Institute of Standards and Technology (MNIST) handwriting image dataset. The ANN consists of 28 × 28 input neurons, 100 hidden neurons, and 10 output neurons, as shown in Fig. 5A. Mediated through the hidden layer, all the three layers are fully connected via synaptic weights. The 784 input neurons correspond to a 28 × 28 MNIST image with 28 × 28 pixels, and the 10 output neurons correspond to 10 Arabic numbers, from “0” to “9.” The ANN is inspired by the retina, in which billions of nerve cells crisscross to form a complex three-layered network (the first is the photoreceptor layer; the second is the layer of intermediate nerve cells, including bipolar/horizontal/amacrine cells; and the third is the ganglion cell layer; Fig. 5B) (50). The neural network in retina implements the visual information processing and transmission through neurons and synapses. When the photoreceptors are excited, the visual signals are mainly transmitted to ganglion cells through bipolar cells and then to the brain through the optical nerve fibers. According to the above biological process, a weight updating algorithm of backpropagation (BP) is used for the ANN simulation based on the dual-modal modulation behavior of the mechano-photonic artificial synapse: (i) synergetic mechanical and optical plasticity and (ii) tunable ΔPSC and persistent PSC by mechanical displacement. The output according to the input image data is evaluated by a current vector divided by the computed matrix result of the input neuron signal vector and the synaptic weight matrix (details in Materials and Methods). After transformation by activating and differentiating, the variations of all the synaptic weights are calculated to provide feedbacks in the ANN to update the synapse weights (i.e., the change of conductance, denoted as ΔG). The mapping input image (number “7”), initial state image, and two output images after supervised learning under different displacements are shown in Fig. 5C. The initial state conductance is randomly distributed before learning. After the supervised learning of image 7 under the synergetic simulation conditions with light illumination and mechanical displacements (D varies from 1 to 1.5 mm), the ΔG value of each pixel (corresponding to image 7) is obviously increased and the body feature of the number is highlighted. This result indicates the feasibility of dual-modal plasticization in emulating the supervised learning function for image recognition. The recognition accuracy by the ANN simulation is further evaluated with increased number of synapses and training samples. Figure 5D demonstrates the recognition accuracy of the input image under different displacement statuses after supervised learning with 360 training samples (randomly selected from the 60,000-image dataset). As D varies from 0.5 to 1.5 mm, the maximum recognition accuracy can be raised from 37 to 54% with the number of synapses at 6 × 10⁵ (training samples are only 360 in this case). The recognition accuracy can be further improved by increasing the training samples. As shown in Fig. 5E, when D is fixed at 1.5 mm (the number of synapses is 6 × 10⁵), the recognition accuracy can be readily increased from 54 to 90% when the number of training samples increases from 360 to 10,000, which is a significant improvement. When the number of training samples increases to 60,000, the maximum recognition accuracy can reach up to 92%. In general, the conductance margin G_max/G_min is considered as the primary factor to determine the recognition accuracy (9). The increased D leads to higher G_max/G_min (as shown in Fig. 4E), based on which we can use it to more readily distinguish the target pixel and achieve a more obvious simulated image feature (i.e., recognition accuracy). Besides, the nonlinearity of the D/P curve is the second important factor affecting the recognition accuracy. A lower nonlinearity means a relatively linear weight update process that enables the input neurons to retain more features of the image. The D/P curve in fig. S17 exhibits typical nonlinear characteristics of synaptic behaviors. Further alleviating the nonlinearity can effectively help increase the recognition accuracy. Other parameters (such as device-to-device variation and cycle-to-cycle variation) are the critical index of device stability (5). Improving the periodicity, stability, and repeatability of the synaptic devices is also highly desired in the simulation of ANN for image recognition. On the basis of above results and discussions, the promoted recognition accuracy demonstrates the effective and sophisticated synergies of mechanical and optical plasticity in the artificial synapse. Further improvement on the recognition accuracy can be expected by optimizing the G_max/G_min, linearity, symmetry, device stability, etc.

Fig. 5. Simulation of ANN for image recognition by the mechano-photonic artificial synapse.

(A) Schematic illustration of the simulated ANN with 784 input neurons, 100 hidden neurons, and 10 output neurons. (B) Schematic illustration of retinal neural network. (C) Examples of the mapping image obtained from ANN: input image, initial state image, output image at D = 1 mm, and output image at D = 1.5 mm. (D) Recognition accuracy of visual signal stimulation with different numbers of synapse. (E) Recognition accuracy of visual signal stimulation with different numbers of training samples.

DISCUSSION

In conclusion, we have successfully demonstrated a mechano-photonic artificial synapse based on Gr/MoS₂ heterostructure with synergistic multimodal synaptic plasticity. Triboelectric potential is used to drive the synaptic transistor and modulate the charge transfer/exchange in the heterostructure, thus readily facilitating the typical behaviors of photonic synapse, including postsynaptic photocurrents, persistent photoconductivity, and photosensitivity. Furthermore, the long-term memory and consecutive neural facilitation are also realized in the mechano-photonic artificial synapse under the synergistic effect of mechanical displacement and spatiotemporal light pulses. An ANN is further simulated to demonstrate the feasibility of mechanical plasticization on promoting the image recognition accuracy. The mechano-photonic artificial synapse offers an efficient method to achieving dual-modal synaptic plasticity, which is a universal plastic strategy readily extended to different types of materials and devices, e.g., memory devices based on interface-functionalized charge-trapping layer, floating-gate optoelectronic transistors, ionic/protonic-gated transistors, etc. The proposed mechano-photonic artificial synapse also paves the way to development of multifunctional and interactive neuromorphic devices.

By introducing proper sequential logic into the mechano-photonic artificial synapse, the sophisticated combination of mechanical and optical plasticization is appliable to implement the temporal mixed-modal (or cross-modal) updating on the synaptic weight. Further understanding on the spatiotemporal correlation of mechanical and optical modulation may push forward the mechanical motion–derived parallel and convolutional neuromorphic computation or in-memory computing. For the integrated TENG component, triboelectric potential is the basis of mechano-plasticization, which is proportional to the mechanical displacement (or applied force) and can be easily quantified. Accordingly, the mechano-photonic artificial synapse in device level is readily extended to multimodal sensory neuron, interactive photonic synapse, artificial visual-somatosensory associative system, etc. From the aspect of system integration level, the mechano-photonic artificial synapse paired with high-throughput semiconductor channels and scalable fabrication process can be potentially integrated into synaptic phototransistor arrays (or optoelectronic active matrix) with globe (or individual) triboelectric potential gating architectures. The on-chip-level synergistic mechanical and photonic plasticization will involve at least dual-modal (or triple-modal) signal processing and computation for neuromorphic visual sensory system and intelligent mechano-assisted image recognition array. The self-powered triboelectric potential gating and high-bandwidth optical programming are capable to push forward the mechano-photonic artificial synapse to go beyond von Neumann architecture with low power and versatility.

MATERIALS AND METHODS

Material transfer and patterning

First, few-layer MoS₂ was transferred onto a SiO₂/Si substrate by standard mechanical exfoliation method. Large-scale monolayer graphene was grown on copper foil (10 cm by 10 cm) by chemical vapor deposition and then transferred onto the MoS₂ flakes through polymethyl methacrylate (PMMA)–assisted wetting transfer method. The PMMA was then removed with acetone. Next, the graphene was patterned by standard electron-beam lithography (EBL) process, followed by electron-beam deposition of ~15-nm Al metal layer as the etching protective mask. Subsequently, graphene was etched into strip shape (3 μm by 20 μm) after removing the portion without protective Al mask by inductively coupled oxygen plasma. Then, the Al protective mask was etched off by 5% KOH (1 min) to achieve the Gr/MoS₂ heterostructure.

Device fabrication

To fabricate the optoelectronic transistor, Cr/Au source-drain electrodes (10 nm/40 nm) were defined on the prepared Gr/MoS₂ heterostructure by EBL and standard lift-off process assisted with PMMA sacrifice layer. The integrated TENG for the mechano-photonic artificial synapse was prepared by attaching PTFE film on the Cu electrode tape as one of the friction layers and a mobile Cu layer as the other friction layer supported by polyethylene terephthalate (PET). Then, the TENG was integrated to the transistor gate by connecting PTFE/Cu to the SiO₂/Si substrate. The mobile Cu/PET layer can move vertically, inducing triboelectric potential (by charge transfer) to replace the gate voltage of Gr/MoS₂ transistor.

Performance characterization

The scanning electron microscope image was observed using Nova NanoSEM 450. The Raman characteristics were measured using a HORIBA/LabRAM HR Evolution spectrograph. The wavelength of the excitation laser was 532 nm. The thickness of MoS₂ was confirmed by atomic force microscopy (Asylum Research MFP-3D). All the electrical characterizations were carried out using a semiconductor parameter analyzer (Agilent B1500A) in a probe station. The displacement of integrated TENG component was controlled by a linear motor. The output of TENGs was characterized using a Keithley 6514 electrometer. The light illumination was applied using commercial green LEDs. The optical power was measured using Thorlabs’ Optical Power Meter. For accurately acquiring the experiment data and avoiding undesired interference signals and errors, all the contact-separation processes of TENG component in the mechano-photonic synapse are controlled automatically by a numerical control linear motor customized for this work. To conduct the operation of contact and separation, one of the TENG friction layers (Cu/PTFE) attached to the Gr/MoS₂ transistor is fixed, while the other friction layer of Cu/PET is controlled by the linear motor to realize the conformal upward/downward action (i.e., contact/separation). The automatic control of the linear motor can also be readily programmed to realize consecutive action orders on demand.

ANN simulation

The multilayer perception–based ANN simulation includes three layers: input layer, hidden layer, and output layer. The input layer is composed of 784 input neurons connected to 28 × 28 pixels of MNIST input images. A total of 100 hidden neurons are set in the hidden layer. The output layer contains 10 output neurons corresponding to numbers from 0 to 9. The three layers are fully connected via synaptic weights, which are updated during the training process by using the BP algorithm. The core idea of the BP algorithm can be described as follows: The output errors can be passed back to the input layer through the hidden layer. By updating the synaptic weight of each unit layer by layer, the network output errors can be reduced to an acceptable level through the BP of errors.

When the MNIST image data reach the input layer, the obtained output current vector (I) is the matrix product of the input vector (light intensity V) and the synaptic weight matrix (W)

$$I={\displaystyle \sum _{i=1}^n}{w}_i{v}_i$$

The current vector (I) is then transformed to the hidden layer input vector (HI) via a sigmoid activation function [HI = f(I)]

$$\text{HI}=\frac{1}{1+e^{-I}}$$

The hidden layer output vector (HO) continues to propagate forward through the synaptic weights (W_HI) connected with the output neurons

$$\text{HO}={\displaystyle \sum _{i=1}^n}{w}_{\text{HI}i}{\text{HI}}_i$$

Then, the output vector (Y) can be obtained through the activation function [Y = f(HO)]

$$Y=\frac{1}{1+e^{-\text{HO}}}$$

Subsequently, through analyzing the difference between the output value (y) of the output vector and the label value (k) of the input image, ∆W is calculated according to the gradient descent method and all the synaptic weights in the ANN are updated through the BP algorithm. The categorical cross-entropy function is applied as the loss function, and prediction accuracy is used to evaluate the model training.

SUPPLEMENTARY MATERIALS

Supplementary material for this article is available at http://advances.sciencemag.org/cgi/content/full/7/12/eabd9117/DC1