Emerging photoelectric devices for neuromorphic vision applications: principles, developments, and outlooks

ABSTRACT

The traditional von Neumann architecture is gradually failing to meet the urgent need for highly parallel computing, high-efficiency, and ultra-low power consumption for the current explosion of data. Brain-inspired neuromorphic computing can break the inherent limitations of traditional computers. Neuromorphic devices are the key hardware units of neuromorphic chips to implement the intelligent computing. In recent years, the development of optogenetics and photosensitive materials has provided new avenues for the research of neuromorphic devices. The emerging optoelectronic neuromorphic devices have received a lot of attentions because they have shown great potential in the field of visual bionics. In this paper, we summarize the latest visual bionic applications of optoelectronic synaptic memristors and transistors based on different photosensitive materials. The basic principle of bio-vision formation is first introduced. Then the device structures and operating mechanisms of optoelectronic memristors and transistors are discussed. Most importantly, the recent progresses of optoelectronic synaptic devices based on various photosensitive materials in the fields of visual perception are described. Finally, the problems and challenges of optoelectronic neuromorphic devices are summarized, and the future development of visual bionics is also proposed.

GRAPHICAL ABSTRACT

1. Introduction

The traditional von Neumann architecture, which is the basis of modern computers, physically separates storage and computation. The data to be computed needs to be extracted from memory and transferred to a processing unit, and then the result of the processing is transferred back to the memory. This operation increases energy consumption, processing time, and the data transfer efficiency is limited [1]. Nowadays, the society is demanding how to process complex data efficiently and rapidly [2]. The von Loymann architecture is increasingly unable to support the high speed of information development. Therefore, a high-performance artificial brain-like computing system is needed to meet the increasing data volume and intelligence requirements. It is known that the human brain consists of about 10¹¹ neurons and about 10¹⁵ synapses [3]. It has the advantages of high efficiency, low power consumption and autonomous cognition. Based on this, brain-inspired neuromorphic computing systems with advantages such as high parallelism and ultra-low power consumption are considered to be the ideal way to achieve efficient artificial intelligence [4]. Synapses, which exist in the biological nervous system to connect and transmit signals, have both computing and storage capabilities [5]. They can help realize various biological functions with ultra-low power consumption and high efficiency [3,6]. For example, in the formation of biological vision, the retina converts the perceived light signals into electrical signals, and then the synapses rapidly transmit visual information layer by layer in the neural network. The transmission from the optic nerve to the visual cortex of the brain for storage and processing consumes very little energy to form biological visual perception [7–11]. Interestingly, it has been reported that approximately 80% of the external environment information obtained by humans is collected by the eyes [12–15]. Thus the visual perceptual system becomes an important way to acquire and learn information from the external environment [7]. Therefore, constructing artificial vision systems with efficient signal processing by retina-like optoelectronic synaptic devices [16] is a promising avenue [17–21]. In recent years, the retina-inspired devices have flourished in enabling many neuromorphic functions such as learning, memory, and pattern recognition in artificial intelligence [22–25]. Such neuromorphic devices can also be beneficial in meeting the increasing demand for edge computing in the era of big data [17–21].

Optoelectronic synaptic devices mainly rely on the optical signals or combined photoelectric signals to mimic synaptic functions [26]. Compared to electrical signals, optical signals have the advantages of low computational requirements, ultra-fast signal transmission speed, and high bandwidth [27,28]. Therefore, optoelectronic synaptic devices are not limited by the trade-off of bandwidth connection density of neuromorphic devices using pure electrical signals [29–33]. They help broaden the bandwidth, reduce the crosstalk, and realize the ultra-fast signal processing [32,34,35]. In addition, conventional neuromorphic visual imaging systems usually consist of photodetectors that convert optical signals into electrical signals, memory units that record visual information, and processing units that process information [36–40]. The physical separation of light perception, information storage, and processing functions leads to severe consumption of energy, space, and time. In contrast, the photoelectric synapse integrates light sensing and synaptic functions. It can not only respond to light stimuli but also realize real-time processing and temporary storage of optical information in parallel [41,42]. This working mode effectively eliminates unnecessary consumption and is very similar to the human visual system [13,43,44]. Currently, there is a growing interest to explore these optoelectronic synaptic devices [45,46]. Among them, the optoelectronic two-terminal memristors [47–49] and three-terminal transistors [50–53] are promising candidates for constructing the future artificial vision systems [54–57]. In this way, artificial visual intelligence with ultra-low power consumption and ultra-high computing speed can be realized.

Photoelectric synaptic memristors and transistors can tune synaptic weights by changing conductance through optical spike stimulation. It enables the realization of various synaptic visual functions, such as long-term plasticity (LTP), short-term plasticity (STP) [30], paired pulse facilitation/depression (PPF/PPD), and peak rate-dependent plasticity, etc. More importantly, the in-depth research on optoelectronic neuromorphic devices has promoted the development of electronic devices in the field of visual bionics [7]. Material science is indispensable for solving various problems faced by modern society. It has contributed greatly to the development of functional systems. Research on various functionalized materials is beneficial to promote the development of synaptic bionics, neuromorphic engineering and related artificial intelligence applications [58,59]. To date, a variety of different photoactive materials including MoS₂, graphene (Gr), carbon nanotubes, metal oxides, organics, halide perovskites, and ferroelectric materials have been investigated for optoelectronic synaptic memristors and transistors [23,28,30,32,56,57,60–67].

In this review, recent progress on optoelectronic memristors and transistors using various photosensitive materials for visual bionic applications is mainly summarized. The principles of biovision generation are introduced at first. Then, the device structures and working mechanisms of optoelectronic memristors and transistors are discussed. The recent advances of these two optoelectronic synapses using different photosensitive materials for realizing the visual perception functions are presented in Figure 1. In the end, the current issues, challenges, and future directions in this area are proposed.

Figure 1.

Schematic diagram for the visual devices from three perspectives in this review. Reproduced by permission from [68], copyright [2020, WILEY‐VCH Verlag GmbH & Co. KGaA, Weinheim]. Reproduced by permission from [69], copyright [2019, WILEY‐VCH Verlag GmbH & Co. KGaA, Weinheim]. Reproduced by permission from [70], copyright [2022, IEEE]. Reproduced by permission from [71], copyright [2022, Tsinghua University Press]. Reproduced by permission from [72], copyright [2021, Wiley‐VCH GmbH].

2. Visual generation principle

The formation of vision consists of two main parts: the retina perceives and preprocesses visual information, and the visual center of the cerebral cortex implements computing and memory for visual information. The two are connected through the optical nerve, as shown in Figure 2 [73]. The entire visual system is a neural network formed by neurons and synapses connections. A significant number of photoreceptor neurons, known as cone neurons and rod neurons, are laminarly distributed in the retina. They convert the incident light signals into neuroelectrical signals. These signals, which are pre-processed by the retina, are transmitted to the brain via the optical nerve [74,75]. Finally, they are further processed by the visual centers of the cerebral cortex to complete the recognition and memory functions. This forms images in the brain of what we see in the outside world. In the process of visual formation, the information transmission is mainly through the rapid release and reception of neurotransmitters in the synapses. It realizes the real-time imaging in the brain from external environment.

Figure 2.

Schematic diagram of the biological vision system. Reproduced by permission from [73], copyright [2022, American Chemical Society].

3. Basic principles of device structures and neuromorphic behaviors

Between two neurons, the action potential generated by the presynaptic neuron travels through the axon to the terminal presynaptic membrane, which then releases the neurotransmitters. These neurotransmitters have different mechanisms of action. They recognize and bind to receptors on the postsynaptic membrane, causing excitatory or inhibitory changes in the postsynaptic membrane potential. When the potential accumulation in the postsynaptic neuron reaches a threshold, the postsynaptic neuron generates an action potential. This triggers excitatory/inhibitory postsynaptic currents (EPSC and IPSC), which complete the signaling process between the two neurons [76,77]. According to the Hebbian learning rule [78], the strength of synaptic connections (synaptic weights) between two interconnected neurons changes after they experience the synchronized firing activities. This phenomenon, in which the efficiency of synaptic information transmission between neurons increases or decreases with the changes in their neural activities, is called as synaptic plasticity [79]. The realization of functions such as visual information learning and memory is carried out by modulating synaptic plasticity through visual neurons [80]. There are many forms of synaptic plasticity which can be divided into STP and LTP according to the length of memory [81,82]. STP refers to the fact that synaptic weights are maintained for only a few seconds to minutes after stimulation, followed by a gradual return to the initial state. Its synaptic weight changes include short-term potentiation (STP) and short-term depression (STD) [83]. PPF and PPD are two important synaptic functions that reflect STP. The PPF refers to the phenomenon that for two consecutive stimuli, the second stimulus triggers a stronger response than the first [30]. PPD is the opposite of it [82]. LTP means that synaptic weights can be maintained for hours to days or even longer after stimulation, and it is closely related to biological learning and memory functions. It is usually divided into long-term potentiation (LTP) and long-term depression (LTD) [84]. They represent excitatory and inhibitory changes in synaptic weight during repeated stimulation, respectively. STP can also be transformed into LTP under certain conditions. In addition, there are other synaptic plasticity behaviors, such as spiking-rate-dependent plasticity (SRDP), spiking-timing-dependent plasticity (STDP), associative learning, and learning-experience [85]. They are the basis of neural signal processing and neural computation at synapses.

Synaptic plasticity is the molecular basis of biological learning and memory. To mimic visual mechanisms, the photoelectric devices with biological synaptic functions can be used. They are the key to realizing the low-energy artificial visual systems with neuromorphic computing abilities. This part mainly introduces the optoelectronic response of two types of synaptic devices (optoelectronic transistors and memristors) and their various synaptic functions.

3.1. Optoelectronic synaptic transistors

Optoelectronic synaptic transistors have three electrodes: the source, gate and drain, respectively. The channel current of an optoelectronic synaptic transistor can be modulated by an electrical gate spike or an external light stimulus. In artificial synapses, the electrical spike applied to the gate or the light spike in the channel is considered as a presynaptic signal stimulus. The change in the conductivity of semiconductor channel is considered to be the change of synaptic weight. The current between the source and drain is used to mimic the postsynaptic current response. The electrical spike applied to the gate induces a transient channel current which is very similar to EPSC in biological synapses. When the active channel layer is illuminated by light, a photocurrent can be generated. The carrier density in channel can be effectively modulated by the both electrical gating and light stimulation. Thus, the device can mimic the synaptic behaviors under photoelectric stimuli [86,87].

3.2. Optoelectronic synaptic memristors

In 1971, Chua [88] proposed the theoretical concept of the memristor. It is a component with memory characteristics. Its working states are related to the operation history. There is no one-to-one correspondence between the output current and input signal under successive measurements. In 2001 and 2005, Terabe et al. [89,90] proposed an atomic switch, which is a two-terminal device. It uses a solid-state electrochemical reaction to control the formation and annihilation of a metallic atom bridge located between two electrodes. The conductance of the device is determined by the history of the previous input signal. More importantly, the structure shows two conductance states similar to biological synaptic memory behavior: one with spontaneous decay of the conductance level after weak signal input, similar to STP, and the other with long-lived stable conductance state, similar to LTP. In 2003, a nanoscale switch was proposed by Sakamoto et al. [91]. The switch consists of a copper sulfide semiconductor (Cu₂S) sandwiched between copper and metal electrodes. Due to the generation and annihilation of conducting paths in Cu₂S, the conductance of the switch can be switched repeatedly by applying positive or negative voltages to the metal electrode. And the generated conductance has a memory effect. However, the first memristor was experimentally implemented in a sandwich structure of Pt/TiO_2–x/Pt until 2008 [92].

As the fourth basic circuit element, the memristor has unique synaptic-like nonlinear transmission characteristics [93]. The existence of multiple resistive intermediate states and gradual transition from a high (low) resistance state to a low (high) resistance state are often exploited for constructing the artificial neural network. In general, the memristors are usually realized using a two-terminal structure which is similar to capacitors. Therefore, the presynaptic and postsynaptic terminals can be respectively mapped as two electrodes of the memristors. The electrical or optical spikes can be applied to the memristors for mimicking the stimulus signal of presynaptic neurons. The conductive states of memristors are used to represent the change of synaptic weight. By varying the amplitude, frequency, and duration of stimulus spikes, the corresponding change of conductive states can be obtained. This continuous change of conductive states corresponds to the plasticity of synapses.

3.3. Neuromorphic behaviors

At first, to mimic the visual neuromorphic system, artificial optoelectronic synapses need to realize the basic synaptic behaviors. Han et al. [94] proposed a light-stimulated synaptic transistor with an ultra-high PPF index based on the Gr/hexagonal boron nitride (h-BN)/perovskite quantum dot (QD) triple-layer heterostructure. The improved performance is attributed to the rate limiting effect of h-BN on the photogenerated carriers. The transistor exhibited typical biological synaptic functions under light stimulation. As shown in Figure 3(a), it is the typical EPSC behavior of biological synapses realized by the transistor triggered by a single light spike. A significant current rise can be detected, corresponding to the increase in postsynaptic currents induced by excitatory neurotransmitters in biological synapses. After removing the presynaptic light spike, a slow relaxation of the photocurrent can be observed. This is similar to the gradual release of neurotransmitters from the postsynaptic membrane in the biological synapse, resulting in a slow recovery of the postsynaptic current to its original state. The PPF behavior was demonstrated on the device by applying two continuous light spikes in Figure 3(b). Obviously, the second EPSC value is significantly higher than the first one. The successive optical spikes can write the signal information, causing a long-term potentiation (LTP) phenomenon in conductivity. In contrast, the successive electrical spikes can erase the written information, causing a long-term depression (LTD) of conductivity, as shown in Figure 3(c). In addition, Wu et al. [27] fabricated an optoelectronic synaptic thin-film transistor that was also able to mimic basic synaptic plasticity behaviors. They combined optoelectronic stimulation to allow the synaptic device to achieve PPD behavior by electrical pulse stimulation, as shown in Figure 3(d). Figure 3(e) shows that varying the frequency of the optical stimulus enables the emulation of the STM and LTM behaviors of the synapse. When the frequency is low, the synaptic device exhibits STM behavior and the current decays rapidly to the original state after removal of the stimulus. However, when the frequency is higher, the device exhibits LTM behavior with a longer current duration. This study demonstrates that the photoelectric combination has a promising future in optoelectronic applications. The phototransistor proposed by Islam et al. [95] can emulate the biological optical synapse. The necessary synaptic plasticity behaviors such as light-induced short-term and long-term potentiation, electrically driven LTD, PPF, and STDP can be achieved. Figure 3(f) shows the emulation of the synaptic STDP behavior. This research opens up a new possibility for machine vision techniques of artificial intelligence. Chen et al. [96] proposed a photoelectric synaptic transistor with the persistent photoconductivity (PPC) effect. They emulated a variety of biological synaptic plasticity behaviors, including EPSC, PPF, STDP and the transition from short-term to long-term plasticity. More importantly, the device implemented classical Pavlovian conditioning reflex behavior, i.e. associative learning [97]. This research provides a new avenue for high-speed, robust and adaptive processing of future optoelectronic neural systems. The MoS2 phototransistor proposed by Nur et al. [98] can mimic the functions of the retinal synapse by converting light spikes into electronic signals. The basic synaptic plasticity behaviors, such as PPF, is attributed to the persistent photoconductivity (PPC) effect of MoS₂. Since it is difficult to show photoinhibition, the PPD and LTD behaviors are also emulated by negative electrical stimulation.

Figure 3.

Schematic illustration of the basic synaptic plasticity functions achieved by photoelectric synaptic transistors. (a) EPSC excited by a single light spike. (b) PPF excited by a pair of light spikes. (c) LTP and LTD characteristic curves. Reproduced by permission from [94], copyright [2022, Wiley‐VCH GmbH] (d) the PPD behavior for device under electrical stimulus. (e) The STM and LTM behaviors realized by frequency‐varied photonic stimuli. Reproduced by permission from [27], copyright [2018, WILEY‐VCH Verlag GmbH & Co. KGaA, Weinheim]. (f) The emulation of STDP. Reproduced by permission from [95], copyright [2020, The Author(s)].

Optical control of memristors opens the way for new applications of optoelectronic neuromorphic computing. Gao et al. [99] proposed an ITO/Nb:SrTiO₃ Schottky junction-based optoelectronic memristor that can exhibit optical responses with neuromorphic characteristics throughout the entire visible spectrum. Figure 4(a) shows the current changes of the optoelectronic memristor under light stimulation. The device is sensitive to blue, green, and red light, as shown in Figure 4(b). However, it is more sensitive to shorter-wavelength light stimuli and thus has a larger current response. Figure 4(c) shows the current response triggered by a pair of light spikes. The results show that the second light spike induces a higher photo-response current, which well realizes the PPF behavior of the synapse. As shown in Figure 4(d,e), the STM, LTM, and its STM-to-LTM transition have been well demonstrated in this optoelectronic synapse by modulating the stimulus numbers and frequencies. In Figure 4(f), the memristor exhibits the typical ‘learning-experience’ behavior like the human brain. Such behavior is consistent with the fact that it usually takes less time for a person to relearn the information that has been previously learned but partially forgotten. More importantly, the relearning process can significantly enhance the stability of memory. In addition, Huang et al. [100] fabricated optoelectronic synaptic devices with photovoltaic effect. A series of important synaptic functions including EPSC, PPF, SRDP and dynamic filtering with zero power consumption were successfully emulated. The device can even perform arithmetic operations of addition, subtraction, multiplication and division. This research is important for realizing ultra-low-power neuromorphic computing.

Figure 4.

Schematic illustration of the basic synaptic plasticity functions achieved by photoelectric synaptic memristor. (a) The evolution of the current in the memristor under light stimulation. (b) The current responses of the memristor to the light with different colors. (c) The PPF characteristic of the memristor under a pair of light spikes. The transition from STM to LTM induced by increasing the (d) Spike number and (e) Frequency of light stimuli. (f) The ‘learning-experience’ behavior realized under light stimulation. Reproduced by permission from [99], copyright [2019, American Chemical Society].

Recent works show that both memristors and transistors can successfully mimic various basic synaptic functions using light spike stimuli. Based on these results, more advanced visual bionic functions can be realized. The fabrication of multifunctional and high-performance photo-sensitive synaptic devices is the key to realizing low-energy artificial vision systems. It is very beneficial to promote the development of machine vision in the field of artificial intelligence.

4. Devices using various optoelectronic materials and their visual bionic applications

The performance of optoelectronic devices depends largely on the design of optoelectronic materials [58,59]. Different photosensitive materials have different detection ranges of light. The material choice affects the response speed and conversion efficiency of optoelectronic devices. Therefore, the selection of suitable photosensitive materials is beneficial to improve the performance of optoelectronic devices. Photosensitive semiconductor materials can convert the light energy into electrical signals. The interaction with light is stronger than that of conductors and insulators, which helps to modulate device performance through light stimulation. Usually, when light is irradiated and then stopped, the concentration of photogenerated carriers produced by photosensitive semiconductor materials needs to be relaxed to reach the equilibrium state. This relaxation time is the optical response time of devices. In general, the response time of optoelectronic devices is mainly determined by the lifetime of the photogenerated carriers inside them. Especially when the photogenerated carriers are trapped, the slow release process will lead to a slow drop in the photocurrent of the devices, i.e. a persistent photoconductivity (PPC) phenomenon.

The retina system is responsible for perceiving and preprocessing visual information, while the visual center of the brain is responsible for recognizing what the eye perceives, as shown in Figure 5(a) [73]. Therefore, bionic research on vision consists of two main directions: fabrication of optoelectronic device arrays to perceive and memorize light information, and construction of artificial neural networks (ANN) to compute and recognize the perceived information. Based on this, this section will focus on the optoelectronic devices based on various types of photosensitive semiconductor materials, such as low-dimensional nanomaterials (including 0D, 1D, and 2D), metal oxides, organic materials, and heterojunction materials for applications in optical sensing systems and neuromorphic computing.

Figure 5.

(a) The visual perception system consists of the retinal system and neuromorphic computing system in the brain. Reproduced by permission from [73], copyright [2022, American Chemical Society]. (b) A pattern obtained in the array. Reproduced by permission from [68], copyright [2020, WILEYVCH Verlag GmbH & Co. KGaA, Weinheim]. (c) EPSC and IPSC triggered by using a vis‐light spike and a UV spike, respectively. (d) Low‐level image pre‐processing procedure of the memristor. (e) Images used in the neuromorphic vision system. Reproduced by permission from [101], copyright [2021, The Authors].

4.1. Zero-dimensional materials

Zero-dimensional materials refer to materials in which the electrons cannot move freely, such as quantum dots (QDs) and nanoparticles. Wang et al. [68] reported a memristor based on InP/ZnS QDs that can switch from non-volatile resistive-switching (RS) mode to volatile threshold-switching (TS) mode under UV light stimulation. The TS memory recovers from the low-resistance state (LRS) to the high-resistance state (HRS) due to the short-term diffusion dynamics of conducting filaments (CFs). However, both HRS and LRS are stable in the RS memory. This shows that the device can optically modulate memory patterns directly through energy band engineering. Emulation of various visual neuron behaviors by the device was also demonstrated. The device can achieve modulation of synaptic weights under UV light stimulation. Based on this point, the authors fabricated a reconfigurable memristor array with initial HRS as a visual information storage system. The pattern was programmed into the array using electrical and UV light stimulation, respectively, as shown in Figure 5(b).

Shan et al. [101] proposed an all-optical modulated memristor using the localized surface plasmon resonance phenomenon of the nanocomposites with Ag nanoparticles in the TiO₂ nanopore membrane. The memristor is capable of visual perception, low-level image pre-processing (contrast enhancement and noise reduction), and high-level image processing (image recognition). Figure 5(c) shows that visible light induces EPSC of the memristor, resulting in the long-term potentiation (LTP) of synaptic weight. In contrast, a UV spike induces IPSC, causing the long-term depression (LTD) of synaptic weight. This proves that the synaptic plasticity of the memristors can be modulated fully by light, which makes it possible to implement both visual sensing and low-level image preprocessing in a single device. They implemented the low-level image pre-processing functions on the memristor array as shown in Figure 5(d). Contrast enhancement and noise reduction of the image are achieved by visible light and UV light processing, respectively. Moreover, the STDP learning function of the device can be reversibly modulated by visible and UV light based on optical gating and electrically driven conductance changes. Therefore, the device can also perform the advanced image recognition function. They constructed a neuromorphic vision system combining visual perception, low-level image preprocessing, and high-level image processing functions using an 80 × 80 memristor array. The image with 10% noise was selected as the real image. As shown in Figure 5(e), the ideal image, contrast-enhanced image, and noise-reduced image can be obtained after processing by this neuromorphic vision system, respectively. They compared the recognition of real images and preprocessed images in the network simulator later. It was found that contrast enhancement and noise reduction made it possible to achieve an image recognition accuracy of 98% in a few learning cycles. This result can help to construct the efficient artificial vision systems.

4.2. One-dimensional materials

One-dimensional materials are those in which electrons are free to move in only one nanoscale direction (linear motion), such as nanowires and nanotubes. Recently, most of the research indicate that ZnO has great potential for nanoscale electronic and optoelectronic devices [102–107]. Therefore, the ZnO nanowires (NWs) with outstanding optoelectronic properties are very beneficial for promoting hardware-based bio-visual neuromorphic networks [102]. Shen et al. [108] reported an optoelectronic synaptic transistor based on ZnO NWs. The light-induced O₂ desorption and the PPC effect in ZnO NWs are responsible for the enhanced synaptic weight of the device. On the other hand, the device is stimulated by the electrical gate spike in the dark, resulting in synaptic depression due to the charge trapping effect in the gate medium. The reversible modulation of synaptic weight indicates that the device can perceive, memorize, and compute optical information in response to optoelectronic stimuli, which means that it can be used in the bionic of visual functions. As with the properties of biological synapses, longer and stronger stimuli typically induce larger EPSCs of the device. To achieve the inhibition of synaptic weights, they obtained IPSCs by applying electrical stimulation with different amplitudes and stimulation times at the gate. The synaptic weight can be extracted from the conductances with different potentiation and depression states as update parameters for training the artificial neural network (ANN) based on this transistor. Pattern recognition of handwritten digits can be obtained with a recognition rate of>90%. It can be seen that this work provides a new approach to the hardware implementation of neuromorphic vision systems. Besides, Sun et al. [69] reported a flexible dual-modulation photoelectrical synaptic transistor with ZnO NWs as the semiconductor layer, as shown in Figure 6(a). The EPSC of the device can be triggered based on the electric-double-layer (EDL) effect [111] from electrical stimulation and the photoconductive effect from UV stimulation. The visual memory array based on this device uses light spikes as presynaptic stimuli, as shown in Figure 6(b). The patterns similar to the input images can be clearly read from its visual memory array. More importantly, the memory retention level of this artificial synapse for the input light can be adjusted with different gate voltages, similar to the function in optic nerve system. This device provides a new idea to mimic visual memory and offers a promising strategy for the future electronic eye.

Figure 6.

(a) The schematic device structure of the ZnO NW synaptic transistor. (b) The visual memory array stimulated by optical presynaptic spike. Reproduced by permission from [69], copyright [2019, WILEY‐VCH Verlag GmbH & Co. KGaA, Weinheim]. (c) The current retention level is regulated by the intensity and number of light pulses. Reproduced by permission from [109], copyright [2020, The Authors]. (d) Long‐term plasticity curves for a sequence of potentiation and depression presynaptic pulses. Reproduced by permission from [110], copyright [2021, American Chemical Society].

Based on the excellent mobility and photosensitivity of the InGaO₃(ZnO)₃ superlattice NWs, Meng et al. [109] proposed an artificial synapse that achieves extremely ultra-low energy consumption. This is comparable to biological synapses and the synaptic electronics available today. Figure 6(c) shows the spike intensity-dependent plasticity and spike number-dependent plasticity of the device. Varying the intensity and number of light spikes can effectively modulate the charge retention level of the device, which is similar to the neurotransmitter release dynamics in biological synapses. The bionic vision system based on this artificial synapse was shown to perform high-performance photodetection, brain-like information processing, non-volatile charge retention, and image memory functions simultaneously. In the past decade, carbon nanotubes have been widely used in biological applications due to their high photosensitivity and biocompatibility [112]. Wan et al. [110] reported a multimodal sensory memory system based on flexible carbon nanotube transistors that can mimic the senses of vision, hearing, and touch. The sensors of the system convert the physical signals such as visual stimuli into presynaptic electrical spikes with important information. Reversible modulation of synaptic weight can be achieved by varying the polarity of the electrical spikes. Figure 6(d) shows the long-term plasticity profiles of the device under a sequence of positive and negative spike stimuli. The synaptic plasticity induced by electrical spikes has been systematically described. The system has bioreceptor-like sensing and synapse-like information processing capabilities that facilitate the construction of ambient interactive artificial intelligence.

4.3. Two-dimensional materials

Two-dimensional (2D) materials are those in which electrons can move freely on the nanoscale (1–100 nm) in only two dimensions (planar motion). Due to their unique atomic structure, mechanical flexibility, and excellent optoelectronic properties, 2D materials have great promise for the fabrication of new optoelectronic devices with excellent performance and reliability [113–117].

Islam et al. [118] reported the UV-vis sensitive phototransistors based on a monolayer MoS₂ channel. The transistor integrates infrared-sensitive PtTe₂ and Si as the gate electrode. Therefore, the optoelectronic synapse can sense, store and process optical data over a wide range of the electromagnetic spectrum. The device achieved basic synaptic plasticity behaviors under optical stimulation, as well as long-term inhibition under electrical drive. Moreover, they obtained different conductance states of light at multiple wavelengths from UV to IR. The different light response ranges show the promise of the synaptic device for multi-color pattern recognition. Artificial neural networks can be trained using the extracted photosynaptic weight parameters. As shown in Figure 7(a), the device structure of this photoelectric synapse is illustrated. Figures 7(b,c) explain the threshold voltage shift mechanisms of this transistor when irradiated with infrared and UV-visible light, respectively. Figure 7(d) shows a single-layer neural network based on the phototransistor for detecting and recognizing multicolor patterns. The activation values of the output neurons corresponding to single-wavelength patterns and mixed-wavelength patterns are shown in Figure 7(e), respectively. The results verify the feasibility of pattern recognition tasks for single-wavelength and mixed-wavelength handwritten digits. Moreover, Xie et al. [119] proposed a water-induced MoS₂ phototransistor with ultra-low operating voltage and ultra-high mobility. This photoelectric synapse has good linearity and symmetry of conductance in response to UV stimulation. Therefore, an artificial neural network (ANN) based on this device was constructed to mimic the recognition function of the visual system. Figure 7(f) shows the recognition accuracy for the handwritten digit dataset in Modified National Institute of Standards and Technology (MNIST). The light-dependent recognition accuracy is as high as 97.2%. This research opens up new paths for future applications of high-performance visual perception systems. In addition, a new compound, MoSSe, can be obtained by replacing the top S atom in the MoS₂ structure with a Se atom [122]. Based on this 2D material, Meng et al. [73] also demonstrated an optoelectronic artificial retinal perception device that integrates visual information perception, storage, and computing functions. The preprocessing, light adaptation, and pattern recognition functions were realized by optoelectronic modulation. These efficient and multifunctional optoelectronic devices have broad application prospects in the future development of artificial intelligence systems. The detection of the polarization property of light by the compound eyes of insects facilitates their visual image processing and navigation control. Based on the strongly anisotropic crystal structure of the direct bandgap semiconductor ReS₂ [123,124], Xie et al. [120] fabricated a phototransistor exhibiting excellent light detection capability and high polarization sensitivity. Figure 7(g) shows the artificial compound eye based on this ReS₂ neuromorphic device. Polarized light at 552 nm and 860 nm was used as the visual input. The average EPSC responses of these two polarized lights at different polarization angles are shown in Figure 7(h). The results indicate that the polarization-sensitive artificial compound eye has reconfigurable visual imaging behavior. Furthermore, the adaptive learning capability of the device is of great importance for polarization navigation. More importantly, the 3D visual polarization imaging function is successfully demonstrated for the first time. Figure 7(i) demonstrates the novel 3D polarization imaging principle proposed by the authors. This research provides new opportunities for future applications of polarization-aware systems in autonomous navigation and machine vision.

Figure 7.

(a) Schematic diagram of the multi-wavelength optoelectronic synapse structure. Energy band diagram of the phototransistor with (b) Infrared light and (c) UV-visible light. (d) Schematic diagram of the single-layer neural network. (e) Schematic diagram of activation values of the output neurons for single wavelength patterns and mixed wavelength patterns. Reproduced by permission from [118], copyright [2022, American Chemical Society]. (f) The recognition accuracy under different light intensities. Reproduced by permission from [119], copyright [2022, Royal Society of Chemistry]. (g) Schematic diagram of the artificial compound eye based on the ReS₂ phototransistor. (h) The EPSC values corresponding to different polarization angles at two optical wavelengths. (i) The three-dimensional polarization imaging principle diagram. Reproduced by permission from [120], copyright [2022, Royal Society of Chemistry]. (j) Optical microscope image of the memristor. (k) Typical I–V curve of the memristor. Reproduced by permission from [121], copyright [2016, Elsevier B.V].

Bismuth oxyiodide (BiOI) is often used for photo-detection because of its excellent photoelectric property [125]. Lei et al. [121] demonstrated an optoelectronic synaptic memristor based on 2D layered BiOI nanosheets. The device structure is shown in Figure 7(j). The bipolar I-V curve of the memristor is shown in Figure 7(k). The memristor has extremely low SET and RESET voltages to switch between LRS and HRS. It not only exhibit short-term and long-term plasticity under light induction, but also mimic the ‘learning experience’ behavior of the human brain. This means that the photonic synapse can emulate the perception, processing and memory functions of the visual system under light stimulation. It provides new materials and strategies for building low-power retina-like vision systems with information perception and processing functions.

From these works above it is evident that nanoscale optoelectronic synapses have great potential in the development of neuromorphic visual systems. Especially, the optoelectronic devices that integrate sensing, processing, memory, and identification functions can reduce the hardware redundancy and energy consumption, and finally increase the efficiency.

4.4. Metal-oxide semiconductor materials

Metal-oxide semiconductor materials have excellent optoelectronic properties and mature preparation processes, which are conducive to large-scale integration of devices. Therefore, the optoelectronic neuromorphic devices based on metal-oxide semiconductor materials have attracted the increasing interests around the world [30,57,96,126]. However, due to the band gap limitation, the oxide semiconductors tend to respond efficiently to ultraviolet light only, but poor or no response in the visible and infrared wavelengths, which limits the application scenarios of the devices [127]. Thus, the development of optoelectronic devices based on metal-oxide semiconductor materials with wide spectral range and high-efficiency response is one of the important directions for future development in the field of visual bionics.

Recently, optoelectronic synaptic transistors based on amorphous oxide semiconductors with intrinsic PPC effect have been extensively investigated due to their long-term retention properties, high electron mobility, and low-cost solution-based processability [127–129]. Among them, the amorphous IGZO (a-IGZO) is recognized as a high-quality channel material due to its high mobility, low processing temperature, and good industrial manufacturing compatibility [130–132]. The PPC effect of a-IGZO can be used to mimic neuromorphic memory behavior. Currently, it is widely used in optoelectronic synaptic transistors for implementing artificial vision systems [11].

To improve the efficiency of mobile edge devices in processing the real-time visual information, Duan et al. [133] proposed an a-IGZO-based optoelectronic synaptic transistor. Similar to retinal photoreceptor neurons, this photoelectric synapse can achieve basic synaptic plasticity behaviors under UV light stimulation. In particular, the photoelectric co-modulation enables reconfigurable excitatory and inhibitory behaviors of the synaptic device, which suggests that steady-state modulation of synaptic weight can be achieved. To demonstrate the ability of the IGZO transistor to work as the building block for edge artificial vision systems, they built a convolutional neural network (CNN) to perform the dataset recognition task. Using the conductance changes of this synaptic device under excitation and inhibition as the weight update rule, a high recognition accuracy of 95.99% was achieved for the Modified National Institute of Standards and Technology (MNIST) handwritten digit dataset. In addition, they also investigated the effect of datasets with different noises on the performance of the CNN. The recognition accuracy was kept within acceptable fidelity. It can be seen that this IGZO-based synaptic transistors have important application prospects in edge artificial vision systems with visual information processing capability.

The implementation of optoelectronic devices for mimicry of photoexcited corneal nociceptor (PCN) and central pain modulation can improve the adaptability of humanoid robots and artificial eyes in the real world. An a-IGZO-based optoelectronic transistor with chitosan/graphene oxide nanocomposite electrolyte as gate dielectric was reported by Ke et al. [134]. When the light intensity is below the noxious threshold level, the PCN can’t be activated. In contrast, when noxious light stimuli cause eye damage, the PCN enhances response sensitivity by lowering the threshold. The transistor successfully emulates the threshold characteristic of the PCN, as shown in Figure 8(a). The transistor can be ‘activated’ only when the EPSC response reaches/exceeds the threshold line as the intensity and stimulation time increase. Even for a light power that does not activate the transistor, the EPSC amplitude increases and exceeds the threshold line as the number of light spikes increases. This is similar to the activation of the PCN by continuous exposure to even mild noxious stimuli. Figure 8(b) shows the application of positive and negative gate biases to tune the central sensitization and analgesic effect. The results provide an important reference for giving visual intelligence to humanoid robots. In the same year, Feng et al. [137] also proposed a vertical coplanar multi-gate ITO phototransistor with an ultra-short channel for visual nociceptor. The device structure is shown in Figure 8(c). From Figure 8(d), it can be seen that the closer the gate is, the more pain-sensitive it is. In addition, they successfully achieved pain-sensitizing behavior by all-optical manipulation, which helps to avoid secondary damage to vision. This device provides a good opportunity for future smart electronic eyes with pain perception capability.

Figure 8.

(a) Emulation of the PCN threshold characteristic. (b) Emulation of the central sensitization and the analgesic effect. Reproduced by permission from [134], copyright [2021, Wiley‐VCH GmbH]. (c) Structure of the vertical coplanar multi-gate ITO phototransistor with ultrashort channel. (d) The light intensity at which the gate reaches the threshold current varies. Reproduced by permission from [137], copyright [2019, Elsevier Ltd]. (e) Transistor performance based on different substrates [13]. (f) Optical PPF and PPD behaviors. Reproduced by permission from [70], copyright [2022, IEEE]. (g) The electrical stimulation in the dark and illumination conditions, respectively. (h) The array triggered by electrical signals for different times. Reproduced by permission from [135], copyright [2022, American Chemical Society]. (i) Schematic diagram of information decoding on the ITO photoelectric synaptic transistor. Reproduced by permission from [136], copyright [2022, Royal Society of Chemistry].

There are still technical limitations in using a-IGZO-based synaptic devices to build artificial visual systems. Most of these devices can only use the high light intensities as presynaptic stimuli [11,30,138]. If the input light intensity is very low [139], the a-IGZO channel layer does not have sufficient conductivity to distinguish the various input lights. Therefore, the a-IGZO conductivity must be increased to differentiate the light intensities with ultra-low intensities. Yu et al. [13] sequentially deposited ultrathin SnO_x layers and p-type PEDOT:PSS layers on the back channel surface of a-IGZO phototransistors. In this way, the PN-junction was formed to improve the photosensitivity and modulate the synaptic weight. Figure 8(e) compares the performance of this photoelectronic synaptic transistor on different substrates. The transistor on transparent substrate can exhibit synaptic behaviors with a minimum optical intensity of 0.3 μW/cm² compared to it on SiO₂/Si substrate with a minimum optical power of 15 μW/cm². This study implies that the conductivity of a-IGZO-based optoelectronic devices can be further improved at low light intensities.

The inability of fully optical-driving neuromorphic devices to achieve bidirectional regulation of conductance limits their further development for visual perception applications. Gu et al. [70] proposed an IGZO phototransistor that enables optical enhancement and suppression for building a fully optical-driving ANN. As shown in Figure 8(f), the shift from optical PPD to optical PPF can be achieved by adjusting the magnitude of the gate bias. Finally, they constructed a fully optical-driving ANN with high accuracy to emulate the visual perception function. This research opens up a new avenue for future applications such as artificial vision systems and intelligent bionic robots.

Kim et al. [140] prepared a memristor with p+-Si as the bottom electrode and Pd as the top electrode using a-IGZO as the switching layer material. They evaluated the visual recognition capability of this a-IGZO memristor in a neural network for MNIST image datasets. Zhu et al. [141] proposed an a-IGZO-based optoelectronic thin-film transistor that can enhance the image quality of the pixels and the real-time processing ability of input visual information through voltage co-modulation approach. The results are important for the development of optoelectronic neuromorphic devices with configurable dynamic functions.

Adaptation of the human eye is the process by which the visual system adjusts its threshold for the perception of light according to the brightness of external stimuli [17,142]. This function allows the human eye to avoid the damaging effects of light in a changing environment. Jin et al. [135] observed negative photoconductivity for the first time in the photoelectric ion-gel-gated In₂O₃ transistor. They designed a transistor array for the artificial visual perception system that can be adaptive to environmental light. As shown in Figure 8(g), the electrical stimulation signals were applied to the array in the dark and light conditions, respectively. The difference in currents before and after stimulation is defined as ΔI_n. Figure 8(h) shows the ΔI_n values triggered by the electrical signals for different stimulus times in the dark and light conditions, respectively. The upper one of Figure 8(h) is used to demonstrate the visual dark adaptation process. As the stimulus number increases, the ΔI_n values of the pixels gradually fall all the way to the red threshold region. This indicates that the device completes the self-adaptation process in the dark. The visual light adaptation process of the device is shown in the lower one of Figure 8(h). The ΔI_n values are at a higher level in the blue threshold region. The experimental results show that the transistor array can adjust the threshold range to achieve adaptive behavior to dark and light, similar to the human eye. This study provides a new way to build an artificial visual perception system with environmental adaption.

The realization of information decoding and processing on optoelectronic devices is important for expanding their applications in artificial neural networks and machine vision enhancement, etc. By using the inorganic solid-state mesoporous silica coating (MSC) as the electrolyte, Ren et al. [136] fabricated a MSC-gated indium-tin oxide (ITO) transistor. The schematic diagram of information decoding is shown in Figure 8(i). The authors decode the signal by co-coupling the optical signal when a ‘0’ or ‘1’ electrical stimulus is applied. The obtained EPSC responses are completely different and can be easily distinguished. The results show that synchronized optical stimulation can improve the recognition accuracy of the input signal. This optical decoding has applications in areas such as bionic vision enhancement. It can be seen from the present studies that metal oxide-based optoelectronic synaptic devices have a promising future for building artificial vision systems.

4.5. Organic semiconductor materials

Organic semiconductors (OSCs), including organic small molecules and organic polymers, have great potential in the field of optoelectronics [143]. As one of the ideal photosensitive conductive layers, OSCs have obvious advantages such as mechanical flexibility, solution processability, and lightweight. Large-area flexible optoelectronic devices based on OSCs can now be prepared by simple and cost-effective methods on a variety of substrates. The spectral sensitivity of OSCs can be made panchromatic or intentionally tuned to a specific wavelength from UV-visible to NIR spectral region. OSCs have inherently good optical absorbance and generation rates [144]. This facilitates the design of broadband or narrowband optoelectronic devices with good photosensitivity. Currently, OSCs have been widely used as photoactive materials for photonic synaptic devices in the fields of imaging, optical communication, or biomedical sensing [143].

The hemispherical structure of the human retina facilitates the high-sensitivity image acquisition, visual information preprocessing, and transmission. Therefore, the implementation of hemispherical high-sensitivity neuromorphic imaging system that mimics the human eye is necessary in artificial intelligence. However, most of the current high-sensitivity optoelectronic synaptic devices are made of inorganic materials that must be constructed on hard, flat glass or silicon substrates [21,43,62,109,145,146]. This makes it impossible to form retina-like three-dimensional hemispherical structures. On the other hand, organic optoelectronic synaptic devices are considered to be the most promising candidates for the realization of hemispherical neuromorphic imaging systems due to their inherent mechanical flexibility, low-temperature processes, and biocompatibility [14,147–150]. However, organic devices generally have poorer photosensitivity under dim conditions compared to inorganic devices. This is attributed to the fact that the exciton binding energies in organic materials [151] are typically higher than those in inorganic materials [152–155], resulting in less efficient dissociation of light-generated excitons [156]. The low photosensitivity inevitably leads to poorer light detection accuracy and lower noise immunity, making the recognition of target images weak. To overcome this drawback, Zhang et al. [157] used orange peel pectin (OPP) as the dielectric layer for dinaphtho[2,3-b:2′,3′-f]thieno[3,2-b]thiophene (DNTT)-based photoelectric transistors. A hemispherical neuromorphic imaging system that can achieve ultra-high photosensitivity under dim light conditions is proposed. The large EDL capacitance due to the high proton conductivity of OPP improves the efficiency of exciton dissociation in the organic semiconductor layer. Similar to the synapses in the human eye, the input light is considered as a presynaptic spike. The synaptic weight of the synaptic transistor is modulated by light and gate voltage. The detection, storage and processing of the input light information can be achieved simultaneously. This behavior is very similar to biological synapses. As the rod cells in the retina are highly sensitive to light stimuli, the high photosensitivity of the device improves light detection accuracy and interference resistance. These advantages facilitate the application of this optoelectronic synapse in high-quality imaging recognition of target images. A transistor array was demonstrated for neuromorphic visual imaging system, as shown in Figure 9(a). Similar to the visual forgetting process, the triggered current gradually decreases after the removal of light. Moreover, the device array is highly noise resistant and can highlight the main information of the image. This is similar to the image pre-processing function of the human retina. A concave hemisphere imaging system needs to be constructed to mimic the retina. Due to the ultrathin nature of the substrate-free and inherent flexibility of organic materials, the array is very similar to the human retina. This ensures that the conformal and stable contact with the curved surface can be established. To demonstrate that the performance is not affected by mechanical deformation, the DNTT-based ultra-flexible transistor array is mounted on glass spheres with different bending radii, as shown in Figure 9(b). As the bending radius decreases, there is no significant performance degradation in the imaging characteristic. To realistically emulate the real-life scenes seen by the human eye, four arrow images with different directions were designed and projected on a concave hemisphere transistor array as shown in Figure 9(c). The arrows can be displayed separately without aberrations and vignetting even in the edge region of the visual field. This study opens a new path for building a visual imaging system like the human retina.

Figure 9.

(a) Schematic diagram of a simplified neuromorphic imaging system based on the OPP photoelectric synaptic transistor array. (b) The array performance based on different-radii spherical surfaces. (c) Four directions are identified on the concave hemisphere device array. Reproduced by permission from [157], copyright [2022, Elsevier Ltd]. (d) The self-powered synapse uses high-pass filtering to achieve image sharpening. Reproduced by permission from [158], copyright [2022, The Authors. Advanced Science published by Wiley‐VCH GmbH]. (e) The EPSC obtained after the light stimulation. Reproduced by permission from [159], copyright [2019, The Author(s)]. (f) Emulation of the visual noise reduction function by the array. Reproduced by permission from [71], copyright [2022, Tsinghua University Press].

To reduce the energy consumption of optoelectrical synapses, Zou et al. [160] fabricated a zero-power optoelectrical synapse based on the ultrathin organic dioctylbenzothien-obenzothiophene (C₈-BTBT) film in 2021. The organic optoelectronic synapse can be self-driven using the photovoltaic effect induced by asymmetric electrode geometry contact [161,162]. The UV-light modulated synaptic behaviors were successfully achieved without external bias. They used the low-contrast raw image to demonstrate the self-driven image-processing capability of the device. After setting the cutoff frequency of the high-pass filter, the final image with significant edge enhancement is obtained. In 2022, the group published another similar research work. The proposed retina-inspired self-powered organic optoelectronic synapse is capable of the similar function, as shown in Figure 9(d) [158]. These two studies realized the image sharpening function of the self-driven light-modulated synaptic device, which provides a prospect for the development of retina-like bionic systems with image preprocessing.

In practical applications, the devices used in artificial vision perception systems generally face the complex device integration problems. Deng et al. [159] selected the organic molecular crystal 5,11-bis(triethylsilylethynyl) anthradithiophene (Dif-TES-ADT) as the photoactive layer to demonstrate a novel organic photo-synaptic device that simultaneously provides photo-sensing and synaptic functions. They constructed an array based on this device to emulate the function of the artificial image perception system. Figure 9(e) shows the EPSC obtained after the array was subjected to light stimulation. The integration of the array is simple, and its emulation of visual recognition and optical image functions is successfully verified. In recent years, organic light-emitting transistors have received much attention as optoelectronic elements for fabricating the new generation of active matrix displays and vision sensors. A long-afterglow [163] organic light-emitting transistor was reported by Chen et al. [164]. It uses the light modulation effect in the IGZO channel layer to power the persistent electroluminescence of the light-emitting material 9,10-bis(4-(9 H-carbazol-9-yl)-2,6-dimethylphenyl)-9,10-diboraanthracene (CzDBA). The array based on this organic light-emitting transistor can be used as a visual UV sensor. It is capable of long-lifetime green light emission in the UV radiation region, demonstrating its great potential as a visual UV microsensor. The human visual system can reduce the noise signal to efficiently select the target information from a large amount of complex information. Hua et al. [71] prepared organic semiconductor transistors using 1,4-bis ((5′-hexyl-2,2’-bithiophen-5-yl) ethyl) benzene (HTEB) monolayer molecular crystals. The light spikes of different numbers, durations and intensities are considered as presynaptic stimuli, while the resulting EPSC is considered as the memory level. This synaptic device has good long-term memory for light spike information, and its generated EPSC can be retained for 109s. In Figure 9(f), an array based on this kind of transistor is implemented to emulate the visual noise reduction function. Different light spike intensities represent signals of different importance. The intensity of light spikes for noisy information is less than that of useful information. The currents of the noise pixels gradually disappeared as the time increased. This result indicates that the information processing process is very similar to that of human vision.

Organic optoelectronic synaptic devices offer outstanding properties, high flexibility, and strong ability to solve real-world problems. New devices using organic semiconductors may open up greater application prospects in areas such as visual perception systems and facilitate the integration of these devices into next-generation flexible and scalable electronics.

4.6. Heterojunction of materials

Currently, the optoelectronic heterojunction devices are a hot research topic for realizing artificial vision bionics. The structure combines the superior physical properties of two different elements or different compositions of semiconductor materials. It is considered as one of the effective ways to realize high-performance optoelectronic devices [165]. The development of artificial optical perception systems is regarded as a key step for the realization of neuromorphic computing for machine vision [43,69,74]. Therefore, the optoelectronic synaptic devices based on heterojunction structures with information perception, processing, and memory functions facilitate the implementation of artificial visual perception at the hardware level. In this section, the recent research advances on optoelectronic synaptic transistors and memristors based on halide perovskite-containing heterojunctions, ferroelectric-containing heterojunctions, and other types of heterojunctions for artificial visual perception are presented. The current status of neuromorphic visual perception will be summarized at the end.

4.6.1. Heterojunctions with halide perovskites

Perovskite materials possess excellent optical and charge transport properties [85,127,144]. They can be classified as zero-dimensional, one-dimensional, two-dimensional, etc. in terms of dimensionality, or organic and inorganic according to the material composition. Especially, they have been focused on the research of photoactive materials in the past decade due to their high carrier mobility [166–168], tunable bandgap [169,170], and solution fabrication processes [171–174]. It is considered as one of the most promising candidates for the fabrication of optoelectronic devices with low-cost and high-performance photovoltaics. Currently, rapid progresses have been made in applying halide perovskites to heterojunction phototransistors [35,56,145,175–177]. This subsection will focus on the applications of heterojunction phototransistors with halide perovskite for neuromorphic visual perception.

The use of heterojunctions in transistors is an effective way to enhance light absorption and improve synaptic performance [11,42,178–180]. The heterojunction channels usually consist of a photo-absorption active layer and a high-mobility charge transport layer. It can provide excellent optical responses and photocarrier separation efficiency. Moreover, heterojunction phototransistors using halide perovskites have the advantage of simple solution-processed methods. It has great potential for the development of optoelectronic devices with large area and low cost. For example, Wang et al. [35] reported light-stimulated synaptic transistors based on inorganic halide perovskite quantum dots and organic semiconductor heterojunctions. The perovskite nanocrystals (NCs) or quantum dots (QDs) have high quantum yields, efficient photon absorption, and excellent optical tunability [181–183]. Blending the two materials improves the separation efficiency of the photoexcited charges and causes a delayed decay of the photocurrent. The transistor can be triggered by visible light. The device can be triggered by visible light. It is capable of responding to light signals in a synaptic-like manner and exhibits adjustable synaptic plasticity. The formation and memory level of vision usually depends on the frequency, intensity and duration of the incident light. This synaptic device can effectively change the channel conductance by the number, intensity and duration of light spikes. Thus, the visual formation behavior can be emulated in this device. Increasing the photoresponsivity and photodetectivity helps to reduce the light energy consumption of phototransistors for one synaptic event to approach the energy consumption level of a biological synapse. In 2020, Yin et al. [184] proposed phototransistors based on organolead halide perovskite (MAPbI₃) and silicon nanomembrane heterojunctions. The phototransistor can be stimulated by a low optical power density of 1 μW/cm². The energy consumption is ~1 pJ when operating at a drain voltage of 0.01 V. Basic synaptic functions can be successfully emulated with optical stimulation. This work contributes to the integration of Silicon-compatible perovskite-based heterojunction devices for future large-scale high-performance neuromorphic devices. In addition, they emulated the learning and forgetting processes of vision under different emotional states, as shown in Figure 10(a). The EPSC of the device can be easily modulated by the duration and intensity of the incident light spikes. More importantly, the device can trigger different EPSCs under different gate voltage modulations stimulated by the same light spikes. Therefore, it is defined that the human emotions are different under different V_G. Emotions can affect the learning and forgetting process. Obviously, the more negative (positive) the emotion is, the faster (slower) the visual information is forgotten. In 2021, Liu et al. [185] reported phototransistors based on all-inorganic halide perovskite CsPbBr₃ and organic semiconductor hybrid films. The phototransistor can implement distinct synaptic functions with 1.6 nW light signals that are weaker than most previous perovskite-based hybrid synaptic phototransistors. The array based on the phototransistors can improve the recognition accuracy of images, as shown in Figure 10(b). As the light wavelength and light intensity change, the contrast of image grayscale is enhanced, which helps to promote the accuracy of image recognition. This study is very suitable for imaging processing of the human visual system. In 2022, Cao et al. [186] reported an ultralow light power consuming CsPbI₂Br perovskite nanocrystal/IGZO heterojunction phototransistor, as shown in Figure 10(c). The designed heterostructure well combines the advantages of the high mobility channel material of IGZO and the highly efficient photoactive layer of perovskite nanocrystals. It can exhibit high photoresponsivity and photodetectivity at a very low detectable light power density of 0.52 nW cm⁻². Thanks to its excellent photodetectivity capability, the power consumption of the device can be reduced to <2.6 pJ for a single light stimulus. This heterojunction phototransistor has the advantages of low optical power consumption, process compatibility, and low cost, opening up new opportunities for the fabrication of high-performance photonic synapses. Currently, the key to developing intelligent vision systems is the improvement of phototransistors to distinguish and recognize the color of light. In particular, the development of photonic synapses with near-infrared (NIR) wavelength selectivity for applications in night vision and robotic vision perception is still to be realized. However, single-carrier (hole-only or electron-only) phototransistors are unable to distinguish colors from visible light to NIR light. Huang et al. [187] proposed an ambipolar phototransistor based on 2D perovskite/organic heterojunction (PEA₂SnI₄/Y6) by a complete solution method. The combination of Y6 and PEA₂SnI₄ broadens the absorption spectrum, making the phototransistor highly responsive to both visible and NIR light. As shown in Figure 10(d), it can distinguish colors according to the type and magnitude of the postsynaptic current (PSC). It is demonstrated that this phototransistor has the ability to detect and distinguish colors of multi-wavelength light. Finally, the authors built a flexible array based on this phototransistor to sense and distinguish images formed by different colors of light. Different colors of light correspond to different ΔPSC values. The process of sensing and recognizing color letters is shown in Figure 10(e). This demonstrates that the array can memorize and distinguish colors like the retina. Moreover, the lead-free and non-toxic properties of the 2D perovskite of PEA₂SnI₄ are very friendly to the environment [188]. This work has great potential for low-cost and large-scale integration of optoelectronic devices for artificial intelligent vision systems. The aim for visual bionics is to eventually apply artificial intelligent vision systems to the real world. Gong et al. [189] proposed a photoelectronic transistor based on CsPbBr₃ quantum dot and black phosphorus nanosheet heterojunction to realize the mimicry of visual nociceptive receptors. The transistor can emulate the threshold, no adaption, relaxation, allodynia, and hyperalgesia characteristics of photonic nociceptive receptors under UV light. Its visual nociceptive perception capability is very beneficial to further promote and improve the implementation of artificial intelligent vision systems.

Figure 10.

(a) Letter recognition with positive, neutral, and negative mood states. Reproduced by permission from [184], copyright [2020, American Chemical Society]. (b) Dynamic current changes of the array manipulated by light wavelength (upper) and intensity (bottom). Reproduced by permission from [185], copyright [2021, American Chemical Society]. (c) Schematic diagram of the phototransistor structure. Reproduced by permission from [186], copyright [2021, Wiley‐VCH GmbH]. (d) The current responses of the PEA₂SnI₄/Y6 heterojunction phototransistor under red, green, blue, and NIR light illumination. (e) The process of perception and recognition of different color letters. Reproduced by permission from [187], copyright [2021, Wiley‐VCH GmbH].

Optoelectronic synaptic devices with the ability to efficiently perceive, process, and memorize visual information are playing an increasingly important role in the development of neuromorphic computing systems. At present, the research of synaptic transistors with perovskite-based heterojunctions is mainly focused on digital pattern recognition for visual neuromorphic computing. For example, Zhang et al. [190] realized phototransistors based on CsPbBr₃ QD and organic semiconductor (Poly[2,5-(2-octyldodecyl)-3,6-diketopyrrolopyrrole-alt-5,5-(2,5-di(thien-2-yl)thieno [3,2-b]thiophene)], DPP-DTT) heterojunctions. They trained the ANN with the handwritten digit dataset in MNIST to accomplish the recognition task. This experiment demonstrates the feasibility of the fabricated device for visual target recognition. Besides, Park et al. [191] implemented optical synaptic transistors based on organic-inorganic halide perovskites (OIHP) and indium-zinc-tin oxide (IZTO) heterojunctions with a large dynamic range of synaptic weight update states. This facilitates well to improve the recognition accuracy of ANNs. They constructed a multilayer perceptual neural network, as shown in Figure 11(a). The training and recognition tasks were performed using the MNIST handwritten digit dataset. This study confirms the feasibility of this photoelectric synapse for neural networks and provides a promising direction for the development of neuromorphic vision systems. Han et al. [94] proposed a light-stimulated synaptic transistor with an ultra-high PPF index by introducing a carrier modulation layer of h-BN into Gr/CsPbBr₃ QDs. Similarly, an ANN was constructed to perform the pattern recognition task. The recognition accuracy of handwritten digit dataset from the MNIST is about 91.5%. Li et al. [192] proposed a flexible optoelectronic synaptic transistor based on lead-free Cs₃Bi₂I₉ NCs and organic semiconductor DPPDTT (poly[2,5(2-octyldodecyl)- 3,6-diketopyrrolopyrrole-alt-5,5-(2,5-di(thien-2-yl)thieno[3,2-b]thiophene)]). Figure 11(b) shows the device performance for the flexible transistor array. This shows that the array has good flexibility stability. They designed an ANN to recognize handwritten digits in MNIST. To evaluate the fault tolerance of this ANN, different noise levels are implemented for the MNIST database, as shown in Figure 11(c). The ANN has good recognition accuracy in different states and different noise levels. The flexible device fabricated in this study has good robustness and strong fault tolerance under the different bending states and noises. High-accuracy image recognition is successfully achieved, further demonstrating its potential in the field of neuromorphic visual computing. In addition to the above researches on digital pattern recognition, Liu et al. [193] proposed a phototransistor based on a hybrid dimensional organohalide perovskite/metal oxide heterojunction. The underlying structure is shown in Figure 11(d). Not only the visual adaptation behavior was achieved by gate modulation, they developed a face recognition gated retinal neuromorphic computing system based on this device, as shown in Figure 11(e). It is able to accurately recognize more than 90% of the faces. This validates the potential of the device in retinal neuromorphic computing applications.

Figure 11.

(a) Schematic illustration of the training and recognition process of the ANN based on organic-inorganic halide perovskite/IZTO optoelectronic heterojunction transistor. Reproduced by permission from [191], copyright [2021, Royal Society of Chemistry]. (b) The image mapping in different states. (c) Handwritten digits with different noise ratios for ANN based on Cs₃Bi₂I₉/DPPDTT optoelectronic heterojunction transistor. Reproduced by permission from [192], copyright [2022, The Authors]. (d) The mixed-dimensional perovskite/metal-oxide structure on the electrolyte. (e) Diagram of the facial recognition access control system. Reproduced by permission from [193], copyright [2022, Elsevier Ltd]. (f) Characterization of the photovoltaic effect of optoelectronic transistor with MoTe₂/α-In₂Se₃ p-n junction. Reproduced by permission from [194], copyright [2021, Tsinghua University Press and Springer-Verlag GmbH Germany, part of Springer Nature].

At present, the main research directions of perovskite heterojunction transistors are focused on enhancing the optical response, broadening the absorption spectrum, and using simple solution methods to achieve large-area, low-cost integration. However, most of the halide perovskites used in the research are lead-containing materials, and their toxicity and environmental instability limit their further application. Therefore, the current limitations remain in terms of complex device structures, stringent processing conditions, and incompatibility with semiconductor fabrication technologies.

4.6.2. Heterojunctions with ferroelectric materials

Ferroelectric materials can be classified as dielectric materials and semiconductors. In 2020, Luo et al. [195] reported a photoelectric synaptic transistor based on the ferroelectric material as dielectric. The transistor can be programmed and switched by external photoelectricity for polarization. This study has demonstrated that the polarization states of the ferroelectric material have good stability. It is concluded that synaptic devices based on ferroelectric materials usually have a series of advantages such as high stability, large switching ratios, small changes in weight updates, and good device reproducibility [196]. Therefore, the fabrication of ferroelectric heterojunction devices has great application prospects for neuromorphic computing.

Emerging ferroelectric semiconductors such as α-In₂Se₃ can integrate ferroelectric and semiconductor properties into a single material. It provides a powerful platform for constructing the ferroelectric devices with multiple functions. Moreover, ferroelectric materials also attract much attention because their polarization can produce stable photovoltaic effects [197]. For example, Wang et al. [194] demonstrated a driving-voltage-free optoelectronic transistor by exploiting the nonvolatile reconfigurable photovoltaic effect of MoTe₂/α-In₂Se₃ ferroelectric p-n junction. The transistor is characterized by the photovoltaic effect under illumination, as shown in Figure 11(f). On this basis, they used the transistor to mimic the visual function of retinal synapses. The basic synaptic plasticity and learning memory rules were achieved in a driving-voltage-free mode. The potential of ferroelectric p – n junctions in constructing low-power passive optoelectronic synaptic devices for neuromorphic machine vision is demonstrated.

To realize various functions of the biological visual neuromorphic system at the single device level, Guo et al. [198] proposed a multifunctional synaptic device based on ferroelectric α-In₂Se₃/GaSe van der Waals heterojunction. The device can mimic the basic behaviors of retinal synapses. In addition, the Pavlovian emulation and wavelength selectivity mean that the device is able to recognize color and process complex multi-input optoelectronic signals. More importantly, the device is also able to emulate the logic and memory functions of the brain’s visual cortex. Logic operations and image recognition with higher accuracy can be achieved in the ANN. This study shows that multifunctional devices can be realized based on ferroelectric van der Waals heterojunctions. It has great potential for efficient processing of complex visual information and simplifying the design of artificial vision systems.

Due to the entanglement of ferroelectric and semiconducting properties of ferroelectric semiconductor materials, the signal from polarization may be annihilated in the strong background signal of the semiconducting property. Therefore, a pulse measurement is recommended instead of the widely used continuous voltage sweep mode. This can effectively highlight the main role of ferroelectric polarization. There are few studies on ferroelectric materials using heterojunctions to fabricate optoelectronic devices. However, their excellent properties prove to be promising in building artificial visual perception systems. Therefore, the role of ferroelectric materials should not be underestimated.

4.6.3. Heterojunctions built by other materials

Most of the research in optoelectronic heterojunction synapses based on other materials that do not contain halide perovskites and ferroelectric materials has focused on visual bionic applications such as visual perception, visual memory, color recognition, and image recognition.

Xu et al. [199] fabricated all-oxide optoelectronic heterojunction memristors with adaptive recognition of visible light to mimic the optical functions of intelligent biological photoreceptors and corneal nociceptors. This is a substantial step in the development of visual cognitive systems for application to robots. Moreover, Xie et al. [72] proposed 0D-quantum-dots/2D-MoS₂ mixed-dimensional heterojunction transistor with visual adaptation. This work has good implications for the future development of artificial vision systems with adaptive capabilities. The development of infrared artificial vision system devices can enhance their potential for applications such as biomedical imaging and robotics engineering. However, the emulation of infrared vision adaptation is a challenge. Recently, the organic/inorganic infrared optoelectronic heterojunction transistor proposed by Huang et al. [200] has successfully emulated adaptation to ambient light. The device array can achieve image recognition of infrared light under ambient light of different luminance. These results indicate that this infrared optoelectronic heterojunction transistor has good prospects for self-adaptive bionic applications. In visual image perception and memory bionics, Hao et al. [201] constructed an array based on optical synaptic transistors with bismuth triiodide (BiI₃)/single-walled carbon nanotube heterojunctions. The array is excited by varying the number of light spikes, as shown in Figure 12(a). Wang et al. [202] developed an array using the proposed TiN_xO_2-x/MoS₂ optoelectronic heterojunction memristor. The array can emulate visual perception and improve the contrast of images. In Figure 12(b), different intensities and durations of light on the array yield different current responses. In addition, the visual memory can be enhanced by increasing the number of views. Figure 12(c) shows the current responses by using different number of views. The result indicates that the more the number of views, the more clearly the image can be remembered. This study effectively emulates superior visual perception and visual memory performance. Zhao et al. [203] built an array using the zinc oxide/poly(3-hexylthiophene) (ZnO/P3HT) optoelectronic heterojunction memristor. Figure 12(d) shows the current attenuation of input images obtained by varying the wavelength, intensity, and frequency of the light. The results show that the array has image recognition and storage capability. Moreover, some works have used the filtering ability of heterojunction devices to build arrays to preprocess images which are called image sharpening [158,204,205]. In this way, it realizes the ability of human retina to highlight the main features of images. This allows selective extraction of useful data from a large amount of raw data. Not only the recognition accuracy is improved, but also the workload of the back-end algorithms in later vision systems is greatly reduced. Zhang et al. [178,179] fabricated photonic heterojunction transistors to emulate the visual learning and memory processes under different emotions. The emotions are represented by different colors of light. The light wavelength that induces a higher EPSC response corresponds to the better mood. Their studies showed that in a good mood, the learned memory level was higher and more difficult to be forgotten.

Figure 12.

(a) The array based on BiI₃/carbon nanotube optoelectronic heterojunction transistor is excited by different optical spike numbers. Reproduced by permission from [201], copyright [2022, Tsinghua University Press]. (b) The current responses of the array based on TiN_xO_2-x/MoS₂ optoelectronic heterojunction memristor with different light intensities and duration time. (c) The current responses of the array with different views. Reproduced by permission from [202], copyright [2021, Wiley‐VCH GmbH]. (d) Input image encoded by light of different wavelengths, intensities and frequencies for ZnO/P3HT optoelectronic heterojunction memristor. Reproduced by permission from [203], copyright [2022, American Chemical Society].

The previous devices for visual perception consume a lot of energy per synaptic event [43,206,207]. Therefore, it is important to develop heterojunction devices capable of low-voltage operation and low-energy consumption. To improve the device photosensitivity, Wang et al. [148] demonstrated an optoelectronic heterojunction transistor based on dinaphtho[2,3-b:2′,3′-f]thieno[3,2-b]thiophene (DNTT) with high charge mobility and 5,15-(2-hydroxyphenyl)-10,20-(4-nitrophenyl)porphyrin (TPP) with high light absorption. As an active layer with high light absorption, the TPP can significantly increase the number of photogenerated carriers in the charge transport layer of DNTT. The authors developed an ultra-sensitive artificial vision array based on the transistors that can detect weak light signals down to 1 µW cm⁻². The transistor can successfully emulate the basic functions of biological synapses at an ultra-low operating voltage of −0.01 V. Even at a very low voltage of −70 μV, the transistor exhibits significant synaptic responses with ultra-low energy consumption of 1.4 fJ. To improve poor electron mobility and limited light-matter interactions of the MoS₂, Luo et al. [208] coupled it with Au nanoparticles to form heterojunctions. The Au nanoparticle/MoS₂ heterojunction greatly enhances the light-matter interactions by utilizing plasmonic resonance, resulting in a significant improvement in carrier mobility and photoelectric response. The heterojunction phototransistor possesses ultra-sensitive currents, ultra-low energy consumption, and long retention time. The BP/CdS heterojunction transistor recently proposed by Zhu et al. [209] has the lowest power consumption compared to all previously reported results. This work provides a new idea for the design of energy-efficient photonic synapses with high performance. These designed heterojunction devices provide new avenues for developing high-performance artificial vision systems.

Human brain-inspired neuromorphic computing is considered a promising alternative to traditional von Neumann computing systems in the era of ‘big data’. It can process unstructured data such as images in parallel by energy-efficient manners [210–214]. Tsai et al. [215] designed an optoelectronic memristor based on a ReSe₂/h-BN/Gr vertically stacked heterostructure. They used MATLAB software to simulate the neural network for pattern recognition based on this optoelectronic heterojunction memristor. The image recognition results for each Hebbian learning cycle are shown in Figure 13(a). High recognition accuracy can be obtained after training. This study demonstrates the feasibility of this optoelectronic heterojunction memristor for application in visual perceptual memory. Graphdiyne (GDY) has excellent optoelectronic and electrochemical properties [219–221]. However, the small area and poor uniformity of the GDY film limit its use for scalable applications. To solve this problem, Zhang et al. [216] fabricated large arrays of wafer-scale GDY/graphene (Gr) vertical heterojunction phototransistors by synthesizing GDY films with high uniformity and controllable thickness on the surface of Gr. In their study, they found that the GDY/Gr heterojunction phototransistor has a near linear and symmetric conductance update trajectory. This helps to construct computational networks for neuromorphic image recognition with high accuracy and strong fault tolerance. The CNN constructed by the authors is able to recognize the handwritten digits from the MNIST dataset with high accuracy after training. In addition, they fabricated a 7 × 6 array of GDY/Gr heterojunction phototransistors to emulate the artificial vision system. The distinguishing function of the visual system is implemented on the basis of image recognition. The conductance changes of the system are measured before and after the input of the reference image and the unknown image, respectively. The authors calculate the conductance changes to determine whether the two match or not, as shown in Figure 13(b). These findings highlight the potential of GDY for applications in artificial vision systems. Actually, before this study, Hou et al. [222] proposed a wafer-scale optical pyrenyl graphdiyne (Pyr-GDY)/Gr/PbS quantum dot heterojunction memristor. They simulated an ANN for supervised learning of handwritten digits in the MNIST database. The ANN can have high recognition accuracy in different bending states. In addition, an array based on this heterojunction memristor has been constructed to achieve real-time detection of visual information, in-situ image memory, and image distinction. This work is an important step in the development of optogenetic neuromorphic computing and adaptive parallel processing networks for intelligent wearable electronics.

Figure 13.

(a) Simulation results of the Hebbian learning rule applied to image recognition. Reproduced by permission from [215], copyright [2021, Wiley‐VCH GmbH]. (b) Principle of distinction between the reference image and the unknown image. Reproduced by permission from [216], copyright [2021, Tsinghua University Press and Springer‐Verlag GmbH Germany, part of Springer Nature]. (c) Schematic diagram showing the metal-oxide heterojunction synapse with central nervous and visual sensory functions. (d) Schematic diagram of the visualized images before, during, and after training. Reproduced by permission from [217], copyright [2022, Elsevier Ltd]. (e) Color image recognition results obtained in three modes. (f) The CNN used for color pattern image recognition. Reproduced by permission from [218], copyright [2022, Wiley‐VCH GmbH].

Liu et al. [217] first proposed a metal oxide (ZnO/In₂O₃) heterojunction transistor that integrates central nervous and visual sensory functions. Figure 13(c) shows a schematic diagram of the metal oxide heterojunction transistor with central nervous and visual sensory functions. The persistent photoconductivity (PPC) effect in the metal oxide heterojunction [127,223] allows the transistor to emulate the visual sensory function under red, green, blue, and UV light stimulation. On this basis, the authors demonstrated a visual ANN combining artificial visual perception and neuromorphic computing. The trained ANN can recognize and classify clothing image datasets, as shown in Figure 13(d). This research simultaneously integrates central neural and visual sensory functions into a single device, reducing the amount of hardware required. It is critical for future intelligent visual robotics that can integrate neuromorphic computing and visual sensory functions. Multispectral color perception capability is important for emulating retinal functions. To this end, Jo et al. [218] proposed an artificial multispectral color recognition system based on ratio-controllable mixed quantum dot (M-QD)/a-IGZO heterojunction transistors. The precise mixing ratio of quantum dots results in differences of the postsynaptic current (PSC) triggered by RGB light, thus enabling the differentiation of the full range of visible colors. On this basis, the authors fabricated a photonic synaptic array using the M-QD/a-IGZO heterojunction transistor to mimic the color-recognizable visual system. Figure 13(e) shows the color image recognitions in three different modes. The visual memory can be enhanced by the spike number. Even though the current value of the response gradually decreases with time, the pattern images are maintained, indicating that color recognition is achievable. Finally, to simulate color pattern recognition in the computing systems, a CNN was implemented, as shown in Figure 13(f). The artificial vision system has a highly accurate color pattern recognition capability. However, it should be noted that when the input light energy is smaller than the band gap of amorphous oxide semiconductors, their PPC characteristics rely mainly on the ionization of oxygen vacancies. This leads to poor optical response of amorphous oxide semiconductor-based synaptic devices, which usually require high-power optical inputs for programming operation. Liang et al. [224] solved this problem by combining highly photosensitive quantum dots with amorphous oxide semiconductors. They reported a fully printed high-performance heterojunction transistor based on InP/ZnSe core/shell QDs and SnO₂. The heterojunction transistor enhances light absorption and improves the response to photoelectricity. It still exhibits significant synaptic responses at low voltages and relatively low energy consumption. Not only the functions of photoelectric perception, memory, learning, and forgetting are realized, but also it can be used for pre-processing and recognition of images. Moreover, all components of the device were produced by an inkjet printing process. This work provides a printable, low-cost, and efficient strategy for fabricating optoelectronic devices to implement artificial visual neuromorphic computing.

Currently, complex device and circuit integration is the bottleneck to realizing artificial visual intelligence. From the above reports, it can be seen that combining various materials to construct heterojunction structures is a promising solution. Not only the device sensitivity can be improved to efficiently detect information, but also the neural properties in visual synapses can be emulated with high precision. Furthermore, optoelectronic heterojunction devices with multiple functions in a single chip can greatly reduce the complexity of visual circuits and increase the speed of image processing [148,185]. It is also worth noting that in order to be useful in practical applications, the heterojunction devices need to be integrated with other types of devices [18,225,226]. Therefore, the development of various heterojunction structures is essential to advance the developments and applications of optoelectronic synaptic devices beyond the scope of the laboratory.

5. Problems, challenges, and prospects for future development

With the rapid development of science and technology, the emergence of new application scenarios such as robotics, driverless cars, and smart cities has placed high demands on artificial vision systems. Optoelectronic synaptic devices have attracted much attention due to their advantages in sensing, storage, and computing integration. These devices combine optical sensing and synaptic functions and can adjust synaptic weights through optical regions from UV to NIR. Meanwhile, the optical wavelength functions of optoelectronic synapses, such as color discrimination, selective non-volatile photodetection, and bidirectional photo-response, are highly desired for artificial visual intelligence [14,227,228]. Furthermore, optoelectronic synaptic devices take advantage of high bandwidth, low-power computational requirements, and low crosstalk to improve response speed, allowing efficient processing of visual information as well as complex memory, learning, and recognition tasks. It facilitates the implementation of artificial visual perception systems at the hardware level. This review first explains the principles of vision generation, and then focuses on the basic principles of two neuromorphic devices, optoelectronic memristor and transistor, and their visual bionic mechanisms. A series of important biological synaptic functions such as STP, LTP, and STDP are emulated using these two kinds of optoelectronic synaptic devices. Afterwards, the device fabrication and neuromorphic applications of the optoelectronic memristor and transistor using various materials and structures are mainly introduced. Almost all devices are capable of using both optical and electrical modulation modes, and these multi-excitation mode devices facilitate robust visual neuromorphic computing with simple device integration and low energy consumption. It has become increasingly desirable to develop artificial intelligence with real-time, fast processing abilities for high-throughput visual information, especially in areas such as self-driving cars. However, it still remains a challenge for developing optoelectronic memristors and transistors with superior opto-synaptic behavior, low-cost processes, low power consumption, and environmental friendliness.

In terms of the use of materials, such as silicon and oxide semiconductors, it is necessary to explore new applications for optoelectronic synaptic devices. The excellent optical and electrical properties of emerging materials such as 2D layered materials, perovskites, and ferroelectric materials should also be developed for optoelectronic synaptic devices. Despite the unique advantages of nanostructured materials, the synergistic integration of hybrid nanostructured materials from macroscopic to the nanoscale is necessary to fully exploit the photodetection function with the synaptic effect. For heterojunction-based optoelectronic synaptic transistors, the problem of realizing long-term synaptic plasticity needs to be addressed due to the rapid de-trapping of photocarriers at the heterointerface, which usually exhibits volatile retention behavior. In addition, optoelectronic synapses capable of realizing bipolar photo-synaptic dynamics by optical means should also be developed. Although realizing photonic neuromorphic computing inevitably requires both positive (e.g. EPSC) and negative (e.g. IPSC) synaptic plasticity, most optoelectronic synapses rarely trigger negative light responses. Electrical stimulation remains necessary for most existing optoelectronic synapses to implement IPSCs to accomplish bidirectional weight updates, which limits the processing speed, bandwidth, and integration density of the devices [11]. Much of the current research on synaptic devices has been conducted using monochromatic light stimulation. Some of the reported optoelectronic synapses can only operate in a relatively narrow spectral range, especially in the UV region [43,69,206,229], which inevitably hinders their potential for use in neuromorphic visual systems across the entire visible range. Most of the reported color-recognizable optoelectronic synaptic systems exhibit some limitations in the spectral selectivity of incident light stimuli [23,230,231]. Polychromatic identification systems in other research work have also been demonstrated, but relatively narrow output differences were obtained by different input wavelengths, which may produce ambiguous readings under optically complex conditions [99,187,232]. In this regard, modulating synaptic plasticity through multispectral selectivity is considered a solution to achieve more effective color perception. However, achieving selectivity for three or more colors requires multiple-channel logic operations and complex circuit architectures and manufacturing processes [227,233]. The complexity of structures and processes may lead to high manufacturing costs, long production cycles, increased power consumption, and low system integration, which in turn lead to reduced pixel density and resolution in vision perception systems. In addition to the aforementioned issues that need to be addressed, optoelectronic synaptic devices for artificial visual perception systems usually exhibit opto-neuromorphic functions over static ranges of light intensities, resulting in immovable thresholds. Therefore, it is difficult to realize an artificial adaptive visual system that can be self-controlled in combination with environmental changes, such as biological light adaptation or dark adaptation to distinguish bright light in bright states or to recognize weak light in dark states. Therefore, the triggering threshold of the visual system must be continuously adjusted with the synaptic weights corresponding to the ambient light conditions. However, the lack of ideal hardware for adaptive visual perception with low complexity and feasible power consumption remains a major obstacle to the implementation of bionic visual neuromorphic systems.

In terms of device fabrication, although considerable progress has been made in optoelectronic synapses as a promising candidate for processing visual signals, they are still in the early stages of research. There are still many challenges for practical applications of optoelectronic synapse-based artificial intelligence systems in neuromorphic visual perception, memory and computational systems, image recognition, autonomous driving, and humanoid robots. There are still many difficulties in miniaturization [234], high power consumption, low compatibility, complex device architectures, fabrication processes that are not fully compatible with commercially mature CMOS technologies. Most of current researches is still focused on the single-device level, only mimicking the basic synaptic plasticity functions [11]. These emerging optoelectronic synaptic devices are still far from large-scale optoelectronic integrated production, and high device integration density is an important requirement for developing miniaturized and high-resolution artificial vision systems. To develop high integration density and high-performance optoelectronic neuromorphic computing systems, optoelectronic synapses must be scaled down to the nanoscale for potential applications. Therefore, nanofabrication should be more widely used in future optoelectronic synaptic devices [235]. Novel photosensitive materials represented by low-dimensional materials and perovskites have attracted extensive attentions in optoelectronic neuromorphic devices because of their excellent optoelectronic properties. Although the new photosensitive materials have improved the photoresponsivity and broadened the response wavelength range of the devices to a certain extent, the immature preparation process makes it difficult to ensure the stability and repeatability of the future visual systems. There are currently only a few 2D materials that can be achieved at the wafer scale by bottom-up methods (e.g. chemical vapor deposition, CVD), and most of the 2D materials in these devices are mechanically exfoliated with random thickness and shape distribution, which will inevitably cause large device-to-device variations. Therefore, most of the synaptic devices based on 2D materials are not suitable for large-scale device arrays for hardware artificial neural networks [236]. Furthermore, organic-inorganic hybrid semiconductors such as organolead halide perovskites have been widely used as light absorption channels for neuromorphic heterojunction transistors, leading to weaker stability and environmental issues [35,237,238]. Meanwhile, high mobility charge transport layers are often fabricated using vacuum-based processes, such as crystalline silicon and 2D nanomaterials, which are incompatible with most non-vacuum processed photo-absorption layers, increasing the fabrication complexity and cost [42,239,240]. Therefore, fully solution-processed high-performance optoelectronic synaptic transistors also present a challenge.

To realize the mimicry of the human eye in artificial intelligence, a flexible and highly sensitive neuromorphic imaging system with image perception and data preprocessing must be implemented. However, the implementation of such a system remains a great challenge due to the limitations of materials, device structures, and conventional imaging modules. Currently, although some photo-synaptic transistors have been reported to show high sensitivity, almost all of them are made of inorganic materials that must be constructed on rigid, planar glass and silicon substrates [21,43,62,109,145,146,241]. Large-area ultra-flexible device arrays based on organic materials are simple to fabricate and fit seamlessly onto the 3D hemispherical surface like the retina of the human eye. However, organic devices are generally less photosensitive (<10²) under low light conditions compared to inorganic devices because the exciton binding energy of organic materials (0.3–1 eV) is typically higher than that of inorganic materials (0.002–0.3 eV) [152,153,156,242,243]. Low photosensitivity inevitably leads to poor photodetection accuracy and low noise immunity, which results in weak target image recognition. Therefore, the above problems seriously hinder the development of neuromorphic imaging systems and their further applications in future artificial intelligence.

Optoelectronic synaptic devices integrating sensing, storage, and computing are urgently needed for the booming artificial intelligence and Internet of Things. Therefore, research on optoelectronic synaptic devices is an exciting topic in the age of intelligence [15]. In terms of future application prospects, various functions of optoelectronic synaptic devices in optoelectronic synaptic plasticity, image enhancement, and erasure, 3D image perception and preprocessing, etc. will be further developed through reasonable material selection and device structure design. Meanwhile, it can also accelerate the development of hardware-based neural networks to neuromorphic computing systems, and help to break the current limitations of the separation of artificial vision sensors and information processing units in the von Neumann architecture. Next efforts are still needed for complex processing, uniformity, and reliability issues.

Biographies

Yi Zhang received her BE from Hunan University of Chinese Medicine in 2021. She is currently a graduate student in School of Physics and Electronics, Central South University under the supervision of Prof. Jie Jiang. Her research focuses on oxide neuromorphic devices.

Zhuohui Huang received her BE from University of Jinan in 2021. She is currently a graduate student in School of Physics and Electronics, Central South University under the supervision of Prof. Jie Jiang. Her research focuses on two-dimensional neuromorphic devices.

Jie Jiang received BE and ME in electronic science and technology and PhD in physics from Hunan University in 2007, 2009, and 2012, respectively. He was a postdoc in Nanyang Technological University (2012–2013) and Auburn University (2014–2015), respectively. He is currently an Associate Professor in School of Physics and Electronics, Central South University. His research focuses on neuromorphic materials and devices.

Funding Statement

This work is supported by the National Natural Science Foundation of China (52172169). Y. Zhang thanks the Fundamental Research Funds for the Central Universities of Central South University (2022ZZTS0768).

Disclosure statement

No potential conflict of interest was reported by the author(s).