Fig. 1. Neuromorphic vision chip based on multimodal resistive memory arrays with ORRAM mode array for in-sensor image preprocessing and ERRAM mode array for high-level image recognition.

An advanced neuromorphic vision system based on MSFP-based multimodal resistive memory arrays with both ORRAM mode for image pre-processing and ERRAM mode for high-level image recognition, simulating the functions of retina cells and visual cortex, respectively. In the biological visual system, the human retina is organised into three nuclear layers and two synaptic layers, in which bipolar cells that include the off and on cells as an inner nuclear layer (INL) connect the outer nuclear layer (ONL, photoreceptors) and the ganglion cell layer (GCL). The images are firstly sensed by the ONL and then are pre-processed by the INL and GCL, and the pre-processed information will be finally transmitted to the visual cortex layer to complete high-level processing. The neuromorphic vision chip consists of an ORRAM mode array with NPM and PPM features for the in-sensor image pre-processing (e.g. contrast enhancement and background denoising) and in-sensor convolution for feature extraction, and ERRAM mode array with analogue resistive switching for in-memory high-level image recognition through convolutional neural network (CNN) operations.