Retina-inspired narrowband perovskite sensor array for panchromatic imaging

Abstract

The retina is the essential part of the human visual system that receives light, converts it to neural signal, and transmits to brain for visual recognition. The red, green, and blue (R/G/B) cone retina cells are natural narrowband photodetectors (PDs) sensitive to R/G/B lights. Connecting with these cone cells, a multilayer neuro-network in the retina provides neuromorphic preprocessing before transmitting to brain. Inspired by this sophistication, we develop the narrowband (NB) imaging sensor combining R/G/B perovskite NB sensor array (mimicking the R/G/B photoreceptors) with a neuromorphic algorithm (mimicking the intermediate neural network) for high-fidelity panchromatic imaging. Compared to commercial sensors, we use perovskite “intrinsic” NB PD to exempt the complex optical filter array. In addition, we use an asymmetric device configuration to collect photocurrent without external bias, enabling a power-free photodetection feature. These results display a promising design for efficient and intelligent panchromatic imaging.

An artificial intrinsic RGB photoreceptor matrix can replicate the neuromorphic panchromatic vision capability of the eye.

INTRODUCTION

Panchromatic image sensing of an object containing both shape and color information is more advantageous than monochrome image sensing for a broad range of applications such as multispectral imaging/telecopy/clairvoyance in satellites (1), surveillance/microscopy smart camera (2), and photography/film for aesthetics and entertainment (3). In nature, the human retinal system is a highly sophisticated panchromatic sensing system with high-level of sensitivity [detectable of single photon (4)], resolution [576 megapixels (5)], and intelligence [capable of chromatic adaption, vision memory, etc. (6)]. Mimicking the human retinal system has been an inspiration for not only hardware research and development but also for the development of signal and image processing algorithms that can result in future improvement in domains such as machine vision (7), artificial intelligence (8), and the Internet of Things (9). The retina system consists of multiple cellular layers, where the outermost layer is the optical nerve cells that connect to the visual cortex to the brain, the intermediate layer consists of sublayer networks responsible for neural signal transduction, and the bottom layer is responsible for light-to-neuronal signal conversion (10). In the human panchromatic vision, cone cells with signature monochromatic sensitivity of red (R), green (G), and blue (B) light at the bottom layer firstly sense the light and convert it to neuronal signals. Then, the intermediate layers further execute the presignal processing within its plexiform structure and transduce the signal to the outermost layer for further transmissionto the brain (11). Specifically, the R/G/B cone cells are able to individually convert photons of different wavelengths into neuronal signals, with a narrow optical band feature. In addition, the intermediate cellular layer can execute both feed-forward and feedback control to simplify the signal being transmitted to the brain (11). Overall, the R/G/B cone cells are the basic units responsible for monochromatic photon detection of different wavelength and the network layer provides processing of the resultant neuronal signals that eventually transport to the brain to perceive the object.

Electronically sensing the object with a similar perception as that of the human eye can be achieved by photodetectors (PDs) that individually respond to the same range of wavelengths in the electromagnetic spectrum where human retina is responsive. Charge-coupled device and complementary metal-oxide semiconductor cameras have shown great progress by using Si or InGaAs-based broadband sensor arrays (12, 13). Unlike the R/G/B retinal cone cells, these broadband sensor arrays (12, 13) cannot “intrinsically” distinguish the color and usually need an external color filter array (CFA) to achieve monochromatic sensing. The insertion of CFA not only increases the cost and manufacturing complexity but also causes spatial information loss due to the coplanar R/G/B dispersion by CFA as well as other issues such as color aliasing or color moiré effect (14), and limited quantum efficiency (15) (1/N) by channel numbers (N) in the CFA layout (16). In these systems, a demosaicing process (digital calculation to process the “missing” color values for each pixel in the raw capture) (17) is typically needed to reconstruct a full-color image from the incomplete color samples output from an image sensor overlaid with a CFA. This additional computational complexity requires higher memory capacity and reduces the overall speed of the camera. In contrast, filter-free narrowband (NB) PDs could circumvent such issues by CFA exemption and optimized spatial arrangement. In analogous to the R/G/B cone cells in the human retina, NB PDs have the “intrinsic” monochromatic sensing capability. This capability can be achieved through mechanisms such as all-dielectric metasurface (18), narrow absorption (19), manipulation of Frenkel exciton dissociation (20), charge collection (21), and surface-trap recombination (22) across different materials including organics, two-dimensional inorganics, and halide perovskites. The halide perovskites bearing exceptional semiconducting properties such as long charge carrier diffusion length, high light extinction coefficient, and benign electronic band structure with high trap tolerance (23) could be excellent NB PD candidate. Through compositional engineering, the NB response band can be tuned to R/G/B regions enabling the realization of individual pixel devices analogous to retinal cone cells.

Here, we demonstrate an R/G/B NB PD array using engineered halide perovskite films fabricated from a volatile solution (VS) system, followed by a multilayer algorithm to mimic human retina system for panchromatic imaging. Briefly, we observed that the perovskites obtained from this VS method exhibit heavily unbalanced electron-hole transfer property that electrons and holes could drift distinct distance toward corresponding electrodes. By sandwiching this unbalanced perovskite into different polar structures (p-i-n or n-i-p), photocurrent collection at electrode can be manipulated upon the wavelength-dependent photogeneration center along the thickness direction, which is due to the effect of wavelength-dependent optical field distribution (WDOFD). This unbalanced transport and WDOFD jointly enable the NB response of the PDs for individual R/G/B distinction. By using a vertical stacking of the six-terminal R/G/B layout, a full-color detection with the exemption of CFA (or demosaicing) has been achieved. Next, to mimic the intermediate network of the retinal system, we pass the raw current signal to a three-sublayer neuromorphic algorithm for feed-forward signal processing. As a result, we demonstrate the photoelectrical capturing and restoration of a 1024-pixel full-color image by a 32 × 32 perovskite NB PD sensor array with three R/G/B channels. The signals were further processed using a neuromorphic network algorithm to mimic the plexiform structure in the intermediate cellular layer in retina. The results of the perovskite array display a high fidelity for panchromatic imaging. This system-level demonstration of the retina imaging process provides substantial insight for implementing panchromatic imaging techniques.

RESULTS

Three types of natural R/G/B cone cells have been mimicked in imaging technology with three channels of R/G/B to realize panchromatic imaging. There can be two types of device architecture for panchromatic imaging. Figure 1A compares the typical parallel (24) and stacking (25) design. The parallel architecture has been ubiquitously applied in the commercial digital camera, with the incident white light being spatially dispersed into R/G/B through a complimentary CFA followed by detection of a color-blind sensor array. Hence, a post logic processing to compromise issues of spatial resolution limit, optical efficiency, and color moiré is needed, and this is provided by a complex back digital circuit (Fig. 1Ai). In contrast, the stacking design allows the white light to penetrate into different photoactive layers to obtain separate monochromatic sensing without spatial dissipation (Fig. 1Ai), which can circumvent the abovementioned issues in those parallel designs. Coincidentally, it should be noted that similar vertical stacking strategies have also been applied in tandem solar cells to overcome the thermodynamic losses (26) and in a light-emitting diode (LED) to have a full-color-tunability (27). The stacking structure is expected to deliver a higher light usage ratio on the unit area but also provide separate electrical channels for panchromatic color reconfiguration using a multilayer algorithm analogous to that occurring in the retinal system. Figure 1Bi specifies the neurotransmitter network at the bottom side of the human eye (Fig. 1Bii). Briefly, the light-to-neuronal signal conversion first takes place at the R/G/B cone cells from the photoreceptor layer, and the network layer consisting of three sublayers of horizontal cells, bipolar cells, and amacrine cells to process the signal, followed by the transduction via the ganglion cell to deliver the signal to the brain. In analogy to this natural scheme, we designed a three-sublayer neuromorphic algorithm to process the current signals from the R/G/B PDs (Fig. 1Ci). In doing so, we developed the NB perovskite PD arrays to mimic the cone cells (Fig. 1D) and transduce their electrical information toward the abovementioned neuromorphic algorithm. A proof-of-concept result of three independent R/G/B channels in mimicking the retina layer of a primitive 15 × 8-pixel NB PD array is presented in fig. S1 (note S1).

Fig. 1. Overall design of retina-inspired NB perovskite PD for panchromatic imaging.

(A) Schematic illustration of (i) typical parallel and (ii) stacking designs of panchromatic imaging sensor. CMOS, complementary metal-oxide semiconductor. (B) (i) Schematic of the multilayer neurotransmitter network at the bottom of the retina in (ii) the human eye. (C) Schematic showing the three-layer neuromorphic algorithm for signal processing and image reconstruction. (D) A photo of a perovskite NB PD array made in this work. Note: C-, H-, B-, A-, G-cells are short for cone, horizontal, bipolar, amacrine, and ganglion cells, respectively. H1, H2, and H3 are the three hidden layers in the neuromorphic algorithm.

The NB PDs are made from thin perovskite films (≤2 μm) of different halogen compositions (MAPbX₃, X = Cl, Br, I or their combination) for bandgap modulation. To obtain highly homogeneous crystalline film for retinal cellular arrays, we used the VS method containing methylamine-liquefied perovskites in volatile acetonitrile (ACN) that enables the realization of ultra-uniform films (detailed in note S2). Through compositional engineering, MAPbX₃ with different halogen compositions were found to display varying absorption edges (Fig. 2A) spanning over the whole visible region from 452 to 700 nm (corresponding to 1.77 to 2.74 eV as plotted in fig. S5). Since the p orbital of halogen in MAPbX₃ contributes to the valence band maximum, tuning the halogen could thereby modulate the bandgap accordingly. We developed the compositions MAPb(BrI₂), MAPb(Br₃), and MAPb(Cl_1.5Br_1.5) with absorption edge of 700, 549, and 470 nm, respectively, as the photoactive layer for the R/G/B NB detection. All the samples exhibit a narrow FWHM (full width half maximum) less than 55 nm from the photoluminescence (PL) spectra (Fig. 2B). It should be noted that the mixed halide perovskite synthesized using traditional nonvolatile methods usually has serious phase separation issues owing to the halide ion migration (28, 29) in the loosely crystallized materials. The phase separation will lead to bandgap splitting interfering the detection range of PD and the defect in poor crystals will cause performance output uniformity issue in the array application. In contrast, the film casted from the VS displays compact and highly crystallized features without any phase separation for all the R/G/B samples. Taking the MAPb(BrI₂) as an example, the x-ray diffraction (XRD) spectra shows strong (100) orientation with a high Lotgering factor (LF) of 0.92 (Fig. 2C), which is nearly three times higher than the reference MAPb(BrI₂) film casted from traditional nonvolatile solution (LF of 0.31, fig. S6A). Similarly, the MAPb(BrI₂) film also displays a mono (100) peak at 14.46°, without characteristic peaks from any of the hypothetical separated I-rich or Br-rich phases (fig. S7). To verify the monophase of the mixed halide R/G/B films, we also use the scanning transmission electron microscopy coupled with a high-angle annular dark-field detector to visualize the atomic distribution in these films. Figure 2D shows the energy-dispersive spectroscopy (EDS) mapping of Cl, Br, I, and Pb in different samples. The binary mixed halogen elements in either red-MAPb(BrI₂) or blue-MAPb(Cl_1.5Br_1.5) films exhibit homogeneous halogen distribution over the grain, and there are no Br-, I-, or Cl-rich phases separated within the grain. The single-halogen green MAPb(Br₃) also shows the identical distribution of Pb and Br in the whole grain, indicating no noticeable impurities or segregation of Pb or Br. These results verify the crystalline phase purity of the perovskite films processed from our VS method, which is important for fabricating array PD device that needs high-standard uniformity and purity to deliver low signal-to-noise ratio. In addition, high reproducibility in array pixel devices also requires high morphological homogeneity of the perovskite crystalline film. We also identify the film homogeneity of R/G/B perovskites processed from the VS method. Figure 2E displays the top view and cross-sectional scanning electron microscopy (SEM) images of R/G/B perovskite films coated on PEDOT:PSS [poly(3,4-ethylenedioxythiophene] polystyrene sulfonate) hole transfer layer substrate. All the R/G/B films display homogeneous and dense features, which will secure each integrated pixel device to have identical PD performance.

Fig. 2. Materials characterization.

(A) UV-Vis (visible) absorption spectra of perovskite film with different compositions. (B to E) (B) UV-Vis absorption and PL spectra, (C) XRD pattern of perovskite thin films, (D) elemental mapping of perovskite crystals, (E) top-view SEM images of perovskite thin-film (upper row) and cross-sectional SEM images (lower row) showing the layout of red-MAPb(BrI₂), green-MAPb(Br₃), and blue-MAPb(Cl_1.5Br_1.5) PD. HAADF, high-angle annular dark-field detector; a.u. arbitrary units.

We then use these VS-processed R/G/B films for making NB PD single-pixel devices, followed by laser scribing–based manufacturing to integrate them into array architecture. In principle, to realize NB photodetection, the device needs to be featured by a narrow external quantum efficiency (EQE) with a peak around the target wavelength window (30). In analogy to R/G/B retina cells, NB PD devices need to have a mono-peak EQE response at 700, 550, and 470 nm, respectively, to avoid the usage of external optical filter. Typical semiconductors with a fixed bandgap tend to absorb shorter wavelength light with energy larger than the bandgap. As schematically shown in fig. S10A, both long-wavelength and short-wavelength photons can be absorbed and contribute to the photocurrent. This makes it difficult to distinguish the photocurrent from different wavelengths and hard to realize an intrinsic NB PD. To selectively collect photocarriers generated from long-wavelength light, the photocarriers generated by short-wavelength light need to be self-consumed (i.e., reducing EQE to zero) while EQE of long-wavelength light needs to be maintained at a high level. There have been prior studies to realize EQE narrowing in both organic and perovskite PDs (21, 22, 30–33). Particularly in perovskite PDs, experiments rely on either thick perovskite films (20 to 25 μm) (32, 33) or single crystals (ultrathick to ca. 300 μm) (22) to quench short-wavelength photocarriers (30) by intrinsic or extrinsic recombination centers (21, 31). Nevertheless, the requirement of ultrathick film and uncontrollable doping is hard to balance with the film homogeneity that is necessary for array device application. In contrast, here we develop the thin-film (<2 μm) perovskites for array structures of NB PDs, by implementing a strategy of unbalanced electron-hole transport. This thin film is highly controllable in uniformity and reaches the application benchmark desired for multipixel array retinal chip.

In principle, along the perovskite thickness direction, there is a WDOFD of the incident light due to the wavelength-dependent variation of optical constants [i.e., refractive index n(λ) and extinction coefficient k(λ)] (34). Taking the red-MAPb(BrI₂) NB PD [with a p-i-n structure of ITO/PEDOT:PSS/MAPb(BrI₂)/PCBM/Au] as an example, Fig. 3A shows the simulated optical field distribution of incident light of different wavelengths along the thickness direction. Short-wavelength light (e.g., from 350 to 550 nm) can be quickly absorbed near the surface region, and beyond 1 μm, there is negligible photocarrier generation from the short wavelength. The long-wavelength light (e.g., 650 nm) penetrates deeper through the perovskite film and generates the photocarriers over the whole thickness region (fig. S11A). Similarly, both green-MAPb(Br₃) and blue-MAPb(Cl_1.5Br_1.5) perovskite films also display this WDOFD effect (figs. S12A and S13A). As exemplified in fig. S14A, in a p-i-n device configuration, short-wavelength generated electrons (generated at front side) need to drift a long distance (through the film thickness) to reach the backside cathode. This makes it possible to minimize the collection efficiency of these short-wavelength generated electrons by using a perovskite with low electron mobility. In parallel, because of the deeper penetration of long-wavelength light, long-wavelength photoelectrons generated close to the cathode side can be collected because of a shorter drift distance (fig. S14B). To ensure a good hole collection from these deeper generation sites, good hole mobility is required for the perovskite.

Fig. 3. Working mechanism and performance of red-MAPb(BrI₂) NB PD.

(A) WDOFD of the incident light due to the wavelength-dependent variation of optical constants of the perovskite. (B) Current density–voltage (J–V) characteristic of the electron-only diode and hole-only diode of red MAPb(BrI₂) measured in the dark. (C) TRPL of electron- and hole-extraction structure of red MAPb(BrI₂). (D) J–V curve of red NB PD measured in the dark and under 650-nm LED light illumination. (E) Noise current of red NB PD. (F) Specific detectivity (D^*) of red NB PD at zero-bias condition, measured from full wavelength spectrum. (G) Zoom-view of J-t curve showing a single light on/off state of red NB PD. (H) Frequency-dependent output response of red NB PD. The dashed line shows the representative −3 dB bandwidth (f_−3dB) of 17.8 kHz. (I) Current density with illumination power curve showing the linearity of red NB PD.

We demonstrate that the NB PD can be achieved using the perovskite with an unbalanced electron-hole mobility (μ_h > μ_e) in a p-i-n configuration (details are demonstrated in note S3). It should be noted that the perovskite film fabricated from using our volatile solvent method exhibits this desired unbalanced photocarrier transport property. We verified this by characterizing the charge transport properties using monotype carrier-type diodes coupled with both electrical and PL investigations (detailed description, calculation, and discussion are specified in note S4). Briefly, Fig. 3B shows the current density–voltage (J-V) characteristics of red MAPb(BrI₂)–based electron-only and hole-only diodes in the dark as an example, from which key transport parameters such as carrier mobility can be extracted using the space charge–limited current method (35). As a result, the red MAPb(BrI₂) displayed a μ_h of 2.61 × 10⁻⁴ cm² V⁻¹ s⁻¹, which is sixfold higher than its μ_e of 4.19 × 10⁻⁵ cm² V⁻¹ s⁻¹. To further confirm the presence of this unbalanced electron-hole transport in a photophysical process, we performed the time-resolved PL (TRPL) spectroscopic studies on the MAPb(BrI₂)-based electron-only (TiO₂/perovskite/PCBM) and hole-only (PEDOT: PSS/perovskite/Spiro-OMeTAD) samples (Fig. 3C). The hole-only sample exhibits a much faster average decay lifetime τ_ave,h of 2.36 ns than that of electron-only sample (τ_ave,e of 18.09 s), which is consistent with the result of their time-integrated PL (fig. S16A). In comparison, we also measured samples with a similar structure using perovskite prepared from the conventional nonvolatile solvent of dimethylformamide and dimethyl sulfoxide. As shown in fig. S16D, conventional MAPb(BrI₂) displays the τ_ave,h and τ_ave,e at similar level ca. 10 ns, which is consistent with its balanced electron-hole mobility nature. The VS-prepared red MAPb(BrI₂) displays an over sevenfold smaller τ_ave,h indicating a much more efficient photogenerated hole transfer and extraction/bleaching from the red MAPb(BrI₂) to the quencher. This is consistent with its higher μ_h that a faster hole transfer toward the quencher quickly reduces their lifetime in the film. Hence, the VS-prepared red MAPb(BrI₂) displays a more efficient hole transfer than the electron. We also characterized the mobility and average lifetime for VS-prepared green-MAPb(Br₃) and blue-MAPb(Cl_1.5Br_1.5) perovskites and observed the similar unbalanced transport behavior (detailed quantitative discussion on mechanisms are included in note S4).

We then used these VS-prepared perovskites coupled with the WDOFD effect to devise the R/G/B NB PDs. By using an asymmetric p-i-n configuration of FTO/PEDOT:PSS/RGB perovskite/PCBM/Au, we observed mono-peak EQE in R/G/B under zero bias (fig. S19). This power-free detection feature is due to the work function difference of cathode and anode materials that form an intrinsic electric field to drive the photogenerated electrons and holes moving toward corresponding electrodes. As a result, under zero bias, we successfully obtained NB responses of 640 to 720, 530 to 570, and 460 to 500 nm lying in the red, green, and blue color regimes, respectively. Figure S20 displays the responsivity (R) of R/G/B NB PDs calculated from the equation

$$R=\mathrm{EQE}\frac{e}{hv}(\mathrm{A}{\mathrm{W}}^{-1})$$ (1)

where e is the elementary charge, h is the Planck’s constant, and v is the frequency of light, respectively. The R/G/B NB PDs display a responsivity peak of 4, 3.5, and 2 mA W⁻¹ at zero bias, respectively, which approaches the state-of-art perovskite NB PDs (e.g., 10 mA W⁻¹) but under large bias as 4 V (22, 32, 33). This result provides the proof of concept of the NB response realized by our thin-film perovskites with unbalanced transport coupled with the WDOFD effect.

Next, to understand the correlation of PDs with the R/G/B retinal cells, we characterize the devices in terms of spectral detectivity, temporal response, and optical linearity. Here, we use the red PD as an example for simplification and include the related demonstrations on green and blue PDs in note S5. Figure 3D shows the J–V plot of the red PD under 650-nm irradiation and dark condition, respectively. It should be noted that under zero bias, the PD displays a low dark current density of ca. 10⁻⁷ A cm⁻² but a high light current of 5 × 10⁻³ A cm⁻² and thus a high on/off ratio (I_on/I_off) over 10³. The specific detectivity (D^*) of PD can be determined from the equation (36)

$$D^{\ast }=\frac{\sqrt{A}}{\mathrm{NEP}}({\mathrm{Hz}}^{-1/2}\mathrm{cm}{\mathrm{W}}^{-1})$$ (2)

where A is the working area of PD (here, A is 0.1 cm²) and NEP is the noise equivalent power, which can be calculated from equation (36)

$$\mathrm{NEP}=\frac{i_{\mathrm{n}}}{R}(\mathrm{W}{\mathrm{Hz}}^{-1/2})$$ (3)

with R being responsivity and i_n being the noise current value. We obtained the i_n from the noise spectra density derived from the Fourier transform of the dark current density versus time at zero bias. It should be noted that the noise current is frequency dependent over the whole frequency range (37), where both frequency-dependent (e.g., flicker, other dynamic disorders at material, interface, and device level) and frequency-independent (e.g., shot noise, thermal Johnson noise) contributions build up the overall noise current. Nevertheless, in most cases of perovskite PDs (21, 22), the frequency-independent contributions dominate at higher-frequency regions, suggesting that an overall “white”-type (frequency-independent) noise approximately equals to the sum of the shot noise and thermal Johnson noise. Consistently in this study, as shown from the noise current spectrum in Fig. 3E, the frequency-dependent noise can be found at the low-frequency regime, while for higher-frequency regime (f > 0.5 Hz), the noise behavior exhibits a white-type feature, where we use the value to determine the noise current. On the basis of this assumption, the red PD displays a noise current of 20 fA Hz^−1/2. Figure 3F displays the specific detectivity spectrum of the red PD under zero bias, with a peak value of 6 × 10¹⁰ Jones (Hz^1/2 cm W⁻¹) which is comparable to that of PD from prior researches but operating at high bias voltage (e.g., 10¹¹ Jones under 5 V) (22, 31). The detectivity bleaching is also present outside the response window, securing the monochromatic light response.

We also characterize the temporal response of the PDs, because the response speed is crucial for practical applications such as image sensors and it sets the limit on the framerate of image capturing (38). Figure S21 shows the current temporal response of red PD operating at zero bias under periodic illumination from a red 650-nm LED. Figure 3G shows the corresponding response zoomed in on one period, with quantified rise time (t_rise, time for photocurrent rises from 10 to 90% of the peak value) and drop time (t_drop, time for photocurrent decreases from 90 to 10% of the peak value). The red PD displays a fast t_rise of 1.29 ms and t_drop of 1.37 ms comparable to typical broadband perovskite-based PDs (39, 40) and almost one order of magnitude faster than that of human eyes (ca.10 ms) (41) for pattern recognition. To further evaluate the response speed, we measured the frequency-dependent current response in Fig. 3H. The response bandwidth (f_−3dB) quantifies the response speed, which is the characteristic light switching frequency at which the PD photocurrent drops to 70.7% of its maximum value at the steady illumination condition (0 Hz) (38). The red NB PD exhibits a f_−3dB of 17.8 kHz, which represents a fast response speed among all the reported perovskite or organic NB PDs (22, 31, 33) (20 to 700 Hz). Lastly, we assess the response linearity of the NB PDs with respect to incident light. Figure 3I shows photocurrent dependence on the incident power with the linear dynamic range (LDR) estimated from the equation (42)

$$\mathrm{LDR}=20\times \log \frac{P_{\max }}{P_{\min }}(\mathrm{dB})$$ (4)

where P_max and P_min are the boundary limits of the optical power within which the current has a linear relationship with the power. Overall, at room temperature, the red PD displays an LDR of 70.4 dB, which is higher than that of InGaAs-based PDs (66 dB, at 4.2 K) and approaching that of Si-based PDs but operating at ultralow temperatures (120 dB, at 77 K) (43). We also characterize these figure of merits in both green and blue PDs, detailed in note S5. As expected, all the R/G/B PDs display a narrow response band (FWHM ≤50 nm), quick response (t_rise, t_drop ≤ 1.6 ms, f_−3dB of 6 to 18 kHz), and good linearity (LDR ≥ 70 dB), indicating good qualification for the next array usage.

We then use these R/G/B NB PDs as pixel devices in an array structure manufactured by the picosecond laser-scribing technique (fig. S26 and note S6). For demonstration purpose, we made three chips of R/G/B containing 10 × 10-pixel devices with a pixel area of 1 mm² (detailed design and readout circuit are presented in fig. S27A). Figure S28 displays the testing setup, where we used a projector (M6 Pico Projector, AAXA Technologies) to generate a panchromatic source image and used a lens to focus the incident light onto the chips. Blue, green, and red chips are sequentially stacked (six-terminal stacking) from front to back, where blue, green, and red channels of current are read out independently. These signals are further transferred to a neuromorphic network analogous to the bipolar, horizontal, and amacrine cells in the human retinal system (Fig. 1Bi) to execute the image restoration. Figure 4A displays the selective spectra response within the red, green, and blue regimes of our R/G/B PDs. It should be noted that the perovskite can be engineered to have a broad bandgap range, which enables panchromatic imaging using multiple color channels. Here, we construct the R/G/B stacking chips as an example. Each R/G/B NB PD chip only detects the corresponding color without any interference with others. To demonstrate this, we tested the stacking chips with a color mixing pattern (fig. S29A). Figure S29, C to E show the photocurrent mapping from red, green, and blue channels, respectively, where there is a good color distinguishment between different lights and no obvious interfering noise signal from other colors. The results also show good color uniformity due to the high film uniformity, which is important for image capturing and reconstruction. To further quantify the color distinguishing ability, we also test the green chip with a color-blind pattern that is normally used for human color vision tests. As can be seen in fig. S31, the chips could distinguish the clear “checkmark” pattern from the original image, and there is two order-of-magnitude difference in current intensity between the checkmark region and the background region. These results suggest good distinguishing in color mixing pattern in primitive 10 × 10-pixel NB PD arrays. Notably, all these results are measured under zero bias conditions so that this light-driven detection without power supply is analogous to the natural retinal cells, and it does not require battery that a typical commercial camera needs. We have also evaluated the uniformity of sensor array by illuminating the 10 × 10-pixel R/G/B NB-PD sensor arrays with monochromatic LED light of 650, 550 and 580 nm, respectively. Figure S32 shows the output two-dimensional current mapping from the R/G/B sensor array, respectively. Uniformity of the sensor array is then quantified by calculating the 1 SD (1σ) from the average current values from the 100 pixels. The uniformity can then be presented by the nonuniformity (NU%) as below:

$$\mathrm{NU}\mathrm{\%}=\frac{1\mathrm{\sigma }}{{\overline{I}}_p}$$ (5)

Fig. 4. Panchromatic imaging test.

(A) Detectivity spectra of red, green, and blue NB PDs showing their respective detection window. (B) Two-dimensional current mapping in (i) red (ii) green (iii) blue channels obtained from red, green, blue NB PD arrays with 32 × 32 = 1024 pixels for each. (C) (i) Original image (ii to iv) restored images by different reconstruction methods: (ii) Direct channel merging. (iii) Neuromorphic processing without filtering scheme (iv). Neuromorphic processing with filtering scheme. (D) Plot of smooth loss and peak signal-to-noise ratio (PSNR) with epoch for the standard model A2 (with 32-64-32 nodes in three hidden layers) and lighter model A2 (with 16-32-16 nodes in three hidden layers), showing that A1 is underfitting. (E) Plot of smooth loss and PSNR with epoch for the standard model A2 and heavier model A3 (with 64-128-64 nodes in three hidden layers), showing that A3 is overfitting.

In the Eq. 5, 1σ is the one standard derivation of 100 current value numbers from 100 pixels, and ${\overline{I}}_p$ refers to the average current values from the 100 pixels. The red, green, and blue detector array exhibits a NU% of 1.58, 4.41, and 4.67%, respectively, which demonstrates a good uniformity of the sensor arrays benefiting from the good perovskite film quality.

To further confirm the down-scalability of pixel size with respect to signal noise ratio and NB PD performance, we constructed 32 × 32-pixel chips for red, green, and blue PD arrays (pixel area of 33 × 33 μm², a photo of a red chip is presented in Fig. 1D). For testing, we use a 949-byte (32 × 32-pixel, 96 dpi, 24-bit depth) color image (sunflower picture, Fig. 4C) as the source image for panchromatic imaging using our R/G/B chips. Figure 4B, i to iii, show the photocurrent mapping under zero bias in red, green, and blue channels obtained from corresponding chips. The photocurrent maps clearly show the R/G/B feature of the original image with sharp contrast and high fidelity. To reconstruct the original panchromatic sunflower image and compare the algorithm excellence between a typical merging reconstruction and a retinal system mimicked multilayer neuromorphic reconstruction, here we use two different reconstruction methods, i.e., (i) channel merging and (ii) machine learning. The channel merging method first normalizes 2D current values to pixel intensity values, with the lowest current shifting to 0, highest to 255, and other values scaling linearly. After further converting floating-point normalized values to integers, the sunflower image is directly reconstructed by stacking pixel intensity integers from R/G/B channels, as shown in Fig. 4Cii). It is worth mentioning that a minimum pixel value of 0 is observed for all three R/G/B channels from the original image (shown in Fig. 4Ci), and the maximum pixel value of 255 is also observed for all channels. These observations guarantee that current values best correlate with pixel values, indicating the best channel merging reconstruction quality. On the other hand, machine learning reconstruction adopts a multilayer perceptron neural network that takes current values as input and predicts pixel values as output accordingly, which can be analogous to the multilayer structures of horizontal, bipolar, and amacrine cells in the retinal system (Fig. 1B). The neural network model relies on a set of training data points that refer to 1024 pairs of current values (features) from NB PD chips and pixel values (labels) from full-color sunflower images. The i.i.d.(44) assumption of the training dataset is satisfied since each current value is independently measured using the same sensor array. A standard neural network architecture (A2) is shown in Fig. 1C where the input layer contains three neurons corresponding to current values from three channels, respectively. In the middle, three hidden layers with respective 32 (H1, in analogy to horizontal cell layer), 64 (H2, in analogy to bipolar cell layer), and 32 (H3, in analogy to amacrine cell layer) neurons perform the black-box-like mapping from input to output. A learning algorithm usually has another testing dataset for evaluating the possible overfitting problem. For data acquiring simplification and concept-proof demonstration, we use the single sunflower image (1024 separate data points) for training and testing the learning algorithm (A2). Another two neural network learning algorithms are created with architectures 16-32-16 (A1) and 64-128-64 (A3) for the middle three hidden layers so as to demonstrate the better model complexity of A2. The learning algorithms are trained with batch sizes of 1 using Adam optimizer with default beta1 0.9 and beta2 on a single RTX 2080 Ti GPU. The learning rate is initially set to 0.001 for the first 100 training epochs and decays to 0.0005 for the second 100 epochs. Smooth L1 loss is adopted to reflect the learning quality. Furthermore, peak signal-to-noise ratio (PSNR), which shows the ratio between the maximum possible power of a signal (original sunflower image) and the power of the same image with noise (reconstructed image), is also calculated along with each training epoch for indicating learning quality.

As shown in Fig. 4D, a lower loss and a higher PSNR are observed in general for the standard model A2, suggesting that the lighter model A1 is underfitting. Similarly, Fig. 4E displays the learning result comparison between A2 and A3, and the generally same level of loss and PSNR are observed, indicating that the heavier model A3 is overfitting. It should be noted that the sharp curve change at epoch 100 is caused by the learning rate switching to a lower one. After 200 epochs, predicted pixel values from the standard model are converted to unsigned integers of 8 bits and then used to reconstruct the sunflower image (shown in Fig. 4Ciii). It is worth mentioning that predicted pixel values have some values above 255 for the R channel in the upper left and right corners, and some values below 0 for the B channel in the middle. After converting to integers, these overflow and underflow values incur significant discrepancies (e.g., 260 converted to 4, −5 converted to 251), shown with notably different patches. Thus, a filtering scheme that sets all overflow values to 255, and underflow values to 0 is adopted before conversion into integers. The reconstruction result after taking the filtering scheme is shown in Fig. 4Civ. PSNRs before (25.49) and after (25.52) the filtering scheme do not have much difference because the number of pixel values having significant discrepancies is small. The PSNRs is much larger than the one from the channel merging reconstructed image because sensor array configuration error and measurement noise are inherently adjusted and mitigated within the machine learning algorithms. As a result, the reconstructed result in Fig. 4Civ shows nearly no difference compared with the original image in Fig. 4Ci. The advantage of the machine learning algorithm over the typical channel merging method is further discussed in note S7. These results demonstrate a promising retina-inspired imaging sensor that combines the R/G/B perovskite NB sensor array (mimicking the R/G/B cone cells) with the neuromorphic algorithm (mimicking the intermediate network of the retinal system), which is capable of high-quality panchromatic imaging.

DISCUSSION

In summary, we demonstrate several fundamental breakthroughs in this study in realizing perovskite NB PD chips for panchromatic imaging, from material synthesis to device design to system innovation. Briefly, we successfully obtain the highly uniform R/G/B perovskite thin films (<2 μm) with heavily unbalanced electron-hole mobility (μ_h/μ_e ~ 5) based on a volatile solvent method. By inserting these films into a p-i-n architecture, the unbalanced electron-hole transport and the WDOFD effect synergistically leads to an NB detection with excellent performance in bandwidth, response speed, and optical linearity. Taking advantage of the good film uniformity, we devise R/G/B image sensing chips using laser-scribing techniques and collect R/G/B photocurrent signals independently. Inspired by the retinal system, we combine the R/G/B perovskite NB chips (mimicking the R/G/B cone cells) with a trilayer algorithm (mimicking the intermediate network of the retinal system) and successfully realized panchromatic imaging. It is also worth noting that our perovskite NB PDs constructed in the asymmetric p-i-n configuration can have NB detection under zero external bias. This light-driven panchromatic imaging may trigger further development in applications such as battery-free cameras, light-driven sensing, artificial retina replaceable for dead retina cells in vision damages, etc.

MATERIALS AND METHODS

Materials

Methylammonium bromide (MABr, 99.5%), methylammonium chloride (MACl, 98%) were purchased from Greatcell Energy Ltd. Lead (II) iodide (PbI₂, 99.99%), lead bromide (PbBr₂, 99.5%), lead chloride (PbCl₂, 99.99%), ACN, chlorobenzene (CB), 4-tert-butylpyridine, γ-butyrolactone, methylammonium solution and (ethanol-MA) (33% in absolute ethanol), were purchased from Sigma-Aldrich. PEDOT:PSS (CLEVIOS P VP AI 4083) was purchased from Heraeus. [6, 6] Phenyl-C61-butyric acid methyl ester (PCBM, 98%) was purchased from Nano-C Inc. ITO glass was purchased from Wuhan Jingge Technology Corp. All the chemicals are used as received without any further purification.

Preparation of perovskite solution

For the preparation of MAPb(BrI₂) solution, a mixture of 184.4-mg PbI₂ and 44.8-mg MABr was treated in CH₃NH₂ environment for 6 hours to obtain a perovskite intermediate in the liquid form. Following that, 100 μl of ACN and 100 μl of ethanol-MA were added into the liquid perovskite intermediate. The solution was then sonicated for ~60 s until a clear perovskite solution with a concentration of 2 M (2 mmol/ml) was obtained for the subsequent spin-coating process. For MAPbBr₃ solution, a mixture of 146.8-mg PbBr₂ and 44.8-mg MABr were used as precursors for a 2 M solution. It should be noted that using presynthesized MAPbBr₃ crystals instead of the precursor mixture is also good in generating high-quality film. Here, for simplicity and generality purpose, we use the precursor mixture to make the film. For MAPb(Br_1.5Cl_1.5), a mixture of 36.7-mg PbBr₂, 83.43-mg PbCl₂ and 44.8-mg MABr powders were used as precursors for a 2 M solution. Perovskite solutions with other compositions [MAPb(Br₃), MAPb(Br_2.5I_0.5), MAPb(Br₂I), MAPb(Br_1.5I_1.5), MAPb(Br_2.5Cl_0.5), MAPb(Br₂Cl), and MAPb(BrCl₂)] can be synthesized by tuning the ratio between precursors accordingly.

Imaging sensor fabrication

The imaging sensor employed a typical p-i-n device architecture. The device with an ITO/PEDOT:PSS/Perovskite/PCBM/Au layout was fabricated similarly with our previous procedure (45). In general, ITO glass substrate (dimension of 2.4 cm by 2.4 cm) was successively cleaned by ethanol, acetone, and deionized water in ultrasonication. Following that, the substrates were treated under ultraviolet (UV) light for plasma cleaning before use. The water solution of PEDOT:PSS was filtered by 0.45-μm Nylon filter and then spin-coated on ITO at 4000 rpm for 35 s. The film was then annealed at 150°C for 10 min to eliminate any moisture in the film. After this, the perovskite layer [MAPb(BrI₂), MAPb(Br₃), and MAPb(Br_1.5Cl_1.5)] was fabricated on PEDOT:PSS at 2000 rpm for 60 s, followed by spinning at 6000 rpm for another 60 s to evaporate the excess solvent and facilitate film crystallization. The film is then annealed at 120°C for 10 min. Following this, PCBM solution (10 mg/ml PCBM in CB) was spin-coated onto perovskite at a speed of 2000 rpm for 35 s. Subsequently, a 1-nm-thick Cr and 7-nm Au layer (16.3 ohms/sq) was sequentially deposited via thermal deposition as the top transparent electrode, according to our prior reports (26). A laser micromachining instrument (OpTek MM2500) is used to fabricate the PD array for the imaging sensor. Specifically, a UV laser (350 nm, 20 W) is used to scribe and separate the large-area PD device into an imaging sensor array containing 32 × 32 square-shaped pixels, with each pixel device of 0.6 mm by 0.6 mm in dimension. MAPb(BrI₂), MAPb(Br₃), and MAPb(Br_1.5Cl_1.5) imaging sensor arrays were fabricated for obtaining the signal of the red, green, and blue channel of a colorful object, respectively. For other array of different number of pixels, we use the same laser scribing method but different device geometric designs accordingly.

Materials characterization

SEM images of perovskite film and device cross section were obtained by a field-emission SEM instrument (Zeiss Merlin LEO 1530). High-resolution transmission electron microscopy images of perovskite crystal were captured by a dual aberration-corrected scanning/transmission electron microscope (S/TEM) operating at an accelerating voltage of 80 kV (FEI Titan³ G2 60–300). The energy-dispersive spectrum elemental mapping was measured with a SuperX EDS system under the scanning TEM mode. XRD data were collected on an x-ray diffractometer (Malvern Panalytical Empyrean) with Cu Kα radiation. UV-visible absorption spectra were collected on a HITACHI UH4150 spectrometer. PL spectra were acquired by using a fluorescence spectrometer (Edinburgh Instrument FLS 1000) at room temperature with 506-nm excitation from a Xenon arc lamp. TRPL measurements were performed using a picosecond pulsed diode laser [505-nm excitation laser for MAPb(BrI₂) and MAPb(Br₃) and 405-nm excitation laser for MAPb(Br_1.5Cl_1.5)] as the excitation source and a time-correlated single-photon counting detector for signal collection. Electrical measurement such as electron/hole mobility is performed with a Keithley 4200 source meter at room temperature in the dark.

Optical field simulation

Optical field distribution in the imaging sensor was simulated using the wave optics module of COMSOL Multiphysics software. Optical constants (refractive index, n and extinction coefficient, k) of the materials used in the device at different wavelengths are required for the simulation. Optical constants of these materials can be found from an online database (46) and published results (47, 48), which were used to run the simulation.

PD performance measurement

Response window and performance of red/green/blue PD based on MAPb(BrI₂)/MAPb(Br₃)/MAPb(Br_1.5Cl_1.5) were characterized by the EQE spectrum using a quantum efficiency measurement system (Newport QuantX-300) at ambient condition. The J-V curve of the PD devices in the dark was measured with a Keithley 4200 source meter. J-V curves of red/green/blue PD were measured under irradiation of LED with wavelength centered at 650, 550, and 480 nm, respectively. A neutral density filter is used to tune the light intensity of LED irradiation. For current density–time (J-t) measurement, a function generator that generates a square wave of desired frequency was used to modulate the LEDs. For the frequency-dependent response of the PD devices, the frequency of the LEDs irradiation varying from 1 to 100 kHz was modulated by controlling the pulse width of the square wave. The current response was recorded by a Tektronix MDO 3104 oscilloscope connected with a 50-ohm input impedance. The dark current of the PDs was recorded by a Keithley 4300 source meter. Noise current was extracted from the Fourier transform of the dark current. During the PD performance measurement, the devices are fully shielded by Al foils to maintain a dark condition.

Imaging sensors test

The R, G, B imaging sensor devices were stacked together (B sensor array on top, G sensor array in the middle, and R sensor array on the bottom) with six terminals, where a mini projector projects the full color image onto the imaging sensor device. The current of each pixel (32 × 32 = 1024 in total on each R, G, B imaging sensor array) was probed and recorded into three 32 × 32 matrixes corresponding to R, G, B channels, separately. The data matrixes are then input to the machine learning algorithm for signal processing and image reconstruction.

Image processing and reconstruction

Two different methods of (i) channel merging and (ii) machine learning are used for image reconstruction of the full color image. Specifically, (i) direct channel merging: the current values from each pixel of the NB PD array were normalized at first and linearly scaled to intensity values in the range of 0 to 255. After converting the floating-point values to integers, the image is directly reconstructed by stacking the pixel intensity integers from R/G/B channels together. For (ii) the machine learning: a multilayer perceptron neural network was adopted (shown in Fig. 1C), which takes current values as input and predicts pixel values as output accordingly. Three different neural network models (A1, A2, and A3) were tested. A2 is a standard model with 32-64-32 nodes in the three hidden layers. A1 refers to a lightweight model with 16-32-16 nodes in the three hidden layers, while A3 refers to a heavier model with 64-128-64 nodes in the hidden layers. The training dataset consists of 1024 pairs of current values (features) from the NB PD array and pixel values (labels) from the full-color sunflower image, which are used for the neural network model. The learning algorithms are trained with batch sizes of 1 using Adam optimizer with default beta1 0.9 and beta2 on a single RTX 2080 Ti GPU. The learning rate is initially set to 0.001 for the first 100 training epochs and decays to 0.0005 for the second 100 epochs. Smooth L1 loss is adopted to reflect the learning quality. Furthermore, PSNR, which shows the ratio between the maximum possible power of a signal (original sunflower image) and the power of the same image with noise (reconstructed image), is also calculated along with each training epoch for the purpose of indicating learning quality.

Supplementary Materials

This PDF file includes:

Notes S1 to S7

Figs. S1 to S32

Tables S1 to S5

References