Feline eye–inspired artificial vision for enhanced camouflage breaking under diverse light conditions

Abstract

Biologically inspired artificial vision research has led to innovative robotic vision systems with low optical aberration, wide field of view, and compact form factor. However, challenges persist in object detection and recognition against complex backgrounds and varied lighting. Inspired by the feline eye, which features a vertically elongated pupil and tapetum lucidum, this study introduces an artificial vision system designed for superior object detection and recognition in a monocular framework. Using a slit-like elliptical aperture and a patterned metal reflector beneath a hemispherical silicon photodiode array, the system reduces excessive light and enhances photosensitivity. This design achieves clear focus under bright light and enhanced sensitivity in dim conditions. Theoretical and experimental analyses demonstrate the system’s ability to filter redundant information and detect camouflaged objects in diverse lighting, representing a substantial advancement in monocular camera technology and the potential of biomimicry in optical innovations.

The feline eye–inspired artificial vision system can present camouflage-breaking performance under various light conditions.

INTRODUCTION

Vision-based operation strategies for autonomous vehicles, drones, and mobile robots have been rapidly revolutionized (1, 2). Rather than passively acquiring image data (e.g., shape, light intensity, and color), these robotic systems need to actively extract and analyze key visual information for object detection/tracking/recognition and perform autonomous decision-making for the next motion (3–5). However, these tasks become substantially difficult under diverse environments and illumination conditions (e.g., indoor and outdoor and daytime and nighttime). This variability can severely affect the contrast between target objects and their backgrounds, mainly due to pixel saturation under bright conditions and low photocurrent in dark conditions. Objects often create indistinct boundaries with their backgrounds, posing detection and differentiation challenges. These challenges can be further amplified when objects have textures similar to their backgrounds, leading to undesired camouflage effects (6). To minimize these issues, artificial vision systems typically adopt computer vision technologies such as high dynamic range imaging, postprocessing (i.e., spatial filter and deep learning methods) (7–10), and binocular vision–based camouflage-breaking techniques (fig. S1) (11). However, separating backgrounds and objects using these computer vision techniques requires high computational cost and large power consumption (12). Thus, there is a remarkable need for hardware that can break camouflage by itself.

Hardware-based approaches have been proposed for enhanced performance of the robotic visual systems, whose designs are inspired by unique structures and functions of natural eyes. In nature, animals have adapted themselves to ecologically complex environments for their survival. As a result, distinctive vision systems optimized for their habitats have been developed through long-term evolution. These natural vision systems could offer potential solutions to tackle limitations of conventional artificial vision systems (13, 14), in terms of depth of field (DoF), field of view (FoV), and optical aberrations (2, 15–19). For instance, the fish eye–inspired graded index ball lens provides a wide FoV within a small form factor (20, 21). Strategies to enhance light sensitivity have also been proposed, such as the design of light-trapping micromirrors inspired by the elephant-nose fish’s retina (21) and the strategy of logarithmic photoresponsivity inspired by the mantis shrimp’s eye (22). In addition, the “W”-shaped pupil, inspired by the cuttlefish eye, compensates for vertically uneven illumination and thus enhances the imaging quality (23). In this regard, we investigated natural vision systems optimized for camouflage breaking under varied illumination conditions, aiming to build advanced artificial vision systems with camouflage-breaking capabilities.

Some animals, like ambush predators with small body sizes, tend to have a vertical pupil (VP), as shown in table S1 (24–29). These pupils with elongated shapes have functional advantages, such as asymmetric DoF and high-definition focus on the targeted object (25–27, 30, 31). This enables easy detection of the prey even when it is camouflaged in natural environments. As another example, some animals active in dark environments, such as owls and nightjars, have a photonic structure (i.e., tapetum lucidum) behind their retina (32, 33), which serves as a biological light reflector. Thus, the retina can absorb both the incident light and the reflected light from tapetum lucidum, which enhances the visual sensitivity to recognize the target object under a dark environment (34–36). Domestic cats, which belong to crepuscular ambush predators mostly active at dawn and dusk (37), have both of these two unique features (i.e., VP and tapetum lucidum). Under bright conditions, the cat’s pupil constricts to form a vertical slit-like shape (38), which prevents dazzling by excess light while camouflage breaking is achieved (Fig. 1A). In dim conditions, the pupil fully dilates into a round shape to ensure sufficient light reception (38), and this fully dilated pupil is also favorable for camouflage breaking (Fig. 1B). In addition, tapetum lucidum reflects the incident light passing through the retina back to the retina (Fig. 1C). This is why the feline eyes glow at nighttime. Figure 1D illustrates the anatomical structure of the feline eye with its key optical components and varying DoF on each visual plane (i.e., sagittal and tangential plane). Figure 1 (E and F) illustrates the detailed difference in feline vision compared to conventional vision in both daytime and nighttime conditions. During the daytime, felines are able to focus on their target object, such as a rat, by maintaining a vertically blurred background with VP (Fig. 1E, right). In contrast, conventional vision, with small circular pupil (CP), often loses focus on the target due to excessive visual information from the complex background (Fig. 1E, left). In low light condition (i.e., nighttime), both feline and conventional vision can focus on their target object with fully dilated pupil, maintaining blurred background. However, only felines can see the target more clearly because their tapetum lucidum improves light sensitivity. Therefore, unlike conventional vision, the feline can distinguish target object from the complex background in both dim and bright environments.

Fig. 1. Structural and functional features of feline eyes.

(A and B) Schematic illustration showing the camouflage-breaking ability of a feline under diverse light condition. (C) Magnified schematic illustration of the tapetum lucidum in the retina. (D) Schematic illustration of the feline eye’s anatomy. (E and F) Schematic illustrations showing the visual ecology of feline and conventional vision during the daytime (E) and nighttime (F).

Here, we present an artificial vision system inspired by the feline eye. The artificial vision system consists of two components: an optical lens with custom-made apertures (e.g., elliptical, full-opening circular, and small circular shape apertures) and a hemispherical silicon photodetector array combined with patterned silver reflectors (HPA-AgR). Similar to the feline pupil, the variable aperture regulates the incoming light intensity and achieves asymmetric DoFs. The system excels in targeted imaging, aided by its unique VP that enhances the contrast of the object against the background, providing an effective solution for overcoming camouflage. The hemispherical shape of the photodiode array, resembling the feline retina, minimizes optical aberrations and reduces the need of complex lens configurations. Furthermore, the challenge of suboptimal photosensitivity resulting from the use of ultrathin silicon image sensors can be resolved by the secondary light absorption. The patterned Ag reflectors inspired from the tapetum lucidum are integrated behind the pixelated active regions, and this artificial tapetum lucidum achieves 52% enhancement in the photoabsorption efficiency in the visible light wavelength range. The system can adjust its imaging features, similar to how a feline eye adapts to its environment, and consistently maintain camouflage-breaking capability. Imaging demonstrations and theoretical simulations confirm the highly sensitive targeted imaging performance under various light environments, confirming the monocular camouflage-breaking feature of the feline eye–inspired vision system.

RESULTS

Optical configuration of the feline eye–inspired artificial vision system

Because of their unique iris dilation and contraction, felines can clearly perceive a target regardless of lighting conditions, distinguishing them from other mammals. For example, in bright environments, most mammals, including humans, have a small CP, resulting in a deep tangential DoF (T-DoF) and sagittal DoF (S-DoF). This condition produces clear images of both the target and the background (fig. S2A). Conversely, in low-light conditions, the dilated pupil aperture in these mammals leads to a narrower T-DoF and S-DoF, enhancing the distinction between the target and the background (fig. S2B). Therefore, the monocular vision system with CP struggles to differentiate between a target and its background in bright environments (fig. S3). In contrast, with its asymmetric aperture design, the feline eye maintains effective target-background differentiation under varying light conditions, even with a monocular system. The feline eye’s VP creates an asymmetric focusing ability between the tangential and sagittal planes (31). During the daytime, the VP’s narrow T-DoF ensures a marked contrast between targets at different distances (Fig. 2A). This mechanism enhances the perception of the target’s features by substantially reducing light intensity offset. At night, the VP also fully dilates to make narrower T-DoF and S-DoF (fig. S4). It maximizes the light intake and maintains a distinction between the target and the background.

Fig. 2. Optical simulation of the characteristics of the feline eye–inspired vision system.

(A) Schematic illustration of the feline eye in daytime with a VP for light adaptation. The yellow plane represents the tangential plane, and the blue plane represents the sagittal plane. (B) The cross-sectional schematic ray diagram of the feline eye–inspired vision system. (C) Optical simulation result of the cross-sectional focal spot according to object distance. The horizontal cross section represents the tangential plane (left), and the vertical cross section represents the sagittal plane (right). (D) Results of the ray tracing simulation for various object distances with small circular (i) pupils and VPs (ii). The cross-shaped object is located at 150, 200, and 250 mm. (E) The result of the ray tracing simulation for camouflage breaking with (i) small CPs and (ii) VPs. The center of the random texture is located at 200 mm, and the background random texture is located at 400 mm.

To design the feline eye–inspired vision system, we simulated imaging systems with different pupils (i.e., VP, small CP, and full CP) using the ray-tracing method (Fig. 2B and see fig. S5 for the pupil information). The optimized structural parameters of the artificial imaging system are as follows: a back focal length (BFL) of 29.9 mm and a radius of curvature (RoC) of the image sensor array of 17.6 mm, and a working distance of 200 mm. This configuration results in an optical system with a FoV of 16° (fig. S6). The lens information for the artificial imaging system is shown in table S2. Figure 2C exhibits a remarkable difference between DoFs of the feline eye–inspired vision system with VP. The DoFs are determined by circle of confusion. In conditions of asymmetric DoF, artificial astigmatism is induced within this system, resulting in distinct circles of confusion in the tangential and sagittal planes. The results of ray tracing from the vertical cross section, representing the T-DoF, show a narrow DoF attributable to the large aperture diameter. In contrast, the horizontal cross section, representing the S-DoF, exhibits deep DoF due to the narrow horizontal width of VP. The optical simulations with three different apertures (i.e., VP, small CP, and full CP) apparently display that VP and full CP integrated vision systems allow narrow DoF for distinguishing the target object and background (fig. S7, B and D). However, the vision system with small CP shows deep DoFs in horizontal and vertical cross sections (fig. S7C). Therefore, in bright environments, a conventional optical system design featuring light adaptation with small CP may encounter difficulties in separating the target object from the background.

Additional optical simulations visualize the influences of apertures. Figure 2D visualizes the asymmetric blurring effect caused by VP, as discussed in Fig. 2C, depending on the object’s position. The results from a small CP show a clear image of the cross-shape object irrespective of the object distances (Fig. 2Di). In contrast, the VP results in a vertically blurred image of the cross-shape object at distances of 150 and 250 mm. However, objects at the focal length of 200 mm are shown clearly (Fig. 2Dii). Last, Fig. 2E illustrates the camouflage-breaking capabilities of the feline eye–inspired monocular vision system compared to the system with CP under various lighting conditions (Fig. 2E and fig. S8). In the case of a pupil system with a small CP under high light conditions, an opening ratio of 3% results in both the background and target appearing clear, making it difficult to distinguish the boundary between them (Fig. 2Ei and fig. S8A). In contrast, with the same aperture area and a 3% opening ratio in the pupil system with VP, the asymmetric DoF results in a blurring effect on backgrounds lying outside the DoF. This effectively separates target objects from their backgrounds with similar textures, aiding in camouflage breaking (Fig. 2Eii and fig. S8B). In practical situations, existing contrast-based autofocus methods can be used in the feline eye–inspired vision system to focus on the object while blurring the background (figs. S9 and S10). Using the VP system, the dilated and constricted mechanisms enable easy breaking of an object’s camouflage, regardless of illumination intensity. The light adaptation strategy is detailed in Materials and Methods and figs. S11 to S13.

Ultrathin photodiode array with artificial reflectors

The curved form of image sensors integrated with a lens system offers numerous advantages, including low optical aberrations, simple system configuration, and miniaturization (20, 39–41). However, achieving curved image sensors relies on the use of ultrathin silicon photodiodes, which inherently has low light absorption due to the silicon’s indirect bandgap structure. There have been several studies to enhance the light absorption in the thin photosensitive layer by installing a reflector in front of or behind the layer (21, 42–46). However, these reflectors were manufactured in a planar form on the rigid substrates, limiting their compatibility with the curved sensor array (47–49). To address this issue, reflectors are patterned into a pixelated array, which corresponds to the active silicon photodetector regions.

Figure 3A shows a photographic image of the device integrated with the artificial reflector transferred onto a hemispherical surface. The hemispherical image plane was optimized to match the system’s focal plane, which has the optimum RoC of 17.6 mm. The inset shows an optical microscope image of the bare photodiode and its circuit diagram. The single pixel consists of a lateral photodiode and a blocking diode, serially connected as a NIPIN doping configuration. Each end of the diode is connected with a row and a column line. An exploded view and cross-sectional SEM image of the device contain detailed thickness information of each component (Fig. 3, B and C). Because of the ultrathin nature of each component, including silicon photodiodes (~1.25 μm), metal (Cr/Au) electrodes (~110 nm), polyimide (PI) encapsulation layers (1 to 1.5 μm), and metal (Ag) reflector (~100 nm), the entire device (~5.5 μm) can endure stress caused by structural deformation without mechanical breakdown (20, 50, 51). Therefore, the device exhibits steady response after the curvy shape deformation and could be conformally laminated on a concave hemispherical surface.

Fig. 3. Structures and optical characteristics of the silicon photodiode device.

(A) Photograph of the fabricated HPA-AgR. The inset shows an individual photodiode pixel with a circuit diagram. (B) Exploded structure of the device with a detailed thickness of each component. (C) Cross-sectional SEM image of the device. (D) Representative schematic illustration and optical microscope image (inset) of the fabrication process. Scale bars, 400 μm. (E) Glowing reflectors under low-light condition. (F) Simulated photoabsorptivity profile of the bare silicon and silicon with reflector in the visible spectrum range. (G) Measured spectral responsivity of the silicon photodiode with and without the reflector. Spectral responsivity was measured over a wavelength range of 350 to 1000 nm (H) I-V curves of the photodiode with and without the reflector. (I) Photosensitivity (I_photo/I_dark) of the photodiode with and without the reflector under illumination with various light intensities. Error bars represent SD of five individual pixels.

The device’s schematic illustrations and optical images are displayed in Fig. 3D, providing a representative overview during the main fabrication process. The Ag reflector was first deposited on the as-prepared PI substrate and patterned into an array (Fig. 3D, left). A doped silicon membrane was picked up from a silicon-on-insulator (SOI) wafer using a poly(dimethylsiloxane) (PDMS) stamp. After aligning the doped Si membrane through the transparent stamp and transferring it to match the position of patterned reflectors, the undoped region was etched away to isolate the active area (Fig. 3D, middle). Then, the electrodes were vacuum-deposited and patterned to make contact to the n-doped region of each side of the pixel (Fig. 3D, right). Here, the top of the blocking diode was concealed with the metal electrode to prevent unwanted light absorption in the area. Similar to the glowing effect observed in the feline eye, the optical image reveals that the artificial reflector glows in low-light conditions (Fig. 3E). The detailed fabrication process of the device is described in Materials and Methods and fig. S14.

When the photodiode absorbs the incident light, the transmitted light is reflected back to the photodiode by the reflector, leading to a secondary absorption, which is similar to the secondary absorption occurring at the tapetum lucidum of the feline eye. Figure 3F exhibits the simulated optical efficiency considering the absorption and responsivity of silicon and irradiance of the light source (fig. S15). Absorptivity of silicon with the Ag reflector exhibits ~58% enhancement compared to the bare silicon. Transmitted light begins to occur at the wavelength longer than 500 nm for the thin silicon owing to its low absorption coefficient. The longer wavelengths that are unabsorbed are compensated for by secondary absorption. The spectral responsivity of each state (i.e., with and without Ag reflector) was also measured in a wavelength range of 350 to 1000 nm (Fig. 3G). The photodiode with integrated back reflector exhibits higher responsivity than photodiode without reflector at wavelength longer than 500 nm, exhibiting the highest value of 0.218 A/W at a wavelength of 630 nm. The total responsivity of photodiode, represented by the integral value of the graph, with the reflector (71.69 A nm W⁻¹) was 52% higher than that of photodiode without the reflector (47.23 A nm W⁻¹), indicating that the experimental results and theoretical calculation values are in excellent agreement.

To evaluate the performance of the silicon photodiode, its dynamic range was also measured. The I-V curves of the silicon photodiode with and without the reflector under various light intensities (0 to 1.52 × 10⁵ W m⁻²) were measured (fig. S16). The photocurrent values at a 1-V bias were compared (fig. S17), and the linear dynamic range (LDR) of the photodiode with and without reflector was calculated to be approximately 87.30 and 86.27 dB, respectively (Supplementary Text). Although the LDR values were similar regardless of the reflector, the photodiode with the reflector showed a photocurrent response at a much lower light level (7.14 × 10⁻³ W m⁻²) compared to the photodiode without the reflector (1.39 × 10⁰ W m⁻²). The I-V curves (Fig. 3H and fig. S18) and photosensitivity (I_photo/I_dark) (Fig. 3I) graphs verify that the photodiode with the reflector can detect the amount of incident light more sensitively than the photodiode without the reflector. Error bars in Fig. 3I represent SD of five individual pixels, and detailed data values are presented in table S3.

Imaging demonstration for enhanced camouflage breaking

For the imaging demonstration, the feline eye–inspired imaging system was constructed on the basis of the design mentioned above (Fig. 4A). Detailed descriptions of imaging demonstration procedures are available in Materials and Methods (fig. S19). The system comprises changeable apertures (i.e., small CP and VP) and HPA-AgR. These two types of apertures are primarily used to demonstrate imaging differences between conventional vision systems and the feline eye–inspired vision system during daytime.

Fig. 4. Imaging demonstration of the feline eye–inspired vision system.

(A) Schematic illustration showing the artificial feline eye–inspired vision system. (B) The obtained image according to various object distance (d_obj) with artificial feline eye–inspired vision system equipped VP and small CP. (C) The obtained image (i) and its gradient map (ii) with a small CP. The mouse object is positioned at d_obj = 200 mm, and the line pattern background is located at d_bg = 300 mm. (D) The obtained image (i) and its gradient map (ii) with a VP. The mouse object is positioned at d_bg = 200 mm, and the line pattern background is located at d_bg = 300 mm. (E and F) The used image of measurement with the GT image and noisy GT of letter object (i.e., F, O, C, U, and S) obtained image (E) and gradient map (F) with a small CP and VP. The letter object is positioned at 200 mm, and the pseudo-random background is located at 350 mm.

To confirm the optical characteristics and focusing performance, the crossbar object at various distances (i.e., 150, 200, and 250 mm) is measured using two types of apertures (i.e., small CP and VP) as shown in fig. S20. The imaging system with a small CP presents a clear cross-shape object image due to its deep T-DoF and S-DoF (Fig. 4B, top) for all tested distances. However, in the imaging system with VP, a clear cross image appears only at 200 mm. Other distances outside the DoF range cause vertically blurred images due to narrow T-DoF (Fig. 4B, bottom).

The vertically blurring effect of the feline eye–inspired imaging system remarkably diminishes background textures located outside the DoF. This phenomenon enhances the contrast between the target objects and the background, enabling effective camouflage breaking. Additional optical measurements were performed to demonstrate this feature using a mouse shape object at 200 mm and a line pattern background at 300 mm. In the imaging system with a small CP, the mouse and the line pattern in the background appear (Fig. 4Ci). However, this clarity, regardless of distance, presents a challenge in distinguishing the background from the target, as shown in the gradient map of measured image (Fig. 4Cii). In contrast, in the imaging system with a VP, the mouse image is highlighted because the object is located within both the T-DoF and S-DoF, while the background lies outside of the T-DoF (Fig. 4D, i and ii). Furthermore, the imaging system equipped with a VP demonstrates substantially higher contrast between the target and backgrounds, including both directional patterns, such as line textures and pseudo-random textured backgrounds. To validate these advantages, the letter shape objects (i.e., “F,” “O,” “C,” “U,” and “S”) are used [Fig. 4E, ground truth image (GT) and noisy GT row]. In the case of the small CP, the letters are shown with a mixed background pattern owing to the deep DoF of the imaging system (Fig. 4E, CP row). However, the imaging system with VP captures the target letter objects more clearly, but the background pattern is unnoticed owing to narrow T-DoF (Fig. 4E, VP row). The feature of edges, represented by the gradient map, highlights the difference in perception between the VP and small CP (Fig. 4F). The gradient map of the small CP presents the edges of background patterns. In contrast, the gradient map of the VP shows concentrated edges at the letter area.

Assessing object recognition capabilities of the feline vision system

Artificial vision systems used in various applications, such as unmanned aerial vehicles, autonomous cars, and humanoid robots, perform a range of image-processing tasks for object tracking/detection and recognition (52). However, in scenarios involving object tracking/detection and recognition, such tasks are more often given under noisy backgrounds and environments than those with objects alone (Fig. 5A). Although computer vision and deep learning algorithms have substantially improved handling of noisy targets, the feline eye–inspired vision system provides intrinsic advantages originated from hardware. The feline eye–inspired artificial vision inherently induces background blurring and camouflage breaking, which can markedly reduce the computational burden.

Fig. 5. Object recognition with a noisy background of the feline eye–inspired vision system.

(A) Schematic illustration enhanced recognition with background blurring and camouflage breaking of feline vision system. (B) The diagram of convolutional neural network (CNN) and fully connected layer (FC) for MNIST and Fashion-MNIST dataset. The CNN is trained with 50,000 datasets. The input image size for the CNN is 28 × 28 pixels, and the output consists of 10 labels each for the MNIST and Fashion-MNIST datasets, respectively. (C) Optical simulation results for the MNIST dataset with small CP and VP for each label. (D) The calculated accuracy rates for the MNIST dataset from image simulations, both with and without noise. (E) Optical simulation results for the Fashion-MNIST dataset with small CP and VP for each label. (F) The calculated accuracy rates for the Fashion-MNIST dataset from image simulations, both with and without noise. (G) The graph of accuracy rates according to epoch for the MNIST dataset (left) and Fashion-MNIST dataset (right). (H) The confusion matrix for the MNIST dataset (left) and Fashion-MNIST dataset (right).

Conventional computer vision algorithms for object tracking were used to assess performance differences based on the shape of the aperture. Seven object tracking algorithms were applied to the images (figs. S21 to S23). Notably, the imaging system equipped with a VP demonstrated remarkably higher accuracy, outperforming the CP in five of seven metrics by a factor of more than 1.5. For further demonstration of object recognition capabilities, a convolutional neural network (CNN) method is used for quantification of the object recognition capability. The CNN recognition model is trained using the Modified National Institute of Standards and Technology (MNIST) and Fashion-MNIST datasets, which feature images without noisy backgrounds (Fig. 5B and fig. S24). Ray-tracing simulations were conducted to generate 10 noisy MNIST and Fashion-MNIST datasets for each label, enabling a comparison of the recognition performance between vision systems equipped with VP and small CP (figs. S25 to S28). For the initial demonstration of the object recognition, the MNIST dataset, which consists of binary data, was used to evaluate the performance of the VP system (Fig. 5C). With the simulated images, accuracy was calculated using a set of 100 images, where each label consists of 10 images. In conditions without background interference, vision systems equipped with a VP and a small CP achieve similar accuracy rates of 96.67 and 97.78%, respectively. However, since the CNN is trained on both the number area and the entire image area, the presence of background leads to a substantial difference in accuracy. Specifically, the imaging system equipped with a VP demonstrates a higher accuracy rate of 94.44%, compared to 88.8%, with the imaging system featuring a small CP (Fig. 5D).

The Fashion-MNIST dataset is used to demonstrate scenarios with redundant grayscale information, such as light saturation and shape information, providing a distinct contrast to the MNIST dataset (Fig. 5E). In conditions without background interference, vision systems equipped with a VP and a small CP achieve similar accuracy rates, with a difference of approximately 2%. Under noisy background conditions, however, the VP system provides more than 10% higher accuracy than the small CP system (Fig. 5F). The imaging system with VP enhances contrast between the target object and background, leading to higher accuracies for both the MNIST and Fashion-MNIST datasets than the small CP equipped system across nearly 50 epochs, as shown in Fig. 5G. At 50 epochs, vision systems equipped with VP exhibit the highest predictive capability for each label, effectively demonstrating their performance in object recognition with background interference, regardless of whether in grayscale or binary, as illustrated in Fig. 5H.

DISCUSSION

Feline eyes, characterized by their elliptically contracted pupil and biological reflector, have inspired remarkable advancements in realizing sensitive artificial vision systems optimized for camouflage breaking and targeted object detection. These feline eye features are key survival tools of felines, allowing for the blurring of unnecessary visual information and improved light sensitivity, thus enabling them to focus on the targeted prey without additional energy consumption required for processing excess visual data.

In our study, we developed a feline-inspired vision system that is optimized to effectively detect and capture target objects against complex backgrounds with minimal energy expenditure. This is achieved by integrating two optical features: tailored apertures and backside reflectors. A single lens, the artificial apertures, and the HPA-AgR were then integrated within a housing to construct the artificial vision module. This integrated module, featuring a VP, efficiently blurs background objects while maintaining focus on the target object. The effectiveness of the VP was further corroborated through a CNN recognition model, which showcased enhanced recognition abilities and accuracy in comparison to a system equipped with a small CP.

However, for industrial applications such as object recognition and tracking, the primary challenges are increasing the pixel density and improving optical characteristics in terms of pixel resolution and FoV. The low-pixel resolution mainly arises from the limitations of fabrication equipment in university laboratories. The pixel fabrication scale at university laboratories is constrained to hundreds of micrometers due to photolithography equipment, whereas commercial flat complementary metal-oxide semiconductor image sensors can achieve pixel scales of a few micrometers. To address this limitation, scanning method, which mechanically rotate the device to capture and merge images into a high-resolution image, has been introduced to overcome the low pixel resolutions, but this often reduces the system’s frame rate, impeding real-time imaging performance. Nonetheless, recent research to implement high-resolution curved image sensors has been actively conducted at the industry level (53–55). Although the curvature of these systems is not yet as high as that of biological vision systems, it is expected that commercially available curved image sensors with higher curvature while maintaining high resolution will be further developed.

Another challenge of our system is the narrow FoV in the feline eye–inspired vision system. The narrow FoV can be addressed by mimicking the head and eye movements observed in animals (56). Methodologies and implementations using mechanically driven camera position adjustment with servo motors have been developed to overcome the limitations of a narrow FoV. In addition, our system is integrated with a curved image sensor, enabling the potential configuration of an artificial eyeball. Thus, methodologies that allow for the rotation of the eyeball itself, mimicking extraocular muscles, can be applied to expand the FoV (57).

With the optimistic advancement of technology, our artificial vision system holds great potential for facilitating the deployment of mobile robots to a variety of unconventional robot applications by replacing humans. Representative examples include unmanned vehicles, surveillance robots, and military drones. With the aid of our artificial vision system, these mobile robotic systems would be able to effectively detect, track, and recognize target objects in dynamically changing environments with complicated backgrounds. As a result, the accuracy and success rate of the robots’ tasks can be remarkably enhanced, which will advance to a society where humans and robots coexist.

MATERIALS AND METHODS

Fabrication of ultrathin photodiode array

A thin film of a PI layer was coated on 3.5 cm by 3.5 cm size SiO₂ wafer by spin coating of polyamic acid solution (12.8 wt % in 80% N-methylpyrrolidone/20% aromatic hydrocarbon solution, Sigma-Aldrich). Ag (100 nm) was deposited on PI and wet-etched to form a patterned reflector. In addition, PI was spin-coated on the top of the reflector and half-cured to prepare a substrate. SOI wafer (1.25-μm-thick silicon layer on a silicon dioxide insulator layer, Soitec) was doped with spin-on-dopant using thermal diffusion. The n-type spin-on-dopant (P509 solution, Filmtronics) was spin-coated and annealed at 200°C for 15 min, followed by thermal diffusion at 975°C for 12 min. The p-type spin-on-dopant (B153 solution, Filmtronics) was spin-coated and annealed at 200°C for 15 min, followed by thermal diffusion at 975°C for 30 min. Before each doping process, a SiO₂ layer was deposited on the SOI wafer using plasma-enhanced chemical vapor deposition, and only the desired region was etched using photolithography. The remaining unetched SiO₂ served as a diffusion barrier. As a result, the individual photodiode had n-p-n configuration with serial connection. After doping, the SOI wafer was immersed in concentrated hydrofluoric acid to remove the buried SiO₂ layer. The doped regions were aligned and transferred onto the as-prepared substrate using a transparent PDMS stamp. The doped regions were isolated into an individual cell by reactive ion etching with photolithography. PI was spin-coated on the top of the diode array as the first encapsulation layer. Through photolithography and dry etching process, a vertical interconnect access (VIA) pattern was formed, and Cr/Au layers (10/100 nm) were deposited on it by thermal evaporation. The Cr/Au layers were wet-etched, forming the first electrode. Additional PI was spin-coated and cured as an intermediate dielectric layer. Second VIA patterning and the deposition of Cr/Au for second metallization were conducted by using the same process as first metallization. The top of the device was encapsulated with 1-μm-thick PI, and then the entire PI was etched into the final device pattern with a reactive ion etching process. The fabricated device was picked up using a water-soluble tape (3M Corp.) and transfer-printed onto the hemispherical PDMS mold. After removing the water-soluble tape, a heat-seal connector (Uniquest Corp.) and a custom-made printed circuit board were connected to each contact pad of the electrodes.

I-V characterization of a single photodiode with artificial tapetum

The I-V characterization was performed by applying DC voltage from −2 to 2 V, and the corresponding current was measured by a semiconductor device analyzer, B1500A (Agilent Technologies, USA). Probe station (MST-5500B, MS Tech, Korea) connected the image sensor array and the semiconductor device analyzer. The halogen lamp with 3100 K color temperature (FOK-100W, Fiber Optic Korea, Korea) was used to measure performance of photodetector. Also, we measured wavelength-dependent performance with Xenon light source (66902, Newport, USA). The dynamic range of the photodetector was measured using a 532-nm laser.

Imaging demonstration of the feline eye–inspired artificial vision

A customized imaging setup was used to validate the optical properties of a feline eye–inspired artificial vision system, particularly for applications such as camouflage breaking. Customized data acquisition (DAQ) board is used to measure photocurrent of each photodiode (fig. S19). In our feline-inspired vision system, the maximum dynamic range can be represented by the bit depth of the DAQ board. Our system features a 24-bit depth for light quantization, allowing it to represent 2²⁴ = 16,777,216 levels of brightness. This bit depth translates to a theoretical maximum dynamic range (DR) of

$${\text{DR}}_{\text{bit}}=20{\text{log}}_{10}\left(2^{24}\right)=20{\text{log}}_{10}\left(16,777,216\right)\approx 144.5\text{dB}$$

A white light source was used to simulate objects and their backgrounds. Two types of setups were used (fig. S20). Each setup included a black shadow mask and a diffuser film attached to the white light source, demonstrating the focusing features of both vertical and CPs. Using this custom light source, we captured images of a cross-shaped object at various distances (150, 200, and 250 mm) to estimate the degree of blur, thereby assessing the DoF. In addition, the background and target were imaged using the light source equipped with the black shadow mask and diffuser film. A shadow-masked target object was positioned near the imaging device.

Optical simulation for the design and analysis of the aperture shape

The geometrical structures of the vertical and circular apertures were designed and simulated using a Monte Carlo–based ray tracing program (Optics Studio 2023, Radiant Zemax). To ensure a fair comparison under identical conditions, both the small CP and the VP were designed with the same aperture area, maintaining consistent light illumination. The radius of the small CP is 1.414 mm. The VP is shaped as an ellipse, with a major axis (a value) of 8 mm and a minor axis (b value) of 0.25 mm. To estimate the optical properties, such as the degree of blur, associated with the two types of pupils, the distance between the BFL was varied from 150 to 250 mm. To simulate the capability of camouflage breaking, the pseudo-random texture is located at 300 mm. Furthermore, the simulation used the MNIST dataset, positioned at the distance of 200 mm, along with a randomly textured background located at 400 mm (figs. S25 to S28). This setup was used to estimate the efficiency and accuracy of object recognition.

Light adaptation for feline eye–inspired vision system with VP

To theoretically confirm the changes in light reaching the image sensor depending on the aperture, we simulated the power of light reaching the image sensor while varying the input light intensity of the object from 0 to 500 lumens (fig. S11). For comparison, we assumed that the relative output power is 0 to 1, where a value of 1 indicates pixel saturation. In the full CP, pixels are saturated at 8.89 input lumens, while in the 3% open VP, pixels are saturated at 274.47 input lumens (fig. S11B). The optical system of the feline-inspired vision system prevents pixel saturation by adjusting the aperture to the VP for strong light intensity and to the CP for weak light intensity (fig. S12). This capability optimizes the sensor’s performance and enables the use of high dynamic range image processing techniques in postprocessing, further extending the effective dynamic range of captured images. To illustrate the concept of a mechanical aperture system, in addition, we have fabricated a prototype featuring reconfigurable aperture (fig. S13).

Supplementary Materials

This PDF file includes:

Supplementary Text

Figs. S1 to S28

Tables S1 to S3