Biologically inspired microlens array camera for high-speed and high-sensitivity imaging

Abstract

Nocturnal and crepuscular fast-eyed insects often exploit multiple optical channels and temporal summation for fast and low-light imaging. Here, we report high-speed and high-sensitive microlens array camera (HS-MAC), inspired by multiple optical channels and temporal summation for insect vision. HS-MAC features cross-talk–free offset microlens arrays on a single rolling shutter CMOS image sensor and performs high-speed and high-sensitivity imaging by using channel fragmentation, temporal summation, and compressive frame reconstruction. The experimental results demonstrate that HS-MAC accurately measures the speed of a color disk rotating at 1950 rpm, recording fast sequences at 9120 fps with low noise equivalent irradiance (0.43 μW/cm²). Besides, HS-MAC visualizes the necking pinch-off of a pool fire flame in dim light conditions below one thousandth of a lux. The compact high-speed low-light camera can offer a distinct route for high-speed and low-light imaging in mobile, surveillance, and biomedical applications.

Nine thousand one hundred twenty fps biologically inspired low-light camera has been developed.

INTRODUCTION

The vision of fast-eyed insects such as Drosophila and Odonata plays an essential role in evading threats, foraging, and navigating under assorted light conditions (1–7). Insect eyes distinctly use multiple optical channels called ommatidia, unlike single-chambered human eyes (8, 9). Visual stimuli from individual optical channels are separately and parallelly processed via the lamina and medulla in the optic lobe. This parallel processing in corresponding sensing neurons effectively increases the update rate for motion detection in the lobula complex (10, 11). Each small facet lens in ommatidia nonetheless hinders capturing sufficient photons (12). In particular, some nocturnal and crepuscular insects, such as Bombus, distinctly use temporal summation of visual signals to increase light sensitivity for individual optical channels (13–16). They integrate the visual signals over time to improve visual reliability in darkness at the cost of motion artifacts (17). Both unique features help fast-eyed insects thrive with high photon sensitivity in low-light conditions (18).

Compact cameras mimicking insect vision exhibit wide field-of-view (FOV), low aberration, high temporal resolution, and multifunctional imaging (19–22). Artificial compound eye cameras feature spherical arrangements of multiple optical channels for notable wide FOV (23–25). Ultrathin microlens array cameras (MACs) inspired from the Xenos peckii vision demonstrate high-resolution and high-sensitivity imaging (26–28). Distinct optical features within multiple channels of insect vision (21, 22) lead to the creation of polarization (29) or multispectral cameras (30–33). Multiple apertures of insect eyes further inspire high-speed cameras by sequentially acquiring subframes from a single frame captured using a rolling shutter complementary metal-oxide semiconductor (CMOS) image sensor (28, 30, 34–36). This approach offers advantages over traditional high-speed cameras with bulky sizes or complex circuits (37–42) in machine vision, even in neuroscience and health care applications (43–45). However, these cameras have technical restrictions for substantially increasing the frame rate over orders of magnitudes and still suffer from rolling shutter distortions for high-speed imaging. They also struggle with photon capture in low-light conditions due to the small aperture. These features pose challenges for achieving high-speed and high-sensitivity imaging.

Here, we report high-speed and high-sensitive MAC (HS-MAC), inspired by insect’s vision. A fast-eyed insect detects swiftly moving objects at a distance in dim light, wherein reflected light sequentially activates photoreceptors in discrete optical channels (Fig. 1A). A series of visual stimuli collected from each channel undergo parallel processing and temporal summation while traversing the lamina and medulla of the optic lobe (46). Parallel processing allows a high update rate for high-speed imaging, and temporal summation further enhances light sensitivity in low-light conditions. Similar to the insect’s vision, HS-MAC performs high-speed and high-sensitive imaging by combining channel fragmentation, temporal summation, and compressive frame reconstruction process (Fig. 1B). Offset microlens arrays (oMLAs) packaged with a single rolling shutter CMOS image sensor allow the high-speed imaging. oMLAs in optical channels project swiftly moving objects onto offset patterned channels with a low signal-to-noise ratio (SNR) in dim conditions. Sequential row-wise exposure of the rolling shutter image sensor divides the individual channels into discrete fragmented arrays, which is defined as channel fragmentation. Spatial information in an ultrashort event is preserved by the fragmented arrays, much above the conventional frame rate. Each fragmented array is then temporally summed during the overlapping period with the adjacent arrays, yielding high SNR fragmented channel images due to the superposition of temporal information. The fragmented channel images (a, b, and c) on a single fragmented array image contain different spatial information about swiftly moving objects due to oMLAs and are stitched together to form a single frame. The compressive image reconstruction process further deblurs the frames by segmenting the overlapped information. Microlens with small offset allows for precise channel fragmentation and substantially improves the frame rate by more than three orders of magnitude within the limited bandwidth. Furthermore, temporal summation and compressive frame reconstruction enhance camera sensitivity in low light by overlapping temporal information, breaking the trade-off between SNR and frame rate.

Fig. 1. Schematic illustration for high-speed and HS-MAC.

(A) Vision in a fast-eyed insect. Reflected light from swiftly moving objects sequentially stimulates the photoreceptors along the individual optical channels called ommatidia, of which the visual signals are separately and parallelly processed via the lamina and medulla. Each neural response is temporally summed to enhance the visual signals. The parallel processing and temporal summation allow fast and low-light imaging in dim light. (B) High-speed and HS-MAC. HS-MAC incorporates channel fragmentation, temporal summation, and compressive frame reconstruction. oMLAs projects swiftly moving objects on a rolling shutter CMOS image sensor via corresponding optical channels. Row-wise exposure horizontally fragments channels into fragmented arrays. The temporal summation of each fragmented array enhances SNR in low-light conditions, resulting in overlapping temporal information in adjacent frames. The frame components of a single fragmented array image are stitched into a single blurred frame, which is subsequently deblurred by compressive image reconstruction.

RESULTS

Fully packaged HS-MAC and comparison to offset-free MACs

HS-MAC was fully packaged with oMLAs (offset distance: 15.5 μm) and a single rolling shutter CMOS image sensor (IMX 477, SONY Corp.) within a compact size (1.53 mm × 8.35 mm × 10.5 mm, 0.3 g) (Fig. 2A). oMLAs with various offset distances were also microfabricated and integrated into HS-MACs to capture array images (Fig. 2B). Each microlens was designed and fabricated to prevent image overlap (26). The image size reduces as the offset distance decreases, allowing for the precise channel fragmentation and enhancing the frame rate. The frame component stitching diagram including the design parameters of oMLAs and HS-MAC is described in fig. S1. The performance of HS-MAC was compared with that of an offset-free MAC. No distortion correction or deblurring process was applied, except for minimal image processing for frame reconstruction, to compare HS-MAC and MAC. Computational resources required were not noticeably different. We also defined the working distance of the camera to capture images in an environment without disparity between microlenses (fig. S2), thereby disregarding depth information. Both MAC and HS-MAC were set to the same working distance. The frame rate of HS-MAC and MAC—both exhibiting unit cell sizes of 750, 493, and 284 μm—are represented in Fig. 2C. HS-MAC shows a remarkable high frame rate as the unit cell size decreases compared to that of MAC. Notably, HS-MAC achieves 9120 fps, which exceeds MAC’s frame rate (480 fps) by a factor of 19. The frame rate of the HS-MAC is 91,200 fps, where the offset distance is equal to the size of a single pixel. Both HS-MAC and MAC use rolling shutter image sensors, which often cause rolling shutter distortion. HS-MAC exhibits significantly lower rolling shutter distortion unlike MAC (Fig. 2D). A free-falling ball with a diameter of 78 mm was captured by HS-MAC and MAC with the same shutter speed, where the exposure time is the reciprocal of MAC’s frame rate. The distortion was calculated by the ratio of the major to the minor axis of the captured ball. The major-to-minor axis ratio of HS-MAC is closer to the ideal value of one compared to MAC, and as the unit cell size decreases, the ratio approaches one. The mean absolute percentage error of HS-MAC is 4% when the unit cell size is 284 μm, indicating a significant decrease of more than 50% in comparison to the error of MAC (10%). This experimental result implies that HS-MAC distinguishes the ball as a nearly spherical shape. This is because HS-MAC captures a single frame in a shorter time compared to MAC by precisely fragmenting the image sensor using the offset. The image quality was also compared between HS-MAC and MAC. Still, pepper images were captured by using both cameras with a unit cell size of 284 μm at a constant exposure time (2.08 ms) (Fig. 2E). The Pearson correlation coefficient (PCC) between the captured images is 0.99, indicating a high degree of similarity. The imaging performance of HS-MAC is equivalent to that of MAC in terms of the measured modulation transfer function (MTF), as shown in Fig. 2F. HS-MAC demonstrates high-speed imaging with low distortion and comparable image quality to the MAC (see figs. S3 and S4 and table S1 for detailed specifications of HS-MAC).

Fig. 2. Fully packaged HS-MAC and comparison to an offset-free MAC.

(A) HS-MAC packaged with an oMLAs having a 15.5-μm offset distance. An enlarged image of a portion of the oMLAs is displayed on the left side. White dashed lines indicate grids to identify the offset. (B) Microfabricated oMLAs and captured array images. Each scale bar provides a consistent reference across all images regardless of the offset. (C) The frame rate of HS-MAC and offset-free MAC as a function of the reciprocal of the unit cell size. The solid line represents the calculated frame rate. HS-MAC exhibits higher frame rate than MAC at equivalent unit cell sizes. HS-MAC achieves remarkable frame rate of up to 91,200 fps, where the offset distance is equivalent to the size of one pixel (corresponding unit cell size: 98.4 μm). (D) Photographs of a free-falling ball using MAC and HS-MAC (unit cell size = 284 μm). MAC and HS-MAC exposed the scene during the reciprocal of MAC’s frame rate. The shape of the ball appears vertically elongated due to the rolling shutter distortion. The ratio of the major axis to the minor axis of the ball was analyzed with different unit cell sizes. Unlike MAC, HS-MAC maintains a nearly spherical shape due to small distortion. (E) Standstill pepper images captured by MAC and HS-MAC (unit cell size of 284 μm, MAC = 480 fps, HS-MAC = 9120 fps). The image of MAC was obtained by averaging the channels along the same horizontal axis. The similarity between MAC and HS-MAC is consistently high (PCC = 0.99). (F) Comparison of the MTF between MAC and HS-MAC. HS-MAC exhibits high-speed imaging with low distortion and comparable MTF to the MAC. a.u., arbitrary unit.

HS-MAC–driven low-light imaging

HS-MAC uses temporal summation, which substantially improves light sensitivity at a constant frame rate. Conventional single lens high-speed cameras restrict the maximum exposure time to one over frame rate, as the photodetector stops the exposure before capturing a subsequent frame. In contrast, HS-MAC breaks this trade-off between the frame rate and the exposure time and thus enhances the light sensitivity by overlapping the exposure time across the subsequent frames for the temporal summation. Temporal summation factor (TSF) is defined as a ratio of HS-MAC’s exposure time ${\mathrm{\tau }}_{\mathrm{e}}$ to the maximum exposure time of a single-lens camera at the same frame rate ${\mathrm{\tau }}_{\mathrm{c}}$ (Fig. 3A). The compressive image reconstruction algorithm was used to restore deblurred frames $\mathbf{f}\in {\mathbf{R}}^{d\times (\mathit{mn})}$ from blurred temporally summed (blurred TS) frames $\mathbf{g}\in {\mathbf{R}}^{c\times (\mathit{mn})}$, where a frame size is m × n and the number of frames are d and c, respectively. g is expressed as a linear forward model as follows

$$\mathbf{g}=\mathbf{\text{Hf}}$$ (1)

where $\mathbf{H}\in {\mathbf{R}}^{c\times d}$ is a sensing matrix (Fig. 3B) (38). For more information about the sensing matrix, see fig. S5. Recovering f directly from g is a challenging task due to the instability of an ill-conditioned problem (47). Instead, $\widehat{\mathbf{f}}$ is obtained by solving the optimization problem as follows:

$$\widehat{\mathbf{f}}=\underset{\mathbf{f}}{\text{argmin}}\frac{\mathrm{\mu }}{2}{\Vert \mathbf{g}-\mathbf{\text{Hf}}\Vert }_2^2+R(\mathbf{f}),$$ (2)

where R is the regularization function and $\mathrm{\mu }$ is the regularization parameter. L1-norm–based total variation (TV) was used for regularization (48–50). A constrained TV deblurring model with the alternating direction method of multipliers was used to address the optimization problem associated with $\mathbf{f}$, where the pixel values lie within a certain dynamic range [$\mathbf{l}$, $\mathbf{u}$] (47, 51). The optimization problem is formulated as follows:

$$\underset{\mathbf{f},\mathrm{\theta },\mathbf{y}\in \mathrm{S}}{\text{min}}\frac{\mathrm{\mu }}{2}{\Vert \mathbf{g}-\mathbf{\text{Hf}}\Vert }_2^2+\text{TV}(\mathbf{\theta }):\mathbf{\theta }=\mathbf{f};\mathbf{y}=\mathbf{f}$$

$$\mathrm{S}=\{\mathbf{x}\in {\mathbf{R}}^{d\times \mathit{mn}}\mid \mathbf{l}\le \mathbf{x}\le \mathbf{u}\}$$ (3)

where θ and y indicate auxiliary variables. The augmented Lagrangian function of (4.3) is defined as follows

$$\mathrm{L}\equiv \frac{\mathrm{\mu }}{2}{\Vert \mathbf{g}-\mathbf{\text{Hf}}\Vert }_2^2+\text{TV}(\mathbf{\theta })$$

$$-{\mathbf{a}}^{\mathrm{T}}(\mathbf{\theta }-\mathbf{f})+\frac{{\mathrm{\beta }}_1}{2}{\Vert \mathbf{\theta }-\mathbf{f}\Vert }_2^2-{\mathbf{b}}^{\mathrm{T}}(\mathbf{y}-\mathbf{f})+\frac{{\mathrm{\beta }}_2}{2}{\Vert \mathbf{y}-\mathbf{f}\Vert }_2^2\mathit{}$$ (4)

where T denotes the transpose of the vector. The detailed algorithm is documented in table S2 and fig. S6.

Fig. 3. Low-light imaging with temporal summation.

(A) Temporal summation for HS-MAC. TSF ${\mathrm{\tau }}_{\mathrm{e}}/{\mathrm{\tau }}_{\mathrm{c}}$ is a ratio of the exposure time of HS-MAC (${\mathrm{\tau }}_{\mathrm{e}}$) to the maximum exposure time of a single lens camera under the same frame rate (${\mathrm{\tau }}_{\mathrm{c}}$). HS-MAC overlaps the exposure time across the subsequent frames for the temporal summation, while a single-lens camera halts the exposure per frame. (B) Temporal summation sampling diagram. Deblurred frames f were calculated by compressive image reconstruction. $\mathbf{g}\in {\mathbf{R}}^{c\times (\mathit{mn})}$ and $\mathbf{f}\in {\mathbf{R}}^{d\times (\mathit{mn})}$ are blurred temporally summed (blurred TS) and deblurred frames, concatenated into mn-vectors, where each frame size is $m\times n$ and the numbers of frames are c and d, respectively. $\mathbf{H}\in {\mathbf{R}}^{c\times d}$ is a sensing matrix, where red and gray boxes indicate values of 1 and 0. (C) NEI of HS-MAC operating at 3040 fps and 9120 fps. A point light source was placed in a dark chamber to calculate NEI. The NEI and TSF are in inverse proportion. Both cameras show the minimum NEI of 0.43 μW/cm² for the maximum exposure time of the CMOS image sensor at 30 fps. (D) Captured blurred TS (top) and deblurred (bottom) frames of a rotating disk at 1953 ± 3 rpm by using HS-MAC at 9120 fps with various TSF. The brightness of all images was normalized for comparison. (E) Comparative full reference image similarities before and after deblurring, depending on TSF. A stationary disk was captured at the maximum exposure level as a reference image. The similarity was calculated by the PCC. The blurred TS frame shows the maximum similarity at TSF = 10 (PCC = 0.93), noticeably decreasing above TSF = 10 due to the motion artifact resulting from the excessive temporal summation. In contrast, the deblurred frames maintain relatively high and consistent similarity even beyond TSF = 10.

The low-light imaging performance of HS-MAC was analyzed by measuring the noise equivalent irradiance (NEI) (Fig. 3C). A small white light-emitting diode was placed at a distance from the HS-MAC within a dark chamber, serving as a point light source, and oMLAs project the dotted patterns on the image sensor. The NEI was defined as the irradiance at SNR of 1 by gradually reducing the light source. The SNR was calculated from image intensities in the bright and dark regions. The integrated spectral range of the irradiance is from 245 to 812 nm, which is beyond the wavelength of the point source to prevent underestimation of NEI. The NEIs of two HS-MACs with 3040 and 9120 fps decrease linearly as TSFs increase on a logarithmic scale for both cases, exhibiting an inverse proportional relationship between TSF and the achievable light intensity. Both cameras achieve the minimum NEI of 0.43 μW/cm² for the maximum exposure time of the 30 fps CMOS image sensor, i.e., 1/30 s, while NEI was measured as 9.68 μW/cm² for 3040 fps and 16.99 μW/cm² for 9120 fps without temporal summation. In comparison, HS-MAC exhibited 23× and 40× lower NEI values with temporal summation for the respective frame rates, highlighting its outstanding sensitivity.

A rotating disk was captured by using HS-MAC at 9120 fps while increasing the TSF to validate the deblurring algorithm (Fig. 3D). The intensities of all captured images were normalized for comparison. Increasing TSF gradually sharpens the disk’s shape in TS frames by capturing more light, while the frame without the temporal summation (TSF = 1) is highly susceptible to noise due to the insufficient photons. The TS frames nonetheless become blurred as TSF increases, resulting from the excessive exposure time. In contrast, the image of a bee printed on the disk is distinguishable after deblurring by compressive image reconstruction, even at TSF = 100. The full reference image similarity between blurred TS and deblurred frames was assessed by the PCC (Fig. 3E) (52). A stationary disk was used as a reference image. The blurred TS frame exhibits the highest image similarity (PCC = 0.93) at TSF = 10, which notably decreases above TSF = 10 due to motion artifacts caused by the excessive temporal summation. In contrast, the deblurred frame maintains relatively high image similarity (PCC = 0.96) even above TSF = 10, remaining relatively constant for higher TSF values. The experimental results demonstrate that HS-MAC effectively captures detailed information about objects under dim light conditions by using the temporal summation and the compressive frame reconstruction.

Experimental demonstration of high-speed and low-light imaging

HS-MAC captured a rotating disk at 9120 fps with TSF = 5 (Fig. 4, A to C). The disk with a diameter of 135 mm rotated at 1953 ± 3 rpm with segments in six different colors (Fig. 4A). A portion of a projected array image and a single channel image taken by HS-MAC are shown in Fig. 4B. A single channel image contains a curved black arrow due to the rolling shutter effect. Sequential frames were reconstructed from a set of fragmented array images (Fig. 4C). HS-MAC distinctly observed the black arrow rotating counterclockwise over time without rolling shutter distortions. The rotating speed of the disk was measured by analyzing pixel intensities on the edge of the disk over time (Fig. 4D). The measured rotating speed is 1950 rpm, falling within an acceptable range of speed jitter. HS-MAC also excelled in capturing the necking pinch-off process of a pool fire flame (0.88 lux) under attenuated lighting conditions (Fig. 4, E to G). The pool fire flame was captured at 20 cm apart from HS-MAC to eliminate the image disparity. Neutral density (ND) filters with optical densities (ODs) of 0, 1.3, and 3.0 were used to attenuate the light intensity of the pool fire flame (Fig. 4E). A cycle of the flame flickering was captured by HS-MAC at 1020 fps during 71.7 ms via different ND filters (Fig. 4F). Each frame captured through different OD was normalized for comparison. SNR of the captured frames decreases as OD increases at TSF = 1, resulting in noisy frames due to lack of light. The shape of the flame even disappears at OD 3.0. However, the temporal summation exceptionally improves the image sensitivity by collecting photons over a longer period of time. HS-MAC clearly observes a set of time-lapse images for the flame pinch-off even at OD = 3.0 (880 μlux) with TSF = 33. The flame pocket detachment process (OD = 3.0, TSF = 33) is also captured from 59.28 to 65.02 ms as displayed in Fig. 4G. The frame tip’s pinching off leads to the separation of the flame pocket. Evidently, HS-MAC successfully captures the exact moment of flame pocket detachment (63.10 ms), even under a considerably reduced light condition.

Fig. 4. Experimental demonstration of high-speed and low-light imaging.

(A) A high-speed rotating disk with segments in different colors at 1953 ± 3 rpm. (B) Photographs of (A) captured by HS-MAC. Left: A portion of a projected array image. Each channel is shifted by the microlens offset. Right: A single channel captured by HS-MAC with 15.5-μm microlens offset (9120 fps). (C) Deblurred sequential frames fully reconstructed from a set of fragmented array images. White dashed lines indicate grids to distinguish a rotation of the disk. HS-MAC operates at 9120 fps and a TSF of 5. (D) Normalized pixel intensities of the red marked region of the rotating disk over time. The measured rotating speed is 1950 rpm, which falls within the acceptable range of speed jitter. (E) Necking pinch-off of a pool fire flame (0.88 lux; 20 cm apart from HS-MAC) under intended dim conditions. (F) Optical images of the pool fire flame via different ND filters with ODs of 0, 1.3, and 3.0 taken at 1020 fps. Left: Captured images at TSF = 1 under different dim conditions. Right: Optical images of a cycle of a flame flickering during 71.7 ms with temporal summation. HS-MAC effectively visualizes necking pinch-off of a pool fire flame under dim light conditions by increasing TSF, while the flame is invisible due to low SNR above OD = 1.3. (G) Sequential snapshots of a fast pinch-off (OD = 3.0, TSF = 33, time span = 5.74 ms). The frame tip is pinched off, resulting in the separation of the flame pocket.

DISCUSSION

We have reported high-speed and HS-MAC, inspired by insect’s vision. HS-MAC captures high-speed images at 9120 fps, outperforming conventional MACs by using channel fragmentation. Temporal summation successfully enhances the sensitivity, allowing HS-MAC to detect extremely low light (0.43 μW/cm²) while maintaining high frame rate. Furthermore, the compressive frame reconstruction process eliminates the blur caused by the temporal summation and provides sharp high-speed frame sequences. In particular, HS-MAC not only allows high-speed and high-sensitivity imaging but also supports three-dimensional (3D) imaging and super-resolution imaging by simply replacing the image processing algorithms of conventional MACs. This versatility facilitates the development of biologically inspired multifunctional camera capable of high-speed imaging at the far field plane and 3D imaging or super-resolution imaging at mid to near field planes. Besides, we anticipate developing multifunctional camera by implementing a sophisticated image processing algorithm that allows for large FOV imaging at mid or near field planes while simultaneously conducting high-speed imaging within overlapping FOV regions between each microlens. Despite the remarkable capabilities of HS-MAC, several limitations warrant further investigation. Disparity between microlenses at short distances may cause wavy distortions in video, requiring improvements in optical design or algorithmic corrections. In addition, various optical improvements can further enhance camera performance, potentially resulting in sharper and higher-resolution image sequences. These strategies include the use of aspherical offset microlenses or achromatic lens designs (53–56) and using hexagonal microlens array or packaging image sensors with an appropriate chief ray angle for oMLAs. Another issue is the computational cost of compressive image reconstruction for deblurring. Further algorithm optimization will substantially enhance the utility of HS-MAC in various applications, including wireless imaging and real-time low-light imaging. Cleary, our work reports the first demonstration of high-speed and high-sensitivity insect eye camera that fully mimics multiple channels and temporal summation of fast-eyed insect’s eyes. This compact, high-speed, and high-sensitive camera can provide a distinct route for diverse imaging applications in healthcare, smartphone, drones, or automobile.

MATERIALS AND METHODS

Microfabrication of HS-MAC

The microfabrication process involves the combination of a metal insulator–metal aperture layer and a microlens array layer (fig. S7). To reduce optical cross-talk noise between individual microlenses, the metal-insulator-metal aperture layer was utilized (57). The fabrication of the oMLAs employs a standard large-area microfabrication technique. Initially, a negative photoresist (DNR L300-D1, Dong-Jin Semichem. Co. Ltd., Korea) was defined photolithographically with an offset-patterned mask on a 4-inch borosilicate wafer. Next, a Cr (5 nm), SiO₂ (100 nm), and Cr (100 nm) layer was evaporated respectively using an electron beam and then lifted off with DNR stripper (DPS-7300, Dong-Jin Semichem. Co. Ltd., Korea). Another DNR photoresist (DNR L4615-D, Dong-Jin Semichem. Co., Ltd., Korea) was then photolithographically defined with the same offset patterned mask. The offset patterned DNR posts were thermally reflowed to form the oMLAs. Last, the oMLAs were integrated in an inverted manner with a rolling shutter image sensor (Sony IMX477), combined with submillimeter-thickness microposts using a flip-chip bonder. We verified that the image sensor was free of pixel defects and operating properly prior to packaging.

Image aquisition and experimental setup

The fully packaged HS-MAC used SONY IMX477 image sensor. This sensor records video at full resolution with a frame rate of 30 fps. The captured data was transferred to a single-board computer (Jetson Nano Development Kit-B01, NVIDIA Corp.). All objects were positioned sufficiently far from the HS-MAC to minimize disparities between channel images.

Supplementary Materials

The PDF file includes:

Figs. S1 to S7

Tables S1 and S2

Legends for movies S1 and S2

Other Supplementary Material for this manuscript includes the following:

Movies S1 and S2