Multispectral imaging with vertical silicon nanowires

Abstract

Multispectral imaging is a powerful tool that extends the capabilities of the human eye. However, multispectral imaging systems generally are expensive and bulky, and multiple exposures are needed. Here, we report the demonstration of a compact multispectral imaging system that uses vertical silicon nanowires to realize a filter array. Multiple filter functions covering visible to near-infrared (NIR) wavelengths are simultaneously defined in a single lithography step using a single material (silicon). Nanowires are then etched and embedded into polydimethylsiloxane (PDMS), thereby realizing a device with eight filter functions. By attaching it to a monochrome silicon image sensor, we successfully realize an all-silicon multispectral imaging system. We demonstrate visible and NIR imaging. We show that the latter is highly sensitive to vegetation and furthermore enables imaging through objects opaque to the eye.

In multispectral imaging, the spectrum is divided into multiple bands, enabling materials or objects in a scene to be identified from the characteristics of their absorption or reflection spectra. This approach permits many applications, including remote sensing¹, vegetation mapping², non-invasive biological imaging³, food quality control⁴ and face recognition⁵. Multispectral imaging systems generally use motorized filter wheels⁶, multiple cameras⁷, line-scanning⁸ or tunable filters⁹ to obtain images in multiple spectral bands. Despite this wide range of uses, relatively few commercial systems are available and most are complicated, expensive and bulky. To address these limitations, several approaches based on filter arrays have been proposed¹⁰,¹¹. An array of containing four NIR filter functions was demonstrated¹². Because these filters were dielectric interference designs, however, the fabrication process was relatively complicated as the deposition of multiple dielectric layers was required for the realization of each filter function. Furthermore, this process had to be repeated to achieve the four filter functions. Plasmonic structures have also been proposed for colour filtering¹³,¹⁴,¹⁵,¹⁶. Their optical characteristics have been measured but, to the best of our knowledge, no imaging experiments have been performed.

Results

Here, we demonstrate the use of vertical silicon nanowires for multispectral imaging. Fig. 1a shows the schematic representation of our approach. It was recently shown that silicon nanostructures show colours¹⁷,¹⁸. We demonstrated that vertical silicon nanowires exhibit colours that span the visible and NIR spectral ranges¹⁸,¹⁹. The effect originates not from diffractive effects of the arrays in which the nanowires are fabricated, but from the wavelength-dependence of the spatial profile of the guided mode¹⁸,¹⁹. The spectral response of each nanowire is therefore determined by its radius. This phenomenon is advantageous for the realization of multispectral filters as a single lithography step can define the optical response of all filters in the array. In our approach, an array of PDMS-embedded vertical silicon nanowires is attached to a monochrome image sensor (Fig. 1a). The unit cell of the array contains eight different spectral filters and a transparent window in its center. This configuration is analogous to the dye-based filter arrays (red/green/blue) used in colour image sensors, but with eight filter functions that span visible to NIR wavelengths, rather than three visible-wavelength filter functions. The device permits an image to be acquired from each spectral channel in a single exposure (Fig. 1a).

Figure 1.

Multispectral filter based on vertical Si nanowires (a) Concept schematic of multispectral imaging system. (b) Fabrication steps for multispectral filter. (c) SEM image of array of etched vertical silicon nanowires (30° tilted view). Array is composed of 20 × 20 unit cells. Each unit cell (75 × 75 μm) has eight different filters (each 23 × 23 μm), plus transparent region in its center. Each filter is composed of 24 × 24 nanowires and has an extent that corresponds to 4 × 4 pixels of the image sensor employed in our experiments. We intentionally choose the size of each filter to correspond to multiple image sensor pixels, rather than to a single pixel, to make it easier to align the device to the sensor chip. Scale bars i-iv are 500 μm, 10 μm, 1 μm and 200 nm. Nanowires with eight different radii (45 nm to 80 nm in 5 nm steps) are fabricated. Heights of all nanowires are 1.67 μm, and pitch is 1 μm. iii shows magnified view of boxed area of panel ii. Radii of nanowires are 45 nm, 50 nm, 70 nm and 75 nm (counter clockwise from upper right corner). iv shows nanowires with radii of 50 nm. (d) Reflection-mode optical microscope image of etched vertical silicon nanowire array on silicon substrate. Scale bars are 200 μm and 20 μm (inset). Inset shows magnified image of unit cell of array. Channels are numbered and contain nanowires with radii as follows. Channel 0:no nanowires, 1:45 nm, 2:50 nm, 3:55 nm, 4:60 nm, 5:65 nm, 6:70 nm, 7:75 nm, 8:80 nm. (e) Transmission-mode optical microscope image of PDMS-embedded vertical silicon nanowire array. Scale bars are 200 μm and 20 μm (inset).

The fabrication process for the multispectral filter is shown as Fig. 1b. The process [19] starts with a silicon wafer with etched vertical silicon nanowires. We next spin coat the substrate with PDMS and cure it. The PDMS film (50 μm thick) is then cut from the substrate with a razor blade. This completes the fabrication process (further described in Methods section and Supplementary Information). It should be noted that the vapor liquid solid (VLS) method²⁰ could present an alternate means for realizing this array of vertical silicon wires²¹. We furthermore note that, while we do not make use of this feature here, the transferal of micro- and nanostructures to PDMS has been shown to enable new applications, due its flexible and stretchable nature²²,²³,²⁴. Fig. 1c shows scanning electron microscope (SEM) images of a vertical silicon nanowire array after etching. An optical microscope image of an array of vertical silicon nanowires on a silicon substrate is shown in Fig. 1d. The color difference between the nanowires with different radii is clearly seen. A transmission-mode microscope image of the vertical silicon nanowires after they are embedded in PDMS is shown as Fig. 1e. The silicon nanowires still show their vivid colors. It can be seen that embedding the nanowires in PDMS results in a small spectral shift to longer wavelengths¹⁹.

After fabrication, the multispectral filter is mounted on the image sensor (Fig. 2a), directly on its microlens array. We measure the spectral response of our multispectral imaging system by sweeping the wavelength of the light from the monochromator from 400 nm to 1000 nm in steps of 10 nm. From the image captured by the CCD at each wavelength, the value of the pixels associated with each filter channel is recorded. This enables us to find the transmission spectra (T(λ)) of our filter channels, which are shown as Fig. 2b along with the image sensor's relative response²⁵. It can be seen that the nanowires act as subtractive color filters, suppressing specific colors (transmission dip). To find the relative response of each filter channel, we multiply the image sensor's response (black dashed line of Fig. 2b) by 1-T(λ). Multiplying it by this factor, rather than T(λ), is convenient because the transmission dips are converted to response peaks. The results are shown as Fig. 2c. It can also be seen that the eight filters (Channels 1–8) cover visible to NIR wavelengths as desired. The filter response in a multispectral imaging experiment will depend on the nature of the application. Our approach enables a variety of filter functions to be realized. As an example, we assume that our goal is for the system to have the spectral response shown as Fig. 2d. We assign Channels VIS1–3 to have the ideal response of the CIE (1964) 10-deg colour matching functions²⁶. We assign Channels NIR1–5 to have Gaussian shapes with full-widths-at-half-maximum (FWHMs) of 100 nm, and center wavelengths that match those of Channels 4–8, respectively. We then use the least squares method to find the linear combination of Channels 1–8 (Fig. 2c) that achieves the most similar response to the model (goal) response of Fig. 2d. The results of this optimization process are shown as Fig. 2e. It can be seen that there are differences between the actual (Fig. 2e) and model (Fig. 2d) responses, but that these are far smaller than the un-optimized result of Fig. 2c. It should be noted that Channels VIS1–3 are reconstructed by linear combinations of Channels 1–4 to avoid them being sensitive to NIR wavelengths.

Figure 2.

Spectral response of multispectral filter array (a) Top: photograph of multispectral filter mounted on 1/4 inch monochrome charge-coupled device (CCD) image sensor. Bottom: image captured by sensor under uniform illumination from monochromator (λ_center = 600 nm). Inset: magnified image of filter area, with ~2 × 2 unit cells shown. Dark squares corresponds to filters containing nanowires with diameters of 55 nm nanowires, as these have transmission dip at λ ≈ 600 nm. (b) Colored lines and symbols: measured filter transmission spectra, which is found by normalizing the value of the pixels associated with each filter channel by those measured from the reference area (Channel 0, no nanowires). Black dashed line: image sensor's relative response from datasheet. (c) Relative response of multispectral imaging system. Channel 1–8 corresponds to the filters in Fig. 1e (d) Model (goal) response of multispectral image system. (e) Actual response of multispectral imaging system after optimization. Channels (VIS 1–3, NIR 1–5) represent linear combination of relative responses (Channel 1–8 in Fig. 2c), optimized via least squares method to make achieve results similar to model response (Fig. 2d).

Imaging experiments are performed using our nanowire-based multispectral system (Fig. 3). No additional optical filters (e.g. IR blocking filter) are used. Nine images of each scene are taken, with the image sensor mechanically translated in a 3 × 3 array. This is done in order to increase the resolution and reduce the pixel calculation error due to the geometrical position mismatch between color filters (Channel 0 and Channels 1–8). We combine Channel VIS1–3 to obtain the linear standard red/green/blue (sRGB) image of Fig. 3a. We then apply colour correction using the results of imaging a Macbeth ColorChecker²⁷ card. Gamma correction is then applied. The resultant colour image produced by our nanowire-based imaging system can be seen to be very similar to that produced by a conventional camera. It can be seen that the resistor colour codes are vivid. Fig. 3a also shows images of the Macbeth ColorChecker card obtained by our nanowire-based imaging system. The measured RMS colour difference ΔE*_ab of the 24 colour patches is 16.3²⁶,²⁸.

Figure 3.

Imaging experiments (a) Visible colour imaging of resistor. Leftmost image is taken by conventional colour digital camera. Linear sRGB image generated from Channels VIS1–3. Colour calibration and gamma correction are applied (gamma = 2.2) to linear sRGB image to produced corrected image. Rightmost images are of Macbeth ColorChecker card by conventional camera (top) and our nanowire-based system (bottom). (b) NDVI imaging of green leaves. Black arrows indicate regions (leaves and square from Macbeth ColorChecker) with similar dark green colours. The leaves and green square appear very similar in colour images, but can be readily distinguished in NDVI image. Rightmost image is combined colour-NDVI image, in which black shading is applied to regions whose NDVI index exceeds 0.3. Note that, due to its material properties, the violet colour patch also has a high NDVI index. (c) Demonstration of NIR imaging through object that is opaque at visible wavelengths (black ink on glass). Both white and IR light emitting diodes (LEDs) are used as light sources. In colour images obtained by conventional camera and our nanowire-based system, object in front of screen is visible, while object behind screen is very difficult to observe. Both front and back objects are observed in NIR image (Channel NIR5, gamma corrected with gamma = 2.2).

Our multispectral system has five NIR channels and can be used for NIR imaging applications. As a proof-of-concept, we present a normalized difference vegetation index (NDVI) image as Fig. 3b. NDVI images are used for mapping vegetation, and represent the ratio between visible and NIR channels²⁹. Vegetation areas exhibit high NDVI values because their NIR reflectance is high and their visible-wavelength reflectance is low (due to high absorption)²⁹. To demonstrate our NDVI capability, we image a scene containing a plant, a single leaf and the Macbeth ColorChecker card. The leaves have dark green colours that are similar to one of the patches of the Macbeth ColorChecker card. In producing the NDVI image, we take Channel VIS3 as the visible channel and Channel NIR3 as the NIR channel. From the NVDI image of Fig. 3b, it is can be seen that the leaves have higher NDVI values than the dark green colour patch. This confirms that ability of our nanowire-based multispectral imaging system to identify vegetation.

We next demonstrate that our nanowire-based system enables imaging to be carried out through objects that are impervious at visible wavelengths. We make use the fact that a number of dyes are transparent at NIR wavelengths³⁰. A glass plate painted with a black permanent marker is used as a screen that is opaque to visible light. A cross-shaped green object is located in front of the black screen, while a donut-shaped green object is located behind it. Illumination is provided by white and IR light emitting diodes (LEDs). The results are shown as Fig. 3c. In the visible-wavelength images collected by a conventional camera and by our nanowire-based system, the object in front of the screen is visible, while the back object is observed only with difficulty. The NIR channel (NIR5) image collected by our nanowire-based system, however, shows both front and back objects.

Discussion

Through the creation of filter based on vertical silicon nanowires, we have demonstrated multispectral imaging from visible to NIR regions. The described method is very practical from a manufacturing standpoint because all filter functions are defined at the same time through a single lithography step, that then only needs to be combined with two simple additional steps (etching and embedding in PDMS) to produce a fully-functional device. We anticipate that this work will facilitate multispectral imaging systems that are more compact and lower in cost than current approaches.

Methods

Fabrication of multispectral filter

A silicon wafer (<100> orientation) is coated with polymethylmethacrylate (PMMA) resist, and electron-beam lithography (Elionix, ELS-7000) is performed. An aluminium etch mask (40 nm thick) is fabricated using thermal evaporation and the lift-off process. The wafer is then dry etched using inductively coupled plasma reactive ion etching (ICP-RIE) with SF₆ and C₄F₈ gases. We next spin coat PDMS (ratio of base to curing agent is 5:1) at 1000 rpm for 60 seconds onto the wafer containing the vertical silicon nanowires. The PDMS is then cured at 230°C on a hotplate for an hour. The cured PDMS film is removed by cutting it from the substrate with a razor blade. The film is mounted on a monochrome CCD image sensor (ICX098BL, Sony) in a camera (DMK21AF04, Imaging Source). The resolution of image sensor is 640 by 480 pixels, and the pixel pitch is 5.6 μm. The cover glass of image sensor is removed to enable the multispectral filter to be directly attached to the surface of the image sensor. No adhesive is needed.

Imaging experiments

A monochromator (Oriel) with a halogen lamp is used for measuring the transmission characteristics of our multispectral filter. A lens (model SMC Pentax-M 1:1.4 50 mm, Pentax) with a focal length of 50 mm and an f-number of 5.6 is used for resistor imaging experiment (Fig. 3a). A lens (model M0814-MP2, Computar) with a focal length of 8 mm and an f-number of 5.6 is used for the NDVI and NIR imaging experiments (Fig. 3b and Fig. 3c). A daylight compact fluorescent light bulb (CFL, model 37919, Westinghouse) is used as the light source for Fig. 3a. The CFL (model 37919, Westinghouse) and an incandescent lamp (power: 40 W, GE) are used as light sources for the NDVI imaging (Fig. 3b). The NIR imaging experiment of Fig. 3c uses both a white LED (model 276–320, RadioShack) and an IR LED (model 276–143, λ_center = 950 nm, RadioShack) as light sources. Our multispectral filter comprises an array of 20 × 20 unit cells, each of which contains 3 × 3 patches (eight filter functions plus transparent region in center). We sample 3 × 3 pixels in each patch. Thus, the resolution of a single-exposure image is 60 × 60 pixels. After the mechanical scanning of the camera is carried out (3 × 3), the image resolution becomes 180 × 180 pixels. Every acquired image uses eight frames averaging for noise reduction. The visible colour reference image is taken by a commercial camera (8 megapixel camera in iPhone 4S, Apple). For visible colour imaging, first we convert the CIE XYZ tristimulus values (Channel VIS3, VIS2 and VIS1) into the linear sRGB space (D65 illumination). This linear sRGB image is multiplied by a color correction matrix. Gamma correction of 1/2.2 is then applied. The color correction matrix is found by imaging the Macbeth ColorChecker card (ColorChecker Classic, X-Rite) and finding the linear sRGB values of the 24 color patches of the acquired image. The matrix that transforms these to the desired linear sRGB values with minimum error is then found.

Author Contributions

H.P. and K.B.C. designed the experiments. H.P. performed the experiments. H.P. and K.B.C. wrote the manuscript. K.B.C. supervised the research.

Supplementary Material

Supplementary Information

Supplementary Information