Skip to main content
PMC home page
Scientific Reports logoLink to Scientific Reports
. 2016 Apr 5;6:24044. doi: 10.1038/srep24044

A four-dimensional snapshot hyperspectral video-endoscope for bio-imaging applications

Hoong-Ta Lim 1, Vadakke Matham Murukeshan 1,a
PMCID: PMC4820774  PMID: 27044607

Abstract

Hyperspectral imaging has proven significance in bio-imaging applications and it has the ability to capture up to several hundred images of different wavelengths offering relevant spectral signatures. To use hyperspectral imaging for in vivo monitoring and diagnosis of the internal body cavities, a snapshot hyperspectral video-endoscope is required. However, such reported systems provide only about 50 wavelengths. We have developed a four-dimensional snapshot hyperspectral video-endoscope with a spectral range of 400–1000 nm, which can detect 756 wavelengths for imaging, significantly more than such systems. Capturing the three-dimensional datacube sequentially gives the fourth dimension. All these are achieved through a flexible two-dimensional to one-dimensional fiber bundle. The potential of this custom designed and fabricated compact biomedical probe is demonstrated by imaging phantom tissue samples in reflectance and fluorescence imaging modalities. It is envisaged that this novel concept and developed probe will contribute significantly towards diagnostic in vivo biomedical imaging in the near future.

Hyperspectral imaging (HSI) can collect required spectral information from each spatial pixel in the region of interest (ROI). By arranging the spectral information according to the positions, a three-dimensional (3-D) datacube can be realized with the spatial information in two dimensions and the spectral information arranged in the third dimension1. Each voxel gives the optical information of one wavelength from one spatial position2. Typically, in order to be defined as a hyperspectral (HS) imager, the resulting datacube should have at least 100–200 images at different wavelengths3. Compared to multispectral imaging which captures at most few tens of wavelength bands3, HSI captures much more wavelengths and thus the spectral gap between adjacent wavelengths reduces and the spectral signatures are more detailed. HSI has mainly three reported configurations which are named as spatial-scanning, spectral-scanning and snapshot methods. They differ in how the 3-D data is acquired to form the 3-D datacube. This leads to each method having a set of characteristics which makes it only suitable for certain applications.

A spatial-scanning HSI system can be a whiskbroom or a pushbroom type and usually uses a dispersive element to separate the incoming spectrum4. In each scan, a whiskbroom HSI system acquires the one-dimensional (1-D) spectral information of one spatial point of the ROI at a time5, using its 1-D single-line detector. After repeating the scan over multiple points over the area of the ROI, the 1-D spectra are arranged according to where they were collected to form the datacube. A pushbroom HSI system has a narrow slit allowing a straight line of light to pass through the slit6. After spectral dispersion, a two-dimensional (2-D) spatial-spectral data is collected by the 2-D detector in each scan. Scanning is repeated after a relative displacement between the sample and the imager, in the direction transverse to the slit, so that the next line can be imaged. After the entire ROI is imaged, the arrangement of the multiple 2-D data according to the sequence in which they were collected forms the datacube. A spatial-scanning imager typically acquires many images from hundreds of wavelengths, giving detailed spectral signatures across a large spectral range. However, the scanning takes a longer time to complete when a larger ROI is imaged. Sequential scanning is required by both whiskbroom and pushbroom imagers. However, the pushbroom configuration is more efficient than the whiskbroom method, as it collects spectral information from a larger area of the sample in each scan7.

A spectral-scanning HS imager uses a tunable filter whose spectral transmission is controlled electronically8. It allows light close to a selected wavelength to be transmitted at a time, which is later detected by the 2-D detector placed behind the tunable filter. Liquid crystal and acousto-optic tunable filters have been used in such HSI systems9,10. Each captured image is a 2-D spatial-spatial data of a narrow wavelength band. Repeated scanning is performed after the tunable filter switches the wavelengths to be transmitted. By arranging the multiple 2-D data according to the transmitted wavelength of the tunable filter, a datacube is formed. Sequential scanning is required for switching the transmitted wavelength of the tunable filter, and the total acquisition time is longer when the number of wavelengths to interrogate becomes larger.

Snapshot HSI systems are able to capture the 3-D data to build a datacube in a single scan11. This is done using different configurations such as integral-field spectroscopy12, image mapping spectroscopy13,14, computed tomographic imaging spectroscopy15 and compressive sensing2. Such systems do not need sequential scanning to build a datacube. The ability of the snapshot imager to capture the 3-D data in one scan has both advantages and limitations. The main benefit of such an imager is that it is much faster than spatial- and spectral-scanning HSI systems, and can be used in real-time applications depending on the exposure time and the detector’s readout rate. Motion artifacts and pixel misregistration can therefore be eliminated15. Each 2-D detector has a limited number of pixels and can only capture that much information in one scan. Thus snapshot imager can only acquire a limited amount of information, and has to sacrifice on the number of spatial points or wavelengths from which the data is collected.

HSI was first used in airborne and spaceborne vehicles for the observation of Earth5. Its ability to capture data to form a datacube consisting of hundreds of images from contiguous and narrow spectral bands for further analysis has since led to many other applications. These include astronomy16,17, examination of historical murals18, quality assessment of food19, and bio-imaging20,21,22. In biomedical imaging, endoscopes have been developed to image sites within the body that are not easily accessible by conventional table-top setup. HSI has been incorporated into such applications using the spectral-scanning23,24 and snapshot methods13. In real-time endoscopic applications such as in vivo disease diagnosis and surgical monitoring, the snapshot methods will be the better choice. However, one major drawback of existing snapshot HS endoscopes is that only ~50 spectral bands can be acquired13.

In this context, we have developed a snapshot HS probe which can be used for endoscopic diagnostics biomedical applications. The custom fabricated probe is flexible along its length and its distal end has a small profile so that it can be inserted into the orifice of the body cavities such as the anus to investigate the gastrointestinal tract. It is a 2-D array (10 × 10) of hexagonally-packed optical fiberlets (individual fibers) arranged in in rows and columns, which are orderly rearranged to form a 1-D array (1 × 100) of fiberlets at the other end. This arrangement allows the 2-D image to be captured by 2-D end of the fiber bundle and be reduced from two spatial dimensions to only one dimension at its 1-D end25. The light from the 1-D end of the fiber bundle enters a spectrograph which disperses the light to be detected by the 2-D detector. This 2-D to 1-D fiber bundle reduces the 3-D data (spatial-spatial-spectral) to a 2-D data (spatial-spectral) so that the 2-D detector takes only one scan to capture all the information. Capturing the data sequentially in real-time adds a fourth dimension to the data collected, which is in the temporal domain26. After data acquisition, custom written software was used to process and rearrange each spectrum acquired in a scan according to the spatial position from which it was acquired on the 2-D end of the fiber bundle. It is to be mentioned that though the use of 2-D to 1-D fiber bundle has been previously reported, it was only used as an optical element in a field or table-top systems26,27,28. These systems using 2-D to 1-D fiber bundles do not have a flexible, long and small probe.

In this paper, the use of a custom fabricated flexible 2-D to 1-D fiber bundle as a compact four-dimensional (4-D) snapshot HS video-endoscope is illustrated for diagnostic biomedical applications. It forms an image of the sample covering ~1.11 × 1.32 mm2 of 100 spatial points at a frame rate of ~6.16 Hz. The spectral range of interest is 400–1000 nm with 756 spectral bands.

Results

Reflectance imaging

The tissue properties of normal tissue and tumor are different and the possible mechanisms include mucosal thickening and higher vasculature density in abnormal lesions29. These can result in the optical reflectance between a normal tissue and tumor to be different, especially when the tumor is located on the tissue surface. In order to simulate this situation of normal and abnormal tissue regions, a black tape was placed on a phantom tissue sample (Fig. 1a) and imaged while being illuminated by two flexible light guides which delivers light from a broadband white light source. The sample is divided into Regions R1 and R2. Region R1 is the phantom tissue sample representing normal tissue, while Region R2 is the black tape representing a tumor located on the tissue surface with different reflectance properties29,30. The sample was manually moved using a mechanical stage towards the right of the 2-D end of the fiber bundle during data acquisition. With respect to the sample, the 2-D end of the fiber bundle was moving to the left (arrow in Fig. 1b). A total of 80 frames were taken at a rate of ~6.16 Hz.

Figure 1.

Figure 1

Open in a new tab

(a) Simulated phantom tissue sample for reflectance imaging and (b) the photograph of the 2-D end of the fiber bundle superimposed on the imaged region of the sample. Region R1 represents normal tissue while Region R2 represents a tumor located on the tissue surface with different reflectance properties. The fiber bundle in (b) shows its initial position and the arrow indicates its relative movement with respective to the sample during data acquisition.

Each frame captured by the detector camera of the snapshot HS video-endoscope was used to build a 3-D datacube. The datacubes were built at the same rate as the frame rate of the detector camera ~6.16 Hz. 80 frames were captured in ~12.81 s. Three cut-datacubes are shown in Fig. 2. These 3-D datacubes show the 4-D data (spatial-spatial-spectral-temporal) captured using the snapshot HS video-endoscope.

Figure 2. Each frame captured by the detector camera was used to build a 3-D datacube.

Figure 2

Open in a new tab

(a–c) show the cut-datacube acquired using frames 21, 35 and 44, which were captured ~3.24 s, 5.52 s and 6.98 s respectively after data acquisition started. These 3-D datacubes show the 4-D data (spatial-spatial-spectral-temporal) captured using the snapshot HS video-endoscope. It can be observed that with respect to the sample, the 2-D end of the fiber bundle was moving towards the left during data acquisition.

Figure 3 shows the reflectance mappings of nine wavelengths and datacubes. Supplementary Videos 1 and 2 are made up of 80 frames each, using reflectance mappings of 575 nm and 700 nm respectively. By looking at the frames in Figs 2 and 3 sequentially and Supplementary Videos 1 and 2, it can be observed that the proposed system was able to perform HS reflectance imaging in a snapshot configuration. The different reflectance between Regions R1 and R2 was captured and they can be differentiated from each another. The 2-D end of the fiber bundle was initially imaging Region R1 of high reflectance, representing normal tissue region. Then it moved to the left with respect to the sample and entered Region R2 of low reflectance, representing the abnormal region of the tissue. The sharp tip on the right of Region R2 can be clearly imaged. This continued in the same direction while imaging Region R2 till the data acquisition stopped. These depict the actual relative motion between them during data acquisition (Fig. 1b). The shape of Region R2 is also correctly represented in the experimental results.

Figure 3. 4-D reflectance mappings of nine selected wavelengths and datacubes taken at different timings.

Figure 3

Open in a new tab

The 2-D end of the fiber bundle was moving towards the left with respect to the sample during data acquisition.

The mean reflectance spectra and standard deviations of Regions R1 and R2 are shown in Fig. 4. Each data set was calculated from 27 spectra. The spectra of Region R1 were acquired from 9 fiberlets whose positions are indicated by the red arrow box in the intensity mapping shown for 500 nm (see Fig. 3) and from Frame 3–5. The spectra of Region R2 were acquired from 9 fiberlets whose positions are indicated by the yellow arrow box in the intensity mapping for 700 nm as shown in Fig. 3 and from Frame 35–37.

Figure 4. Reflectance spectra of Regions R1 and R2.

Figure 4

Open in a new tab

Mean reflectance spectrum (center bold line) and standard deviation (surrounding black areas) along with insets.

The spectra in Fig. 4 show that the 4-D HS imaging probe could capture the detailed reflectance spectra of Regions R1 and R2 while there was a relative motion between the sample and the 2-D end of the fiber bundle. It can be observed that Region R1 (phantom tissue sample) had a much higher reflectance than Region R2 (black tape) along the entire spectral range of interest. The average standard deviations of the reflectance spectra of Regions R1 and R2 were about ±2.49% and ±0.42% respectively.

The results in this section show that 4-D HS imaging probe was able to capture the HS reflectance of different parts of the imaged sample throughout the duration of data acquisition. Reflectance intensity mappings of appropriate wavelengths can be selected to spectrally distinguish one region from another for diagnostic applications. The spectral information collected from known samples can be stored in a data library and be used for identification and quantification in future. The capabilities of the proposed imaging probe are also demonstrated in another experiment conducted on a chicken breast tissue with blood clot. The results are detailed in the Supplementary Discussion (Supplementary Figs 1–3 and Supplementary Videos 3 and 4).

Fluorescence imaging

Endogenous fluorophores such as flavin, collagen and porphyrins29,31,32 are natural fluorophores. Autofluorescence occurs when these endogenous fluorophores are excited by a laser having an appropriate excitation wavelength. The differences in the tissue microarchitecture and concentration of endogenous fluorophore between a normal tissue and tumor29 can lead to variations in autofluorescence29,31,33. Depending on the excitation wavelength, autofluorescence in tumor can be lower29,33 or higher33 compared to normal tissues. This is due to the excited endogenous fluorophores having lower or higher concentrations in tumor respectively33. The magnitude of autofluorescence variations is dependent on the tumor stage since the differences become greater as the tumor progresses. Therefore, detection and staging of tumor can be done using fluorescence imaging to capture the autofluorescence in tissues by looking out for changes in the fluorescence intensity. In this experiment, a phantom tissue sample with applications of fluorescent powder (Fig. 5a), simulating different stages of caner growth in colon, was imaged while being illuminated by two flexible light guides delivering light from a 532-nm laser. The fluorescent powder has emission wavelengths from ~500–700 nm, and falls within the emission maxima of biological endogenous fluorophores (280–690 nm)32. The sample is divided into Regions F1, F2 and F3. Region F1 is simulated with a thick region of fluorescent powder (higher concentration) representing normal tissue having normal autofluorescence intensity. Region F2 is simulated with a thin region of fluorescent powder (lower concentration) representing tumor growth in the intermediate stage with reduced autofluorescence intensity. Region F3 is the simulated phantom tissue sample representing tumor growth in the advanced stage with very weak autofluorescence. The sample was manually moved using a mechanical stage first upwards, then towards the left of the 2-D end of the fiber bundle during data acquisition. With respect to the sample, the 2-D end of the fiber bundle was first moving downwards then to the right (arrows in Fig. 5b). A total of 160 frames were taken at a rate of ~6.16 Hz.

Figure 5.

Figure 5

Open in a new tab

(a) Simulated phantom tissue sample for fluorescence imaging and (b) the photograph of the 2-D end of the fiber bundle superimposed on the sample. Region F1 represents normal tissue with normal autofluorescence intensity. Region F2 represents tumor growth in the intermediate stage with reduced autofluorescence intensity. Region F3 represents tumor growth in the advanced stage with very weak autofluorescence. The 2-D end of the fiber bundle was initially placed on the top of the marked area. The arrows in (b) indicate its relative movement with respect to the sample during data acquisition. (a,b) are images of the same sample but appear to have different colors due to the different illuminations and cameras used.

In this experiment, the snapshot hyperspectral video imager captured 160 frames in ~25.79 s. Each frame was used to build a 3-D datacube. Although the full spectral range of 400–1000 nm was captured, only the data from 570–600 nm is used to build the three cut-datacubes shown in Fig. 6. Figure 6a representing Region F1 indicates normal tissue with detection of high fluorescence intensity. Figure 6b representing Region F3 indicates tumor growth in the advanced stage with detection of very low fluorescence intensity. Figure 6c was acquired from Region F2 which represents tumor growth in the intermediate stage with detection of low fluorescence intensity. These 3-D datacubes show the 4-D data (spatial-spatial-spectral-temporal) captured using the snapshot HS video-endoscope.

Figure 6. Each frame captured by the detector camera was used to build a 3-D datacube.

Figure 6

Open in a new tab

(a–c) show the cut-datacube acquired using frames 18, 58 and 128, which were captured ~2.92 s, 9.25 s and 20.61 s respectively after data acquisition started. (a) was acquired from Region F1 representing normal tissue with detection of high fluorescence intensity. (b) was acquired from Region F3 representing tumor growth in the advanced stage with detection of very low fluorescence intensity. (c) was acquired from Region F2 representing tumor growth in the intermediate stage with detection of low fluorescence intensity. These 3-D datacubes show the 4-D data (spatial-spatial-spectral-temporal) captured using the snapshot HS video-endoscope.

Figure 7 shows the fluorescence mappings of nine datacubes at 585 nm. Supplementary Videos 5 and 6 are made up of 125 frames each (Frame 11–135), using fluorescence mappings of 575 nm and 585 nm respectively. By looking at the frames in Fig. 7 sequentially and Supplementary Videos 5 and 6, it can be observed that the proposed snapshot system can be used to perform HS fluorescence imaging. The fluorescence intensities of Regions F1, F2 and F3 were captured and the information can be used to differentiate one from another. The 2-D end of the fiber bundle was initially imaging Region F3 of very low fluorescence intensity. It moved downwards with respect to the sample and entered Region F1 of high fluorescence intensity. Then it entered Region F3 before moving towards the right with respect to the sample and entered Region F2 of low fluorescence intensity. It continued in this path until it entered Region F3 before data acquisition stopped.

Figure 7. Fluorescence mappings of nine datacubes at 585 nm taken at different timings.

Figure 7

Open in a new tab

With respect to the sample, the 2-D end of the fiber bundle was moving downwards before moving towards the right during data acquisition.

These depict the actual relative motion between them during data acquisition (Fig. 5b). It is to be noted that there was uneven illumination on the phantom sample where the illumination on the left was stronger. The shapes of Regions F1 and F2 are also correctly represented in the experimental results.

The mean fluorescence spectra (solid lines) and standard deviations (dotted lines) of Regions F1, F2 and F3 are shown in Fig. 8. Each data set was calculated from 27 spectra and normalized to the maximum value of the mean fluorescence spectrum of Region F1. The spectra were acquired from the same 9 fiberlets whose positions are indicated by the green and red arrow boxes in Fig. 7. For the purpose of comparing the spectral intensity of Regions F1, F2 and F3, the effect of uneven illumination on the sample is reduced by acquiring the spectra from the same fiberlets. The spectra of Regions F1, F2 and F3 were acquired from Frame 17–19, 127–129 and 57–59 respectively.

Figure 8. Fluorescence spectrum of Regions F1, F2 and F3.

Figure 8

Open in a new tab

Mean fluorescence spectrum (center bold line) and standard deviation (dotted lines) with inset of Region F3.

The spectra in Fig. 8 illustrate that the 4-D HS imaging probe could capture the detailed fluorescence spectra of Regions F1, F2 and F3 while there was a relative motion between the sample and the 2-D end of the fiber bundle. The peak fluorescence wavelength was ~585 nm. It can be observed that strong fluorescence was detected from Region F1 (thick fluorescent powder region with higher concentration) representing normal tissue with normal autofluorescence intensity. A relatively weaker fluorescence was detected from Region F2 (thin fluorescent powder region with lower concentration) representing tumor growth in the intermediate stage with reduced autofluorescence intensity. A very weak fluorescence was detected from Region F3 (phantom tissue sample) representing tumor growth in the advanced stage with very weak autofluorescence. The average standard deviations of the fluorescence spectra of Regions F1, F2 and F3 from 500–700 nm are about ±0.0368, ±0.0213 and ±0.0026 respectively (Fig. 8).

The results in this section show that 4-D HS imaging probe captured the HS fluorescence of different parts of the imaged sample throughout the duration of data acquisition. The system can capture the fluorescence spectra which can reveal the type and concentration of fluorophores in the samples. This can in turn leads to tumor staging and other related disease diagnostics applications.

Discussion

Based on the technical specifications of an existing pushbroom HS imager, we have a custom-fabricated 2-D to 1-D fiber bundle that converts the pushbroom HS imager into a snapshot configuration. The fiber bundle is also flexible and has a small distal end enabling it to be used as an imaging probe that can be inserted into the body for minimally invasive and in vivo investigations. By acquiring data frames continuously, these factors come together to give a snapshot HS video-endoscope with potential bio-imaging applications.

We have given a detailed instrumentation of the proposed system and demonstrated its feasibility. Reflectance and fluorescence imaging were conducted when the light source and the probe were both on the same side as that of the imaged samples. This is the expected configuration during in vivo imaging of the internal body cavity. Further, it is to be noted that this probe can be integrated with a control and locomotion option as in conventional endoscopes avoiding the need for sample movement when this is to be used inside body cavities.

In both reflectance and fluorescence imaging, we were able to attain a frame rate of ~6.16 Hz, and each frame was converted into a 3-D datacube with 756 spectral bands. The 3-D datacubes and intensity mappings can provide vast amount of information, which includes the spatial features (size and shape), spectral signatures (756 bands), speed and direction of the imaged samples. The spectral information can also be seen in the line plots. These promising results confirm the successful implementation of such a 2-D to 1-D fiber bundle serving for its use as a snapshot HS video-endoscopic probe. The snapshot HS video-endoscope illustrated in this paper, used the flexible 2-D to 1-D fiber bundle for potential bio-imaging applications for the first time. It also captures 756 spectral bands which are significantly more than existing snapshot HS video-endoscopes which can generally capture only ~50 spectral bands. With more spectral bands available, limitations such as a reduced spectral range, insensitivity to certain narrow spectral band and inability to capture detailed spectral signatures, can be avoided.

The use of such a HS video-endoscope with a flexible 2-D to 1-D fiber bundle can be a potential alternative to conventional fiber optic imaging systems. The information collected by a HS video-endoscope has an additional spectral dimension of the order of several hundred wavelength bands. Conventional video-endoscope using color camera gives only little spectral information from three bands. In this aspect, HS video-endoscopes can be especially useful when detailed spectrum is required for classification and quantification to give functional information such as hemoglobin saturation9. It is also valuable in cases where multiple excitation sources are used to excite multiple fluorescent tags and the HS data can be used to differentiate the fluorescent tags even when the excitation and emission spectra are over-lapping but distinct34. Currently, many conventional setups in the field of optogenetics and neuronal imaging can only image one fluorescent tag in each frame35,36,37. Such studies can benefit by using HS video-endoscope with a flexible and compact distal end. It can be used in more complex and non-invasive studies to capture detailed spectral information from multiple fluorescent tags.

A future improvement to the probe system is to use smaller fiberlets so that more can be packed within the fiber bundle. The current system images 100 fiberlets on the 1-D end using all the 1004 pixel columns. While the maximum number of fiberlets that can be effectively imaged by the snapshot imager is 1004, which is the number of pixel column of the sensor array. By using smaller fiberlets, more can be packed along the 1-D end. Spectral information from more spatial points will be collected and the data collection by the sensor array becomes more efficient. The image quality in terms of spatial dimensions is also expected to be better. Another possible improvement to the system is to increase its frame rate from the current ~ 6.16 Hz to 20 Hz so that it can be real-time. The current frame rate is limited by the detector camera and exposure time. This can be made a complete real-time system by replacing the detector camera with another one having faster readout rate and by using a lower exposure time.

Methods

HS imaging system

The snapshot HS endoscopic probe shown in Fig. 9a was developed and installed in-house. This has two main parts, the HS imager and the 2-D to 1-D fiber bundle probe. The HS imager is the major part of a pushbroom HS imager which was previously reported by our group20, without the 3-axis motorized stage. During hardware installation, the video camera (UI-1550LE-C-HQ, iDS) and the detector camera (LucaEM DL-604M-OEM, Andor) had been positioned to produce focused images at the same time. The forelens was a doublet (2-50145, Navitar) kept in a fine focus adaptor (2-16265, Navitar). This adaptor was attached to bottom side of the quadrocular adaptor (Y-QT, Nikon), which housed a sliding mirror. The sliding mirror was pushed into the quadrocular adaptor and directed light towards the video camera (Path 1 in Fig. 9a). The video camera allowed direct video imaging and was used to position the 1-D end of the fiber bundle. With the 1-D end of the fiber bundle in place, the sliding mirror was pulled out of the quadrocular adaptor so that light can travel straight towards the spectrograph (ImSpectors V10E, Specim) and the detector camera (Path 2 in Fig. 9a). The spectrograph dispersed the light and the detector camera recorded the information required to build a datacube with each scan. A broadband light source (MI-150, Edmund Optics) and a 532-nm diode-pumped solid-state laser were used during reflectance and fluorescence imaging, respectively. The samples were illuminated using two flexible light-guides. During fluorescence imaging, a 550-nm long-pass filter (FEL0550, Thorlabs) was kept before the forelens. Spectral calibration was carried out and the probe system was found to be able to detect 756 spectral bands within a range of 400–1000 nm.

Figure 9.

Figure 9

Open in a new tab

(a) Schematic diagram of HS imaging system, (b) photograph of 2-D to 1-D fiber bundle placed near a 23 mm diameter coin, (c) photograph of 2-D end-face showing all fiberlets and (d) photograph of 1-D end-face showing all fiberlets. 100 fiberlets were used in the fiber bundle, but only 99 fiberlets can be used for imaging (Fiberlet 4 was damaged and inactive).

The 2-D to 1-D fiber bundle was custom fabricated by Polymicro TechnologiesTM according to author’s specifications. The 100 optical fiberlets in the fiber bundle have core and buffer diameters of 100 μm and 125 μm, respectively (FVP100110125, Molex). They were arranged in a 10 × 10 hexagonally-packed fashion in the 2-D end and numbered from 1 to 100 across the column towards the right and then down the row (Fig. 9c). The vertical and horizontal core-to-core spacing on the 2-D end-face were ~110 μm and ~125 μm, respectively. The packing ratio on the 2-D end-face was ~55%. The fiberlets were then rearranged by row and column in a 1 × 100 fashion in the 1-D end and numbered correspondingly from 1 to 100 towards the right (Fig. 9d). The core-to-core spacing was ~125 μm. Fiberlet 4 was damaged and found to be inactive as indicated by the dark spot shown in Fig. 9c,d, thus cannot be used for imaging. The flexible bundle has a length of ~1 m with a 3.5 mm diameter polyvinyl chloride jacket. The optical fiberlets in the 1-D end were encased in a stainless steel holder with an end-face area of 5 × 20 mm2 and length of 30 mm. While in the 2-D end, they were enclosed in a cylindrical stainless steel holder with a diameter of 5 mm and length of 30 mm.

The full line-of-view of the HS imager was measured to have a length of ~12.5 mm. After selecting the optical fiberlet having a buffer diameter of 125 μm, it was estimated that the HS imager would only image ~100 optical fiberlets placed in a straight line. Hexagonally-packed optical fiberlets result in a bundle with a higher packing ratio compared to one which is packed in a square array. Therefore 100 optical fiberlets were used to make the fiber bundle and arranged in a hexagonally-packed fashion in the 2D-end.

Spatial calibrations

The spatial calibration was conducted initially on the 1-D end of the fiber bundle after alignment and was fixed in place. The detector camera had a sensor array of 1002 rows (y-axis, spectral) and 1004 columns (x-axis, spatial), and it captured a 2-D spectral-spatial data (Fig. 10a) from the fiberlets in 1-D end of the fiber bundle. Each colored vertical line in Fig. 10a came from the light exiting the core of each fiberlet, which was then spectrally dispersed along the y-axis of the sensor array. The y-axis was later converted to the calibrated spectral bands20. The position of each colored vertical lines along the x-axis indicated the pixel columns used to image each fiberlet. Spectral information from each fiberlet would be acquired from the corresponding pixel columns during data processing. It can be observed from Fig. 10a that all the 1004 pixel columns of the sensor array was used to image the 100 fiberlets on the 1-D end. ~10 x-pixels were used to image each fiberlet. The dark line in Fig. 10a indicated the position of Fiberlet 4, which was damaged and thus inactive.

Figure 10.

Figure 10

Open in a new tab

(a) Reference image taken by detector camera, (b) photograph of fiberlets on 2-D end-face and (c) digital mask of 2-D end-face. Both (b,c) are ~1.11 × 1.32 mm2. The numbers on some of the fiberlets in (b,c) indicate the numbering of the fiberlets. The white triangle in (b) indicates the position of Fiberlet 4 which was inactive and thus appeared to be dark.

The second spatial calibration was done on the 2-D end of the fiber bundle. Figure 10b shows the photograph of all the fiberlets on the 2-D end-face which were illuminated from the other end of the fiber bundle. Using Fig. 10b, a digital mask of the 2-D end-face was created (Fig. 10c), which contained the position and numbering of some fiberlets. The spectrum acquired from each fiberlet in Fig. 10a will eventually be placed in the corresponding position in Fig. 10c.

Sample preparation

In reflectance imaging, the sample used was a simulated tissue (obtained from Simulab Corporation) placed on a glass slide. This phantom tissue is a standard test sample used here for proof of concept studies. A black tape was then placed on the tissue phantom. A 99% reflectance standard (SRS-99-010, Labsphere) was also imaged for reflectance imaging. In fluorescence imaging, fluorescent powder (Ultra-orange/yellow fluorescent power, Medtech Forensics) was coated on the tissue phantom to simulate the different stages of tumor as illustrated in the manuscript. All the imaged objects were kept at a distance of ~0.5 mm away from the 2-D end-face of the fiber bundle. The samples were manually moved using high-resolution mechanical translational stages during imaging. Details regarding the data acquisition, processing and visualization are given in Supplementary Discussion.

Additional Information

How to cite this article: Lim, H.-T. and Murukeshan, V. M. A four-dimensional snapshot hyperspectral video-endoscope for bio-imaging applications. Sci. Rep. 6, 24044; doi: 10.1038/srep24044 (2016).

Supplementary Material

Supplementary Information
srep24044-s1.pdf (856.3KB, pdf)
Supplementary Video 1
Download video file (25MB, avi)
Supplementary Video 2
Download video file (25MB, avi)
Supplementary Video 3
Download video file (36MB, avi)
Supplementary Video 4
Download video file (36MB, avi)
Supplementary Video 5
Download video file (47.8MB, avi)
Supplementary Video 6
Download video file (47.8MB, avi)

Acknowledgments

The authors acknowledge the financial support from COLE (NTU)-EDB and AcRF Tier 1 (RG162/15), Ministry of Education (MOE), Singapore.

Footnotes

Author Contributions H.T.L. and V.M.M. designed and conceived the research. H.T.L. performed all the experiments and took all the data and images. Both authors prepared, corrected and finalized the manuscript. V.M.M. supervised the project as principal investigator.

References

  1. Hagen N., Kester R. T., Gao L. & Tkaczyk T. S. Snapshot advantage: a review of the light collection improvement for parallel high-dimensional measurement systems. Opt. Eng. 51, 111702 (2012). [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Gehm M., John R., Brady D., Willett R. & Schulz T. Single-shot compressive spectral imaging with a dual-disperser architecture. Opt. Express 15, 14013–14027 (2007). [DOI] [PubMed] [Google Scholar]
  3. Fresse V., Houzet D. & Gravier C. GPU architecture evaluation for multispectral and hyperspectral image analysis. Conference on Design and Architectures for Signal and Image Processing, 121–127 (2010). [Google Scholar]
  4. Dell’Endice F., Nieke J., Koetz B., Schaepman M. E. & Itten K. Improving radiometry of imaging spectrometers by using programmable spectral regions of interest. ISPRS J. Photogramm. 64, 632–639 (2009). [Google Scholar]
  5. Nieke J., Schwarzer H. H., Neumann A. & Zimmermann G. Imaging spaceborne and airborne sensor systems in the beginning of the next century. Proc. SPIE 3221, 581–592 (1997). [Google Scholar]
  6. Kosec M., Bürmen M., Tomaževič D., Pernuš F. & Likar B. Characterization of a spectrograph based hyperspectral imaging system. Opt. Express 21, 12085–12099 (2013). [DOI] [PubMed] [Google Scholar]
  7. Puschell J. J. Hyperspectral imagers for current and future missions. Proc. SPIE 4041, 121–132 (2000). [Google Scholar]
  8. Gat N. Imaging spectroscopy using tunable filters: a review. Proc. SPIE 4056, 50–64 (2000). [Google Scholar]
  9. Sorg B. S., Moeller B. J., Donovan O., Cao Y. & Dewhirst M. W. Hyperspectral imaging of hemoglobin saturation in tumor microvasculature and tumor hypoxia development. J. Biomed. Opt. 10, 044004 (2005). [DOI] [PubMed] [Google Scholar]
  10. Kong S. G., Martin M. & Vo-Dinh T. Hyperspectral fluorescence imaging for mouse skin tumor detection. ETRI J. 28, 770–776 (2006). [Google Scholar]
  11. Lu G. & Fei B. Medical hyperspectral imaging: a review. J. Biomed. Opt. 19, 010901 (2014). [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Ren D. & Allington-Smith J. On the application of integral field unit design theory for imaging spectroscopy. Publ. Astron. Soc. Pac. 114, 866–878 (2002). [Google Scholar]
  13. Kester R. T., Bedard N., Gao L. & Tkaczyk T. S. Real-time snapshot hyperspectral imaging endoscope. J. Biomed. Opt. 16, 056005 (2011). [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Gao L., Smith R. T. & Tkaczyk T. S. Snapshot hyperspectral retinal camera with the image mapping spectrometer (IMS). Biomed. Opt. Express 3, 48–54 (2012). [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Johnson W. R., Wilson D. W., Fink W., Humayun M. & Bearman G. Snapshot hyperspectral imaging in ophthalmology. J. Biomed. Opt. 12, 014036 (2007). [DOI] [PubMed] [Google Scholar]
  16. Barden S. C. & Wade R. A. DensePak and spectral imaging with fiber optics. Conference on Fiber Optics in Astronomy 3, 113–124 (1988). [Google Scholar]
  17. Vanderriest C. Integral field spectroscopy with optical fibres. Conference on Tridimensional Optical Spectroscopic Methods in Astrophysics 71, 209–218 (1995). [Google Scholar]
  18. Sun M. et al. What’s wrong with the murals at the Mogao Grottoes: a near-infrared hyperspectral imaging method. Sci. Rep. 5, 14371 (2015). [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Lorente D. et al. Recent advances and applications of hyperspectral imaging for fruit and vegetable quality assessment. Food Bioprocess Tech. 5, 1121–1142 (2012). [Google Scholar]
  20. Lim H.-T. & Murukeshan V. M. Pushbroom hyperspectral imaging system with selectable region of interest for medical imaging. J. Biomed. Opt. 20, 046010 (2015). [DOI] [PubMed] [Google Scholar]
  21. Jahr W., Schmid B., Schmied C., Fahrbach F. O. & Huisken J. Hyperspectral light sheet microscopy. Nat. Commun. 6, 7990 (2015). [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Vasefi F. et al. Polarization-sensitive hyperspectral imaging in vivo: a multimode dermoscope for skin analysis. Sci. Rep. 4, 4924 (2014). [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Leitner R. et al. Multi-spectral video endoscopy system for the detection of cancerous tissue. Pattern Recogn. Lett. 34, 85–93 (2013). [Google Scholar]
  24. Martin M. E. et al. Development of an advanced hyperspectral imaging (HSI) system with applications for cancer detection. Ann. Biomed. Eng. 34, 1061–1068 (2006). [DOI] [PubMed] [Google Scholar]
  25. Nelson M. P. & Myrick M. L. Fabrication and evaluation of a dimension-reduction fiberoptic system for chemical imaging applications. Rev. Sci. Instrum. 70, 2836–2844 (1999). [Google Scholar]
  26. Gat N., Scriven G., Garman J., Li M. D. & Zhang J. Development of four-dimensional imaging spectrometers (4D-IS). Proc. SPIE 6302, 63020M (2006). [Google Scholar]
  27. Fletcher-Holmes D. W. & Harvey A. R. Snapshot foveal hyperspectral imager. Proc. SPIE 4816, 407–414 (2002). [Google Scholar]
  28. Khoobehi B., Khoobehi A. & Fournier P. Snapshot hyperspectral imaging to measure oxygen saturation in the retina using fiber bundle and multi-slit spectrometer. Proc. SPIE 8229, 82291E (2012). [Google Scholar]
  29. Uedo N., Higashino K., Ishihara R., Takeuchi Y. & Iishi H. Diagnosis of colonic adenomas by new autofluorescence imaging system: a pilot study. Digest. Endosc. 19, S134–S138 (2007). [Google Scholar]
  30. Kashida H. & Kudo S.-e. Early colorectal cancer: concept, diagnosis, and management. Int. J. Clin. Oncol. 11, 1–8 (2006). [DOI] [PubMed] [Google Scholar]
  31. Palmer G. M., Keely P. J., Breslin T. M. & Ramanujam N. Autofluorescence spectroscopy of normal and malignant human breast cell lines. Photochem. Photobiol. 78, 462–469 (2003). [DOI] [PubMed] [Google Scholar]
  32. Vishwanath K. & Ramanujam N. Fluorescence spectroscopy in vivo. In: Encyclopedia of Analytical Chemistry (ed. Robert A. M.) 4–6 (John Wiley & Sons Ltd, 2006). [Google Scholar]
  33. Li B.-H. & Xie S.-S. Autofluorescence excitation-emission matrices for diagnosis of colonic cancer. World J. Gastroentero 11, 3931–3934 (2005). [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Schultz R. A. et al. Hyperspectral imaging: a novel approach for microscopic analysis. Cytometry 43, 239–247 (2001). [DOI] [PubMed] [Google Scholar]
  35. Massaad C. A. et al. Fluorescently-tagged anti-ganglioside antibody selectively identifies peripheral nerve in living animals. Sci. Rep. 5, 15766 (2015). [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Mehta A. D., Jung J. C., Flusberg B. A. & Schnitzer M. J. Fiber optic in vivo imaging in the mammalian nervous system. Curr. Opin. Neurobiol. 14, 617–628 (2004). [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Klimas A. & Entcheva E. Toward microendoscopy-inspired cardiac optogenetics in vivo: technical overview and perspective. J. Biomed. Opt. 19, 080701 (2014). [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplementary Information
srep24044-s1.pdf (856.3KB, pdf)
Supplementary Video 1
Download video file (25MB, avi)
Supplementary Video 2
Download video file (25MB, avi)
Supplementary Video 3
Download video file (36MB, avi)
Supplementary Video 4
Download video file (36MB, avi)
Supplementary Video 5
Download video file (47.8MB, avi)
Supplementary Video 6
Download video file (47.8MB, avi)

Articles from Scientific Reports are provided here courtesy of Nature Publishing Group

RESOURCES