Pencil-like imaging spectrometer for bio-samples sensing

Abstract

Spectrally-resolved imaging techniques are becoming central to the investigation of bio-samples. In this paper, we demonstrate the use of a WIFI-camera as a detection module to assemble a pencil-like imaging spectrometer, which weighs only 140 g and has a size of 3.1 cm in diameter and 15.5 cm in length. The spectrometer is standalone, and works wirelessly. A smartphone or network computer can serve as the data receiver and processor. The wavelength resolution of the spectrometer is about 17 nm, providing repeatable measurements of spatial two-dimensional images at various wavelengths for various bio-samples, including bananas, meat, and human hands. The detected spectral range is 400 nm - 675 nm and a white LED array lamp is selected as the light source. Based on the detected spectra, we can monitor the impacts of chlorophyll, myoglobin, and hemoglobin on bananas, pork, and human hands, respectively. For human hand scanning, a 3D spectral image data cube, which exhibits excellent signal to background noise ratio, can be obtained within 16 sec. We envisage that the adaptation of imaging spectrometer devices to the widely-accepted smartphone technology will help to carry out on-site studies in various applications. Besides, our pencil-like imaging spectrometer is cost-effective (<$300) and easy to assemble. This portable imaging spectrometer can facilitate the collection of large amounts of spectral image data. With the help of machine learning, we can realize object recognition based on spectral classification in the future.

1. Introduction

In the past decade, interest in smartphone-based systems have emerged in numerous applications [1-2]. Utilizing their integrated cameras as detectors, smartphones have the ability to be developed into various optical detection tools. Recently, scientific optical instruments based on smartphones are of growing interest as new tools for bio-sensing [3], micro-imaging [4], fruit-monitoring [5], and alcohol detection [6]. These portable instruments enable people to perform rapid detection.

In 2014, a giant Chinese technology company announced a conceptual smart chopstick. Acting as a food safety device, the smart chopsticks may distinguish the origin of oil, water, and other foods. We believe that optical spectrometers could be a possible technology to such a desired product, since optical sensing can be utilized to study the molecules in food [7], water [8], drug [9], etc. Therefore, a portable spectrometer, with its inherent cost-effectiveness and ease of use, has the potential to become a personal analyzer in daily life. However, few people engage in research of imaging spectroscopy based on smartphones. As a branch of the spectroscopy domain, imaging spectroscopy is of great importance to applications in ocean color sensing [10] and bio-imaging [11], etc. Imaging spectroscopy involves a high throughput optical instrument, which can yield a 3D spectral image data cube (i.e. 2D image data at many wavelength bands). Combining spectral and 2D imaging information, imaging spectroscopy is capable of extracting wavelength-dependent features [12].

Usually a home-made cradle was utilized to hold a smartphone in alignment with other optical elements, and point-of-care tests can be performed in a remote location. However, the incorporation of spectrometer into a smartphone may lead to some inconvenience in use (e.g. adjusting the cradle to align the diffraction element and smartphone camera before spectral detection, spectral detection may impede the audio or video communication), while the core characteristic of point-of-care test systems is easy-to-use. Since only the camera of the smartphone is required for optical detection in such a spectrometer system, the smartphone can be replaced by a WIFI-camera. In this way, the smartphone serves as data receiver and processor, and the standalone spectrometer can be assembled by using professional opto-mechanical elements rather than a home-made cradle. Hence, performance of spectrometer can be more stable. Furthermore, WIFI-camera is usually utilized in a video monitoring system and is suitable for long term use. Therefore, we develop a pencil-like imaging spectrometer (called “PLIS” hereafter; 140 g in weight, 3.1 cm in diameter and 15.5 cm in length) for bio-sample sensing by using a WIFI camera as detection module. The prototype is a high-throughput, wireless, cost-effective, and portable optical sensing platform. The imaging spectrometer can be operated as if using a pencil.

The spectral resolution of the prototype is about 17 nm and the spectral range is 400 nm - 675 nm. Using this instrument, we carried out optical sensing on bananas, pork, and human hands. Based on the experimental spectral results, we can perform testing of ripeness in bananas, detection of myoglobin in pork, and monitoring hemoglobin in human hands. In addition, through manual scanning of the imaging spectrometer, a 3D spectral image data cube can be obtained. Compared with a commercial portable imaging spectrometer, our system can be available at low cost (less than $300). When high spatial resolution is not required, PLIS has the potential to become good choice for field testing that were not feasible earlier. Furthermore, image processing based on machine learning has become a hot topic recently. Utilization of the present imaging spectrometer can facilitate the collection of large amounts of spectral image data. Combined with machine learning, theses spectral image data may lead to some ‘smarter’ instrument.

2. Methods

As shown in Fig. 1, we apply a classical imaging spectrometer setup [13], which is comprised of an imaging module, a diffraction module, and a detection module. As the imaging spectrometer scans over an object, the light emitted (e.g. fluorescence; or reflected) from a line region on the object enters the system through an imaging lens (1, f = 5 mm). The light from the object then gives an image onto a slit (2, width = 60 µm). A doublet lens (3, f = 50 mm) collimates the light passing through the slit. A prism-grating-prism sub-module (4-6, 300 lines/mm for the grating) is then utilized to spread the spectrum over the detection module. The detection module will be described later. All of the optical elements have an external diameter of 25.4 mm and can be installed in optical lens tubes. The optical elements of the imaging (diffraction) module are housed in an optical tube of SM1L20 (SM1L20C) purchased from ThorLabs company.

Fig. 1.

Schematic illustration of the pencil-like imaging spectrometer. Optical elements 1: Imaging lens; 2: slit; 3: doublet lens; 4: prism; 5: grating; (6) prism; 7: CCTV lens; 8: CMOS. The imaging spectrometer can be held like a pencil to scan an object. A white LED array lamp is utilized as the light source, which illuminates the object at an angle of 45 degree. Lights reflected from a line-region (indicated by black dot dash line) on the object (human hand is as an application example in our work) enter the system, which yields a spectral image on the CMOS chip. The spectrum of white LED is also shown as an inset in the lower-left corner of the figure.

Photo of our prototype for the PLIS is shown in Fig. 2(a). The length and diameter are 15.5 cm and 3.1 cm, respectively, and it only weighs 140 g. A WIFI-camera was selected as the detector. The focal length of the WIFI-camera lens is about 3.8 mm, which is similar to the focal length of the camera in a smartphone. The selected WIFI-camera is usually applied in Closed Circuit Television Inspection (CCTV), but still can provide a suitable spectral image, which will be described later. The photo captured by the WIFI-camera can be transmitted to a smartphone through a wireless network in real time. The frame per second (fps) rate for photos with the size of 1280 × 720 is 15 fps. Herein, a smartphone (MX4 Pro, Meizu, Inc., China) was used as the data receiver. As shown in Fig. 2(a), diffraction patterns from the 0th to 2nd order were displayed on the screen of the smartphone. In our study, the 1st order diffraction pattern is utilized to extract the spectral image information. Hence, the 1st order diffraction pattern is defined as the spectral image in this paper. The 1st order diffraction pattern covers approximately 55 pixels in the dispersive direction (defined as pixel index axis) and is about 186 pixels wide. Herein, each pixel index was correlated with a specific wavelength band, and each column represents a spectrum from one spot on the detected object. As a result, overall spectra at 186 spots can be obtained at the same time.

Fig. 2.

(a). A photo of the prototype pencil-like imaging spectrometer; the length and diameter are 15.5 cm and 3.1 cm, respectively, and the weight is 140 g. The diffraction image can be transmitted to a smartphone through wireless network. (b). The spectrum of a mercury lamp acquired by the imaging spectrometer. Inset shows the original spectral image. One vertical line of the spectral image was selected to plot the spectral curve. (c). The spectrum of the same mercury lamp acquired by a commercial spectrometer. It was found that the wavelengths for the spectral peaks are consistent with (b).

In order to acquire real spectra, wavelength calibration is needed to translate pixel index to wavelength. Aiming our prototype at a mercury lamp, a spectral image was captured and shown in the inset of Fig. 2(b). There were six horizontal lines (red, lighter red, orange, green and blue and purple lines), which represent 620 nm, 596 nm, 579 nm, 546 nm, 436 nm, and 405 nm light from the mercury lamp [14]. The pixel index values of these six lines were 357, 363, 366, 374, 400, and 408. Based on a 3th order polynomial fitting [5], the pixel index values can be translated into wavelength values. After wavelength calibration, each pixel index corresponded to a specific wavelength. The color image was then transformed to a grey image (image not shown); the grey values of one column in the grey image were defined as the spectral intensity. Combining the wavelength corresponding to the pixel index and the spectral intensity, the spectral curve could be plotted, as shown in Fig. 2(b). Due to the short focal length of the lens in the WIFI camera, the spectral resolution was limited. From Fig. 2(b) one sees that the peaks of 581 nm and 598 nm could barely be separated. Therefore, the spectral resolution could be estimated to be about 17 nm. We also measured the spectrum of a mercury lamp by a commercial spectrometer, the results are shown in Fig. 2(c). It was found that the wavelengths for the spectral peaks are consistent with (b). The profile of spectral curve in Fig. 2(b) differed from the profile of accurate spectra in Fig. 2(c). The reason is the optical intensity response spectra of the CMOS sensor is altered by RGB filer. In practice, the reflectance spectrum is utilized. Assuming the spectrum of incident light is denoted as R_s(λ), the reflected spectrum from any object is denoted as R_f(λ). We then obtained the reflectance spectrum through R_f(λ)/R_s(λ).

3. Results and discussion

3.1 Ripeness testing

To illustrate the performance of the PLIS, the reflectance spectra of bananas was firstly investigated. Herein, we sequentially measured two bananas: one was green, and the other was fully ripe. During the measurement, a white LED array lamp (about 1200 lumens) was used as the light source. The incident angle was about 45 degree and the distance between the light source and the samples was 30 cm. In this way, uniform illumination condition can be achieved. During the experiment, all samples were placed on a black board. In order to avoid the disturbance of external light, all measurements were performed in a darkroom. It is worth mentioning that the background can hardly affect the detection of the reflected spectra from bio-samples in our experimental set-up (see section 3.3 for details).

The spectral image captured by the imaging spectrometer was sent to the smartphone via WIFI networking. The exposure time was set to 0.06 sec. After wavelength calibration, the reflectance spectra for the green and ripe bananas could be obtained, as shown in Fig. 3(a) and 3(b), respectively. We believe the difference between the spectra of Fig. 3(a) and 3(b) originates from the chemical compounds in the banana peel. It is well-known that the content of chlorophyll in fruit is directly related to its maturity [15]. Due to the strong absorption of chlorophyll in the 640 nm - 680 nm range [16], the reflectance spectrum of the green banana exhibited a sharp downward trend at the red light band, and this result was almost identical to previous data published by Merzlyak et.al. [Fig. 2(a) of Ref.17], i.e. the reflectance after 650 nm increases when chlorophyll decreases. In contrast, the measured data for the ripe banana presented a strong reflectance at the red light band. This is because as the banana ripened, the chlorophyll was essentially converted to carotenoids, which don’t absorb light in the >600 nm range [16]. According the above result, the PLIS could be used to test the ripeness. It is necessary to note that, due to a built-in infrared cut-off filter (cutoff after 675nm) in front of the CMOS detector, the reflectance sharply declined when the wavelength is longer than 675 nm.

Fig. 3.

The pencil-like imaging spectrometer was utilized for ripeness testing and myoglobin detection. A white LED array lamp was used as the light source. (a) and (b) show the reflectance spectra from green and ripe bananas, respectively. The green banana exhibited a sharp downward trend at the red light band. (c) and (d) show the reflectance spectra from the muscle and fat parts of pork, respectively. The insets in (c) and (d) illustrate the reflectance around 540 nm range. There is an absorption around 540 nm in (c) due to the myoglobin in the muscle part of pork.

3.2 Myoglobin detection

A similar measurement was carried out to detect myoglobin in pork. As shown in Fig. 3(c), a reflectance spectrum, which was derived from a muscle part of pork, showed optical intensity attenuation around 540 nm. The muscle part contains a large amount of myoglobin, and myoglobin has an absorption around 540 nm [18, 19]. This measurement result was in good agreement with the absorption characteristics of myoglobin. Furthermore, investigation into the fat part was performed as a comparison test. Since less myoglobin was present in the fat part, the absorption of fat around 540 nm band was very small (<0.009 cm −1) [20]. Hence, the reflectance spectrum exhibited an upward spectral curve between 530 nm and 550 nm, as shown in the inset of Fig. 3(d). Based on the above result, our PLIS is shown to have the ability to detect myoglobin in pork.

3.3 Push-broom scanning for human hand

To demonstrate spectral imaging feasibility based on push-broom scanning, the first author held the imaging spectrometer to scan his left hand, which was put flat on a black board with the palm facing up. Light from an external LED lamp was incident upon the hand. The imaging spectrometer was firstly aimed at the wrist, and then scanned over the left hand. During the scanning, the camera was operated in video mode. One frame of the video represented a reflectance spectral image for one line region of the hand. After scanning, a video could be captured. This video consisted of 200 spectral images and lasted 16 sec. The scanning length was about 20 cm. Stitching all of the spectral images, a 3D spectral image data cube could be obtained, as shown in Fig. 4(a), in which the spatial image was derived from summing the spectral images over all wavelength bands. Since the scanning was manually performed, image blur and distortion were inevitable. However, five fingers and a palm could be distinguished. The 3D spectral image data cube pave a way for a deep investigation into the detected object. For example, we can study the signal to background noise ratio. With the help of the spatial image, we can select one horizontal line region to plot its reflectance intensity line profile, as shown by the inset in Fig. 4(a). The intensity of background is almost zero. From the intensity line profile, one sees that our PLIS detects no reflected light at the background. Therefore, the reflectance spectra obtained by our PLIS can provide accurate data for deep investigation.

Fig. 4.

(a). Schematic illustration of the 3D spectral image data cube. Herein, the spatial image represents the total reflectance image, see detail in text. The inset show the reflectance intensity line profile denoted by the green dotted rectangle region. (b) and (c) show the reflectance spectra derived from fingertip and palm, respectively; (d). The photo taken during the scanning experiment; (e). The reflected image at the 580 nm band, see more detail in the main text.

The reflectance spectra for the fingertip and palm are shown in Fig. 4(b) and 4(c), respectively. As the blood oxygen saturation was 98% (measured by an oximeter in advance), the fingertip was almost full of oxygenated hemoglobin (HbO₂). The HbO₂, which absorbed a large amount of light of 540 nm and 580 nm [21], led to optical attenuation in the reflectance spectrum around 540 nm and 580 nm, as shown in Fig. 4(b). Furthermore, as shown in Fig. 4(d), since the red color of the fingertip was more saturated than that of the palm, the concentration of blood at the palm should be lower. The reflectance spectrum of palm is shown in Fig. 4(c), in which less blood led to less absorption around 540 nm and 580 nm. As the reflectance at 580 nm was related to the absorption of hemoglobin, we could study the concentration of blood based on the spectral image at the 580 nm band (denoted as $R(580 \mathrm{nm})$), in which a smaller numerical value referred to a higher concentration. Hence, $1\text{-}R(580 \mathrm{nm})$ could provide a qualitative assessment of the concentration of blood. As shown in Fig. 4(e), the numerical value of $1\text{-}R(580 \mathrm{nm})$at the middle region was smaller compared with that at the finger region, indicating a lower concentration in the middle of palm. As shown in Fig. 4, our system can be utilized to obtain a 3D spectral image data cube for a human hand via manually push-broom scanning. Similarly, this manually scanning can be used to perform point-of-care tests on other parts of the body. This spectral image data may provide useful information for health monitoring or biomedical sensing.

4. Conclusion

In this paper, we have demonstrated the PLIS for bio-sensing. Firstly, the details of our prototype were introduced. This imaging spectrometer was then utilized in fruit and pork measurements. The experiments presented in this paper have demonstrated the ability to detect chlorophyll and myoglobin. Furthermore, we have utilized this PLIS to scan a human hand, and a 3D spectral image data cube (spatial two-dimensional images at various wavelengths) has been obtained. Due to the regional differences in blood concentration, the reflectance showed differences in optical intensity attenuation around 540 nm and 580 nm. We could utilize the spectral image at the 580 nm band to predict the concentration of blood. By using an infrared camera, the spectral range for our PLIS can be expanded. For tissue samples, infrared light (1310 nm or 1680 nm) shows greater penetration depth compared with visible light [22–24]. Our PLIS with an infrared camera can be utilized to study deep human tissue.

Having presented these basic capabilities, it is worth mentioning that our imaging spectrometer platform is still not yet optimized in terms of spectral resolution because the prototype was made from commercially-available optical components. We expect significant spectral resolution improvements in the future by using an appropriate OEM WIFI-camera with a long focal length lens. Additionally, the sensitivity of CMOS in the imaging spectrometer limits its application range. By using an enhanced sensitivity detector, this instrument may be suited for fluorescent detection in bio-sensing.

After obtaining spectral image cube data, another important issue is to perform classification of spectra. Recently, machine-learning algorithms have been applied in the analysis of spectral data, see e.g [25, 26]. As a combination of machine-learning algorithm and spectral image data cube, the present portable imaging spectrometer would find more interesting applications in the future.

Disclosures

The authors declare that there are no conflicts of interest related to this article.

Funding

This work is supported by National Key R&D Program of China (2016YFA0201701/2016YFA0201700), the National Natural Science Foundation of China (11621101), the Fundamental Research Funds for the Central Universities (BUCTRC201601), the scientific research fund of Hainan University (No. kyqd1653), and Natural Science Foundation of Hainan Province (617022).