Quantification of Plant Chlorophyll Content Using Google Glass

Abstract

Measuring plant chlorophyll concentration is a well-known and commonly used method in agriculture and environmental applications for monitoring plant health, which also correlates with many other plant parameters including, e.g., carotenoids, nitrogen, maximum green fluorescence, etc. Direct chlorophyll measurement using chemical extraction is destructive, complex and time-consuming, which has led to the development of mobile optical readers, providing non-destructive but at the same time relatively expensive tools for evaluation of plant chlorophyll levels. Here we demonstrate accurate measurement of chlorophyll concentration in plant leaves using Google Glass and a custom-developed software application together with a cost-effective leaf holder and multi-spectral illuminator device. Two images, taken using Google Glass, of a leaf placed in our portable illuminator device under red and white (i.e., broadband) light-emitting-diode (LED) illumination are uploaded to our servers for remote digital processing and chlorophyll quantification, with results returned to the user in less than 10 seconds. Intensity measurements extracted from the uploaded images are mapped against gold-standard colorimetric measurements made through a commercially available reader to generate calibration curves for plant leaf chlorophyll concentration. Using five plant species to calibrate our system, we demonstrate that our approach can accurately and rapidly estimate chlorophyll concentration of fifteen different plant species under both indoor and outdoor lighting conditions. This Google Glass based chlorophyll measurement platform can display the results in spatiotemporal and tabular forms and would be highly useful for monitoring of plant health in environmental and agriculture related applications, including e.g., urban plant monitoring, indirect measurements of the effects of climate change, and as an early indicator for water, soil, and air quality degradation.

Introduction

Large-scale industrialization over the past century has led to a variety of expanding environmental impacts (e.g., air, soil and water pollution, deforestation, desertification), resulting in both urban and rural public health safety concerns and significant alarm over human-driven planet change^1–11. Additionally, global warming and its effects have led to changes in precipitation distribution and an increase in average temperatures globally,^12,13 significantly impacting plant and animal ecosystems in various ways such as e.g., plant growth rates, soil mineralization, metabolic rates, life-cycle changes and animal migration patterns^14–18. Due to their ubiquitousness and resilient nature, plant health and growth rates have been used as indicators for various environmental factors^19–21. For agriculture applications, rapid plant monitoring has remained a subject of great importance for maintaining plant health and identifying potential emerging diseases, which can affect plant storage dynamics and crop production efficiency^22–24. Mainstream efforts focus on indirectly measuring plant chlorophyll concentration, which is considered an important metric for general plant health. Abnormal levels of chlorophyll may be indicators of important plant stress agents such as e.g., plant diseases, climate change, lack of or excess amounts of nutrients, light, or water, or the presence of toxic substances (e.g., cadmium)^25–29.

The gold standard for direct measurement of plant chlorophyll concentration is through chemical extraction of the chlorophyll from plant specimens, whereby plant leaves are mechanically dissociated and dissolved using chemicals (e.g., acetone), their chlorophyll filtered from the other plant compounds, and subsequently measured using a spectrophotometer. This process is inherently destructive, expensive, complex, and, due to its sample preparation steps, requires trained personnel within a controlled lab environment. To combat these disadvantages, over the past three decades several indirect methods for estimating chlorophyll levels of plants have been developed^30–33. Hyperspectral satellite imaging techniques^34–36 using indirect chlorophyll estimation systems have been effectively used to control and monitor crop fields and forested regions, deforestation processes, and the spread of invasive species. While effective at the macro scale, these methods require complex and expensive hardware and time-consuming data acquisition and processing. In recent years, various hand-held optical systems (e.g. GreenSeeker™, atLeaf+, or SPAD 502) have also been developed to form portable instruments for estimating the chlorophyll information of plants in a local region^37–41. One of the most reliable and most frequently used one of these is the commercially available SPAD (Special Product Analysis Division) chlorophyll meter, created by Konica Minolta Inc.⁴¹, which uses ratio-metric analysis of the light absorption of a leaf under 650 nm and 940 nm wavelength illumination for estimation of site-specific chlorophyll concentration levels on the leaf. While widely used for agriculture and plant physiology studies^42–51, this device is relatively expensive and only provides an estimation of chlorophyll levels for a small (3×2 mm²) area on the leaf surface, necessitating multiple measurements across the leaf surface for estimation of the overall leaf chlorophyll concentration.

Recent technological advances in wireless platforms such as e.g., smart phones, tablets and wearable devices, have opened the gates for creating advanced micro-analysis and diagnostics platforms as well as analytical measurement tools for various biological and medical applications^52–70. Among these, the Google Glass (see Figure 1a), a cloud-connected wearable computer integrated with a camera and various spatio-temporal sensors, is recently emerging as a platform to integrate various diagnostic and biomedical applications into everyday activities^71–73. Utilizing Google Glass and a custom-designed and cost-effective leaf holder and multi-spectral illuminator device, here we demonstrate a rapid, accurate and non-destructive leaf chlorophyll measurement platform. Our hand-held leaf holder external device (see Figures 1b-c) is used to illuminate an inserted leaf uniformly using red (center wavelength: 645 nm; bandwidth: 16 nm) and white (with a wide spectrum spanning >400–700 nm) LEDs such that the images captured by Google Glass can automatically estimate the leaf’s chlorophyll content using a custom-developed Android application. To measure the performance of our system, we compared it against the Konica Minolta SPAD-502 Plus meter using their standard SPAD chlorophyll index value, which directly maps to chlorophyll levels in plants^{42,44,46–48}. Using different plant species across a range of SPAD indices, we demonstrated the ability of our Google Glass based chlorophyll estimator to generate calibration curves (for indoor and outdoor conditions, separately) matching the sensitivity of a commercial SPAD-502 meter. After this calibration step, we also validated that our approach can accurately and blindly estimate the chlorophyll indices of fifteen different plant species, selected from the UCLA Mildred E. Mathias Botanical Garden, under both indoor and outdoor lighting conditions. This Google Glass based rapid and non-destructive chlorophyll measurement platform can prove useful for urban plant monitoring, indirectly measuring the effects of climate change, as well as for early detection of water, soil, and air quality degradation.

Figure 1.

Hardware used during our experiments. a) An overview of Google Glass features. b) Schematic of our custom developed 3D-printed leaf holder and illuminator device. c) An example of using the combination of the Google Glass and our device to quantify the chlorophyll content of plants.

Methods

System Overview

Our approach determines plant leaf chlorophyll concentration using Google Glass images of leaves by leveraging chlorophyll’s low light absorption in the green part of the optical spectrum⁷⁴. Our system is composed of Google Glass as the imager, used with a custom developed software application for image capture, processing, and result display (see Figure 1a), and a handheld 3D-printed leaf holder and illuminator unit which is used to enhance imaging contrast for various illumination conditions (see Figures 1b-c). The Google Glass sends the captured images of the leaves of interest to a remote server for rapid processing and estimation of their chlorophyll content; subsequently, the server sends the results back to the originating Google Glass to be displayed to the user. Our Glass application minimizes operator error using a simple gesture-based hands-free interface for easy positioning of the leaf illuminator unit.

In our experiments, we represent our measured leaf chlorophyll content in the form of SPAD indices. The SPAD index standard is used by our gold standard SPAD-502 chlorophyll meter and has both plant-independent and plant-specific mappings to chlorophyll concentration levels^{42,44,46–48,50}. To calibrate our platform for use under various ambient lighting conditions, we measure each leaf using a SPAD-502 meter (Konica Minolta, Japan) and map the intensity values calculated from Google Glass images against these SPAD indices.

Hardware

The hardware used in our system is comprised of Google Glass and a custom-designed hand-held leaf holder and illuminator unit. Both devices are shown in detail in Figure 1. Figure 1a labels important features of the Google Glass used as part of this chlorophyll quantification system. Note that we utilize only the built-in hardware of the Glass and do not attach any external hardware to it.

Figure 1b labels our leaf illuminator device, which is used to create uniform red and white LED illumination on the leaf sub-region to be measured. The entire leaf illuminator device can be assembled under a cost of 30 USD even for low-volume manufacturing, and can be divided into three parts: the main body, external cap and internal electronic board. Both the main body and external cap were built using a 3D Fused Deposition Modeling (FDM) printer (Stratasys, Dimension Elite) that uses ABS plastic. The main body protects the electronics and forms a uniform light pattern internally. The external light isolation cap is added to reduce intensity changes on the internally illuminated leaf area due to external lighting conditions. This cap is attached onto the main body by using magnets placed on the main body and the inner part of the cap, enabling easy attachment and detachment for leaf placement between the cap and main body. The leaf can thus be placed into position without inflicting any damage to the leaf by first removing the cap, then placing the leaf on the main body, and finally replacing the cap. The region of interest (ROI) this device can image on the leaf surface is approximately a circular area of 5 cm². An electronic board placed in the main body holds and can power two red 645 nm wavelength LEDs (Digikey #475-1322-1-ND) or two broadband/white LEDs (Digikey #492-1180-ND). A switch allows the device to alternate between red and white illumination configurations; both sets of LEDs are never on at the same time. The illumination light from these LEDs is reflected off an aluminum foil attached to the body of the device and directly reflected towards the leaf ROI. A diffuser material is placed in between the reflected light and the leaf in order to generate a uniform pattern on the leaf surface. To achieve the necessary illumination levels for constant illumination under various exterior lighting conditions, the device is powered by three Alkaline AAA batteries placed in the upper part of the device and power regulated using a voltage regulator.

Google Glass application for chlorophyll measurements

After installing our custom-designed Google Glass app, the user can run this app by either using the touch-pad on the side of Glass to select the application from the main menu or using the voice command interface with a spoken “ OK Glass, image a leaf ”. Subsequently, the user will place the leaf of interest into the location designated in the accompanying device (as shown in Figure 1b) in preparation for imaging.

Once the application starts, the user is first prompted to select the ambient light option that best fits to the environment where leaf images will be taken (see Figure 2). In this proof-of-concept application, we have limited this selection to two general lighting environments, indoors or outdoors, which in the future could include more options to select from. After making this selection, our Glass application automatically shows a camera preview with a text message overlay requesting the user to take a picture of the leaf under red illumination. The user then turns on the red LEDs on the leaf illuminator device and, to obtain the best repeatability in the measurement, the camera preview overlays a blue rectangular template for the user to match with the device profile (as shown in Figure 2). With this alignment, the relative distance and orientation between the leaf and the Glass camera exhibit minimal variations. If fit to match the displayed template, the distance between the camera and the leaf ROI is approximately 10 cm. Additionally, in order to ensure an optimal estimate of the chlorophyll levels, we prevent strong external light sources from pointing toward the leaf through the cap opening.

Figure 2.

Application flow chart of our Chlorophyll Estimator on Glass. The blue cycle on top shows the user experience while running the application. The grey dashed box shows the processing of the data performed by the server.

After taking a picture of the leaf under red LED illumination, our Glass application opens the camera preview again, this time with a text message overlay that prompts the user to take an image of the same leaf under white LED illumination. Having taken both of these images for the same leaf of interest, the user can then upload them to a remote server using the Google Glass’s wireless connection (e.g., Wi-Fi), where they are digitally processed to create a chlorophyll concentration estimate (which will be detailed in the next sub-section). After processing the uploaded images, the calculated SPAD index is returned back to the originating Google Glass in the form of a timeline card, which displays an image of the ROI, the date and time of image capture, the validity of the ROI region, and the estimated SPAD index (as shown in Figure 2).

Remote server-based leaf image processing

Due to the limited computational performance of Google Glass, a remote server is used to perform the post-processing of the data after image capture (see Figure 2, data processing section). All the Glass images are automatically compressed using the JPEG format, with each image generally not exceeding 1.2 MB in size. Once the leaf images are received, the server first validates each image using its ROI. To enforce the relative distance between the Glass and the leaf, the ROI is first digitally cropped using the template location and subsequently scanned for circular shapes with parameters corresponding to the physical dimensions of the circular cap of our leaf illuminator unit. Upon successful detection of the circular cap, we further limit our ROI by cropping it to a 110-pixel radius circular ROI, incorporating a 40-pixel margin to avoid border/edge effects and enhance the repeatability between experiments.

After successful detection of our illuminated leaf ROI, we then extract only the red channel information for the images taken under the red and white LED illumination. We calculate, for each image separately, the average of all the non-zero elements in the circularly masked leaf ROI. These two values are subsequently correlated to SPAD indices using calibration curves for different lighting conditions, where each calibration curve is generated by sampling several leaves from five different plant species (all selected from the UCLA Mildred E. Mathias Botanical Garden as will be detailed in our Results and Discussion Section) covering a wide SPAD index range. To reduce variability caused by differing plant physiology and lighting conditions, the SPAD indices obtained from the red and white illumination images are then averaged to produce a final SPAD index for each leaf under test. This entire image processing step, leading to a quantified chlorophyll index, takes less than 10 sec on our server (CPU: Intel Core i5-760, RAM: 16GB).

Results and Discussion

The digital calibration process of our Google Glass based platform to generate SPAD values for chlorophyll content was performed using five plant species (i.e., Xylosma congestum, Vivurnum suspensum, Gingko biloba, Asparagus asparagoides, and Hypoestes artistata) obtained from the UCLA Mildred E. Mathias Botanical Garden (see Figure 3). These plants were specifically chosen to have a wide variety of leaf sizes, internal structures, leaf thicknesses and most importantly, SPAD indices. In this calibration process, we processed twenty leaves from each plant species using the Google Glass device. As SPAD correlates better with chlorophyll for fully grown leaves, we only used fully grown leaves from the branches of each plant. Due to the Glass camera’s high sensitivity to lighting conditions, our system requires a different calibration curve for indoor and outdoor conditions. In order to ensure high repeatability in our measurements, all of our outdoor experiments were performed between 10am and 12pm in a location shaded from direct sunlight exposure.

Figure 3.

Calibration measurements and curves for chlorophyll quantification using five different plant species for indoor and outdoor lighting conditions using our Google Glass-based imaging system. Each data point represents the average measurement at a specific region of interest on the leaf. a) Xylosma congestum, b) Vivurnum suspensum, c) Gingko biloba, d) Asparagus asparagoides, and e) Hypoestes artistata.

Figure 3 shows the two characteristic curves obtained for each one of our five calibration plant species, for indoor and outdoor lighting conditions. For each calibration curve, a total of twelve images for each leaf were taken (three images under red and white LED illumination, for indoor and outdoor lighting conditions), leading to a total of 240 Glass images per plant species. Since the leaf area used by the SPAD-502 instrument is ~6 mm², i.e., much smaller than our imaging area (~5 cm²), we use the average of ten SPAD measurements taken using SPAD-502 for each leaf ROI to better calibrate our Glass based chlorophyll measurements. As shown in Figure 3, a strong correlation is obtained for each plant species between the average SPAD index measured using the commercially available SPAD-502 instrument and the average intensity of the red image channel measured using our Google Glass based chlorophyll measurement platform. The SPAD index variability, shown in Figure 3 as standard deviation bars, refers to the variability of the ten measurements taken using the SPAD-502 instrument to cover our device’s large imaging area (~5 cm²) on the leaf. The Glass measurement standard deviations, however, represent the variability in the average intensity values of the three different images taken for each illumination condition and leaf ROI. Because of the significantly larger leaf area that Google Glass can image using our leaf illuminator unit, our measurements show better repeatability as compared to the SPAD-502. Both indoor and outdoor results of the Glass measurements demonstrate a strong correlation to the SPAD-502 measurements. In general, the calibration results obtained for indoor imaging conditions provide better fit compared to the outdoor measurements, possibly due to a more stable and controlled lighting environment indoors.

The SPAD indices for these calibration plants shown in Fig. 3 cover a wide dynamic range. Next, we combined this data to generate plant-independent calibration curves as shown in Figure 4. To obtain these curves, we used one hundred individual leaves and twelve hundred pictures in total: three images of each leaf for both red and white LED illumination under indoor and outdoor lighting conditions. As Figure 4 illustrates, our Google Glass measurements provide a good correlation to the measurements made by the SPAD-502 instrument across a wide range of chlorophyll index values, ranging from 19.1 to 68.2. The calibration curves for red illumination (i.e., red curve) and white illumination (i.e., blue curve) are shown for indoor and outdoor illumination conditions in Figures 4a and 4b, respectively. Optimizing curve fitting based off R² value resulted in exponential and quadratic calibration curves for indoor and outdoor illumination conditions irrespective of the choice of leaf illumination LED (red vs. white). In general, the red LED illumination data provided a better fit, but were also sparser than the white LED illumination data. We also note that although our indoor test results generated more accurate plant-specific calibration curves (Fig. 3), for our plant-independent calibration curves our outdoor curves exhibit slightly superior fit, most likely due to the poor fit of the Hypoestes artistata for indoor conditions.

Figure 4.

Calibration curves for our chlorophyll quantification platform. Red-colored curves represent red LED illumination data. Blue-colored curves represent white LED illumination data. a) Indoor illumination conditions, with exponential curves. b) Outdoor illumination conditions, with quadratic curves.

After this calibration step, next we tested the performance of our Google Glass based chlorophyll quantification system by performing blind tests with sixty fresh leaves taken from 15 different species, where fifty of them were leaves from the same 5 species used in our calibration process and the remaining 10 were leaves taken from 10 other species (Baphia racemosa, Crolalaria agatiflora, Justicia leonardii, Alstonia venenata, Tristaniopsis laurina, Drypetes australasia, Melicytus ramiflorus (male), Montanoa guatemalensis, Tithonia diversifolia, Ceratozamia hildae) that were randomly chosen from the UCLA Mildred E. Mathias Botanical Garden. We tested each leaf three times by taking images in red and white illumination configurations, as specified in Figures 1 and 2. Our Google Glass based blind measurement results are shown in Figure 5, where we achieved a decent correlation to the blind measurements made by a SPAD-502 chlorophyll reader.

Figure 5.

Blind test results of our Google Glass based chlorophyll measurement platform. a) Indoors measurements. b) Outdoors measurements.

In these blind tests, for the same 5 plant species that were used in our calibration process, we obtained a mean SPAD index error of 2.8 and 2.2 with standard deviations of 1.96 and 1.46, for indoor and outdoor lighting conditions, respectively; if we include all the 15 species in our tests, we achieved a mean error of 3.0 and 2.2 with standard deviations of 2.01 and 1.40, for indoor and outdoor conditions, respectively. These numbers indicate that our outdoor tests have slightly superior performance, as also predicted by the better calibration curve fit for the outdoor case compared to the indoor lighting conditions (see Fig. 4). Additionally, our high correlation to the measurements of SPAD-502 reader is maintained for randomly selected plant species not used in our calibration process, achieving similar accuracies when compared with the plant species that were included in our calibration experiments. The fact that our calibration performs very well with other plant species suggests that our selection of calibration plants is diverse enough for accurate measurement of the chlorophyll content in various plants.

We believe that the wireless connectivity requirement of our Glass based chlorophyll measurement platform does not pose a limitation for this technique. The total number of wireless internet users reached 2.1 billion in 2013,⁷⁵ and this growth is not limited only to developed countries. In fact, the mobile-broadband subscriptions in developing countries increased from 472 million to 1.16 billion between 2011 and 2013.⁷⁵ Additionally, there are fast-paced projects (e.g., Google’s Project Loon⁷⁶) that aim to widely deliver wireless connectivity to remote and rural parts of the world. Considering the widespread growth of wireless connectivity over the past decade and the new projects bringing remote and rural parts of the world online, we are confident that the use of Google Glass as a chlorophyll measurement platform is very well suited for today’s highly connected digital world.

One additional advantage of this platform is that our custom-designed Glass application can also provide GPS (Global Positioning System) information of chlorophyll measurements, which is important for spatio-temporal mapping, tracking and analysis of the results. The same feature also permits the users to continue their chlorophyll measurements without stopping or waiting for a local result since they can later correlate their tests with the GPS coordinates on a map and thus be able to exactly determine where each test was captured, at what time, etc. Furthermore, since it is an imaging-based design, our Google Glass chlorophyll measurement system can also be used to determine leaf skeletal structure information in addition to chlorophyll concentration. Finally, based on the correlation between the SPAD index value and the nitrogen content of the plant,^39,42–44 our Glass based platform can also be used to indirectly monitor the soil nutrition content and the crop growth process.

Conclusions

In conclusion, our custom-designed Google Glass app together with a 3D-printed hand-held portable leaf holder and illuminator device can rapidly, accurately and non-destructively estimate chlorophyll levels in various plant species over a wide range of chlorophyll concentrations. Our Glass measurements and SPAD estimations successfully exploit chlorophyll’s characteristic spectral signature by utilizing the red channel intensity of our captured images as the main indicator for chlorophyll concentration. Using this methodology, we successfully generated plant-independent calibration curves, accurately mapping our captured images of plant leaves to standard SPAD indices. Our results suggest that this method can be extended to a wider SPAD range by including plants with higher and lower SPAD indices. This Google Glass based platform can be used as a chlorophyll concentration measurement tool for quick and accurate assessment of plant health under different lighting conditions, providing a good alternative to existing more complex, expensive or time consuming methods.