Cell Phone based Colorimetric Analysis for Point-of-Care Settings

Abstract

Cell phones show considerable promise for point-of-care (POC) diagnostic procedures because they are accessible, connected, and computationally powerful. Cell phone image processing methods are being developed for the detection and quantification of a wide range of targets, employing methods from microscopy to fluorescence techniques. However, most of the lab-based biological and biochemical assays still lack a robust and repeatable cell phone analogue. Existing solutions require external smartphone hardware to obtain quantitative results, imposing a design tradeoff between accessibility and accuracy. Here, we develop a cell phone imaging algorithm that enables analysis of assays that would typically be evaluated via spectroscopy. The developed technique uses the saturation parameter of hue-saturation-value color space to enable POC diagnosis. Through the analysis of over 10,000 images, we show that the saturation method consistently outperforms existing algorithms under a wide range of operating field conditions. The performance improvement is also proven analytically via the mathematic relationship between the saturation method and existing techniques. The method presented here is a step forward towards the development of POC diagnostics by reducing the required equipment, improving the limit of detection (LOD), and increasing the precision of quantitative results.

Table of Contents:

Cell phone imaging based method for bioanalytical applications under various operating field conditions.

Introduction

Cell phones have emerged as highly capable evaluation tools at the point-of-care (POC) for a wide range of applications, including cell phone microscopy^1–6, cytometric analysis, lateral flow assays^7–9, paper-based microfluidic devices^10–12, and colorimetric tests^13–17. Cell phones are an attractive option because they combine sophisticated onboard sensors, powerful processing capability, and wireless connectivity into a portable device. Distributed cell phone sensing networks can generate spatiotemporal maps of measurement results, providing information for epidemiology studies, environmental monitoring, and other applications where real-time measurements are critical^14,18,19. Self-contained smartphone applications, which solely use the cell phone processor, have been reported and are capable of providing a readout for biomedical assays in remote locations, where communication with offsite computer systems and medical professionals may be unavailable²⁰. In particular, camera technology has improved at an exponential rate, and the camera sensor has been used to obtain useful quantitative and semi-quantitative measurements with low limits-of-detection (LOD) for fluorescent and non-fluorescent assays. Cell phone image algorithms have been successfully applied for analytes including the human immunodeficiency virus (HIV) p24 capsid protein²⁰, tuberculosis DNA¹⁰, mercury and chlorine contaminants²¹, glucose and protein concentration in urine^15,19, ovarian cancer antigens²², and water-borne pathogens²³. As a result, most mobile phone-based methods inherently involve some form of cell phone image processing, which is consequentially becoming an important field.

Image-based tests attempt to replace bulky laboratory equipment with an easily accessible smartphone device. They could be a viable solution for healthcare in the developing world because they can allow untrained users to collect data for transmission to medical professionals. Cell phone image processing algorithms are accessible, user-friendly, and equipment-free²⁴. However, in practice cell phone image tests have limitations that severely restrict their utility. Cell phone cameras automatically perform a variety of preprocessing algorithms, including demosaicing, gamma correction, sharpening, and lossy image compression. These algorithms are optimized for image appearance rather than for quantitative image-based measurements and cannot be bypassed or reversed easily²⁵.

Ambient lighting and focal distance to the sample have also been identified as major factors influencing the quality of a readout, leading to approaches that require external imaging equipment to standardize conditions during image capture¹¹. In some cases, alternative approaches must include a calibration step that is repeated before each analysis^26,27. The tilt angle of the camera relative to the sample, the separation distance, changes in ambient light intensity, and local variations of pixel intensity values have all been identified as practical limitations^26,28. The most convenient color space for cell phone image processing is the red-green-blue (RGB) color space, because cell phone images are natively stored as arrays of RGB pixel intensities, commonly referred to as color channels. Unfortunately, RGB values are highly susceptible to ambient lighting noise, particularly when the illumination varies between different regions of the image²⁹. This problem is most commonly mitigated through the introduction of external smartphone attachments^30,31. These attachments include LEDs with optical diffusers to provide constant illumination, an imaging chamber to block ambient light variations, rigid components to hold the phone in place, and optical filters or lenses.

Software-based strategies to combat the adverse effects of unpredictable image capture conditions are also available, but are mainly used in conjunction with external imaging equipment and attachments. Image processing algorithms have been reported to address ambient lighting issues, including dynamic thresholding, image morphology methods, pixel intensity normalization, and frequency-domain techniques^32,33. These methods are applied to normalize outputs for multiplexed assays and are also employed in cell phone microscopy to prepare for cell counting. For visible light spectroscopy, statistical methods are the preferred analysis technique. The parameter used to quantify analytes, or the output metric, is often a function of the arithmetic mean pixel intensity (MPI) inside a region-of-interest (ROI) for one or more of the RGB image channels. The MPI of an individual RGB channel has been successfully used directly as the output metric, but alternative approaches can improve system performance^34,35. A common alternative is to use a ratio of RGB channel MPIs, referred to as RGB ratio tests¹⁰. Principal component analysis has been used, but this method is application-specific and has not been used for quantification²¹. Instead, color space transformation has proven to be a very effective, simple, and general method for quantitative colorimetric analysis. In color space transformation analysis, RGB values are transformed to an alternative set of coordinates before MPI analysis is performed^15,16. The standardized CIE 1931 chromaticity space has enabled quantification of protein and glucose without any imaging equipment, but is only sensitive to changes in color¹⁵. An analogous method, using the hue channel of hue-saturation-value (HSV) space, has also been used for the same purpose with similar limitations³⁶. Assays that respond with a chromatic intensity change but no wavelength shift cannot be analyzed using these methods. Assays that would normally use absorbance spectroscopy are primarily limited to noise-sensitive RGB analysis and RGB ratio tests at the POC.

Here, we develop a color space transform technique that can be used to analyze intensity change assays with significant improved precision when compared to existing RGB-based methods. This algorithm transforms images to HSV space and uses the MPI of the saturation channel as the output metric. HSV space is a well-established cylindrical-coordinate representation of RGB space, with an invertible transformation from RGB to HSV coordinates. In HSV space, the hue channel corresponds to the color or wavelength of the sample, the saturation channel corresponds to the chromatic intensity, and the value channel is a measure of the overall light intensity²³. The saturation channel has been used to provide readout for colorimetric tests and was found to increase the correlation and decrease the LOD of the BCA assay.³⁷ However, saturation is not a well-established, common measurement technique for bioanalytic applications and it has not been investigated as an enabler for equipment-free imaging. Here, we mathematically show that saturation can achieve similar improvements for a wide range of colorimetric applications, and we show that it is equivalent to the optimal form of a popular class of application-specific RGB ratio test. We analytically show that, while ambient light intensity variations are present in all three RGB channels, saturation is robust to ambient lighting conditions. We verify this result empirically with 10,000 ROI images taken under a range of image capture conditions.

We investigate whether the robust noise performance of saturation is sufficient to enable equipment-free colorimetric testing by processing images taken under a range of ambient light levels, against a variety of backgrounds, with and without external imaging equipment, and with three separate cell phone cameras. To make our analysis of image capture conditions complete, we record a stream of 2,000 images while recording the gyroscope and accelerometer sensors to investigate the effect of camera angle and distance to the sample. We also provide a discussion of ROI histogram properties to explain local pixel intensity variations. This analysis provides general recommendations for image capture methods when MPI analysis is used, and we also find that saturation outperforms RGB methods under all experimental conditions. To validate our technique, we apply saturation MPI analysis to a direct ELISA for the HIV p24 capsid protein. We obtain a 2x improvement in LOD and we demonstrate that the saturation parameter of HSV space can enable an equipment-free evaluation for p24 antigen.

Methods

Cell Phone Image Processing Algorithm

OpenCV, a cross-platform computer vision library, was used to implement the cell phone image processing algorithm because the same library is available for the Windows, OS X, Linux, iOS, and Android operating systems. This allows the app to be used for the two main types of POC smartphone-enabled systems, both on the cell phone device and on a remote server. As a proof-of-concept, an Android application was written to show that saturation computations for a typical ROI are computationally within range of low-end smartphone processors. However, for data collection purposes, a semi-automated application was written for desktop systems that processes cell phone images.

Our image processing system has four components: image capture, ROI extraction, ROI processing, and quantification (Figure 1). The image capture phase consists of reading the CMOS camera sensor data on the cell phone and is handled by the operating system of the phone. At this stage, unavoidable preprocessing algorithms including demosaicing, sharpening, gamma correction, JPEG compression, and white balancing occur. The ROI extraction process removes portions of the image that are not relevant to the test and distinguishes between multiple samples if they are present. The ROI processing step is used to perform color-space transformations, image filtering, thresholding and morphology, and other image processing algorithms. Finally, quantification occurs when some characteristic of the processed ROI is used to estimate a physical quantity. In our application, we use image-based measurements to estimate absorbance readings from a spectrophotometer and hence protein concentrations measured by a direct ELISA.

Figure 1:

Diagram of the saturation algorithm. First, images of a 96 well plate or other assay container are captured using a smartphone camera. Regions of interest are extracted through manual or automatic methods and are converted to HSV space. After the conversion process, the standard MPI analysis is applied to the saturation channel and the values are used to determine absorbance and concentration of the sample.

Image Capture with Different Smartphones

We performed image capture using three smartphones: the Android Moto G with a 5 megapixel (MP) camera, the iPhone 6 with a 12 MP camera, and the Samsung Galaxy Edge 7 with a 12 MP camera. To facilitate data collection, we developed a desktop GUI application using Python and OpenCV to enable manual ROI extraction. Manual ROI extraction was chosen because human performance is the gold standard for image segmentation problems and because a robust ROI segmentation algorithm that works for the 96 well plate is not currently available. Users can draw and drag circular ROIs that are superimposed on top of an image, which are then removed from the source image, saved individually, and batch processed separately. In this case, the ROI area was defined as the portion of the image corresponding to a single well of the 96 well plate. This is the portion of the plate that is read by the plate reader. This process enables the collection of large amounts of data, by allowing all ROIs to be isolated from a 96 well plate. Furthermore, this enables meaningful comparisons between samples in a plate, since image capture conditions are approximately the same for all 96 ROIs of a source image. While optical aberrations have been reported at the edges of cell phone images, the effects primarily concern microscopy applications, not statistical image processing methods and so do not have a significant effect. Therefore, we imaged the entire 96 well plate. A skilled user required approximately one minute to extract the 96 ROIs from an image using the extraction GUI. To ensure that human error was not responsible for features in the dataset, we extracted the same ROI ten times and measured the standard deviation of the resulting MPI values. We found that the standard deviation is substantially less than one MPI unit, therefore our ROI collection method is sound. While a similar user interface would be impractical on a smartphone, known methods can be employed for ROI extraction on mobile devices^15,17. Such methods typically involve imaging a small number of samples at a time, which is the equivalent of extracting fewer ROIs with the desktop application. The saturation algorithm is contained entirely in the ROI processing step and is not dependent on the type of ROI extraction, provided that the extraction is adequate. Therefore, results from the desktop system are directly applicable to mobile systems.

Our ROI processing consists of transforming the RGB coordinates to HSV color space and computing the MPI, which is the arithmetic mean of pixel intensities, in the saturation color channel. The MPI values are then used to quantify the biological analyte. RGB coordinates can be transformed to normalized HSV space via Equations 1–3. We used the non-normalized HSV space, which simply requires multiplication by 256. Equation 4 shows the process to obtain the MPI of channel α, denoted as α_MPI.

$$S=\frac{\max \left(R,G,B\right)-\text{min}\left(R,G,B\right)}{\max \left(R,G,B\right)}$$ (1)

$$H=\{\begin{array}{cc}60\left[\frac{G-B}{\max \left(R,G,B\right)-\min \left(R,G,B\right)}\right]\text{mod}\left(6\right)& \max \left(R,G,B\right)=R\\ 60\left[\frac{B-R}{\max \left(R,G,B\right)-\min \left(R,G,B\right)}+2\right]& \max \left(R,G,B\right)=G\\ 60\left[\frac{R-G}{\max \left(R,G,B\right)-\min \left(R,G,B\right)}+4\right]& \max \left(R,G,B\right)=B\end{array}$$ (2)

$$V=\text{max}\left(R,G,B\right)$$ (3)

$${\alpha }_{MPI}=\frac{1}{N}\sum _{n=0}^N{\alpha }_n$$ (4)

Image Capture at Various Imaging Conditions

Three sets of data were collected under a wide range of capture conditions. We considered four variables for image capture conditions – ambient lighting noise, lighting level, distance to the sample, and the background color. Ambient lighting noise and light level were controlled within an imaging box, which was rapid-prototyped using a 3D printer and constructed according to well-established principles^30,31,38. The light from six LEDs was passed through an optical diffuser to obtain a constant illumination level for the entire plate. The light level was controlled by a variable current driver powering the LEDs. Images taken of the sample within the enclosure have the lowest ambient lighting noise and the best image capture conditions. Ambient lighting noise was introduced to the system by imaging the plates in the open, without the imaging box. The background color was considered because some studies have discussed this factor as a design parameter for colorimetric paper-based microfluidic devices^10,33. Based on existing work, a dark background outside the imaging box was assumed to represent the worst-case image capture conditions.

Inside the imaging box, the distance and angle to the sample were controlled by the imaging box itself. The camera was laid flat against the imaging hole in the enclosure, which ensured that there was no angle to the sample and that the distance was constant. Outside the imaging box, we held the cell phone parallel to the well plate at a pre-marked distance from the plate. To quantitatively understand the effects of distance and angle to the sample, we collected a stream of 2,000 images of the plate while moving the camera and tracking the sensor measurements of the cell phone.

Measure of Algorithm Performance

We considered three measures of algorithm performance. First, image-based metrics should be strongly associated with conventional measurements. We analyzed this by plotting MPI values against absorbance for the same test, forming a linear regression model, and calculating the correlation. Immediately after each image was taken of the 96 well plate, the optical density was measured using a spectrophotometer and the saturation MPI values were compared with the spectrophotometer readout. This procedure is a well-established method to test cell phone based absorbance spectroscopy algorithms³⁹. If image-based metrics have a strong relationship with laboratory measurements, and the laboratory measurements are an accurate predictor of biological analytes, then the image metrics will also measure the biological analytes, albeit with a larger standard error.

Image-based metrics should also be able to represent biological values, such as protein concentrations, in a similar way to conventional measurements. To show how MPI values compare to OD and to each other, we plotted the five number summary of the output values vs. concentration in a boxplot. This shows the variance of the MPI response and provides a way to compute the instrument LOD, or the magnitude of MPI response that is significant at low concentrations.

The pixel intensity distribution within the ROI should be Gaussian or at least unimodal with low skewness. If this is not true, then the well-established MPI method is not valid and will not accurately represent the central tendency. In practice, this results in outliers that increase the mean-squared error of the linear regression model and the variance within each concentration category. We verified this requirement by computing the image pixel intensity histograms for all RGB and HSV channels and ensuring that most ROIs did not have a significant departure from normality.

HRP Antibody-Conjugate Concentration Curve

Initially, we prepared various samples from a serial dilution of the horseradish peroxidase (HRP) antibody-conjugate (Supplementary Materials, Assay Protocol 1). These results were useful for showing the response of image metrics when exposed to a wide range of absorbance values. We also used these results to measure the effect of distance and angle to the sample during image capture.

Application to HIV p24 Capsid Protein Quantification

To validate the algorithm, HSV analysis was applied to quantify a direct ELISA for HIV-1 p24 antigen. The p24 antigen is the capsid protein that forms the structural virion component of HIV. It is widely used targeted in several HIV-1/−2/-O ELISA diagnostic kits using colorimetric quantification^40–43, and serves as means to detect HIV viremia⁴⁴, as it is present in early stages of infection^12,45. The diagnostic window is shortened, with detection found prior to antibody seroconversion, when HIV antibodies are detectable^40–46. This comprehensive approach using combined antigen/anti-HIV antibody detection, leveraged with rapid point-of-care diagnostic platforms is further enhanced using cell phone absorbance quantification, and could potentially be used to detect a multitude of different pathogens or diseases. For anti-p24 HIV direct ELISA curve (Supplementary Materials, Assay Protocol 2), images were taken for a variety of image capture conditions, including variable light level, presence and absence of an image box, variable distance to the sample, and variable background color. This set of data corresponds to absorbance values in the linear response range for MPI analysis.

Results and Discussion

Strength of Relationship with Absorbance

We applied the saturation algorithm to a calibration dataset from a standard dilution and to a direct ELISA for the HIV p24 protein. For the calibration dataset, we generated scatterplots for the set of ROIs in each image with a variety of phones and image capture conditions. The Supplementary Materials contain three image sets, where each image is of a 96 well plate and contains a number of sample ROIs, each of which has an absorbance value measured via spectrophotometer and an MPI value measured via cell phone camera. We computed the Pearson product moment correlation between absorbance and MPI.

Analysis of the wide-range HRP antibody-conjugate curves (Supplementary Table 1, Supplementary Figure 1) shows that there is a strong relationship between OD and MPI for a wide range of absorbance values. A logarithmic model provides the best prediction over a wide optical density range with high r-squared correlation (r² > 0.98), but there is a linear region in the MPI response from 0.2 OD to 2.0 OD (Supplementary Table 2 and Table 3, and Supplementary Figure 1, 2, and 4). The scatterplots also reveal that some of the RGB information is redundant, since each color channel MPI shows a response to changes in absorbance (Figure 2b). Even though all the RGB channels respond, RGB analysis is typically done only with the channel with the largest dynamic range. In this case, the blue MPI had the largest dynamic range and was therefore chosen as the RGB metric. In HSV space, the hue parameter is constant, and the value parameter is approximately constant. Only the saturation parameter responds to changes in absorbance, and the dynamic range of saturation MPI (Figure 2a) is larger than the dynamic range of the blue MPI variation (Figure 2b).

Figure 2:

Typical absorbance and concentration results for RGB and HSV pixel intensity values. The MPI-absorbance curve is approximately linear when OD < 2.0 for (A) hue, saturation, and value MPIs and (B) red, green, and blue MPIs. When images are taken both inside and outside the imaging box, saturation (C) retains its slope and intercept while blue (D) displays a much larger variation, shown by the size of the boxplot boxes (N = 42 images, each with 8 replicates. Each box in (C) and (D) contains 336 measurements. Each scatterplot in (A) and (B) has N = 64)

Within the linear range, we used a linear regression model for the responses in Figure 2 and found that the correlation between MPI and absorbance is higher for both RGB and HSV MPIs when the enclosure is used to block out ambient lighting noise (Figure 3a). Within the enclosure, the correlation coefficient of the logarithmic model is equally high for both methods (r² > 0.9). However, outside the imaging box, RGB MPI values fail to maintain a predictable relationship with absorbance, with correlation values dropping below 0.5. The effect is particularly destructive when a black background is used (Figure 3a). In terms of the strength of correlation, the saturation MPI is not affected and remains very strong (r² > 0.9) even under non-ideal image capture conditions outside the box. The difference between images taken inside and outside the lighting box can be visualized with Figure 3b.

Figure 3:

(A) R² Correlation between MPI and Absorbance (OD) for 42 iOS images and (B) examples of the RGB color image, blue channel, and saturation channel from images number 4 and 32. In (A), shading (left part) indicates images taken within the enclosure, whereas right part indicates images taken outside the enclosure. Each correlation was obtained using (N = 64) measurements from the linear region of the MPI curve.

Sensitivity to Camera Distance, Tilt, and Motion

We captured a video stream of 2,000 images while we moved the cell phone, varied the distance to the sample, and manipulated the angle between the camera and the sample. We measured the roll, pitch, yaw, acceleration, vertical height, and rotation rate of the phone by interfacing with the sensors on the iPhone 6S. To understand the effect that these factors have on MPI analysis, we plotted the readout quality (measured by r² correlation) over the course of the video and we also correlated the sensor measurements with the assay readout quality. This can be seen in Figure 4.

Figure 4:

Analysis of image capture conditions. The correlation matrix (top) shows the Pearson product-moment correlation of measurement quality (rows) with the sensor data (columns). Measurement quality is represented by the r² and standard error of the linear regression with blue (r²_B, σ_B) and saturation (r²_S, σ_S). To interpret the matrix entries, note that each row corresponds to a measurement quality feature and each column corresponds to a sensor. If a matrix entry is shaded red, then an increase in the corresponding sensor is associated with an increase in the measurement quality feature. A blue entry indicates that an increase in the sensor is associated with a decrease in the quality feature. For instance, an increase in H is associated with a decrease in r² and an increase in standard error. The plot (bottom) shows the readout quality over the video and the sensor data results. At t = 4 seconds and t = 10 seconds, the distance to the sample (H) decreased by 0.8 meters. Between t = 15 and 22 seconds, the phone was tilted in the pitch and roll directions by up to 30 degrees. Note that the effects of camera motion when H changes degrade the measurement quality until the camera is still.

We found that as distance to the sample (H) increases, r² decreases and the standard error increases for both saturation and blue MPI. Increases in pitch and roll are also associated with a decrease in r², and a low standard error tends to accompany a high correlation, as desired (Figure 4). The correlation matrix for r² and standard error against sensor measurements shows that a decrease in the angle and distance to the sample is correlated with an increase in r² and a decrease in standard error. For this reason, we recommend maximizing the area occupied by the ROIs, taking the image as close to the sample as possible, and taking the image directly above the sample when performing MPI analysis. These recommendations were used when taking images for the HIV p24 analysis, with good results.

Histogram Properties

The histograms of each of the RGB channels were all similar in terms of variance and multimodality. We observed that the distributions of saturation and blue pixel intensities were closely related and appeared to be approximately Gaussian. ROI pixel intensity distributions inside the box had lower variance than distributions outside the box, except for the hue channel. The histogram of the hue channel had a constant, small variance both inside and outside of the imaging box, explaining the success of colorimetric tests based on the chrominance signal. For the saturation and blue channels, we found that the ROIs with a multimodal pixel intensity histogram were the ones that produced the outliers in the concentration response and absorbance response (Figure 5). We found that the ROIs that had multimodal distributions tended to be farthest from the center of the image and contained reflections through the plastic 96 well plate. This provides further evidence for our earlier recommendation that images be taken with minimal distance to the ROI. However, the majority of our RGB and HSV distributions are unimodal with low skewness, therefore the simple MPI analysis was appropriate and the results are valid. The regression analysis is justified by the central limit theorem applied to the means of the MPI distributions.

Figure 5:

Typical distributions for the saturation channel. The vast majority of saturation and blue ROIs had distributions similar to plots A, B, and C, but some ROIs produced multimodal distributions similar to D and E. In the scatterplot, we have labeled the data points associated with distributions A-E. MPI values that came from multimodal distributions (red) tend to be outliers, while those from normal distributions (green) tend to cluster closer to the regression line.

Concentration Response and Limit-of-Detection (LOD)

We plotted the sets of saturation and blue MPI values against the dilution levels (Figure 2c, Figure 2d) to show how MPI responds to changes in concentration. Each source image generates up to 96 ROI sample images, but each image also included up to 8 duplicates of the same concentration. The plots show all of the saturation and blue MPI values associated with a given concentration, over a set of 42 images, which had variable lighting conditions and background color (Supplementary Table 2). The spread is shown as a five number summary boxplot, where 50% of the data falls within the box and the bars represent upper and lower extremes. We found a large variance for RGB MPI values of images taken outside the enclosure, to the point where an unknown concentration cannot be correctly characterized and the LOD is affected. The serial dilution results show that the slope and variance of the saturation response are more consistent than those of the blue response when images are taken outside the box. This indicates that saturation MPI is more consistent under varying measurement conditions.

For the HIV p24 capsid protein quantification task, we computed the LOD, which is a measure of the smallest significant variation in test readout at low concentrations. We used the variance of blanks to calculate the LOD and the limit of quantification (LOQ). Although this computation assumes that the response is homoscedastic and the variance of the MPI response does change with concentration, at low concentrations the variance is relatively constant (Figure 6, Supplementary Table 4). Within the box, we found an LOD of 1.8 ug/mL for saturation MPI and an LOD of 3.7 ug/mL for blue MPI. Outside the box, we found an LOD of 2.3 ug/mL for saturation and 4.9 ug/mL for blue MPI. Due to saturation’s higher dynamic range (147 pixel intensity units vs 128 units), 50% lower LOD, and 50% lower LOQ, saturation is a better metric than blue for quantitative purposes.

Figure 6:

Response of saturation and blue MPI to concentration of p24 HIV capsid protein inside and outside enclosure. Saturation MPI inside the box (A) had an LOD = 1.8 ug and LOQ = 2.3 ug (N = 40). Saturation MPI outside the box (B) had an LOD = 2.2 ug and LOQ = 3.1 ug (N = 24). Blue MPI inside the box (C) had LOD = 3.7 ug and LOQ = 4.5 ug (N = 40). Blue MPI outside the box (D) had LOD = 4.9 ug and LOQ = 5.4 ug (N = 24).

The absorbance values from the HIV p24 quantification task were outside the linear range of the MPI-absorbance response identified in the calibration dataset (Figure 7), but still had sufficiently high variation to identify concentration ranges. Interestingly, the LOD for absorbance, computed graphically using the method of blanks (Figure 7) increased for the blue channel when the assay was completed outside the box but did not increase for the saturation channel. The r² value of the blue channel decreased, but the r² value of saturation did not.

Figure 7:

Linear regression for the MPI and absorbance points from the HIV p24 assay. These measurements are outside the linear range identified in Figure 3 since all OD < 0.6. The saturation and blue thresholds are those corresponding to the mean of the blanks ±3.3 standard deviations. The LOD in terms of absorbance is the intersection of the regression line with the threshold. In particular, note that the LOD for saturation is lower than the LOD for blue. Inside the box, r² for saturation was 0.97 and r² for blue was 0.95. Outside, r² for saturation was 0.97 and r² for blue was 0.87.

In a practical setting, we wish to estimate the concentration of an unknown sample. If absorbance values are available, this can be done by fitting a concentration-absorbance standard curve to a calibration dataset. Then, one would determine the concentration of an unknown sample by inverting the curve for a given absorbance value. We may quantify the precision of this measurement by reporting the 95% prediction interval of our concentration estimate.

If MPI values are available, a similar analysis is possible because we may convert MPI values to absorbance values using the regression model. Alternatively, we may also directly model the relationship between MPI and concentration. However, this is equivalent to the first procedure when the concentration-absorbance standard curve is one-to-one and invertible. In the practical case where the concentration-absorbance standard curve contains some measurement uncertainty, a direct model will potentially offer better performance and a smaller prediction interval because it does not necessarily introduce the uncertainty from the standard curve.

For an ideal standard curve and the MPI analysis method, the 95% prediction interval for a concentration estimate is determined by the 95% prediction interval of the MPI-absorbance regression model. Given an MPI value with an unknown absorbance, the actual absorbance will fall within this interval with 95% probability. By finding the concentration values associated with the upper and lower bounds of the 95% absorbance prediction interval, we obtain a concentration interval inside which the actual sample concentration will fall with 95% probability. For a practical concentration-absorbance relationship, we must add the prediction interval of the standard curve.

Figure 8 shows the process of estimating an unknown concentration for the HIV p24 assay for a sample with a measured saturation MPI of 50. Using the regression model, this MPI value corresponds to an absorbance of 0.25 OD, resulting in a concentration estimate of 3.4 μg/mL. However, the 95% prediction interval of the regression line specifies that the actual sample absorbance may lie between 0.12 OD and 0.35 OD, which corresponds to a 95% concentration prediction interval of 1.6 μg/mL to 5.6 μg/mL.

Figure 8:

Link between the regression analysis and concentration measurement. The concentration estimate, upper bound and lower bound for the concentration 95% prediction interval when MPI = 50 are shown with dashed red lines. The 95% prediction interval of the regression model is shown as a shaded region in the MPI-absorbance plot. The calibration standard curve is shown in the absorbance-concentration plot. Observe that if the 95% prediction interval shrinks, the concentration estimate performance improves because a given MPI value maps to a narrower range of concentration values.

Note that an increase in r² and a decrease in standard error cause the prediction interval to shrink. This causes a corresponding reduction in the concentration prediction interval, explaining the improvement in LOD when using saturation rather than RGB models, shown in Figure 8. The inverse is also true; a reduction in r² and increase in standard error will reduce performance by widening the prediction interval. Empirically, we found that the LOD is compromised when the r² correlation drops below 0.9 (Figure 7).

Properties of Saturation

Additive Noise

To the performance improvement from saturation, consider the effect of additive noise on the RGB and HSV MPI values. Images are captured in the RGB format, so additive noise takes the form of a superimposed noise image that corrupts each pixel. Additive noise may be introduced by uneven illumination, shading, tilt and distances to the sample. If the sample is illuminated by diffuse, full spectrum white light, then the local ambient lighting variations will be approximately the same for each of the RGB channels. If δ_n represents the additive noise at pixel n with a mean of δ_MPI, then the RGB MPI for the blue channel of a noisy image (BN_MPI) and the saturation channel of a noisy image (SN_MPI) are:

$$B{N}_{MPI}=\frac{1}{N}\sum _{n=0}^N{B}_n+{\delta }_n$$

$$B{N}_{MPI}=B_{MPI}+{\delta }_{MPI}$$

$$S{N}_{MPI}=\frac{1}{N}\sum _{n=0}^N\frac{\max \left(R_n,G_n,B_n\right)+{\delta }_n-\min \left(R_n,G_n,B_n\right)-{\delta }_n}{\max \left(R_n,G_n,B_n\right)+{\delta }_n}$$

$$S{N}_{MPI}=\frac{1}{N}\sum _{n=0}^N\left(\frac{\max \left(R_n,G_n,B_n\right)-\min \left(R_n,G_n,B_n\right)}{\max \left(R_n,G_n,B_n\right)}\right)\left(\frac{\max \left(R_n,G_n,B_n\right)}{\max \left(R_n,G_n,B_n\right)+{\delta }_n}\right)$$

$$S{N}_{MPI}=\frac{1}{N}\sum _{n=0}^N{S}_n\left(\frac{\max \left(R_n,G_n,B_n\right)}{\max \left(R_n,G_n,B_n\right)+{\delta }_n}\right)$$

If max(R_n,G_n,B_n) >> δ_n, as is often the case, then the saturation MPI will not be affected by additive noise because the multiplicative distortion approaches unity. On the other hand, RGB MPIs add the mean of the noise to the output metric. Additive noise is present during image capture and can be due to diffraction, reflection, ambient lighting variations, and illumination by multiple sources of incoherent light. Regardless of the noise source, saturation MPI exhibits a reduced sensitivity to additive pixel intensity variations.

Multiplicative Noise – Ambient Lighting Level

Saturation also has desirable characteristics when exposed to multiplicative noise, which occurs when the overall lighting level changes. This is the primary cause of differences between RGB MPI curves from different images, since image sets taken under similar conditions tend to have low variance (Supplementary Figure 3, Supplementary Figure 5). A lower overall light level compresses the entire curve, resulting in a downward shift. When an image is illuminated by white light and the intensity of the light changes due to shadows or large-scale changes in light intensity, the effect is multiplicative because low light levels result in pixel values close to zero, while bright light results in proportionally larger RGB values. For multiplicative noise where the light level changes by some value α_n:

$$B{N}_{MPI}=\frac{1}{N}\sum _{n=0}^N{\alpha }_n{B}_n$$

$$S{N}_{MPI}=\frac{1}{N}\sum _{n=0}^N\frac{{\alpha }_n\max \left(R_n,G_n,B_n\right)-{\alpha }_n\min \left(R_n,G_n,B_n\right)}{{\alpha }_n\max \left(R_n,G_n,B_n\right)}=S_{MPI}$$

The saturation MPI is not affected by multiplicative noise and is therefore insensitive to the ambient light level. This enables equipment-free imaging because the ambient lighting level no longer has to be standardized to compare the output against a standard curve, explaining the consistency of results in Figure 2c. This built-in pixel intensity normalization is not present for RGB analysis. An extreme example can be found in Image 20 of Supplementary Table 3, where the saturation MPI values and dynamic range are comparable to other tests under better conditions, even though the light level is very low. The RGB response had a greatly reduced dynamic range and was much different from the response for ideal capture conditions.

Relationship with RGB Ratio Tests

For the spectroscopy task considered here, the absorbance changes while the color and wavelength do not. Hue is the HSV space analogue for wavelength, and it did not change (Figure 2a). An interesting result, which can be observed in Supplementary Tables 1, 2, and 3, is that when the hue in HSV space does not change, the maximum and minimum components of the RGB images always come from the same channels. This can also be obtained analytically using Equation 2, which describes the HSV hue computation from RGB coordinates. We start by assuming that one of the color channels is maximum. Hue is a nonnegative quantity, and the denominator is always positive. Therefore, the minimum color channel is constrained because the numerator cannot be negative. For instance, when red is maximized, blue must be the minimum so that G – B is nonnegative. This has practical implications for absorbance spectroscopy. Specifically, this leads to the result that saturation represents the most optimal RGB ratio test in this case. The saturation MPI can be written as:

$$S_{MPI}=\sum _{n=0}^{n=N}\frac{S_n}{N}=\sum _{n=0}^{n=N}\frac{1}{N}\left(1-\frac{\min \left(R_n,G_n,B_n\right)}{\max \left(R_n,G_n,B_n\right)}\right)$$

$$S_{MPI}=1-\frac{1}{N}\sum _{n=0}^{n=N}\frac{\min \left(R_n,G_n,B_n\right)}{\max \left(R_n,G_n,B_n\right)}$$

If the wavelength does not vary, min(R,G,B) will always come from the same color channel, as will max(R,G,B). Therefore, saturation is fundamentally related to the particular RGB ratio with the greatest difference between numerator and denominator, and so there will be the largest possible variation in response. Therefore, the saturation MPI is an optimized RGB ratio test that maximizes the dynamic range and is adjusted to pass through the origin. Since the HSV color space transformation does not increase the variance of the MPI response except at very high concentrations (Figure 6), the saturation MPI is equivalent to the most optimal RGB ratio test.

The properties of the saturation MPI make it a good alternative to RGB-based output values for the analysis of colorimetric assays where the absorbance changes but the wavelength does not. For this set of assays, saturation is strongly related to the ratio tests commonly in use, but has reduced sensitivity to additive and multiplicative noise. Ambient lighting variations, which are commonly cited as the most problematic factor for equipment-free evaluation, have a greatly reduced effect on saturation, as shown by the analytic results and the correlation with absorbance. This causes the LOD of a saturation-based test to be lower than the one for an RGB-based test. Multiplicative noise and light level shifts are greatly reduced when using saturation, which has important consequences for repeatability and practical POC systems. Since the ambient light level is no longer a factor affecting the readout, the intensity of the light source no longer needs to be strictly controlled. A practical model for equipment-free POC evaluation must have sufficiently high correlation and low standard error to accurately detect the target (Figure 8), but must also have stable model parameters from calibration dataset to the test dataset. The saturation method substantially improves the first problem. However, although it is better than RGB, saturation does not completely solve the second issue. Methods to correct for lighting bias and to stabilize parameters do exist, but they degrade performance of the model by decreasing the dynamic range or increasing the variance^32,33. Alternatively, it is possible to re-calibrate the model for a new set of environmental conditions^26,27, which is similar to what we did in this study. Since saturation improves the performance of the model, it may help offset the cost of a stable model. If re-calibration or colorimetric calibration strips are an option, then saturation enables equipment-free evaluations with reasonable LOD. Saturation’s performance without equipment is approximately equivalent to RGB performance with equipment, making POC applications more convenient and accessible if precise quantitation is not needed. If precise measurements are desired, saturation still improves performance though external equipment may be necessary. If environmental conditions warrant bias correction, saturation may help harden the model against the performance issues incurred by stabilization efforts.

Provided that the color does not change, the saturation MPI can be used for any color sample, since the argument relating it to a RGB ratio test is valid for all hue values. However, saturation MPI analysis cannot be used for color change assays, as saturation does not contain chrominance information. Instead, the existing CIE or hue MPI method should be used.

Conclusion

We developed a cell phone based image preprocessing method that produces an MPI output with smaller variances, lower LODs, and a higher dynamic range than existing methods. Using several thousand images, we compared saturation analysis with existing RGB methods and found both analytically and empirically that it has improved performance in the presence of additive and multiplicative ambient light noise. We also showed that saturation analysis can be interpreted as an optimized version of existing RGB ratio tests, and verified that the ideal image capture conditions include constant white light, a clean white background, minimal distance to the sample and zero angular displacement of the camera. When we applied the test to an ELISA for the p24 HIV capsid protein, saturation analysis enabled an equipment-free test evaluation with an LOD two times lower than the one available with RGB methods. The saturation MPI represents an improvement in repeatability, practicality, and image capture noise rejection. Saturation analysis is not affected by many of the major limiting factors for image-based tests, such as ambient lighting variations, shading, and variable light levels. We anticipate that the favorable properties of saturation analysis will encourage and enable cell phone image-based POC tests with less equipment overhead and lower LODs.