Speckle illumination SFDI for projector-free optical property mapping

Abstract

Spatial frequency domain imaging can map tissue scattering and absorption properties over a wide field of view, making it useful for clinical applications such as wound assessment and surgical guidance. This technique has previously required the projection of fully characterized illumination patterns. Here, we show that random and unknown speckle illumination can be used to sample the modulation transfer function of tissues at known spatial frequencies, allowing the quantitative mapping of optical properties with simple laser diode illumination. We compute low- and high-spatial frequency response parameters from the local power spectral density for each pixel and use a lookup table to accurately estimate absorption and scattering coefficients in tissue phantoms, in vivo human hand, and ex vivo swine esophagus. Because speckle patterns can be generated over a large depth of field and field of view with simple coherent illumination, this approach may enable optical property mapping in new form-factors and applications, including endoscopy.

Tissue optical properties, especially the absorption (μ_a) and reduced scattering coefficients (${\mu }_s^{\prime }$), provide useful information on tissue composition, oxygenation, and metabolism [1–3]. Absorption and scattering maps are being used for an increasing variety of clinical applications, including image-guided surgery, wound monitoring, and assessment of surgical margins [4–6]. Moreover, unlike color values, optical properties are absolute measurements that can be directly compared across different imaging platforms, study sites, and time scales, facilitating their statistical interpretation via machine learning.

Over the last decade, spatial frequency domain imaging (SFDI) has emerged as a powerful tool for optical property measurements. Conventional SFDI acquires images of tissue under sinusoidal illumination at different spatial frequencies and phase offsets, sampling the modulation transfer function (MTF) of the tissue [7]. Because the bulk tissue scattering and absorption properties preferentially attenuate the high- and low-spatial frequencies of the MTF, respectively, sampling the MTF at a range of spatial frequencies allows the decoupling of the two effects. Images are subsequently demodulated and calibrated, and model inversion is performed using a lookup table (LUT). SFDI can rapidly generate wide-field optical property maps using a camera and projector in a non-contact configuration. These advantages have led to the exploration of SFDI for a number of clinical and research applications [1,4–6,8,9].

Despite its advantages, there are several practical challenges to translating SFDI to clinical settings. To generate an optical property map, conventional SFDI requires a minimum of six images at each wavelength (three phase offsets at two different spatial frequencies) and performs a pixel-wise LUT search. Progress has been made toward improving the acquisition and processing time. For example, the acquisition requirements can be relaxed by estimating the MTF from a single spatial frequency using signal processing [10] and content-aware machine learning [11]. Processing speed has also been improved with GPU-based implementations and machine learning [12–14]. A second major challenge is that SFDI requires the projection of structured illumination with precisely controlled imaging geometry and known spatial frequencies. This requirement has made the translation of SFDI to endoscopy and other space-constrained applications particularly difficult. Previous work has implemented SFDI in custom benchtop endoscopy systems with an added projection channel [15,16], but this approach requires either rigid relay optics or low-pixel-count fiber imaging bundles, each of which take up significant cross-sectional area of the endoscope.

An alternative to projecting images of patterns to sample the MTF is to utilize the speckle patterns formed by the interference of coherent illumination. Analyzing tissue response to speckle patterns is often used to estimate blood flow in laser speckle contrast imaging [17,18]. This approach has the important advantage of being amenable to low cost and compact implementation, for example, by laser illumination coupled through an optical fiber. Moreover, compared to image projection, coherent illumination can produce patterns with high-spatial frequencies over a large depth of field. Recently, Jain et al. [19] analyzed the response of turbid liquid phantoms to laser illumination and found that speckle patterns are blurred in agreement with a model based on the optical properties of the sample. However, this technique has several limitations. First, this work developed a forward model that correlates with sample measurements but did not directly measure optical properties or evaluate accuracy. Second, this model analyzes full images, producing one response measurement for an entire image, and thus is not capable of producing an optical property map and only works on homogeneous samples.

In this study, we present speckle illumination spatial frequency domain imaging (si-SFDI), which maps the optical properties of turbid media from unknown laser speckle patterns. For the same imaging geometry and optical properties, the integral of the power spectral density (PSD) should be constant for any speckle pattern randomization. Therefore, for the same imaging geometry but different optical properties, relative changes in the MTF can be sampled at consistent spatial frequencies via a phantom calibration, but without knowledge of the exact speckle pattern. This phenomenon can be exploited to reconstruct tissue optical property maps using as few as one speckle pattern. There are two main contributions of this study. First, we develop an inverse model to accurately measure tissue optical properties from random speckle images. Second, we apply this technique to heterogeneous, biological tissues and evaluate its performance. To our knowledge, this is the first attempt to directly map optical properties from unknown laser speckle patterns.

The experimental setup is shown in Fig. 1 (a). A 520 nm wavelength laser diode is used for illumination (Opt Lasers micro RGB laser module) and a 1392 × 1040 pixel sensor with a camera lens ( f /# = 2.1) is used for imaging, resulting in 138 μm pixels in the object space when the sample is 30 cm away. A tunable lens (Optotune EL-10–30-TC) focuses the laser beam onto a 220-grit diffuser (Thorlabs DG20–220). By varying focal length, the tunable lens alters the laser spot size on the diffuser surface and, subsequently, the size of the speckles in the object space. Speckle size was adjusted to be an average of 3 pixels per speckle grain width to satisfy the Nyquist criterion. The diffuser is mounted on a NEMA 11 stepper motor, which rotates at random angles to vary the speckle pattern. The light is spread by a diverging lens (Thorlabs LD1613-A) to match the 60° camera field of view. Additionally, the illumination and detection paths are cross-polarized to reduce specular reflections. The laser illumination path is mounted at a 30 deg angle on the side of a commercial SFDI imaging head (Modulim Reflect RS), which has a projector at a 12° angle and was used for measuring ground truth optical properties at 526 nm using three-phase 0 and 0.2 mm⁻¹ sinusoidal illumination.

Fig. 1.

Overview of si-SFDI. (a) Experimental setup. A laser diode (L) is focused by a tunable lens (TL) on a rotating diffuser (D) mounted on a stepper motor (SM). A diverging lens (DL) spreads the light to match imaging vergence. Two linear cross polarizers (P1 and P2) reduce specular reflections. (b) Processing flow. The mean autocorrelation function (ACF) is calculated on a sliding window for N speckle images. Results are radially averaged, and the fast Fourier transform (FFT) is taken to produce a local power spectral density (PSD). After calibration, AC and DC reflectances (R_d,AC and R_d,DC) are used to estimate optical properties from a lookup table (LUT).

Figure 1(b) summarizes the si-SFDI algorithm. Laser speckle illumination, which is randomized by object and diffuser movements, is a wide-sense stationary process [19–21] with a constant autocorrelation function (ACF) and PSD. We characterize the tissue response in the spatial frequency domain by analyzing the PSD of each speckle image. This allows tissue optical properties to be calculated without knowledge of the exact illumination pattern. The image of the remitted signal, v(x, y), can be expressed as the convolution of the impulse response h(x, y) of the sample and system with the illumination speckle pattern u(x, y), or v(x, y) = h(x, y) * u(x, y). To reduce computational cost, we estimate the ACF utilizing the Wiener–Khinchin theorem:

$$a_v(x,y)={\mathcal{F}}^{-1}\left[V^{\ast }\left(k_x,k_y\right)\cdot V\left(k_x,k_y\right)\right],$$ (1)

where V and V * represent the Fourier transform of the output image v and its conjugate. Taking the magnitude of the Fourier transform of the a_v, we obtain the PSD S_V (k_x, k_y). Assuming the impulse response is radially symmetrical, the PSD is

$$S_V\left(k_r\right)={\left|H\left(k_r\right)\right|}^2\cdot S_U\left(k_r\right).$$ (2)

Thus, given a measured image of a reference phantom with known optical properties under coherent illumination, the ratio between the PSDs can be calculated and subsequently scaled by the reference model response predicted by Monte Carlo simulations. In this way, we characterize the sample response H(k_r) without measuring the input pattern u(x, y).

In order to reduce noise in the 0-lag PSD, we apply a low-pass median filter with a kernel size of 3 pixels to the raw image. Additionally, accuracy and spatial resolution of optical property mapping can be improved by considering N speckle patterns for each sample (Fig. 1). To generate a PSD for each pixel in the image, we first compute a normalized ACF for a 73 × 73-pixel sliding window, which samples a minimum spatial frequency 0.2 mm⁻¹ due to radially averaging of the ACF about the center pixel. This spatial frequency is commonly used for SFDI [22]. The mean ACF ($\overline{\text{ACF}}$) is measured as the average of N windows at each window location. We further improve signal-to-noise ratio by assuming the response to diffuse optical properties is radially symmetric and averaging the 2D ACF about the radius from each pixel. The same process is performed for a reference phantom with known optical properties. To reconstruct optical properties from the PSD maps, we use a 2D LUT with a DC and AC response parameter, similar to conventional SFDI. The DC response (M_DC) is the remitted signal under planar illumination and can be approximated as the mean of N raw speckle images. We subtracted the estimated M_DC from each speckle image so that the calculated ACF had a zero mean. We define the AC response (M_AC) as the sum of the frequency response corresponding to the first two points of the PSD curve (0.1 mm⁻¹ ≤ f ≤ 0.5 mm⁻¹, where f stands for spatial frequency). Similar to [7], the DC and AC reflectance (R_d) can then be calculated using $R_d=\frac{M}{M_{\text{ref}}}\cdot R_{d,\text{ref},\text{pred}}$, where M_ref is the response parameter of the reference phantom, and R_d,ref,pred represents the reflectance value predicted by White Monte Carlo models for the reference. Optical properties are then fit using a LUT that relates (μ_a, ${\mu }_s^{\prime }$) to (R_d,DC, R_d,AC).

We first validated si-SFDI on homogeneous samples by imaging 16 tissue-mimicking phantoms with unique combinations of optical properties. The phantoms were fabricated by polydimethylsiloxane (PDMS) with titanium dioxide and India ink as scattering and absorption agents, respectively. For each phantom, we compared optical properties estimated by si-SFDI to those computed by conventional SFDI in a central 150 × 150 pixel region of interest (ROI). For phantoms, the mismatch in illumination wavelengths (520 nm for si-SFDI and 526 nm for SFDI) was corrected, which is discussed in more detail in Supplement 1. With four input images (N = 4), si-SFDI results are plotted against SFDI ground truth in Fig. 2. We found that the si-SFDI predicted optical properties fit a y = x ground truth curve with an R² value of 0.996 for absorption and 0.991 for reduced scattering. The variance of the si-SFDI measurements increases for larger ${\mu }_s^{\prime }$, likely due to decreased blurring of the speckle patterns, making the measurements more sensitive to local speckle grains. Analyzing the same ROI across all 16 phantoms, we observed an average pixel error of 6.9% for μ_a and 5.7% for ${\mu }_s^{\prime }$ when using four speckle patterns (N = 4). Using a single speckle pattern, the average pixel errors increased to 12.9% for μ_a and 11.0% for ${\mu }_s^{\prime }$. Optical property measurement statistics for each phantom pair estimated by one-image and four-image si-SFDI are shownin Fig. S1, Supplement 1.

Fig. 2.

Optical property measurements from si-SFDI on 16 homogenous tissue phantoms. (a) si-SFDI measurements of absorption and (b) si-SFDI measurements of reduced scattering versus conventional SFDI ground truth.

To assess the accuracy of si-SFDI in heterogeneous samples with irregular surface topography, we tested the technique on in vivo human hand and ex vivo swine gastroesophageal junction [Fig. 3(a), (b)]. Side-by-side comparisons of four-image si-SFDI with ground truth SFDI show that for flat regions, there is excellent agreement over the whole field of view. si-SFDI recorded a sharp change in optical properties between the stomach and esophagus regions of the swine sample, in agreement with SFDI, and representative of the expected change in tissue type on either side of the junction.

Fig. 3.

Comparison of one- and four-image si-SFDI optical property estimates with SFDI ground truth in (a) in vivo human hand, and (b) ex vivo pig gastroesophageal junction. N indicates the number of input patterns. (c) Absolute percentage difference as a function of number of speckle patterns used in the si-SFDI calculation for the ROI indicated by the white boxes in (a) and (b). Error bars are standard deviations. Black boxes in (a) highlight a scar that did not produce contrast in SFDI.

To explore the effect of the number of speckle patterns on the accuracy of optical property calculations, we analyzed a relatively flat region on the back of the hand and at the gastroesophageal junction [250 × 250 pixels, highlighted by the white boxes in Figs. 3(a) and 3(b)] and plotted the error with varying numbers of images [Fig. 3(c)]. This error was calculated as the absolute percentage difference. The errors for both absorption and scattering measurements decrease significantly when additional speckle patterns are included, but beyond N = 4 the improvements become small—the decrease in errors from N = 4 to N = 10 images is only approximately 2%. Analyzing additional speckle patterns has the effect of both increasing signal to noise and improving the spatial resolution of optical property mapping, since additional patterns improve the sampling of the local ACF in our window-based approach. Single-image si-SFDI results (N = 1) are also shown in Figs. 3(a) and 3(b). As expected, the results show fair agreement with SFDI, however, with more noise and image artifacts than N = 4. To further investigate the accuracy and image quality of si-SFDI applied to heterogeneous samples, we calculate the structured similarity (SSIM) index for the pig esophagus and human hand sample (Table 1). Overall, si-SFDI demonstrates high SSIM scores compared to profile-corrected SFDI as reference. On average, si-SFDI with four images (N = 4) achieves 6.3% higher SSIM than N = 1, and this improvement becomes 8.2% with N = 10.

Table 1.

SSIM of Pig Esophagus and Human Hand Optical Properties Obtained by si-SFDI^a

	Pig Esophagus		Human Hand
Number of Speckle Patterns(N)	μa	${\mu }_s^{\prime }$	μa	${\mu }_s^{\prime }$
N = 1	0.790	0.870	0.777	0.817
N = 4	0.839	0.901	0.846	0.871
N = 10	0.855	0.912	0.874	0.893

^aSFDI is used as reference.

For both hand and esophagus samples (Fig. 3), we observe a larger discrepancy between si-SFDI and ground truth absorption than in scattering measurements. This may be partially due to the mismatch in wavelengths used (520 nm and 526 nm for si-SFDI and SFDI, respectively). For example, the extinction coefficients of hemoglobin are different at these two wavelengths, with a ratio of approximately 1 to 1.2 [23]. This mismatch could also contribute to the differences in scattering, but likely to a lesser extent [24]. Moreover, in areas of the sample with irregular surface topography, we found larger errors in absorption than in scattering estimates. This is expected because absorption depends mostly on M_DC, while scattering depends mostly on M_AC, which is less affected by surface angle variations. Additionally, the difference in illumination angle for the laser diode compared to the SFDI projector is expected to contribute to reconstruction errors. These effects are further explored and discussed in Supplement 1. Compared to profile-corrected SFDI, the pixel error rate of si-SFDI is below 20% when the surface is flat (<10 deg) and increases with larger angles.

We observed a small region with large errors on the hand sample [highlighted by the black boxes in Fig. 3(a)]. From inspection of the hand, we found a scar in this region that does not generate contrast in conventional SFDI but was captured with si-SFDI. This difference in contrast may be due to the si-SFDI incorporating higher spatial frequencies than SFDI, which would lead to a shallower sampling of tissue optical properties [7] and also contributions from sub-diffuse scattering parameters. A recent study similarly uncovered a scar when using high-spatial frequency structured illumination [25]. We note that even at the high-spatial frequencies analyzed in si-SFDI, we did not observe a change in optical property estimation when blood flow was occluded (Supplement 1).

There are several areas for improvement of the si-SFDI technique. First, multiple speckle patterns are required for the most accurate optical property estimate. Although this can be achieved by using a rotating diffuser or phase randomization, it inevitably limits the imaging speed. However, instead of averaging multiple speckle patterns to approximate the DC response, one can use a laser speckle reducer to approximate planar illumination and accurately estimate M_DC from a single image. In this scenario, only two images would be required. Moreover, if the average illumination power of these two images is different and known, sensitivity to ambient lighting can be reduced. Alternatively, planar illumination may be approximated from coherent illumination using a content-aware deep learning approach [26], allowing accurate single-frame si-SFDI. Second, the processing speed of si-SFDI is slow due to window-based computations. It takes 2 s to compute a 1040 × 1392-pixel optical property map from a single speckle pattern (N = 1) using a four-core 3.6 GHz processor running the algorithm in MATLAB (R2020a, MathWorks). However, because each window computation is independent, this computation is highly parallelizable and can be accelerated by using a graphics processing unit. Third, for many clinical applications, the topography of the sample will need to be estimated and accounted for. This may be accomplished in simple setups with recent monocular depth estimation techniques [27,28].

To summarize, we demonstrated a wide-field, projector-free technique for non-contact optical property mapping using laser speckle patterns as structured illumination. When applied to both homogeneous phantoms and heterogeneous biological tissues, si-SFDI accurately measures scattering and absorption parameters. With further optimization, this technique can facilitate the translation of quantitative optical property mapping in more clinically friendly systems.