High-fidelity and high-speed wavefront shaping by leveraging complex media

Abstract

High-precision light manipulation is crucial for delivering information through complex media. However, existing spatial light modulation devices face a fundamental speed-fidelity tradeoff. Digital micromirror devices have emerged as a promising candidate for high-speed wavefront shaping but at the cost of compromised fidelity due to the limited control degrees of freedom. Here, we leverage the sparse-to-random transformation through complex media to overcome the dimensionality limitation of spatial light modulation devices. We demonstrate that pattern compression by sparsity-constrained wavefront optimization allows sparse and robust wavefront representations in complex media, improving the projection fidelity without sacrificing frame rate, hardware complexity, or optimization time. Our method is generalizable to different pattern types and complex media, supporting consistent performance with up to 89% and 126% improvements in projection accuracy and speckle suppression, respectively. The proposed optimization framework could enable high-fidelity high-speed wavefront shaping through different scattering media and platforms without changes to the existing holographic setups, facilitating a wide range of physics and real-world applications.

The speed-fidelity tradeoff in wavefront shaping can be mitigated by leveraging the random property of complex media.

INTRODUCTION

Light scattering is ubiquitous in fog, biological tissues, and other complex media with inhomogeneous and disordered structures, which prohibits direct access to the scene beyond a short transport mean free path, e.g., 100 μm in biological tissues (1–4). Over the past two decades, precise manipulation of light has been demonstrated in and through various complex media, promising a wide range of applications in microendoscopy (5–10), noninvasive deep-tissue imaging (11–19), holographic optical tweezers (20, 21), microfabrication (22–24), and optical telecommunications (25, 26). The rapid progress in wavefront shaping in complex media can be partly attributed to the increasing availability and performance of spatial light modulation devices such as liquid crystal–based spatial light modulators (LC-SLMs) and digital micromirror devices (DMDs). These devices compensate for the scattering process by generating conjugated light fields through transmission matrix (TM) inversion (5, 6, 11, 17, 27, 28), optical phase conjugation (time reversal) (13, 29–31), or iterative wavefront optimization (16, 19, 32–34). For applications that require light manipulation with high spatiotemporal precision, such as holographic optogenetics (35, 36), multimode fiber-based endoscopy (5–10), and holographic three-dimensional (3D) printing (22–24), high-speed and high-fidelity wavefront shaping through complex media is in high demand for fast and precise projection of optimized light fields.

However, almost any existing spatial light modulation devices have a fundamental tradeoff between speed and accuracy due to hardware limitations, including data transfer rates, driving voltages, and heat dissipation. This tradeoff between speed and accuracy is evinced by the competition between the frame rate, pixel count, and modulation depth in the spatial light modulation devices, resulting in dimensionality limitation in wavefront shaping problems (37). For example, LC-SLMs feature high-precision (8 to 12 bits) phase modulation and have been demonstrated for high-fidelity wavefront shaping in various complex media (6, 17, 28, 38–40), yet the frame rates are limited to 50 to 600 Hz. While a 350-kHz 1D SLM has been demonstrated for wavefront shaping in complex media (41), its high frame rate is at the cost of a total 1088 degrees of freedom, which limits the enhancement ratio and, consequently, the focusing quality. On the other hand, DMDs can achieve a frame rate of up to 22 kHz enabled by a high-speed micro-electro-mechanical system, while the precision can be unsatisfactory for high-fidelity wavefront shaping due to the limited modulation depth (1 to 2 bits) in amplitude. To enable high-precision wavefront shaping in high-speed applications, a wide variety of approaches, including the Lee hologram method (42, 43), the superpixel method (44), and the island algorithm (45), have been proposed to convert a binary DMD pattern into a complex wavefront, which is commonly used in TM-based approaches.

Despite the development of holographic coding schemes for DMDs, the light shaping performance remains constrained by the speed-fidelity tradeoff that is ultimately dictated by the frame rate, pixel count, and pixel modulation depth of the spatial light modulation devices, analogous to the space-bandwidth product in imaging and holography (46). Such speed-fidelity tradeoff in wavefront shaping becomes pronounced for high-bandwidth signals, such as high-speed projection of complex patterns (38–40, 47–49), which approaches the intrinsic dimensionality limitation of the devices. To address the artifacts arising from the limited degrees of freedom of a single frame (binary pixel modulation depth), end-to-end (50) and deep learning–based methods (51) were proposed to take advantage of scientific understanding (mathematical models) or observations (training datasets) of a specific system to accommodate the underlying artifacts. However, these methods either require precise calibration of a specific forward model corresponding to a predefined system configuration or substantial training datasets from specific types of complex media, which makes these methods system- and data-dependent, limiting their generalizability to various systems and complex media platforms. Fidelity can also be improved by directly performing binary optimization (52, 53) of the binary mask (which is nondifferentiable) for the entire DMD frame, which is at the cost of considerable computational complexity and convergence to suboptimal solutions. To compensate for the limited modulation depth of a single frame in a more system- and data-agnostic way, temporal multiplexing (47, 49) has been exploited to remedy the low projection quality but at the cost of a roughly 10-fold reduction in the frame rate. An alternative approach, that is also system- and data-agnostic yet at the full frame rate, is the utilization of the phase-only constraint in DMD-based wavefront shaping techniques. This method involves optimizing the wavefront using only the phase information, while the amplitude is set to a constant value. Optimizing the wavefront with a phase-only constraint is one of the most popular and successful methods in computer-generated holography, microscopy, and wavefront shaping in complex media using phase-only SLMs. While this approach has been successfully demonstrated on DMDs (7, 8, 43), it omits the possibility of simultaneous amplitude and phase modulation (54, 55) and could be susceptible to ill-posedness and ill-conditionedness of the inverse problem without proper regularization (10, 56), leading to suboptimal inverse solutions with limited projection fidelity.

To fill this gap, we leverage the intrinsic random multiplexing in complex media to remedy the dimensionality limitation problem of spatial light modulation devices, allowing for a sparse and robust wavefront representation to achieve high-fidelity projection through complex media at a full DMD frame rate. We propose a sparsity-constrained optimization framework that accounts for two physical properties: (i) the limited degrees of freedom of spatial light modulation devices and (ii) the sparse-to-random transformation caused by the light scattering in complex media (Fig. 1A). To overcome the speed-fidelity tradeoff, we investigate the underlying limitations of DMD-based wavefront shaping and the sparse representations of wavefronts in a random basis provided by complex media, yielding a robust and efficient optimization framework that enables high-fidelity projection of high-bandwidth signals without compromising the frame rate (Fig. 1B and the “Sparsity-constrained wavefront optimization” section). By incorporating the dimensionality limitation through l₁ regularization, our approach demonstrates consistently higher-fidelity projections across different types of complex media, showing up to an 89% increase in projection accuracy and a 126% improvement in speckle suppression through graded-index multimode fibers, step-index multimode fibers, and diffusers at the speed of 22 kHz.

Fig. 1. Design principle of sparsity-constrained light shaping through complex media.

(A) The challenge of achieving arbitrary light manipulation in complex media can be addressed by recognizing and leveraging two physical properties in the optimization framework: the dimensionality limitation of wavefront shaping and the sparse-to-random transformation of complex media. (B) The wavefront optimization problem in complex media involves optimizing the pattern displayed on a spatial light modulation device to generate a given target pattern in or through a scattering medium. To achieve high-fidelity pattern projection at a full frame rate, a sparsity constraint in the Fourier plane of the DMD is introduced through l₁ regularization as a physics prior, which leverages the fact that patterns are compressible in a random basis provided by complex media to overcome the limited control degrees of freedom.

RESULTS

Sparsity constraint for DMD-based wavefront shaping

In experiments, we observe that DMDs show higher fidelity when the targets are sparser in the Fourier plane due to the limited degrees of freedom (Fig. 2C and fig. S1). To gain quantitative insight into the relationship between the wavefront fidelity and the sparsity in the Fourier plane, we conduct a simulation and an experiment (see the “Simulation and implementation of Lee hologram method” section for implementation details) of DMD-based wavefront shaping using the Lee hologram method (42, 43) in a Fourier domain setup without complex media, similar to Fourier transform holography (Fig. 2A) (57). Here, we use the number of foci in the Fourier plane of a wavefront (M) to evaluate the wavefront sparsity. The simulation result in Fig. 2B illustrates that projecting more foci in the Fourier plane simultaneously leads to a decreased projection quality, which is in agreement with the experimental result shown in Fig. 2C and fig. S1. Note S1 details a simplified theoretical explanation for our experimental observations by deriving the wavefront error as a result of the limited control degrees of freedom of DMDs. These observations and analyses demonstrate the intrinsic tradeoff between fidelity and the complexity (i.e., bandwidth) of the pattern projected by DMDs. Because of the preference of sparse patterns, such dimensionality limitation of DMD-based wavefront shaping can be potentially described by the sparsity of the wavefront in the Fourier plane. Compared to phase-only and binary constraints, incorporating the intrinsic sparsity constraint of the hardware as a l₁ minimization offers a balance between experimental projection fidelity and numerical optimality. It allows wavefronts with nonuniform amplitude distributions while being a practical solution for DMDs. In addition, the sparsity constraint in the form of l₁ minimization can potentially converge better and faster, which will be important for noisy measurements and applications that requires short precomputing time as we will discuss in the “High-fidelity light shaping via sparsity-constrained optimization” section. Now that we gain quantitative and physical insights into the sparsity constraints for DMD-based wavefront shaping, next, we investigate how to build on these insights in the optimization framework to enable high-fidelity high-speed light manipulation through complex media in the following sections.

Fig. 2. Numerical and experimental illustration of the sparsity constraint of wavefront shaping and the sparse-to-random transformation of complex media.

(A) Schematics illustrating sparsity-constrained DMD-based wavefront shaping. Without scattering, the set of attainable wavefronts is limited by the degrees of freedom of the DMD, which can be expressed as a sparsity constraint in the Fourier domain. However, this dimensionality limitation can be overcome by the sparse-to-random transformation through a complex medium, which supports a sparse representation of a generic pattern in a random basis. (B) Simulation of wavefront shaping in the Fourier plane without scattering. The wavefront mean squared error (MSE) increases with a larger number of foci (M). The number of foci in the Fourier plane represents the wavefront sparsity and is equivalent to the number of nonzero input modes in the presence of complex media. Ten different random distributions of foci are simulated, and the resulting MSE is averaged. (C) Experimental results of different pattern sparsity in the Fourier plane using the Lee hologram method. (D) Simulation of pattern reconstruction with different sampling ratios (the fraction of the number of input modes) through a diffuser. (E) Experimental results of wavefront shaping in media with different levels of scattering strength, including free space and a diffuser with 1500 and 600 grit polishes (DG10-1500 and DG10-600, Thorlabs, respectively). Scale bars, 10 μm.

Sparse-to-random transformation via complex media

Given the observation that higher fidelity is associated with lower bandwidth (sparse) targets (Fig. 2, B and C, and fig. S1), we seek a way to convert the high-bandwidth signals (arbitrary, generic patterns) to low-bandwidth ones to accommodate the sparsity constraint of DMDs. We find that, rather than being detrimental to precise wavefront shaping, complex media can support sparse representations of generic patterns in a random basis defined by a set of speckle patterns, which we refer to as the sparse-to-random transformation in this article. The theoretical foundation of the sparse-to-random transformation is attributable to the theory of random projections for dimensionality reduction (58). This property, together with l₁ minimization, allows the recovery of a generic pattern through scattering by a relatively sparse and robust wavefront at the input end, which overcomes the hardware sparsity constraint of DMDs (Fig. 2, D and E). These key insights are consistent with the theory of compressive sensing [incoherence and random sensing (59, 60)] and have been supported by the numerical and experimental observations (Fig. 2, D and E, and fig. S2) that patterns are compressible in the random basis provided by scattering media.

Consistent with the results in Fig. 2 (B and C), without the complex media, the simulated projection quality is poor due to the dimensionality limitation of DMDs shown in the simulation in Fig. 2D. However, using the same number of foci in the Fourier plane (i.e., same M), the light shaping fidelity in the output plane is substantially improved with the addition of complex media with a sampling ratio M/N = 1.0 (defined as the ratio between the number of foci M and the total number of the input modes N). Note that the number of foci is equivalent to the number of nonzero input modes generated by the wavefront in the presence of complex media. Such enhancement in the shaping fidelity is experimentally validated in Fig. 2E. Moreover, this sparse-to-random transformation is more pronounced in complex media with stronger scattering, leading to considerable enhancements in the quality of the dense pattern projection (Fig. 2E and fig. S3). To investigate the effectiveness of a sparse input in generating generic images, we progressively decrease the sampling ratio and compare the performance with the reference image directly generated from the inverse solution. Figure 2D shows that reduction of input modes does not necessarily lead to reduction of projection accuracy, and a relatively low sampling ratio is sufficient to yield an image of comparable quality to the reference image (around 0.25 in this example). A few more numerical examples of pattern compression in the random basis provided by complex media are shown in fig. S2. These numerical and experimental results show that (i) most patterns are compressible using the random basis provided by the complex media, and (ii) the sparse-to-random transformation of the complex media, together with l₁ minimization, allows sparse reconstruction of generic patterns, which conveniently compensates for the dimensionality limitation of DMDs. Note S2 details a more thorough analysis and discussion on the pattern compression property of complex media.

High-fidelity light shaping via sparsity-constrained optimization

By leveraging the sparse-to-random transformation in complex media, we propose a sparsity-constrained optimization framework which incorporates the dimensionality limitation of devices in the form of l₁ regularization (Fig. 1B and the “Sparsity-constrained wavefront optimization” section). We test the performance of the sparsity-constrained optimization framework (referred to as GD + L1) in the experimental setup shown in fig. S5 (see the “Experimental setup and characterization of transmission matrix” section for more details) using a graded-index multimode fiber (GIF50C, Thorlabs) as the complex medium and compare it with two commonly used methods: Gerchberg-Saxton (GS) algorithm (61, 62) with phase-only constraint at the conjugate plane of the DMD and gradient descent (GD) method (50, 63) without constraints. The implementation details of three methods are provided in notes S3 and S4. The sparsity-constrained optimization consistently outperforms the GS algorithm and the GD method for various targets (Figs. 3A and 4), achieving considerably higher-fidelity light manipulation through the fiber (higher accuracy and better speckle suppression; see the “Extension to different complex media” and the “Extension to different target patterns” sections for more details).

Fig. 3. Analysis of the projection performance of the sparsity-constrained optimization.

(A) Simulation and experimental results of projection through a graded-index multimode fiber (GIF50C, Thorlabs). (B) Optimization curves in simulation and experiment. The sparsity-constrained method (GD + L1) achieves consistent high-quality fiber output in simulation and experiment, which is attributed to the sparse representation it finds in the random basis provided by the complex medium [i.e., the predicted fiber input in (A)]. The scale bars for experimental outputs, predicted fiber inputs, and predicted fiber outputs are 10 μm, and the scale bar for wavefront inverse solutions is 1 mm.

Fig. 4. Experimental demonstration of enhanced projection quality in different complex media.

(A) Graded-index multimode fiber. (B) Step-index multimode fiber. (C) Diffuser. Scale bars, 10 μm.

Here, we dive in the graded-index multimode fiber experiment to gain quantitative insights into why sparsity-constrained optimization yields considerably better results and why this can be potentially extended to almost any existing DMD-based wavefront shaping systems, different complex media types, and different light manipulation target pools. The root cause of the improvement can be dissected from three perspectives. First, from the perspective of compressive sensing, generic patterns are compressible in a random basis (59, 60), and complex media provides a natural platform as an analogue randomizing compressor (64, 65) (for more examples, see fig. S2). Thus, transforming a wavefront by a random matrix together with l₁ minimization is an effective compression strategy for a wide variety of patterns through complex media.

Second, from the perspective of DMD shaping fidelity, the inherent limitation of DMDs can be approximated by the number of nonzero modes in the Fourier domain (Fig. 2, B and C, and note S1), which fits well with the l₁ minimization of the compression strategy. Our objective function leverages this synergy between the sparse representation of solutions and the dimensionality limitation of wavefront shaping. The random property of complex media allows the conversion of a high-bandwidth signal to a low-bandwidth one in a random basis with a sampling rate far below the Nyquist sampling rate. Such sparsity perfectly remedies the dimensionality limitation of DMDs, and these two properties can be seamlessly combined and efficiently solved in the shared domain of the Fourier plane of the DMD and the input plane of scattering media. Such synergy results in better solution optimality (i.e., wavefront shaping fidelity) and better solution feasibility (i.e., the consistency between the predicted outputs and the experimental outputs), as presented in Fig. 3A. A comprehensive discussion on the tradeoff between the solution feasibility and optimality is entailed in note S5.

Third, from the perspective of robustness and practicality, compared to other constraints (e.g., phase-only or binary), l₁ minimization has the theoretical guarantee that it can stably and accurately reconstruct nearly sparse signals from markedly undersampled data in an incoherent domain (60). Figure 3B illustrates the robustness of this sparsity-constrained optimization method, of which the mean squared error (MSE) curves consistently descend and converge in both the simulation and the experiment. This is in contrast to the GS method, which achieves less effective convergence due to the unnecessarily strong phase-only constraint, and the GD method, which obtains inconsistent simulated and experimental results because it overlooks the hardware constraint (for a detailed discussion, see note S5). Besides the robustness, this simple objective function also has a closed-form expression of its gradient, which promises high-speed, high-fidelity wavefront shaping without the cost of high computational complexity. A derivation of the gradient of the loss function is entailed in note S4, and a detailed analysis of the computational time can be found in note S6.

Extension to different complex media

To show its broad applicability, we test our method on different types of complex media, including graded-index multimode fibers (GIF50C, Thorlabs), step-index multimode fibers (FG050LGA, Thorlabs), and diffusers (DG10-600, Thorlabs). We use the same experimental setup for all three experiments, as shown in fig. S5. The experimental TM of each medium is characterized as described in the “Experimental setup and characterization of transmission matrix” section. After calibrating the TMs, we optimize the projections through each medium using the three methods described in the “High-fidelity light shaping via sparsity-constrained optimization” section. The implementation details of the optimization are provided in note S3. Despite differences in the underlying scattering mechanisms, the proposed method demonstrates consistent improvements in the projection quality in Fig. 4 and Table 1 by leveraging the sparse-to-random transformation exhibited in all three media to overcome the dimensionality limitation of wavefront shaping. The proposed method shows a substantial improvement of up to 2.73 dB in peak signal-to-noise-ratio (PSNR), which translates to an 89% enhancement in projection accuracy. A complete panel of the projected images in Table 1 can be found in fig. S11 and table S1. To characterize the effectiveness of the proposed method in reducing background speckle noise in the projected images, we calculate the speckle suppression in Table 1, which is defined as the ratio of the light intensity of the foreground and background. It is also referred to as Weber contrast in visual perception and imaging processing (66). The proposed method achieves a remarkable enhancement of speckle suppression up to 126% compared to the other two methods. Such consistent improvement in different types of complex media demonstrates the generalizability of the proposed method, and it can be readily adopted for various light manipulation projects involving complex media.

Table 1. Quantitative evaluation of projection quality through various complex media.

The average peak signal-to-noise ratio (PSNR), multiscale structural similarity (MS-SSIM), and Weber contrast values are computed on the target patterns shown in fig. S11. Bolded values highlight performance improvement. GI-MMF, graded-index multimode fiber; SI-MMF, step-index multimode fiber. The breakdown results are entailed in table S1.

	GI-MMF			SI-MMF			Diffuser
	GS	GD	GD + L1	GS	GD	GD + L1	GS	GD	GD + L1
PSNR	16.44	15.09	17.48	16.20	15.65	17.60	15.63	15.20	18.39
MS-SSIM	0.66	0.50	0.73	0.58	0.50	0.65	0.44	0.38	0.62
WC*	9.47	6.13	15.01	4.68	4.18	7.19	4.09	4.66	9.24

*Weber contrast (WC) is defined as (I − I_b)/I_b (66), where I and I_b are the luminance of the foreground and the background, respectively.

Extension to different target patterns

In Fig. 5, we generate diffraction-limited foci through a graded-index fiber (GIF50C, Thorlabs), which is an important application in endoscopic imaging. Compared to standard matrix inversion method with phase-only conjugate wavefronts (67), our method effectively suppresses the residual field in the background region and achieves a slightly higher focusing efficiency. This is consistent with the discussion in the “High-fidelity light shaping via sparsity-constrained optimization” section that our method effectively identifies the sparse and robust representations of generic patterns in the random basis. The relatively small improvement is owing to the already sparse solution in the random speckle basis, which is also the fiber input domain, for a diffraction-limited point as shown in fig. S3. For a more complicated pattern of which the inverse solution is denser in the random speckle basis (e.g., the second row in fig. S3), the effect of imposing a proper sparsity constraint in the optimization is substantial. In addition to diffraction-limited point focusing, we also demonstrate a wide variety of masks and patterns that can be applied in holographic optogenetics, compressive imaging, and optical communications through scattering, as illustrated in Figs. 6 and 7. In all of the examples, our method achieves a consistent improvement throughout with a higher image contrast and lower speckle noise.

Fig. 5. Diffraction-limited foci through a graded-index multimode fiber.

Three exemplary foci at different locations are demonstrated using different methods: Inv (phase), matrix inversion with phase-only wavefronts. GD + L1, sparsity-constrained optimization method. The insets are the corresponding log-scale images. The values are the average power ratio (80) of 77 foci on a grid with a 5 μm spacing at the distal end. Scale bars, 10 μm.

Fig. 6. Potential applications of high-fidelity high-speed wavefront shaping through scattering.

(A) Generation of masks for holographic optogenetics. (B) Generation of periodic patterns for compressive imaging (e.g., single-pixel imaging). MS-SSIM, multiscale structural similarity. Scale bars, 10 μm.

Fig. 7. Experimental demonstration of gray-level image projection through scattering.

The experiment involves testing 50 images extracted from the Fashion-MNIST dataset, with five examples from each of the 10 categories. Ten exemplary projected images are presented. Scale bar, 10 μm. Table 2 displays the quantitative evaluation of the projected image quality.

Table 2. Quantitative evaluation of projection quality of the Fashion-MNIST images through multimode fiber.

Average PSNR and MS-SSIM of the Fashion-MNIST images that are projected using a graded-index multimode fiber. The values in the parentheses are the SD. Bolded values highlight performance improvement.

Method	GS	GD	GD + L1
PSNR	18.36 (2.89)	17.22 (2.95)	19.30 (3.13)
MS-SSIM	0.64 (0.09)	0.55 (0.09)	0.72 (0.10)

DISCUSSION

The presented results demonstrate that the proposed sparsity-constrained wavefront optimization framework can substantially enhance the projection quality across various types of complex media. It is worth noting that our method shares a resemblance with compressive sensing using random matrices together with l₁ minimization for sparse signal recovery (60). Complex media serve as a natural, non-engineered randomizer for encoding wavefront information (64, 65) in a low-dimensional space composed of random speckles. Finding this sparse representation is the key to achieving high-fidelity wavefront shaping with limited degrees of freedom. Our method successfully leverages the intrinsic dimensionality limitation of DMDs and pattern compression by the sparse-to-random transformation through complex media, yielding substantially enhanced projection fidelity with single-shot wavefront shaping. Notably, the concept of sparsity here is twofold: (i) the dimensionality limitation of spatial light modulation devices and (ii) the low-dimensional representations of generic patterns in the random basis formed by complex media. It differs from the definition in the propagation invariant mode domain that the intermodal coupling is limited to the adjacent modes in multimode fibers (68). Subsequently, we discuss the advantages of our method, followed by its potential limitations, in the context of real-world applications.

First, the problem we intend to solve in this work is the intrinsic tradeoff of hardware performance, speed, and fidelity, in holographic wavefront shaping stemming from the dimensionality limitation. In other words, we aim to address the system bandwidth limit determined by the spatiotemporal complexity, i.e., the capability of generating complex spatial wavefronts with a high temporal resolution. Our optimization framework achieves high-fidelity projection of complex patterns at DMD’s full frame rate via pattern compression by the complex media. The high-fidelity projection of complex patterns at submillisecond and micrometer-level spatiotemporal resolution could potentially amplify a wide range of applications for monitoring or controlling targets dynamically, including optogenetics (35, 36, 51, 69, 70), microfabrication (22–24), optical manipulation (20, 21), and compressive endoscopic imaging (9, 71–73). Alternatively, for applications that do not require such a high frame rate, the excess frame rate can be leveraged for extended projection depth in microscopy (74) or higher spectral resolution in hyperspectral imaging (75).

In addition, our method is computationally efficient due to its simple mathematical formulation compared to the alternative methods for generating high-bandwidth signals through complex media (e.g., phase retrieval algorithms, binary optimization, and neural networks). This means that the high-speed, high-fidelity performance does not come with high computational complexity, making it a practical solution in the real-world applications. As benchmarked in note S6, it takes 0.5 s to optimize 1000 target patterns with a low-end graphics processing unit. Although further acceleration is required to grant “on-the-fly” computing for applications that require instant calibration (76), such precomputing speed and fidelity enhancement at DMD’s full speed can be readily used for applications that demand higher spatiotemporal precision but show resilience to changes in TM, e.g., deep optogenetics and imaging in living animals (7, 69).

Besides, our method can be generalized to different system configurations and types of complex media because it builds on two intrinsic properties—the dimensionality limitation of DMDs and pattern compression by complex media. By identifying and incorporating the sparsity constraints associated with limited degrees of freedom, our optimization framework can serve as a robust physics prior for dimensionality reduction of problems in combination with other methods such as temporal multiplexing, end-to-end methods, and deep learning–based methods. Such synergy is possible because our method approaches the problem by explicitly incorporating the dimensionality limitation into the wavefront optimization problem, making it a good complement to the existing wavefront shaping toolsets.

While our method has shown promising results, there is still room for further enhancement. First, the sparsity constraint in our approach approximates the wavefront error caused by limited amplitude modulation depth, but a real wavefront error also depends on the distribution of its angular spectrum as discussed in note S1. To improve the accuracy, one possible approach is to design a wavefront loss function that considers the effect of spectrum distribution. Despite this limitation, the sparsity constraint has a strong merit in its simplicity, offering an elegant understanding of the dimensionality limitation of spatial light modulation devices and ease of implementation. Second, the method is based on the characterization of the TM, which is susceptible to perturbations especially in a dynamic system. With the recent advancement in the compressive sampling techniques based on memory effect (68, 77, 78) and single-ended calibration techniques based on reciprocity-induced symmetry (10), the characterization process can be considerably accelerated. In view of some slowly varying random scattering systems such as multimode fiber-based endoscopes in which the TM can sustain for hours and the calibration time is not the bottleneck (7), the speed-up offered by our method remains beneficial. Last, compared to phase-only constraints, one inherent drawback of allowing complex wavefront solutions in optimization is the lower power efficiency due to the rejection of partial incident light on the DMD (fig. S8B). To improve power efficiency, one can increase the strength of the sparsity constraint in the optimization or penalize low efficiency by calculating the normalized power of the wavefront solution on the DMD plane ($\widehat{v}$ in Eq. 3). From the hardware perspective, this issue can be potentially mitigated by a higher laser power as DMDs have a high damage threshold.

METHODS

Sparsity-constrained wavefront optimization

As shown schematically in Fig. 1B, our proposed sparsity-constrained wavefront optimization method seeks to solve for an inverse solution $\widehat{\mathbf{x}}$ that minimizes the loss function $\mathcal{L}$ given a desired projection pattern y_t

$$\widehat{\mathbf{x}}=\underset{\mathbf{x}}{\text{argmin}} \mathcal{L}(\mathbf{x};{\mathbf{y}}_{\mathrm{t}})$$ (1)

The loss function $\mathcal{L}$ consists of two terms: a data fidelity term that penalizes the difference between the target projection pattern y_t and the pattern y estimated by the forward model y = Tx and a physics prior term that represents the sparsity constraint through l₁ regularization, also known as LASSO regression (79)

$$\mathcal{L}(\mathbf{x};{\mathbf{y}}_{\mathrm{t}})=\frac{1}{2}{||{\mathbf{y}}_t-\left|\mathbf{Tx}\right|||}_2^2+\mathrm{\lambda }{||\mathbf{x}||}_1$$ (2)

The introduction of l₁ regularization tends to suppress the coefficients of the less representative features to zero. The resulting solution falls onto a low-dimension manifold and therefore has a sparse representation. This optimization problem can be solved using the GD method, and the inverse solution $\widehat{x}$ can be used to obtain the estimated wavefront $\widehat{v}$ in the image plane by performing an inverse Fourier transform

$$\widehat{v}(\mathrm{\xi },\mathrm{\eta })={\mathcal{F}}^{-1}\left\{\widehat{x}(\mathrm{\xi }\prime ,\mathrm{\eta }\prime )\right\}$$ (3)

Here, (ξ, η) and (ξ′, η′) are Cartesian coordinates in the image plane and in the Fourier plane, respectively. Last, the estimated wavefront is encoded as a binary DMD hologram using the Lee hologram method (54)

$$\widehat{u}(\mathrm{\xi },\mathrm{\eta })=\frac{1}{2}+\frac{1}{2}\text{sgn}\left\{\text{cos}\left[k_0·(\mathrm{\xi }+\mathrm{\eta })-\mathrm{\varphi }(\mathrm{\xi },\mathrm{\eta })\right]-\text{cos}\left[w(\mathrm{\xi },\mathrm{\eta })\right]\right\}$$ (4)

where $\mathrm{\varphi }(\mathrm{\xi },\mathrm{\eta })=\text{Arg}\left[\widehat{v}(\mathrm{\xi },\mathrm{\eta })\right]$ is the phase of the estimated wavefront, $w(\mathrm{\xi },\mathrm{\eta })=\text{arcsin}\left[\frac{\mid \widehat{v}\mid }{{\mid \widehat{v}\mid }_{\text{max}}}\right]$ is the arcsine of the normalized amplitude, and k₀ is the modulated carrier frequency determining the angle of the first diffraction order.

Simulation and implementation of Lee hologram method

The simulations of binary Lee holograms demonstrated in Fig. 2 (B and D) and figs. S2 and S3 are constructed using scalar diffraction theory. To compute the wavefront in the Fourier plane generated by a binary Lee hologram in the image plane, we perform two steps: (i) Fourier transform of the binary hologram and (ii) screening the field outside the aperture of the spatial filter centered at the first diffraction order in the Fourier domain. Our simulations use a binary hologram with 512 × 512 pixels and a superpixel size of 4 × 4 in the Lee hologram method.

To model the transmission of light through a complex medium characterized by a TM as shown in Fig. 2D, we convert the simulated wavefront in the Fourier plane ($\mathcal{F}${u(ξ, η)}) to the pixel-based input mode domain of the TM by calculating the overlap integral with each input pixel mode ψ_i

$$x_i=\iint ‍{\mathrm{\psi }}_i^{*}\mathcal{F}\left\{u\right\}\mathit{dA}$$ (5)

where x = [x₁, x₂, …, x_n]^T is a vectorized input of the TM. In the pattern reconstruction simulation shown in Fig. 2D, the solutions are obtained by selecting the input modes with the M greatest absolute values of the coefficients and setting the remaining coefficients to zero.

The experimental setup of the Lee hologram method is depicted in fig. S5. The standard configuration consists of a DMD and a 4f system with a spatial filter located in the Fourier plane. We use the same parameters for the number of pixels and the size of superpixels as in our simulation. In our setup, we use an objective (OBJ1) to couple the wavefront into the complex medium. To observe the wavefront generated by the Lee hologram method in the Fourier plane, we remove the complex medium shown in fig. S5 and adjust two objectives to be confocal. To examine the image projection through complex media, we use the same setup as depicted in fig. S5.

Experimental setup and characterization of TM

In the experimental setup depicted in fig. S5, a 100-mW, 488-nm continuous-wave laser (Sapphire 488 SF NX, Coherent) is used for illumination. The laser beam is expanded by a 4f system (L1 and L2) with ×10 magnification to match a circular region of 7 cm in diameter, equivalently 512 pixels, on the DMD (V-7001, Vialux). The Lee hologram method is applied to generate a predefined complex wavefront in the first diffraction order, and the other diffraction orders are blocked with a spatial filter in the Fourier plane. An objective (RMS20X, Olympus) is used to focus the wavefront onto the input plane of a complex medium, and another objective (RMS10X, Olympus) collects the resulting speckle in the output plane. The speckle image is formed on a monochrome camera (Mako G-040B, Allied Vision) after passing through a 4f system (OBJ2 and L5) with ×16.7 magnification. For the fibers used in the experiments (GIF50C and FG050LGA, Thorlabs), the length is approximately 15 cm.

To determine the TM of the complex medium, we perform raster scanning at the proximal end in the Fourier plane of the DMD and acquire the corresponding complex-field speckles at the distal end (27, 67). For each of the complex media used in the experiments, we scan 1941 foci with a spacing of 1.0 μm across a circular region with a diameter of 50 μm. To achieve a diffraction-limited beam during raster scanning, we calibrate the wavefront aberration caused by the DMD using Zernike polynomials of 20th order. The resulting speckles are split into two orthogonal linear polarization states by a beam displacer and measured using off-axis holography. To reduce phase instability, we measure a reference speckle to characterize and compensate the temporal phase variation caused by environmental vibration. Last, we combine the two submatrices associated with the two polarization states in the output to generate the TM.

Supplementary Materials

This PDF file includes:

Supplementary Notes S1 to S7

Figs. S1 to S12

Table S1

References