Long-Distance Autonomous Navigation of Optical Microrobotic Swarms in Complex Environments

Abstract

The local force field generated by light endows optical microrobots with remarkable flexibility and adaptivity, promising significant advancements in precise medicine and cell transport. Nevertheless, the automated navigation of multiple optical microrobots in intricate, dynamic environments over extended distances remains a challenge. In this study, we introduce a versatile control strategy aimed at navigating optical microrobotic swarms to distant targets under obstacles of varying sizes, shapes, and velocities. By confining all microrobots within a manipulation domain, we ensure swarm integrity while mitigating the effects of Brownian motion. Obstacle’s elliptical approximation is developed to facilitate efficient obstacle avoidance for microrobotic swarms. Additionally, we integrate several supplementary functions to enhance swarm robustness and intelligence, addressing uncertainties such as swarm collapse, particle immobilization, and anomalous laser-obstacle interactions in real microscopic environments. We further demonstrate the efficacy and versatility of our proposed strategy by achieving autonomous long-distance navigation to a series of targets. This strategy is compatible with both optical trapping- and nudging-based microrobotic swarms, representing a significant advancement in enabling optical microrobots to undertake complex tasks such as drug delivery and nanosurgery and understanding collective motions.

Graphical abstarct

The study presents a control strategy for navigating optical microrobotic swarms through complex environments. By confining microrobots within a manipulation domain, the strategy ensures swarm integrity and efficient obstacle avoidance. It addresses challenges like swarm collapse and anomalous interactions, enhancing robustness. This approach advances optical microrobots’ capabilities in precise medicine, cell transport, drug delivery, and nanosurgery.

1. Introduction

Optical microrobots autonomously execute a variety of tasks within the microscopic world through adaptive control of the external light field.^[1] In comparison to microrobots driven by global fields like magnetic,^[2] electric,^[3] and acoustic forces,^[4] optical microrobots hold a distinct advantage due to their ability to individually control each agent, enabled by their local force field.^[5] This feature renders them highly adaptable for navigating complex and ever-changing microenvironments. Since the advent of optical tweezers in 1970,^[6] various optical manipulation techniques have emerged, including plasmonic,^[7] optoelectronic,^[8] and optothermal^[9] methods. By fine-tuning properties such as incident light, substrates, solutions, and particles, manipulated particles can exhibit diverse working modes^[10], including trapping, nudging, pulling, rolling, and rotating. For optical microrobots, techniques like trapping^[11] and nudging^[12] are commonly employed to induce directional translational movement in microparticles. Nonetheless, navigating optical microrobotic swarms over long distance, particularly in real microscopic environments with arbitrary and dynamic obstacles, remains a significant challenge, which necessitates an automatic optical setup equipped with effective obstacle-avoidance and feedback-control algorithms.

Numerous algorithms have been developed for obstacle avoidance in optical microrobots, categorized into global and local path planning methods.^[13] Global path planning algorithms like A*,^[14] rapid-exploring random tree (RRT),^{[11e, 15]} and machine-learning-based^[16] methods can theoretically compute the most efficient path considering all observed obstacles and targets. However, their heavy computational load makes them not suitable for real-time control, particularly in dynamically changing environments.^[17] To enhance adaptability in complex settings, local path planning methods such as the potential field method^[18] and fuzzy logic approach^[19] are employed. Currently, efforts focus on refining obstacle-avoidance algorithms to effectively coordinate larger swarms of microrobots in increasingly complex environments. As for the existing feedback-control algorithms for automating optical setups, they often overlook integration with sample stage movement or the generation of multiple laser beams.^{[11b–d, 20]} This oversight hampers the long-distance navigation of optical microrobotic swarms. Furthermore, it has remained challenging to ensure all manipulated microrobots remain within the same camera view, which is crucial to prevent any agents from being left behind during long-distance transport, particularly in biological applications.

In this study, we present and experimentally validate a versatile strategy for navigating optical microrobotic swarms intact over long distances in complex environments. Our approach involves obstacle’s elliptical approximation to generalize our obstacle-avoidance algorithm while maximizing space for particle movement. By designing interaction rules governing particle-particle, particle-obstacle, and particle-manipulation domain interactions, our swarm can navigate through obstacles of varying shapes, sizes, and speeds while maintaining its intended moving direction and avoiding collisions. The proposed algorithm is seamlessly integrated into an automatic optical setup comprising a camera, spatial light modulator (SLM), and motorized stage, demonstrating its effectiveness in real-world microscopic scenarios with obstacles and uncertainties. First, we conduct quantitative analyses to investigate the effects of laser power on group’s transportation efficiency and the tradeoff between throughput (i.e., number of manipulated particles) and group speed in optical microrobots. Then, we validate that both trapping- and nudging-based microrobots can be successfully navigated over long distances in complex environments without losing any agents. In addition, we incorporate auxiliary functions into our control algorithms to address issues such as swarm collapse, particle immobilization, and abnormal laser-obstacle interactions, which usually exist in real microscopic environments while not being captured in simulations. Furthermore, we showcase the ability of optical microrobots to autonomously search for targets over long distances, approaching and aggregating around them in complex environments. Our proposed strategy is versatile and applicable to various types of optical microrobots, offering potential applications in cell transport, nanosurgery, and drug delivery.

2. Working principle and experimental setup

We aim to devise a control strategy that can be universally applied to various types of microrobots, powered by focused laser beams, to achieve long-distance and collision-free movement as a cohesive unit through optical trapping or nudging in dynamically evolving environments (Figure 1A). Optical microrobots primarily operate under optical microscopes for the real-time imaging and feedback control, which constrain the robots within a rectangular field of view typically spanning several hundred microns. Challenge arises when the microrobots attempt to move longer distances, such as centimeter scales. Consequently, continual adjustment of the field of view via the motorized sample stage motion becomes imperative. Accordingly, all microrobots must be confined with a designated manipulation domain to ensure none of them evades optical detection or lags during long-distance transport (Figure 1B and Supplementary Text S1).

Figure 1. Working principle and experimental setup.

(A) Schematic showing the concept of our optical microrobotic swarms. Each microrobot is driven by an individual laser beam, enabling tailored movement through diverse obstacles to reach specific targets. (B) Schematic demonstrating the versatility of our platform accommodating both optical trapping and nudging mechanisms (left), with a crucial emphasis on confining all microrobots within a manipulation domain to ensure intact transport (right). The buffered region with certain thickness is introduced to further secure the integrity of the whole microrobotic swarm. (C) Schematic of the experimental setup and the real-time feedback control. “M” and “L” represent mirrors and lenses, respectively.

Our experimental setup relies on real-time feedback control, with an update frequency of approximately 0.132 seconds (Figure 1C). At the onset of each iteration loop, the camera on the optical microscope captures a real-time image, which is then routed to the computer. Subsequently, the control program processes the image, identifying and evaluating the presence of microrobots, obstacles, and targets. Should a target be identified, all microrobots pivot towards it. Nonetheless, during their navigation towards the target, individual microrobots might temporarily alter their moving directions due to neighboring microrobots, obstacles, or the manipulation domain’s boundaries. The methodology for determining microrobot moving direction is elaborated in Figures 2 and 4, as well as in the supplementary text S1. Given that the movement of each microrobot is finally achieved by controlling the relative position between the laser’s center and the microrobot’s center, the real-time laser pattern is subsequently calculated after determining the moving direction of each microrobot. Following this, the control program generates a hologram transmitted to SLM for the corresponding laser pattern generation. Concurrently, a signal is transmitted to the motorized stage if any microrobot is detected to contact a manipulation domain boundary (Figure 2). Ultimately, the microrobots reposition themselves, initiating the subsequent iteration loop until the swarm fulfills its predefined objective. The details of implementing these processes are elaborated in Figure S1 and the supplementary text S1.

Figure 2. Working principles of the optical microrobotic swarms within the manipulation domain.

Schematic (left) and time-elapsed optical images (right) of (A) activating the movement of the motorized stage when microrobot(s) reach the open boundary (blue) or reflecting the microrobot(s) towards the domain’s center when microrobot(s) reach any reflective boundaries; (B) rotating the manipulation domain to align it with the group moving direction (${\widehat{\mathit{v}}}_{\mathrm{g}\mathrm{r}\mathrm{o}\mathrm{u}\mathrm{p}}$); (C) temporarily activating the aggregation mode when microrobots reach two opposite sides of the domain concurrently; and (D) increasing the intensity of the laser that follows the immobilized microrobot. ${\widehat{\mathit{v}}}_{\mathrm{s}\mathrm{t}\mathrm{a}\mathrm{g}\mathrm{e}}$ and ${\widehat{\mathit{v}}}_{\mathrm{p}\mathrm{a}\mathrm{r}\mathrm{t}\mathrm{i}\mathrm{c}\mathrm{l}\mathrm{e}}$ represent the moving direction of the motorized stage and single microrobot, respectively. Scale bar: 5 μm.

Figure 4. Obstacle avoidance with elliptical approximation.

(A) Illustration outlining the principle of obstacle avoidance using elliptical approximation. Solid red and blue regions represent repulsion and interaction areas, respectively. Stars denote critical tangential points on the ellipse relative to ${\widehat{\mathit{v}}}_{\mathrm{g}\mathrm{r}\mathrm{o}\mathrm{u}\mathrm{p}}$ (indicated by yellow arrows and dashed lines). Selection between two potential tangential directions when (B) only one critical point or (C) both critical points are within the manipulation domain. (D) Temporary adjustment of ${\widehat{\mathit{v}}}_{\mathrm{g}\mathrm{r}\mathrm{o}\mathrm{u}\mathrm{p}}$ for large obstacles with critical points on opposite sides of the manipulation domain. (E) Optical image illustrating the differentiation between microrobots (blue lines) and obstacles (red lines) by the control program. “Extra manipulation laser” indicates the introduction of additional lasers to trap yeast cells entering the manipulation domain, preventing them from being attracted by microrobots’ driving laser beams. (F) Sequential optical images depicting the transport of ten microrobots over a long distance in a complex environment. All scale bars are 5 μm.

As depicted in Figure 1B, the shift in the field of view required by long-range navigation necessitates the establishment of a manipulation domain to confine all microrobots, thereby preventing any potential loss of microrobots during transport. Compared with the conventional velocity-alignment interaction rule to achieve compact and directional swarms^[21], this domain also serves to eliminate the influence of Brownian motion on the transport process, rendering the swarm’s movement deterministic (Supplementary Text S2). Specifically, we define a square manipulation domain featuring one open boundary (blue side in Figure 2A) and three reflective boundaries (grey sides in Figure 2A). The orientation of the open boundary is determined by the group direction of the microrobotic swarms, denoted as ${\widehat{\mathit{v}}}_{\mathrm{g}\mathrm{r}\mathrm{o}\mathrm{u}\mathrm{p}}$, which signifies the direction that all microrobots should adhere to but might be updated based on real-time information regarding targets and obstacles (Supplementary Text S1.2.1). The open boundary aligns perpendicular to ${\widehat{\mathit{v}}}_{\mathrm{g}\mathrm{r}\mathrm{o}\mathrm{u}\mathrm{p}}$, extending outward from the domain. Consequently, upon any contact between certain microrobot(s) and the open boundary, the stage moves along ${\widehat{\mathit{v}}}_{\mathrm{g}\mathrm{r}\mathrm{o}\mathrm{u}\mathrm{p}}$ for a predefined distance (Movie S1). Following the stage movement, microrobot(s) previously in contact with the open boundary become distanced from it again. Subsequent stage movement occurs only upon new contact between microrobot(s) and the open boundary. Additionally, when microrobots contact reflective boundaries, their movements temporarily orient towards the center of the manipulation domain. This temporary adjustment persists for 35 iteration loops (approximately 4.62 seconds), facilitating the return of microrobots to the swarm without excessively impeding progress towards the target. After 35 loops, microrobots realign their direction with ${\widehat{\mathit{v}}}_{\mathrm{g}\mathrm{r}\mathrm{o}\mathrm{u}\mathrm{p}}$. This approach ensures the entire microrobotic swarm advances exclusively along ${\widehat{\mathit{v}}}_{\mathrm{g}\mathrm{r}\mathrm{o}\mathrm{u}\mathrm{p}}$, effectively mitigating the impact of Brownian motion during long-distance transport (Supplementary Text S2). Furthermore, we enable the manipulation domain to rotate in accordance with ${\widehat{\mathit{v}}}_{\mathrm{g}\mathrm{r}\mathrm{o}\mathrm{u}\mathrm{p}}$, guaranteeing the open boundary remains perpendicular to ${\widehat{\mathit{v}}}_{\mathrm{g}\mathrm{r}\mathrm{o}\mathrm{u}\mathrm{p}}$ (Figure 2B and Movie S1).

In addition, when the swarm significantly disperses and the microrobots simultaneously touch both the open and reflective boundaries, there is a risk of microrobots near the reflective boundary escaping the manipulation domain. To prevent swarm’s collapse, we introduce an additional operational mode termed “aggregation mode” (Figure 2C and Movie S1). In this mode, when microrobots contact two opposite sides of the domain concurrently, all microrobots temporarily realign towards the domain’s center. For simplicity, we also designate one aggregation period to last 35 iteration loops, after which the swarm reunites, and their directional alignment returns to ${\widehat{\mathit{v}}}_{\mathrm{g}\mathrm{r}\mathrm{o}\mathrm{u}\mathrm{p}}$. It should be noted that the manipulation domain’s size and shape can be dynamically adjusted to meet specific applications or working conditions. Based on hardware limitations, particularly the performance of the spatial light modulator and the size of the mirrors on the light path (Figure 1), the domain size should be maximized to transport as many microrobots as possible while ensuring effective manipulation at any position within the domain. The domain shape should maintain a straight open boundary, while the reflective boundaries can take any shape, provided they form a closed figure with the open boundary. Additionally, microrobots operating along substrates may become immobilized during movement. To address this, we integrate an extra function within our feedback control (Figure 2D and Movie S1). If a microrobot exhibits minimal movement over a specified period (e.g., 8 loops), which arises from its immobilization on the substrate, the driving laser increases the intensity until the robot overcomes the adhesion force and resumes its free movement.

Apart from showcasing microrobots based on optical nudging (Figure 2), we demonstrate that the same strategy facilitates the long-distance navigation of microrobots based on optical trapping (Figure S2 and Movie S2). The methodologies of realizing optical trapping and nudging in our experiments are elucidated in the Methods section. Beyond these two specific manipulations demonstrated here, the proposed algorithm is compatible with many other optical manipulation techniques, such as conventional optical tweezers^[6], opto-electronic tweezers^[22], and opto-thermophoretic tweezers^[23] etc. This compatibility arises because our method fundamentally tailors laser patterns based on real-time image analysis. Overall, leveraging these four strategies enables the optical microrobotic swarm to achieve autonomous long-distance navigation along any chosen direction without losing any participants.

3. Transport efficiency of optical microrobotic swarms

The efficiency of navigating optical microrobotic swarms in free space is mainly determined by the magnitude of the group velocity ($\left|{\mathit{v}}_{\mathrm{g}\mathrm{r}\mathrm{o}\mathrm{u}\mathrm{p}}\right|$) and the number of delivered microrobots (N). Maximizing both parameters is crucial, as it signifies the simultaneous delivery of a greater number of microrobots at higher speeds. Through a series of experiments, we investigated various parameters, including input laser power, the number of manipulated microrobots, and the magnification of the objective lenses, to optimize the navigation efficiency.

For the microrobotic swarms moving within a domain (Figure 2), $\left|{\mathit{v}}_{\mathrm{g}\mathrm{r}\mathrm{o}\mathrm{u}\mathrm{p}}\right|$ is chiefly influenced by two key parameters: the input laser power and the frequency of activation of the aggregation mode, denoted as $f_{\mathrm{g}}$ (Figure S3). We initiated our experiments by varying laser powers, employing ten microrobots with a diameter of 1.97 μm, and operating under 60x magnification objective lens (Movie S3). The movement of the motorized sample stage and the activation of the aggregation mode at different intervals were recorded, enabling subsequent calculations of $\left|{\mathit{v}}_{\mathrm{g}\mathrm{r}\mathrm{o}\mathrm{u}\mathrm{p}}\right|$ and $f_{\mathrm{g}}$ (Figure S4). As depicted in Figure 3A, $\left|{\mathit{v}}_{\mathrm{g}\mathrm{r}\mathrm{o}\mathrm{u}\mathrm{p}}\right|$ initially ascends but eventually declines with increasing laser power, whereas $f_{\mathrm{g}}$ exhibits an ascending trend followed by saturation. This observed trend is specific to the optical manipulation technique utilized here, predominantly relying on the interplay of two optothermal forces,^[10a] i.e., opto-thermoelectric and depletion forces. These forces exhibit long-range characteristics owing to the substantial temperature gradient fields established on the plasmonic substrate.^[5] With the utilization of the SLM to generate multiple laser beams for customized microrobot movements, each microrobot experiences nudging not only from its designated laser beam but also from neighboring beams. Consequently, as the input laser intensity escalates, the repulsion force among microrobots increases rapidly, dispersing the swarm and prompting aggregation mode. As $f_{\mathrm{g}}$ increases (Figure 3A), the swarm allocates less time toward the target, thereby suppressing $\left|{\mathit{v}}_{\mathrm{g}\mathrm{r}\mathrm{o}\mathrm{u}\mathrm{p}}\right|$. The dual effect of altering laser power governs the observed trend in $\left|{\mathit{v}}_{\mathrm{g}\mathrm{r}\mathrm{o}\mathrm{u}\mathrm{p}}\right|$ shown in Figure 3A. In simulated scenarios devoid of these specific long-range repulsion forces (Figure S5A), $\left|{\mathit{v}}_{\mathrm{g}\mathrm{r}\mathrm{o}\mathrm{u}\mathrm{p}}\right|$ indeed exhibits an anticipated linear growth pattern with increasing input laser power. However, even in the absence of long-range forces, practical constraints imposed by equipment performance limit the achievable $\left|{\mathit{v}}_{\mathrm{g}\mathrm{r}\mathrm{o}\mathrm{u}\mathrm{p}}\right|$ in experiments. Factors such as the finite frame rate of the camera for real-time particle position capture and the restricted speed of the motorized stage to follow rapid particle movements contribute to this limitation.

Figure 3. Quantitative analysis of optical microrobotic swarm navigation efficiency through experimental investigations.

Assessment of group velocity magnitude ($\left|{\mathit{v}}_{\mathrm{g}\mathrm{r}\mathrm{o}\mathrm{u}\mathrm{p}}\right|$) and aggregation frequency ($f_{\mathrm{g}}$) across (A) varying input laser powers ($I_0$ represents the laser power applied for one microrobot), (B) diverse particle densities, and (C) different magnifications. For (A), N = 10 and magnification is 60x. For (B), N = 5/ 10/ 15/ 20/ 25, magnification is 60x and $I_0=0.16\mathrm{m}\mathrm{W}$. For (C), $I_0=0.16\mathrm{m}\mathrm{W}$.

Furthermore, enhancing N, the number of delivered microrobots, involves two approaches: increasing particle density or expanding the physical size of the manipulation domain once particle density reaches saturation. As shown in Figure S3, both strategies inevitably lead to a decrease in $\left|{\mathit{v}}_{\mathrm{g}\mathrm{r}\mathrm{o}\mathrm{u}\mathrm{p}}\right|$, indicating a theoretical tradeoff between $\left|{\mathit{v}}_{\mathrm{g}\mathrm{r}\mathrm{o}\mathrm{u}\mathrm{p}}\right|$ and N. Primarily, the determination of the manipulation domain’s size depends on the characteristics of the SLM and other optical components. The domain cannot be excessively large as diffracted laser spots may suffer from poor quality when distant from the zero-order spot. Moreover, the optimized manipulation domain consistently occupies the same portion of the field of view, i.e., a fixed number of pixels, irrespective of its physical size. Consequently, under the optical microscope, we should first maximize the manipulation domain based on the optical system’s properties and then adjust the domain’s physical size by modifying the objective’s magnification (i.e., altering the pixel’s physical size). Based on this understanding, $\left|{\mathit{v}}_{\mathrm{g}\mathrm{r}\mathrm{o}\mathrm{u}\mathrm{p}}\right|$ and $f_{\mathrm{g}}$ are first measured at different particle densities with a constant 60x magnification (Figure 3B and Movie S4), revealing an inverse relationship between $\left|{\mathit{v}}_{\mathrm{g}\mathrm{r}\mathrm{o}\mathrm{u}\mathrm{p}}\right|$ and particle density. As particle density increases, more microrobots crowd in the manipulation domain, elevating $f_{\mathrm{g}}$ and consequently reducing $\left|{\mathit{v}}_{\mathrm{g}\mathrm{r}\mathrm{o}\mathrm{u}\mathrm{p}}\right|$. This trend is also observed in simulations without long-range repulsion forces, albeit with a slower decline in $\left|{\mathit{v}}_{\mathrm{g}\mathrm{r}\mathrm{o}\mathrm{u}\mathrm{p}}\right|$ (Figure S5B). Additionally, density can further increase by decreasing the minimum spacing between microrobots or reducing the size of the microrobots. More discussion about density is provided in Supplementary Text S3. Then, the same measurements are conducted at different magnifications while keeping the domain’s size the same in pixels (Figure 3C and Movie S5). Decreasing objective magnification while maintaining particle density increases N but also augments detection errors as microrobots appear smaller under lower magnification. Large detection errors could elevate $f_{\mathrm{g}}$ and then reduce $\left|{\mathit{v}}_{\mathrm{g}\mathrm{r}\mathrm{o}\mathrm{u}\mathrm{p}}\right|$. Compared to the data at 40x magnification, $\left|{\mathit{v}}_{\mathrm{g}\mathrm{r}\mathrm{o}\mathrm{u}\mathrm{p}}\right|$ does increase under 60x magnification due to improved detection accuracy and reduced $f_{\mathrm{g}}$. However, at 100x magnification, the manipulation domain’s physical size becomes relatively small compared to microrobot diameter (Movie S5). Consequently, the volume-exclusion function (Supplementary Text S1.2.5), activated when microrobot gaps are smaller than one diameter, triggers more frequently, increasing $f_{\mathrm{g}}$ and decreasing $\left|{\mathit{v}}_{\mathrm{g}\mathrm{r}\mathrm{o}\mathrm{u}\mathrm{p}}\right|$ (Figure 3C). These results emphasize the necessity to adapt laser power, particle density, and objective magnification for different optical microrobotic swarms and applications.

4. Obstacle avoidance

To bypass obstacles of arbitrary sizes, shapes, and speeds, we introduce an obstacle-avoidance strategy based on elliptical approximation. Herein, all obstacles within the field of view are represented as corresponding ellipses, with their orientation and velocity updated in real-time within each frame. In contrast to circular approximation methods,^{[11j, 17]} our approach offers more available space for microrobots to maneuver, particularly in accommodating elongated obstacles and boundaries. It should be noted that elliptical approximation for obstacle avoidance has been developed previously.^[24] However, their concepts are based on the potential fields, which are different from what we proposed here, as detailed below. Crucially, despite moving collectively, each microrobot will respond to the obstacles individually, selecting its own optimized path (Figure S1).

As depicted in Figure 4A, each obstacle has a distinct repulsion area (solid red region) and an interaction area (solid blue region). The repulsion area encompasses the entire obstacle to avoid potential collisions between the obstacle and microrobots. When microrobots are detected to enter the repulsion area owing to Brownian motion or forces from neighboring microrobots, their moving direction is altered to become perpendicular to the tangential line at the contact point, facilitating prompt exit from this area (Figure 4A). Conversely, the interaction area spans only half of the ellipse. Microrobots entering this region select one of the tangential directions to circumvent the obstacle. If the interaction area encompasses the entire ellipse, microrobots may continuously orbit the obstacle until Brownian motion randomly enables departure from this zone. To solve this issue, we designate two critical points on each ellipse (depicted as stars in Figure 4A), which are the tangential points on the ellipse relative to ${\widehat{\mathit{v}}}_{\mathrm{g}\mathrm{r}\mathrm{o}\mathrm{u}\mathrm{p}}$. Upon crossing either of these points, microrobots swiftly recover to ${\widehat{\mathit{v}}}_{\mathrm{g}\mathrm{r}\mathrm{o}\mathrm{u}\mathrm{p}}$, eliminating further interaction with the obstacle.

To select one direction between two potential tangential directions in the interaction area, the relative positions of the manipulation domain and the critical points of the ellipse play a crucial role. Adhering to the primary rule that no microrobot should exit the domain, if only one critical point lies within the domain, microrobots will opt for tangential directions leading toward this inside critical point (Figure 4B and Movie S6). In cases where both critical points are situated inside the domain, microrobots choose the tangential direction toward the critical point nearest to them, minimizing the distance needed to bypass the obstacle (Figure 4C and Movie S6). Furthermore, if neither critical point resides inside the domain, two possible conditions arise. If both critical points align on the same side of the domain, it indicates no interaction between the microrobotic swarm and the obstacle (unless the obstacle rapidly approaches the domain), necessitating no change in direction. However, if these critical points occupy opposite sides of the domain, indicating a large obstacle, a temporary adjustment is required. In such cases, ${\widehat{\mathit{v}}}_{\mathrm{g}\mathrm{r}\mathrm{o}\mathrm{u}\mathrm{p}}$ temporarily shifts along the long axis of the obstacle (Figure 4D and Movie S6), with the swarm choosing a direction that forms an acute angle with the original ${\widehat{\mathit{v}}}_{\mathrm{g}\mathrm{r}\mathrm{o}\mathrm{u}\mathrm{p}}$. The swarm reverts to its original ${\widehat{\mathit{v}}}_{\mathrm{g}\mathrm{r}\mathrm{o}\mathrm{u}\mathrm{p}}$ once one critical point of the obstacle enters the domain, thereby resembling the scenario illustrated in Figure 4B. Additionally, considerations are made for microrobotic swarms navigating dynamic obstacles (Figure S6). When obstacles rotate rapidly, potentially leading to collisions with microrobots, they will be approximated as circles rather than ellipses (Movie S7). For obstacles exhibiting high translational speed, the swarm temporarily adjusts its ${\widehat{\mathit{v}}}_{\mathrm{g}\mathrm{r}\mathrm{o}\mathrm{u}\mathrm{p}}$ to bypass them (Movie S8). Further elaboration on the entire obstacle-avoidance rule is provided in Supplementary Text S1.2.4 and S1.2.5.

We have successfully validated the effectiveness of our obstacle-avoidance rule in a real and intricate environment comprising a mixture of silica microparticles and yeast cells. At the program’s beginning, silica microparticles initially within the manipulation domain are designated as microrobots (blue circles in Figure 4E), while all other particles are considered as obstacles. During the movement of the microrobotic swarms, obstacles are dynamically approximated as ellipses in real-time (red lines in Figure 4E). Notably, both types of obstacles are responsive to laser beams: microparticles are nudged, while yeast cells are trapped. To prevent yeast cells from being drawn towards the laser beams and colliding with microrobots, an additional function is incorporated into our control program. This function introduces extra laser beams to trap all yeast cells entering the manipulation domain (red arrows in Figure 4E). As depicted in Figure 4F and Movie S9, a group of ten microrobots successfully maintains cohesion and navigates within a dynamically evolving environment, following arbitrary ${\widehat{\mathit{v}}}_{\mathrm{g}\mathrm{r}\mathrm{o}\mathrm{u}\mathrm{p}}$. Furthermore, our observations indicate that the microrobotic swarm can handle floating obstacles without encountering collisions (Figure S7 and Movie S10).

5. Target Navigation

To prototype task execution in complex environments, we have integrated target navigation functionality into our control algorithms (Supplementary Text S1). As depicted in Figure 5A, target navigation operates in a loop comprising four distinct steps. When no target is detected within the field of view, the optical microrobotic swarm follows a predefined search direction to locate the target. Upon detection of a new target, the swarm adjusts ${\widehat{\mathit{v}}}_{\mathrm{g}\mathrm{r}\mathrm{o}\mathrm{u}\mathrm{p}}$ to approach it. As targets may be dynamic, ${\widehat{\mathit{v}}}_{\mathrm{g}\mathrm{r}\mathrm{o}\mathrm{u}\mathrm{p}}$ is updated every 0.132 seconds (i.e., one iteration loop) to ensure precise navigation towards the target. To prevent unexpected collisions during the approach, target information is added to the obstacle information list, allowing obstacle avoidance to prevent physical contact. Once the swarm’s center overlaps with the target’s center, the built-in aggregation mode is activated. All microrobots converge to surround the target and execute predefined tasks. Upon task completion, the target is automatically labeled as “inactive” (Supplementary Text S1.2), ensuring the swarm does not mistakenly return to the same target. Simultaneously, the swarm leaves the target, proceeding to approach the next target within the same field of view or continuing its search for new targets.

Figure 5. Target Navigation in Complex Environment.

(A) Schematic illustrating the operational principle of target navigation for optical microrobotic swarms. The process involves the swarm initially searching for a target until a new target enters its field of view. Subsequently, the swarm aligns itself towards the target by adjusting ${\widehat{\mathit{v}}}_{\mathrm{g}\mathrm{r}\mathrm{o}\mathrm{u}\mathrm{p}}$. Upon overlapping of the swarm and target centers, aggregation occurs to prototype executing a certain task. Upon completion, the swarm leaves the target, which is then labeled as inactive. (B) Sequential optical images demonstrating the successful navigation of the optical microrobotic swarm towards three targets over a long distance within a complex environment. Scale bar: 5 μm.

We have successfully validated the target navigation capabilities in real complex environments (Figure 5B and Movie S11). In this experiment, 9.5 μm polystyrene particles are dispersed on the substrate to serve as targets. Initially, with no target within the field of view, the swarm initiates a search along a predefined direction. Upon identification of two new targets by the program, the swarm navigates towards and aggregates around the closer target. After 200 iteration loops, the swarm completes its task at the first target, which is then labeled as “inactive”, and proceeds to approach the next target. Following a similar approach, the swarm repeats the process, searching for and aggregating around new targets detected in subsequent field views. This iterative approach demonstrates the prototype execution of specific tasks. For instance, by loading drug molecules onto the microrobots’ surface or leveraging self-heating microrobots, our control algorithm can be utilized to enhance the efficiency of precise drug delivery or cancer therapy based on optical manipulation.^[25]

6. Conclusion

We have devised a versatile strategy for navigating optical microrobotic swarms through complex and dynamically changing environments across long distances. By confining all microrobots within a manipulation domain, we ensure the preservation of microrobots during navigation while eliminating the influence of Brownian motion on swarm movement. Introducing an elliptical approximation enables microrobots to bypass arbitrary obstacles with greater spatial freedom. Furthermore, our control program has been enriched with multiple supplementary functions to enhance the robustness of microrobotic swarms in real-world scenarios: Incorporating an aggregation mode ensures swarm integrity (Figure 2C); the detection of immobilized microrobots aids in their reactivation (Figure 2D); and additional laser beams are employed to trap obstacles that may be drawn towards microrobots, avoiding direct collisions (Figure 4E). Moreover, the capability to temporarily adjust the group’s moving direction enables adaptation to large or fast-moving obstacles (Figure 4D and Figure S6). Based on this, we further achieve target navigation of optical microrobotic swarms in complex environments.

Compared to previous control methods, our strategy offers simultaneous control of multiple optical microrobots, facilitating their transport to any location within the large-scale sample. Importantly, our control program is universally applicable to various types of microrobotic swarms propelled by focused light fields, irrespective of their operation mechanisms. As examples, we have demonstrated microrobots based on optical trapping and nudging mechanisms. Looking ahead, we aim to integrate global path planning into our control program to navigate more complex macroscopic environments, such as mazes and veins. Initially, global path planning will be executed under low magnification, providing a broad field of view to determine the macroscopic path. Subsequently, we will switch to high magnification for precise microrobot manipulation, guiding them along the global path to their intended destinations. Besides, we will also try to exploit functionalized microparticles, navigating them towards target cells in biological environments to demonstrate precise medicine. Overall, our approach extends the utility of optical microrobots to various applications from drug delivery and cell transport to fundamental studies in cell-cell interactions and collective motions.

7. Methods

Sample preparation for different manipulation modes:

Both optical nudging and optical trapping methods are employed in this study, with their techniques outlined in Ref. [10a] and ^[26], respectively. The microrobots utilized are unmodified silica spheres with a diameter of 1.97 μm (Bangs Laboratories, SS04002) while the targets are 9.51-μm polystyrene spheres (Bangs Laboratories, PS07N).

For optical nudging, its operation relies on the synergy of light-induced thermoelectric and depletion forces.^[10a] The thermoelectric force repels the particles away from the laser spot and necessitates the presence of 5 wt% phosphate-buffered saline (PBS, Sigma-Aldrich, 806552) in the solution. Concurrently, the depletion force aids in confining the particles on the substrate surface, requiring 5 wt% polyethylene glycol (PEG, Sigma-Aldrich, 8.18897) in the solution. Both forces arise from the spatial redistribution of ions or molecules under light-induced temperature gradient fields. To establish the thermal field, a 5.5-nm gold-nanoparticle film is employed on glass, achieved through a two-step process: deposition of a 5.5-nm gold film using a thermal evaporator (Kurt J. Lesker, NANO 36) at a base pressure of 1 × 10⁻⁵ Torr, with a deposition rate of 0.5 Å s⁻¹, followed by annealing at 550 °C for 2 hours. Additionally, the substrate is immersed in 1 wt% Bovine-Serum-Albumin (BSA, Sigma-Aldrich, A8531) at room temperature for one day to mitigate particle immobilization.

For optical trapping, its operation relies on opto-thermoelectric forces.^[26] The solution comprises 10 mM cetyltrimethylammonium chloride (CTAC), providing the micelles and ions required for establishing the opto-thermoelectric field under the light-induced thermal field. The substrate remains the same as mentioned above but without BSA modification. It is important to note that the thermoelectric forces play an opposite role in optical nudging and trapping in this study: one repels the microrobot while the other attracts it. This distinction arises from the different solutes dispersed in the solution (PBS versus CTAC), resulting in different electric field directions. Further details can be found in Ref. [10a] and ^[26].

Optical setup:

The optical setup is depicted in Figure 1C. A red laser (660 nm, Laser Quantum, Opus 660) expanded by a factor of 5 is directed towards a liquid crystal on silicon-spatial light modulator (LCOS-SLM, Hamamatsu, X13138-01) with a resolution of 1392×1040 pixels. The laser pattern diffracted by the SLM traverses a 4f lens setup (f₁/f₂ = 0.75) before reaching an inverted optical microscope (Nikon, Ti2). Three types of microscope objectives are employed: a 40x magnification lens (Nikon, Plan Fluor 40X), a 60x magnification oil-based lens (Nikon, CFI Plan Fluor 60XS Oil), and a 100x magnification oil-based lens (Nikon, CFI Plan Fluor 100XS Oil). The charge-coupled device camera utilized is from Lumenera, INFINITY 2, while the motorized stage is from Prior, H117E1N5. These components are interconnected with a computer, enabling digital control of the SLM, camera, and stage.

Control algorithm:

The control algorithm is thoroughly elucidated in the supplementary text S1 and visually represented in Figure S1. In essence, our control algorithm aims to efficiently guide multiple microrobots toward designated targets across complex environments while traversing long distances, all while avoiding any unwanted physical contact between microrobots, obstacles, and targets.

Simulation:

The simulation is conducted using MATLAB, with the control algorithm integrated as outlined in the supplementary text S1. Key mathematical expressions of the algorithm are detailed therein. Additionally, to accurately model microrobots’ motions in real-world scenarios, factors such as Brownian motion and long-range physical repulsion force are accounted for. Leveraging the particle manipulation technique from our prior work on collective motion,^[21c] we employ the same code for modeling both Brownian motion and the long-range physical repulsion force. Thus, detailed quantitative analysis and mathematical expressions regarding these effects are provided in the section “Modelling and simulations of collective motion”, and supplementary text S1 and S2 of Ref. [21c].

Data Availability Statement

The data that support the findings of this study are available from the corresponding author upon reasonable request.