Intelligent Acoustofluidics Enabled Mini-bioreactors for Human Brain Organoids

Abstract

Acoustofluidics, by combining acoustics and microfluidics provides a unique means to manipulate cells and liquids for broad applications in biomedical sciences and translational medicine. However, it is challenging to standardize and maintain excellent performance of current acoustofluidic devices and systems due to a multiplicity of factors including device-to-device variation, manual operation, environmental factors, sample variability, etc. Herein, to address these challenges, we propose “intelligent acoustofluidics” - an automated system that involves acoustofluidic device design, sensor fusion, and intelligent controller integration. As a proof-of-concept, we developed intelligent acoustofluidics based mini-bioreactors for human brain organoid culture. Our mini-bioreactors consist of three components: (1) rotors for contact-free rotational manipulation via an acoustic spiral phase vortex approach, (2) a camera for real-time tracking of rotational actions, and (3) a reinforcement learning-based controller for closed-loop regulation of rotational manipulation. After training the reinforcement learning-based controller in simulation and experimental environments, our mini-bioreactors can achieve the automated rotation of rotors in well-plates. Importantly, our mini-bioreactors can enable excellent control over rotational mode, direction, and speed of rotors, regardless of fluctuations of rotor weight, liquid volume, and operating temperature. Moreover, we demonstrated our mini-bioreactors can stably maintain the rotational speed of brain organoids during long-term culture, and enhance neural differentiation and uniformity of brain organoids. Comparing with current acoustofluidics, our intelligent system has a superior performance in terms of automation, robustness, and accuracy, highlighting the potential of novel intelligent systems in bioelectronics and microfluidics experimentation.

Graphical Abstract

1. Introduction

During the past three decades, microfluidics or lab-on-a-chip systems have been attracting increasing interest and emerging as a formidable tool in basic research and for translational applications in chemistry, biology, and medicine.^1–3 Compared with traditional laboratory techniques, microfluidics technology brings many advantages including reduced sample consumption, system miniaturization, low cost, high throughput, rapid turnaround time, and a precisely controlled microenvironment.^4–6 So far, the innovative design, fabrication, and integration of microfluidic devices and/or lab-on-a-chip systems have enabled a wide range of applications in disease diagnostics, treatment screening, drug delivery, biomedical assays, artificial organs, chemical reactions, combinatorial synthesis, environmental health, microscale energy systems, etc.^7–11 As an active microfluidic technique, acoustofluidics has been developed to improve the versatility and functionality of current microfluidics technologies.^12,13 By integrating acoustic waves into microfluidic devices or lab-on-a-chip systems, acoustofluidics technology provides a unique means to manipulate cells and handle liquids.^13–18 For example, acoustic radiation force has been employed for the trapping, separation, transportation, sorting, and enrichment of cells and particles.^19–25 Moreover, acoustic streaming has been used for the pumping, moving, and mixing of liquids,^26–29 as well as detaching, clustering, and rotation of cells^30–34. Acoustofluidic technology offers useful and unique features and advantages: (1) Based on the nature of acoustic waves, this method provides excellent biocompatibility, contactless operation, and label-free manipulation;³⁵ (2) Acoustofluidic devices consist of simple, compact, and inexpensive microfluidic devices and acoustic transducers, which can be fabricated through standardized processes in a mass-producible fashion;^36–38 (3) Acoustofluidic devices can also be integrated with simple and compact power supply and detection systems into a portable and user-friendly system allowing portable and daily usage in the research laboratory and hospital.^36,37 Therefore, intensive ongoing efforts have been made to further push acoustofluidic and microfluidic technologies from research labs to real-world applications in daily-life and hospital settings.

Current acoustofluidic devices and systems, however, have not yet fully exploited their translational potential, due to several hurdles: (1) Device-to-device variation: Currently, acoustofluidic devices are made through standard MEMS fabrication processes in research labs and manufacturing facilities. However, the regulation of device-to-device variation is commonly required among devices to meet quality standards.³⁹ For example, to provide good working performance, one acoustofluidic device may require slightly different input settings (e.g., frequency, power, phase angle, or flow rate) than another device.^40–42 The device-to-device variation raises the bar for the standardization of acoustofluidic technology. (2) Environmental and temporal factors: The working performance of acoustofluidic devices also depends on environmental and temporal factors. ^39,43 Taking the bulk acoustic wave-based cell separation devices as an example, the separation device may have an excellent cell separation performance at room temperature, but provide very poor separation results at physiological temperatures due to the heating effect of the device. ^28,44 This is because the temperature increase changes the resonant frequency of the separation device. Similarly, high or low temperature may also impact the resonant frequency of the separation device, resulting in the poor working performance of the device. Therefore, considerable engineering effort is needed to mitigate the impact of environmental and temporal factors and maintain the excellent performance of acoustofluidics in daily life settings. (3) Automation: Tremendous manual operation and intervention are required for most acoustofluidic experiments, especially to circumvent the impact of environmental and temporal factors, and device-to-device variation.⁴⁵ These manual operations are labor-intensive, expensive, and significantly limit broad applications in an environment outside of a microfluidic research laboratory. Thus, there is an urgent need to develop automated and standardized acoustofluidic devices and systems for basic research and translational applications in real-world settings.

Machine learning (ML), a subset of artificial intelligence (AI), may provide new opportunities for the development of microfluidic and acoustofluidic technologies because this method enables computer algorithms to learn from experience. Recent engineering efforts have been made to combine a variety of ML methods (e.g., supervised learning, and reinforcement learning) with microfluidic devices and lab-on-a-chip systems for broad applications in biomedical sciences from several aspects.^46,47 Supervised learning has been used to process and analyze data from high throughput microfluidic experiments, addressing the throughput, accuracy, and robustness issues of current manual data handling processes.⁴⁸ Supervised learning is a subset of ML methods that learn the mapping function from the input to the output using the training dataset, and then predict questions such as image processing and classification. For example, after being trained with microfluidic experimental results, convolutional neural network (CNN) algorithms have been employed as analyzing tools for cell classification in image-based microfluidic cytometer devices,^49–51 cell counting in optofluidic devices⁵² and acoustofluidic chips⁵³, and automated detection and sorting of microencapsulation⁵⁴. Similarly, supervised learning has been employed for the optimization and prediction of microfluidic design and experimentation⁵⁵. For example, deep artificial neural network (DNN) algorithms have been utilized to design microchannel architectures for customized acoustic field and microparticle patterning,⁵⁶ microfluidic geometry design,⁵⁷ and flow modeling⁵⁸. Different from supervised learning that focuses on mapping inputs to outputs based on labeled input-output pairs, reinforcement learning (RL) learns from the agent-environment interaction through maximizing the notion of cumulative reward, enabling the most effective method for control problems.^59,60 Recently, the RL method has been used to address the challenges of dynamic microfluidic system control.^61,62 For example, Deep Q-Networks, and model-free episodic controllers have been employed to establish an automated control system for the precise regulation of dynamic laminar flow and microfluidic droplet generation.⁶¹ Compared to manual operation, this RL-based system demonstrated a significant improvement in consistency, robustness, and repeatability during long-term microfluidic experiments. Despite there only being very few attempts, the fusion of RL algorithms and microfluidics has started to show promising potential in building automated microfluidic systems and maintaining microfluidic performance. Therefore, ML methods, especially RL algorithms, seem to hold a promising solution to address the above-discussed challenges in acoustofluidic experiments.

Herein, we introduce a new concept “intelligent acoustofluidics”, which is an RL-based closed-loop system for the standardization, optimization, and maintenance of acoustofluidic devices and systems. Particularly, we aim to develop proof-of-concept intelligent acoustofluidics enabled mini-bioreactors to improve human brain organoid culture. As is known, the rotational culture of organoids using bioreactors or orbital shakers is a very important step for organoid differentiation and maturation. However, the low rotational speed of organoids may lead to insufficient oxygen/nutrient perfusion and poor differentiation, whereas high rotational speed may lead to high mechanical shear stress and surface cell death. Current bulk orbital shakers or bioreactors fail to provide a uniform rotational culture environment for each organoid, limiting the yield of successful organoid formation. The recent advance of miniaturized spinning bioreactor has provided a versatile, simple, cost-effective platform by engineering a multi-well spinning device to a standard 12-well tissue culture plate, providing a better engineering method for brain region-specific organoid formation.⁶³ The emerging microfluidic systems also bring new insides for brain organoids culture, providing enhanced perfusion as well as reduced necrosis and promoted differentiation.^64,65 However, challenges remain in improving the organoid culture conditions. The organoids are growing from several hundred microns to several millimeters in diameter during the spinning or rotational culture. The increased weight of organoids may slow down the rotational speed of the suspended organoid cultures, causing inconsistency of organoid culture conditions. The increased size of organoids may cause hypoxia and necrosis to further impact the viability of the organoids. Therefore, there is a great need to maintain a stable rotational culture of organoids (e.g., rotational mode, and speed) and to improve the perfusion of nutrients and oxygen for reducing organoid hypoxia and necrosis during the long-term culture of organoids.

To address these issues to improve organoid cultures, we explore intelligent acoustofluidics enabled mini-bioreactors. Compared with current bioreactors and orbital shakers, our intelligent mini-bioreactors have several unique advantages for organoid culture. (1) Closed-loop control of rotational culture: Current organoid culture systems lack a mechanism to check and maintain culture conditions during long-term culture. Through the fusion of a CCD camera, acoustofluidic rotors, and a controller, our intelligent mini-bioreactors can track, tune, and maintain the organoid culture condition (e.g., rotational speed) in real-time. (2) Mitigation of inconsistency: Current bulk organoid culture systems have difficulty in providing a consistent rotational culture condition during a long-term culture due to the growth of organoids and manual operations. Through the integration of RL algorithms, our intelligent mini-bioreactors minimize system variations to stably maintain the rotational speed of organoids with high consistency. (3) Introduction of perfusion scaffolds: Through the introduction of 3D printed organoid scaffolds into the rotor device, our method enhances medium perfusion to reduce or avoid hypoxia and necrosis within the organoids. (4) Incorporation of contact-free rotors: The employment of an acoustic spiral phase plate (ASPP) ^66,67 enables the contact-free rotation of rotors via the regulation of highly biocompatible acoustic pressure fields, avoiding organoid damage from mechanical contacts. (5) Generation of better organoids: Compared with conventional organoid culture methods, our intelligent mini-bioreactors could generate human cerebral organoids with a more uniform size distribution of ventricular/subventricular zone (VZ/SVZ), and better differentiation commitment to the forebrain identity. Moreover, this novel intelligent system control concept is not limited to acoustofluidics, it can also potentially change the technical form of bioelectronics and biodevices. The integration of advanced RL algorithms enables the automated operation of different devices and actuators, reducing manual intervention and ensuring consistency under a variety of factors including device-to-device variation, manual operation, environmental factor, sample variability, etc. Compared to current systems heavily dependent on manual intervention, this intelligent system can provide better performance in terms of automation, robustness, and accuracy. Most importantly, such intelligent systems may significantly contribute to pushing biodevices and sensors from the research laboratory to their translational applications in real-world settings.

2. Experiment

Fabrication of rotor devices.

With the optimized simulation design, ASPP-based rotor devices were fabricated using a stereolithography 3D printer with clear resin (Formlabs, Massachusetts). The ASPP of the rotor was designed to operate at 2MHz consisting of 2 parts: a cylindrical base plate of 14 mm diameter and 0.5 mm thickness, and a spiral phase plate with 4 spiral identical portions with diameter 12 mm, angular width π/2, and height 2.2611 mm. Each spiral portion ensures a 2π phase delay ramp along the angular direction. Therefore, an 8π circulation of the phase delay is produced around the ASPP axis.

Experiment setup and operation.

During the rotational experiment, an ASPP-based rotor was placed at the interface between fluids and air. The ASPP was irradiated from below by a lead zirconate titanate (PZT) transducer (dimension, 20 mm x 10 mm x 1mm; resonant frequency, 2 MHz, PZT-41, Yuhai Electronics Ceramics, Co. Ltd, China). The PZT transducer was immersed in water to minimize the power loss during propagation and was driven by a 2 MHz RF signal. The RF signal was generated by a function generator (AFG3021C, Tektronix, Oregon) and amplified by a power amplifier (LZY-22+, Mini-circuit, New York) to drive the acoustofluidics rotational device. The function generator was connected to the computer via a USB interface module (NI-VISA, PyVisa) and output signals were controlled by a customized Python control program. The rotational status of the ASPP was monitored from above by a CCD camera (Adesso CyberTrack H4) connected to the computer as feedback. During the rotational control experiment, the frequency and power of the RF signal were selected by the closed-loop controller and generated by the function generator, based on the feedback rotational status from the video camera.

Problem formulation.

The goal of the RL-based controller is to maintain a stable rotational manipulation of organoids (e.g., rotational speed) for long-term culture. We formulate the acoustic power and rotational speed as a Markov Decision Process (MDP)⁶⁸ for maximizing the overall reward over time, the details are as below:

1. State: The state of the MDP is defined as the current RF signal parameters and rotational speed of the ASPP-based rotor as

   $$S_t=(F_t^{out},P_t^{out},V_t)$$

   where S_t is the state at time $t,F_t^{out},P_t^{out}$ are the frequency and power of the output RF signal, and V_t is the current rotational speed of the ASPP-based rotor.
2. Action: The action of the MDP is defined as the frequency and power of the resetting RF signal as

   $$A_t=(F_t^{set},P_t^{set})$$

   where A_tS_t is the action at time $t,F_t^{set},P_t^{set}$ are the frequency and power of the RF signal selected by the controller based on the current state observation, can be represented as

   $$A_t=\pi \left(S_t\right)$$

   where π is the control policy for the rotation system.
3. Reward: The reward of the MDP can be divided into two parts based on the control goal, the penalty for the rotational speed variation from the set goal, and the penalty for the influence of acoustic power. Specifically, as the rotational speed deviates from the set rotational speed or more power is provided to the PZT transducer (which is undesirable for the health of the organoid), the reward decreases. The surrounding microenvironment of the BO can be regarded as stable when the rotational speed is within a small range. Thus, we set this range as [-D, D], where D is the threshold for the stable environment value. Within this range, the penalty from the rotational speed will be zero. Otherwise, the penalty will be calculated based on the difference between the set and the current rotational speed. By joining the two parts together, we set the overall reward of the MDP as

   $$R_t\left(S_t\right)=-\beta P_t^{out}+\left\{\begin{array}{ll}0,& \left|V_t-V_{goal}\right|<D\\ -\left|V_t-V_{goal}\right|,& \left|V_t-V_{goal}\right|>D\end{array}\right.$$

   where R_t is the reward for time t and β is the weight of the acoustic power. β represents the importance of acoustic power when compared to rotational speed control. In our case, stable rotational speed control is more important than acoustic power. Thus, β is set to be a small value and will be minimized after achieving the rotational speed threshold.
4. Reward maximization: Our goal is to maximize the overall discount reward overtime by deriving the optimal rotational control policy for acoustic amplitude optimization and stable rotational speed control, mathematically, the objective is

   $$\underset{\pi }{\max }\sum _{t^{\prime }=0}^{\mathrm{\infty }}{\gamma }^{t^{\prime }}{R}_{t+t^{\prime }}(S_{t+t^{\prime }})$$

   where γ is the discount factor. Since the whole system is complex and it is impossible to directly solve the problem, we adopt an RL algorithm to optimize the control policy.

RL for rotational control policy.

Rotational control is a continuous problem and requires a continuous selection of acoustic parameters. Thus, we adopt Deep Deterministic Policy Gradients (DDPG) method,⁶⁹ which is a natural solution for rotational control. Because the action space of DDPG is continuous, we can directly obtain the setting parameters of the RF signal from the outputs of DDPG. With DDPG, the network only has two outputs, namely, the real-valued set-points of frequency and amplitude of the RF signals. We first used the simulation environment to learn the optimal policy and reduce the experiment cost. The parameters of the simulation environment were adapted from the experiment data. We then transferred the acquired policy to the real experiment for further training and optimization. During the acoustic rotation experiments, once a frame is captured by the camera, the current rotation speed of the rotor is determined. The measured speed is then fed to the controller. The controller calculates the optimal action according to the trained policy and selects the acoustic parameters that drive the rotor towards the speed setpoint. The signal of selected acoustic parameters is output to the PZT transducer until the next image frame is received by the controller. The controller continues maintaining the rotational speed by continuously approaching the rotational speed setpoint.

Human embryonic stem cell culture.

Matrigel (Corning) coated 6 well plates were employed to culture Human embryonic stem cell line WA01 (WiCell) with mTESR plus medium (Stemcell Technologies). The medium was changed every other day. The WA01 cells were passaged every week using ReLeSR kit (Stemcell Technologies). WA01 cells before passage 42 were used to generate brain organoids.

Brain organoid culture.

Brain organoids were prepared using STEMdiff™ Cerebral Organoid Kit. Embryonic bodies (EBs) were generated with ~9,000 cells using a U-bottom ultra-low-binding 96-well plate (Corning, NY). EBs were prepared from the single-cell suspension of a human embryonic stem cell line WA01 (WiCell). Cells were counted and resuspended in EB formation media. EBs were fed every other day with EB formation media for 3 days. Subsequently, EBs were seeded onto the 3D printed scaffold of rotors and transferred to Induction Medium in a 24-well Ultra-low Attachment Plate (Corning, NY). EBs were cultured for an additional 2 days and were then embedded in liquid Matrigel (Growth Factor Reduced, Corning, NY). Embedded organoids were maintained in an expansion medium for 3 days. On Day 10, organoids were switched to Maturation Medium and cultured on the acoustic intelligent rotor set at 60 RPM. Organoids were fed every 3 – 4 days with a maturation medium.

Human cerebral brain organoids (hCOs) were fabricated as we previously reported.^70–72 Briefly, 9,000 WA01 cells were harvested from culture and aggregated into embryonic bodies using Aggrewell-800 plates (Stemcell Technologies) in EB formation medium (Stemcell Technologies) supplemented with 10 μM Y-27632 (SelleckChem). The aggregated WA01 cells form EBs in one day. The EBs were then derived into human cerebral organoids using the STEMdiff Cerebral organoid kit (Stemcell Technologies)

Cryo-section of organoids.

After 35-days of culture in conventional/intelligent acoustofluidic methods, the culture samples were washed three times in 1x Phosphate-Buffered Saline (PBS, Gibco) and then fixed in 4% paraformaldehyde in 1x PBS (Thermo Scientific) overnight at 4 °C. The samples were rinsed three times with 1x PBS and then transferred into 30% sucrose (w/v) solution overnight at 4 °C for cryoprotect. The organoids were then incubated with 7.5% gelatin (w/v) 10% sucrose (w/v) in PBS solution at 37 °C for 1 hour. Finally, the samples were snap-frozen onto a cryomold (Sakura Finetek) in a dry ice/ethanol slurry. The frozen samples were then sectioned at 30 μm thickness on a cryostat (Leica).

Immunofluorescence staining.

To characterize the brain organoids cultured using conventional or intelligent acoustofluidic methods, sectioned samples were placed onto charged glass slides and washed three times with 1x PBS. The samples were then treated with 3N hydrochloric acid (HCl) for 15 minutes for antigen retrieval. Following HCl treatment, the samples were washed again twice with 1XPBS and subjected to blocking (0.3% Triton-X100, 5% normal goat serum in 1XPBS) for 1 hour, followed by primary antibody incubation in a humidified chamber at 4 °C overnight. The samples were then washed 3 times with 1XPBS followed by secondary antibody incubation at room temperature for 1 hour before being washed and coverslipped with gold anti-fade mounting medium containing DAPI (Invitrogen). Detailed antibody information and dilution factors can be found in Supplementary Table S1.

Gene expression analysis.

Mature human brain organoids were lysed for RNA extraction using RNeasy Mini Kit (Qiagen). RNA was first reversed transcribed into complementary DNA (cDNA) using cDNA synthesis kit (Quantabio). The cDNA was then analyzed by quantitative PCR using SYBR green kit (Applied Biosystems). Gene expression fold change was analyzed by normalizing against housekeeping gene GAPDH.

Statistical analysis.

Students’ t-test in GraphPad Prism 8 was employed to statistical compare the two sample groups. Statistical significance was shown as following: * (p<0.05), ** (p<0.01), *** (p<0.001).

3. Results and discussion

Intelligent acoustofluidics enabled mini-bioreactors.

The ‘intelligent acoustofluidics’ concept involves the development of automated systems with the support of acoustofluidic devices, sensors, and intelligent controllers to interact and learn from biomedical environments (Figure 1a). The intelligent acoustofluidic system observes the biomedical environment through its sensors and outputs actions to acoustofluidic devices based on the observations. The biomedical environment is actively driven by the actions of acoustofluidic devices, and a reward signal representing the quality of the action is returned based on the environmental changes. The agent repeatedly interacts with an environment and iteratively maximizes the rewards obtained from the environment. As a result, the intelligent acoustofluidic system can be automatically maintained at the best performance regardless of a multiplicity of variance factors. To improve the organoid culture, we developed intelligent acoustofluidics enabled mini-bioreactors consisting of three components (Figure 1b): (1) rotors for rotational manipulation via an acoustic spiral phase plate (ASPP) which can be digitally regulated by RF signals, (2) a CCD camera for real-time tracking of rotational actions and sending dynamic data to the controller of intelligent mini-bioreactors, and (3) an RL-based controller of the intelligent mini-bioreactors for tracking and closed-loop regulation of rotational manipulation. After trained the RL-based controller in simulation and experimental environments, our intelligent mini-bioreactors took in the observations (e.g., the rotational speed of rotors) from the camera and outputted actions (e.g., acoustic frequency, and amplitude) to control the rotation of the rotors. By maximizing the rewards, our intelligent mini-bioreactors can improve its performance and control the rotor at the desired rotational speed. By using this strategy, our intelligent mini-bioreactors can automatically tune and stably maintain the rotational speed of rotors by desire regardless of device-to-device variation, environmental factors, and operational variation.

Figure 1. Concept of intelligent acoustofluidics.

(a) The intelligent acoustofluidics system involves acoustofluidic device design, sensor fusion, and intelligent controller integration. The agent (RL controller) observes the biomedical environment through the sensors and outputs actions to the acoustofluidic devices based on the observations. The biomedical environment is actively driven by the actions of acoustofluidic devices, and a reward signal representing the quality of the action is returned based on the environmental changes. (b) Illustration of intelligent acoustofluidics enabled mini-bioreactors for human brain organoid culture.

Design and fabrication of rotors.

Based on our acoustofluidic simulation and experimental experience,^70–72 we designed and optimized the ASPP-based rotors. Each rotor consisted of an ASPP and a hollow scaffold for organoid culture in one intelligent mini-bioreactor. The ASPP was designed as a chiral plate with an azimuthally varying thickness $e=\mathcal{l}\epsilon [1-\varphi /(2\pi \left)\right]$, with $\mathcal{l}$ an integer for topological charge and ϕ the azimuthal angle in the transverse plane with respect to the beam propagation axis, $\epsilon =\frac{1}{f(c_1^{-1}-c_2^{-1})}$, with $f$ the frequency of the sound wave, c₁ the sound speed in the medium surrounding the ASPP and c₂ the sound speed in the material. A normally incident axisymmetric acoustic wave propagating through the ASPP carried a pressure field with an azimuthal dependence of the form $\mathrm{e}\mathrm{x}\mathrm{p}\left(i\mathcal{l}\varphi \right)$, also known as an acoustic vortex with topological charge $\mathcal{l}$. Under a plane acoustic field, the ASPP generated orbital angular momentum (OAM) to transfer incident acoustic waves into the acoustic vortex (Figure 2a). Due to angular momentum conservation, such an acoustical vortex carrying nonzero acoustic OAM induced a nonzero acoustic radiation torque exerted on the ASPP. This acoustic torque induced mechanical actions to drive rotation around the beam propagation axis.

Figure 2. Rotational manipulation via an acoustic spiral phase approach.

(a) Illustration of the mechanism to generate an acoustic vortex with orbital angular momentum by an acoustic spiral phase plate. (b) Acoustic phase and intensity distribution generated by acoustic spiral phase plates with different numbers of spiral portions (l=1,2,4,6). (c) Experimental and numerical relationships between acoustic torque and spiral portion number. The acoustic spiral phase plate with a 4-spiral portion was selected as the optimized design to generate maximal torque with minimal fabrication complexity.

A numerical model was used to calculate the field distribution of acoustic pressure and phase of the acoustic vortex generated by the ASPP. This model considered the effects of the longitudinal vibrations in the liquid and the interface between the ASPP and the liquid in the cell culture well. The working frequency of the ASPP was set to 2MHz, within the frequency range of widely used medical ultrasonic imaging. The diameter of the ASPP was set to 14 mm to fit in the 24 well plates. The parameter that we needed to optimize was the topological charge $\mathcal{l}$. The acoustical phase and intensity distributions of ASPPs with different topological charges ($\mathcal{l}=\mathrm{1,2},\mathrm{4,6}$) were simulated (Figure 2b). The phase patterns of the acoustical vortex twisted azimuthally at the same number of topological charges. While the intensity patterns of the acoustic vortex presented high areas surrounding the lowest center. As our ASPPs of different topological charges were able to generate acoustic vortex, we further investigated the acoustic torque exerted on the ASPP of different topological charges. From the calculation, the acoustic torque applied to the ASPP was proportional to the reflection angle of the spiral surface. (Figure 2c, the acoustic torque was normalized to $\mathcal{l}=1$) The ASPP with the same operation frequency (2 MHz) and diameter (14 mm), but with different numbers of topological charges ($\mathcal{l}=\mathrm{1,2},\mathrm{4,6}$), were produced to perform the experimental validation. As the numbers of identical spiral portions increased before $\mathcal{l}=4$, the corresponding acoustic torque increased correspondingly (Figure 2c). To minimize acoustic effects on the biological cultures, we chose an ASPP with a topological charge of $\mathcal{l}=4$ as our design, since this design can generate maximal torque with minimal acoustic power.

Tracking rotation.

To speed up the training process and avoid excessive interaction with the real environment, we developed a simulation environment to pretrain the RL algorithm. To build the simulation environment, the rotational speeds of rotors were quantified under different input frequencies and amplitudes. An ASPP-based rotor was employed to investigate the dependence of rotational speed on the input voltage (Figure 3a). When applying acoustic waves, the rotor was driven to rotate by the torque generated by the acoustic vortex. At a steady-state, the SPP rotates at constant angular velocity Ω in the clockwise direction (when looking at the SPP from the top) (Figure 3b). The rotational speed under different voltage inputs was further quantified with this setup (Figure 3c). When the voltage is lower than 21 Vpp, the acoustic torque was not enough to drive the movement of the rotor. As the voltage increased beyond 21 Vpp, the rotor started to rotate and showed a proportional relation with the input amplitude. When the voltage was adjusted from 25 Vpp to 32 Vpp, the rotor rotated at a suitable speed range (57~80 RPM) for organoid development.^65,73,74 A simulation environment for the rotor was developed based on the collected data. The simulation environment took the rotational speed and the signal amplitude as the input and estimated the resulting rotational speed and acceleration of the rotor as the output. A random variation with a standard normal distribution was also added to the output to account for disturbances and uncertainty of the system.

Figure 3. Tracking rotational speed.

(a) Illustration of the imaging setup for tracking acoustofluidic rotation. (b) Snapshots of the top view of the acoustic spiral phase plate through a full rotation. The rational speed of the rotor can be measured by tracking the red ink mark on the rotor (indicated with yellow arrows). (c) Experimental results describe the dependence of the rotational speed of the acoustic spiral phase plate on the acoustic amplitude. Scale bar: 2 mm.

Tuning rotational speed.

After optimizing the ASPP-based rotor design, the RL-based controller was developed in the simulation and experimental environments and then integrated into the acoustic rotational system. The RL-based controller was developed in a Markov decision process (MDP) framework. The control agent took in rotational speed (state) as input, and output RF signal (e.g., frequency and amplitude) to the PZT transducer as action (Figure 4a). The change of amplitude (action) altered the rotational speed (state) and consequently altered the decision of the control agent. By narrowing the difference between current rotational speed and speed setpoint, the rotational behavior of rotors was controlled by desire. The RL-based controller was first implemented and tested in the pre-built simulation environment. During the training process, the controller continuously altered the RF signals applied onto the PZT transducer, resulting in altered rotational speed. After a fixed number of steps (250 steps in each episode), the environment was reset and the rotor was reset to a random rotational speed (0~80 RPM), and the controller restarted its controlling task. After each episode, we trained the policy neural networks according to the accumulated rewards and learning algorithm and stored the trained network. The reward learning curve was acquired from the simulation environment, after 100 episodes of training, the algorithm maximized the rewards at around 0 (Figure 4b).

Figure 4. RL-based control of rotational speed.

(a) Illustration of the RL algorithm architecture. The agent takes rotational speed from the sensors and outputs the amplitude and frequency of RF signal actions. The actions influence the rotational speed, and a reward signal representing the quality of action is returned to the agent. The agent learns the optimal control policy by maximizing the cumulative rewards. (b) Curve mapping the cumulative rewards in the training process. (c) A representative result showing the customized rotational speed regulation with a trained RL controller. (d) A representative result showing the stability of our system under a sudden weight disturbance (corresponding to Movie S1). The insert images shows a sudden weight load at second 9 (the rotational speed drops to about 40 RPM), and the system recovered (the rotational speed comes back to about 60 RPM) within about 10 seconds.

Then, the acquired policy from the simulation environment was transferred to the real experimental environment. The rotation speed of the rotor is monitored by the camera using a classical object tracking algorithm, giving observation to the computer, and reward is also calculated based on the rotation speed. As the rotation speed approaches the desired speed, the reward rises. The computer (agent) takes in the observations from the camera and output actions to the signal generators to control the rotation of the rotor device. By maximizing the rewards, the agent improves performance and controls the rotor at the desired rotational speed. After further 50 episodes of training in the real environment, the controller successfully controlled the rotational speed in the real system. With the further trained policy from the real environment, we demonstrated the customized rotational speed control (Figure 4c). With the RL controller, one could control the rotational speed of the rotor in the desired pattern.

After demonstrating the precise tuning of rotational speed, we also investigated the stability of our system. It is hard to maintain a constant rotational speed of a developing organoid, because the organoid gains weight during the long-term development process. Thus, we tested the stability of our intelligent mini-bioreactors by disturbing the rotational speed with a sudden weight load (Figure 4d). For example, the intelligent mini-bioreactor was first set and maintained to a rotational speed at 60 RPM. After a weight loading on top of the rotor, the rotational speed decreased to 40 RPM. Our intelligent mini-bioreactor automatically detected the speed variation, tuned the acoustic input signals, and recovered the rotational speed to the initial constant rotational speed (60 RPM) within 10 seconds (Movie S1). Thus, we demonstrated our intelligent mini-bioreactor can maintain stable rotational speed regardless of weight load, paving the way for the stable rotational culture of developing organoids.

Spinning culture of human cerebral organoids.

We further applied our system to improve brain organoid culture conditions, based on our early experiences of organoid culture protocols.^75,76 Our intelligent mini-bioreactors were able to spin the brain organoid with a stably controlled axis and speed. The EBs were loaded and cultured into the 3D printed scaffold under the rotor on Day 3. Each intelligent mini-bioreactor started to spin the 3D printed scaffold that attached with EBs on Day 10 with a stably maintained rotational speed (Figure 5a). Here the rotational speed was set as 60 RPM for organoid rational culture because this optimized speed range was showed to increase oxygenation, reduce “dead cores” of the organoid, and minimize the potential damage of the organoid viability.^65,73,74 During the long-term culture process, the human cerebral organoids were growing and extending on the printed 3D scaffolds of rotors (Figure 5b schematics). The extension of organoids on the 3D printed scaffold was recorded on Day 10, Day 19, and Day 35 (Figure 5b microscopic images). We measured the rotational speed of each rotor with the growing organoids on Day 10, Day 19, and Day 35 (Figure 5b plots). We found our intelligent mini-bioreactors were able to maintain the rotational speed at 60 RPM regardless of the brain organoid growth during the long-term culture process. The intelligent mini-bioreactors are compatible with brain organoids generation and maintained stable rotational speed and axis for a long period (more than 30 days).

Figure 5. Organoid culture using our intelligent mini-bioreactors.

(a) The culture protocol of human cerebral organoids (hCOs) using our intelligent mini-bioreactor. (b) The schematics and experimental results of hCO growth during the culture process of our intelligent mini-bioreactor. The hCOs grew on the printed 3D scaffolds of rotors (images at Day 10, Day 19, and Day 35), while the rotational speed was precisely controlled at 60 RPM during the whole organoid development process. The extent of organoids was marked with the blue dashed curves. Scale bar: 500 μm.

After cultured within our intelligent mini-bioreactors for 35 days, we characterized human cerebral organoids generated by our intelligent acoustofluidic mini-bioreactor (AhCOs) via cryo-section, immunofluorescence staining, and gene expression analysis. We also compared the differences between AhCOs and human cerebral organoids produced by a conventional method (hCOs). The PAX6/MAP2 staining of organoid cryo-section indicated the AhCO had more uniformed VZ/SVZ zones than the hCO (Figure 6a). Moreover, we found that the distribution of VZ/SVZ zones within AhCOs was more uniformed than that within hCOs via quantitative measurement of more than 10 organoids (Figure 6b). We further employed qRT-PCR to determine detailed gene marker expression differences between AhCOs and hCOs. Organoids at Day 35 (neuron maturation stage) were collected, lysed for RNA-extraction, and analyzed for gene expression (Figure 6c,d). We found that AhCOs have higher expressions of dorsal neural progenitor marker PAX6 as well as cortical neuron marker TBR1 and CTIP1 (P<0.05), indicating enhanced neural differentiation. Moreover, compared to hCOs that express heterogeneous differentiation markers indicative of multiple brain regions, AhCOs showed a higher forebrain fate commitment, with lower gene expression for other brain regions. Thus, our results demonstrated a better differentiation of organoids using our intelligent mini-bioreactors than conventional organoid culture methods.

Figure 6. Comparison of AhCOs and hCOs.

(a) Cross-section staining of the neuron (MAP2, in green) and neural progenitor cells (PAX6, in red) distribution within human cerebral brain organoids (hCOs) regulated by intelligent acoustofluidic mini-bioreactor (AhCOs) and conventional hCOs at Day 35. (b) The quantified size distribution of VZ/SVZ inside AhCOs and hCOs. (c) Gene markers expression level related to forebrain neural differentiation: PAX6, TBR1 and CTIP2. (d) Gene markers expression level related to various brain regions: dorsal forebrain (dFore), ventral forebrain (vFore), midbrain (Mid), hindbrain (Hind), and derm layers including mesoderm (Meso) and endoderm (Endo) inside AhCOs and hCOs on Day 35. Scale bar: 500 μm.

4. Conclusion

In this paper, we introduce a novel intelligent acoustofluidic concept that combines acoustofluidic device design, sensor fusion, and intelligent system integration. As a demonstration, we developed intelligent acoustofluidics enabled mini-bioreactors for human brain organoid culture. By integrating this system with an advanced controlling algorithm, we demonstrated stable control of the rotational speed of brain organoids over an extended period. The results were quite promising in terms of controller stability and convergence over rotational mode, direction, and speed of rotors, regardless of the fluctuations of rotor weight, organoid mass, and liquid volume. As a brain organoid culture system, the intelligent mini-bioreactors provides excellent control over brain organoids rotation and enables the culture of brain organoids with a more uniform VZ/SVZ size distribution. Such a system proves the advantageous performance over the conventional acoustofluidic experimental setup. The automation, robustness, and accuracy brought by the integration of intelligent systems enable optimal experiments regardless of any error factors, including device-to-device variations, manual operation, environmental factor, sample variability, etc. In the future, we expect that the implementation of novel intelligent systems will offer optimal performance over active biomedical devices and advanced fields of microfluidics, bioelectronics, and biomedical systems.