Robot motor learning shows emergence of frequency-modulated, robust swimming with an invariant Strouhal number

Abstract

Fish locomotion emerges from diverse interactions among deformable structures, surrounding fluids and neuromuscular activations, i.e. fluid–structure interactions (FSI) controlled by fish's motor systems. Previous studies suggested that such motor-controlled FSI may possess embodied traits. However, their implications in motor learning, neuromuscular control, gait generation, and swimming performance remain to be uncovered. Using robot models, we studied the embodied traits in fish-inspired swimming. We developed modular robots with various designs and used central pattern generators (CPGs) to control the torque acting on robot body. We used reinforcement learning to learn CPG parameters for maximizing the swimming speed. The results showed that motor frequency converged faster than other parameters, and the emergent swimming gaits were robust against disruptions applied to motor control. For all robots and frequencies tested, swimming speed was proportional to the mean undulation velocity of body and caudal-fin combined, yielding an invariant, undulation-based Strouhal number. The Strouhal number also revealed two fundamental classes of undulatory swimming in both biological and robotic fishes. The robot actuators were also demonstrated to function as motors, virtual springs and virtual masses. These results provide novel insights in understanding fish-inspired locomotion.

1. Introduction

The diversification of fish locomotion rests on 530 million years of evolution that successfully explored various forms of fluid–structure interaction (FSI) for underwater locomotion [1,2]. The most common forms of fish locomotion, i.e. body and/or caudal-fin (BCF) swimming [3], include active structures such as fins and elongated bodies; they are deformed by an array of distributed muscle forces, leading to interactions with surrounding fluids and their passive deformation by hydrodynamic forces, forming a FSI loop (figure 1ai) [4]. Undulatory swimming gaits emerge from such FSI under rhythmic motor control, which includes central and peripheral neural systems that combine feedforward control (from motor signal to swimming gaits) with peripheral sensory feedback (pressure force due to the interaction between body and water) [5]. This combination offers high robustness of rhythmogenesis and gait coordination against neural disruptions, as evidenced by a recent study using a robot to emulate lamprey neuromechanics [6].

Figure 1.

Overview of µBot motor learning experiment and µBot design. (a) Framework of µBot motor learning experiments for swimming, which includes (i) the fluid–structure interaction (FSI) between the robot and fluid system; (ii) the emergence of swimming gaits and performance; and (iii) the experimental set-up for robot learning, where the FSI is motor-controlled according to a motor program (central pattern generator, CPG) which was optimized via policy-gradient reinforcement learning (RL) method. (b) Design of µBots (µBot-6 used as an example) with a cross-section view of the body segment. (c) µBot motor program designed as a CPG with its parameters labelled (µBot-2 used as an example). Dashed box represents a CPG module corresponding to a single actuator, which includes two neurons inhibiting each other. Parameters labelled with colours were learned experimentally, while those in black were fixed a priori.

The interplay between fish's motor system and physics of FSI is complex, especially considering the diversity in fishes' morphologies and swimming behaviours [7]. However, previous studies have hinted that such motor-controlled FSI may possess embodied traits, which are properties of swimming behaviours that are independent of body size, morphologies and neuromuscular control. For example, most BCF fish species exhibit similar midline kinematics during steady swimming and share major kinematic features, suggesting existence of unifying locomotor hydrodynamic mechanisms across various morphologies [8]. Studies have shown that swimming speed of multiple fish species (e.g. dace [9], green jack [10] and odontocete cetaceans [11]) is sensitive and proportional to caudal-fin undulation frequency, suggesting simplicity in speed control. The Strouhal number, which characterizes wake patterns and hydrodynamics of undulatory swimmers [12], is typically in a narrow band for a single species at various speeds [13], presumably due to relatively small variations in caudal-fin beating amplitude and proportionality between speed and undulation frequency [9]. Furthermore, fishes actively modulate their body stiffness at various undulation frequencies, enabling them to match their body's natural frequency with the undulation frequency of caudal-fin and therefore minimizes the mechanical cost of bending [14]. These embodied traits govern fish or fish-inspired locomotion and may offer simplicity and high predictability for gait generation and swimming performance.

However, current understandings of FSI-embodied traits are mainly based on scattered observations [15]. The implications of these traits in motor learning, neuromuscular control, gait generation and swimming performance have not been thoroughly examined [16], and many questions remain to be addressed. For example, is the observed proportionality between speed and caudal-fin frequency a result of robustness in gait generation and swimming speed? Does it translate to simplicity in motor learning? i.e. fast convergence of frequency? Can we also establish a trackable, quantitative mapping between motor control, swimming gaits, and speed? Additionally, whether and how do the muscle functions vary with motor control frequency or speed? Answering the above questions has critical importance for advancing the biomechanics of fish swimming and their informed mimicry in bioinspired robotics.

Robot models (sometimes referred to as robophysical models) offer a unique opportunity to provide quantitative results to address the above questions that are difficult to obtain in biological fish [17]. Using robot models, not only can we generate variations in morphology, but we also have full motor control and the ability to monitor internal variables and swimming gaits. This allows for systematic testing and validation of embodied properties and their implications. Moreover, the discoveries made in robotics can inform the study of living organisms, leading to new hypotheses and insights (i.e. robotics-inspired biology [18]).

In this work, we aimed to systematically study the FSI-embodied traits in fish-inspired BCF swimming. Instead of directly measuring the physical properties and states of FSI, we considered the relationship among the inputs (i.e. the motor control) and outputs of the FSI (i.e. swimming gaits and speed, figure 1a). We designed and assembled modular robots for BCF swimming, named μBot (figure 1b). The robots were directly torque-controlled by electromagnetic motors receiving voltage inputs generated by central pattern generators (CPGs) (figure 1c) so that the motor control of μBot followed a feedforward control pattern. We generated variations in the robot designs. For each robot, we first used reinforcement learning (RL) to optimize the CPG parameters for maximizing the swimming performance (forward or backward swimming speed, figure 1aiii). Note that, rather than being optimized directly, the swimming gaits emerged from the motor-controlled FSI (figure 1ai and aii). Next, we measured the frequency response of the robots, and analysed the changes in swimming gaits, speed, actuator work loops and cost of transport (CoT). Finally, we applied disruptions to the robot motor control and evaluated the robustness in the gait generation by performing sensitivity analysis on the emergent swimming gaits and speed.

2. Material and methods

2.1. µBot design and experimental set-up

µBots were developed as modular, robophysical models for BCF swimming [19–21]. Compared with existing swimming robots (e.g. [22–27]), a distinctive aspect of µBots is the modular design which enabled rapid prototyping with different design configurations (such as body length, shape and stiffness). Another feature is that the swimming gaits are not prescribed; instead, they emerge from FSI via motor control. In this work, we assembled and tested µBots with various body actuators or degrees-of-freedom (µBot-d.f., d.f. = 2, 4, 6) as the number of body segments is one of the most important parameters that determine the body kinematics [28]. A fully assembled µBot was 2.5 cm in width and 3.5 cm in depth, while its length and weight depended on the number of body segments or d.f. (µBot-2: 117 mm, 39 g; µBot-4: 170 mm, 59 g; µBot-6: 224 mm, 78 g). A µBot-4 with the caudal-fin removed was also experimentally studied. Details are found in electronic supplementary material.

The experimental platform also included a water tank (58 cm W × 56 cm H × 305 cm L), a monochrome camera (acA2000-165umNIR, Basler AG Inc, Ahrensburg, Germany) with a 760 nm filter, IR light sources and a motor controller (figure 1aiii). The laptop was used to send motor signals to the robot through the motor controller and capture robot motion through the camera. The serial communication for robot control and motion capturing was achieved with Matlab 2020a (MathWorks, Natick, MA, USA), operating at a 50 Hz frequency. Details are provided in electronic supplementary material.

2.2. µBot motor control and experimental learning

The robot motor control signals were generated by a CPG [29]. The CPG network for µBot-2 is illustrated in figure 1c. Each actuator was controlled by a CPG module, while more modules were added for µBot-4 and µBot-6. The equations of each neuron were represented as

$$\begin{array}{rl}\tau {\dot{U}}_{i,j}+U_{i,j}=& E_i-{\beta }_i{V}_{i,j}-\alpha y_{i,3-j}+\omega y_{i-1,j}\\ \tau {\dot{V}}_{i,j}+V_{i,j}=& y_{i,j}\\ y_{i,j}=& max(0,U_{i,j})\\ y_{i,\mathrm{out}}=& y_{i,1}-y_{i,2}\\ i=1,2,\dots ,n;j=& 1,2\end{array}$$ 2.1

where U and V are the states; n is CPG module number; τ is the time constant; E_i represents external stimulus for ith module; β_i is adaption coefficient of ith module; α is mutual inhibition weight; ω is the inter-module connection weight of the neuron; y_i,out is the output of the ith CPG module [19].

The rhythmic motor signals generated by the CPG can be characterized by their frequency, amplitudes and intersegmental phases. In the CPG, τ determines the frequency of the output, β_i sets the intersegmental phases, and E_i controls the amplitudes. To simplify the learning process, all the other parameters were fixed (electronic supplementary material, table S1). The parameter vector to be learned was ν = [τ, β₂, … , β_n, E₁, … ,E_n].

In our previous work, we used parameter-exploring policy gradient (PGPE) method for motor learning [19]. Although PGPE showed good convergence, it required empirical tuning of the heuristic learning rate. To reduce the number of empirically tuned hyper-parameters, here we used EM-based policy hyper-parameter exploration (EPHE) algorithm [30]. In EPHE, the policy parameter ν is sampled from normal distribution N (ν | h), where h = [µ σ]^T is the hyper-parameter vector. µ and σ are the mean and standard deviation of normal distribution of ν. At each episode, M policy parameters vectors v^m were sampled from N (ν | h). Then K best parameters were selected from the sorted reward R(v^m) and h was updated as

$$\mathit{\mu }={\displaystyle \frac{{\sum }_{k=1}^K[R({\mathit{v}}^{\mathit{k}}){\mathit{v}}^{\mathit{k}}]}{{\sum }_{k=1}^{K}R({\mathit{v}}^{\mathit{k}})}}$$ 2.2

and

$$\mathit{\sigma }=\sqrt{{\displaystyle \frac{{\sum }_{k=1}^K[R({\mathit{v}}^{\mathit{k}}){({\mathit{v}}^{\mathit{k}}-\mathit{\mu })}^2]}{{\sum }_{k=1}^{K}R({\mathit{v}}^{\mathit{k}})}}},$$ 2.3

where v^k is k^th sampled policy parameters and R(v^k) denotes the reward for v^k. In the experiments, the reward was set as the average swimming speed over a 1 s interval during steady swimming. Extending the time interval to 3 s showed only negligible fluctuations (less than 2%). A learning experiment was stopped when the following convergence criterion was reached: changes of the average swimming speed were within 2 mm s⁻¹ for three successive episodes. This criterion was applied primarily because of limit in the speed measurement error (i.e. approximately 1.5 mm s⁻¹) so that we chose a value slightly higher than that. Note that other performance measures, such as CoT, propulsive efficiency and power, can also be potentially used as the reward; however, they require considerable change in both the experimental set-up and learning methods (also see electronic supplementary material for relevant discussion). These will be studied in our future work.

Number of sampled roll-outs in one episode (M) and number of selected best samples (K) were hyper-parameters that needed to be empirically tuned for different µBots (electronic supplementary material, table S2). With the above setting, the experimental learning usually converged within 20 episodes.

3. Results

3.1. Experimental motor learning

Here we performed motor learning experiments to obtain the optimal motor programs for fastest swimming. For each robot, experiments were conducted for maximizing forward and backward swimming speed (electronic supplementary material, movie S1). Each learning experiment was repeated three times with varying initial values of CPG parameters. The repeated experiments always converged to nearly identical speed (electronic supplementary material, figure S1) and swimming gaits (electronic supplementary material, figure S2) (the only exception was µBot-4 without the caudal-fin, electronic supplementary material, figure S1d for speed and S2d for gaits). For forward swimming, the optimized swimming speed increased as the d.f. increased, while the swimming speed in body length per second (BL s⁻¹) decreased from 1.3 BL s⁻¹ (µBot-2) to 1.0 BL s⁻¹ (µBot-4), and to 0.8 BL s⁻¹ (µBot-6) (electronic supplementary material, figure S3). Removing the caudal-fin decreased forward speed from 1 BL s⁻¹ (173 mm s⁻¹) to 0.37 BL s⁻¹ (52 mm s⁻¹). For backward swimming, µBot-2 was unable to swim backwards, while µBot-4 and µBot-6 reached maximal backward speed at 0.16 BL s⁻¹ (27 mm s⁻¹) and 0.25 BL s⁻¹ (55 mm s⁻¹), respectively (electronic supplementary material, figure S3 and movie S2). Both forward swimming without caudal-fin and backward swimming underscored the importance of caudal-fin (or the asymmetric design between the head and caudal-fin) in improving the swimming speed.

For each learning task, the parameter determining the motor control frequency converged faster than the parameters determining intersegmental phases and amplitudes (electronic supplementary material, figures S4 and S5). In terms of the learned motor inputs, the optimized frequencies in the three experiments were almost identical, while the intersegmental phases and amplitudes had large variations among the experiments (electronic supplementary material, figure S6). These results indicated that FSI-embodiment promoted fast convergence of motor control frequency, which may lead to simplicity in motor learning.

3.2. Frequency response—swimming gaits and speed

We performed frequency response experiments in forward swimming to examine the frequency modulation of swimming gaits (electronic supplementary material, movie S3). This was done by changing frequency (adjusting τ) of the optimized CPGs (1–11 Hz). The caudal-fin tip velocity showed strong correlation to motor frequency (figure 2a) and peaked nearly at optimized frequencies for highest swimming speed (µBot-2: 7.3 Hz, µBot-4: 8.2 Hz, µBot-6: 8.8 Hz, figure 2b). Caudal-fin tip velocity and swimming speed also exhibited similar trends with an approximately linear relationship in the high-frequency (or speed) region (5.5–11 Hz, grey shadow, figure 2c), and a nonlinear relationship in the low-frequency region (1–5.5 Hz, green shadow).

Figure 2.

(a) Dependency of caudal-fin tip velocity on motor frequency. (b) Dependency of swimming speed on motor frequency. (c) Relationship between swimming speed and caudal-fin tip velocity. At the high-frequency region (grey shadow, 5.5–11 Hz), there was a linear relationship between the swimming speed and the caudal-fin tip velocity, as illustrated by the linear regression (green line) and the 95% prediction intervals (pink lines). However, at the low-frequency region (green shadow, 1–5.5 Hz), the linear relationship did not hold. The two regions also corresponded to the two modes of swimming gaits presented in (d) and (f). (d) Midline kinematics of high frequency standing-wave ‘resonant’ gaits (SW ‘Res’ Gaits) at optimized frequency (7.3, 8.2, 8.8 Hz for µBot-2, µBot-4, and µBot-6, respectively). (e) Illustration of SW ‘Res’ Gaits. The robot joints are marked as filled colour dots, and the gait nodal points and oscillatory points are illustrated as dotted lines and dashed lines, respectively. The oscillatory points consistently coincided with the robot joints, which was not the case for nodal points. (f) Midline kinematics of travelling-wave ‘fish-like’ gaits (TW ‘FL’ Gaits) at low frequency (3.1, 2.1, 2.7 Hz for µBot-2, µBot-4 and µBot-6, respectively). The lateral displacements of all the midline kinematics are scaled up two times for clearer illustrations. Kinematic snapshots in (d) and (f) show the displacement of the body midline at 10 equally spaced time intervals during a single tail beat cycle.

It was further found that the above two regions were associated with two modes of swimming gaits (electronic supplementary material, movie S4). At high frequency, the swimming gaits of all three µBots resembled standing waves (figure 2d), i.e. lateral displacements along the body long axis alternated between large amplitude (oscillatory points) and small amplitude (nodal points). Note that the nodal points did not necessarily coincide with the body mechanical joints (figure 2e). The head and caudal-fin tips were end oscillatory points that consistently exhibited large displacements. Intuitively, a nodal point can be considered as a virtual fulcrum point, and together with its two adjacent oscillatory points, composes a virtual lever. Therefore, the entire robot behaved as a chain of oscillating levers; each lever rotated around a virtual fulcrum, and together they drove and overcame the hydrodynamic load on the distal caudal-fin.

Since these gaits emerged within the high-frequency region including the resonant frequencies which are known to produce standing waves [31], we named these gaits standing wave ‘resonant’ gaits (SW ‘Res’ Gaits). With SW ‘Res’ Gaits, swimming speed was proportional to caudal-fin tip velocity (figure 2c), and such linear predictability may underscore dominance of the caudal-fin and negligible contribution from the µBot body in thrust generation (see electronic supplementary material for details).

At low frequency, µBots exhibited backward travelling waves along their bodies (figure 2f), which resembled those commonly observed in fishes [1]. Therefore, we named these gaits travelling wave ‘fish-like’ gaits (TW ‘FL’ Gaits). With TW ‘FL’ Gaits, the swimming speed was no longer proportional to the caudal-fin tip velocity (figure 2c). However, as will be shown below, the linear predictability which existed for SW ‘Res’ Gaits can be re-established by including the body undulatory velocity, suggesting the non-eligible contribution to thrust generation from the µBot body that exhibited backward travelling waves.

Taken together, these results indicated that optimal swimming can be achieved by frequency modulation, and FSI-embodiment promoted the emergence of two distinct swimming gaits.

3.3. Invariant body and/or caudal-fin Strouhal number

Here we explored whether there are invariant hydrodynamic properties that are independent of morphological design, motor control and swimming gaits in μBots' swimming. We found that the averaged lateral undulatory velocity of all body segments and caudal-fin combined (BCF lateral undulatory velocity, electronic supplementary material), instead of caudal-fin tip velocity alone, proportionally predicted the swimming speed for all µBots and all tested frequencies (figure 3a). This led to an invariant BCF Strouhal number (St_BCF = 0.182, linear regression with coefficient of determination R² = 0.941), defined based on the undulation of body and caudal-fin combined (electronic supplementary material). Note that R² = 0.875 if the caudal-fin tip velocity was used for the linear regression.

Figure 3.

(a) Relationship between swimming speed and average BCF lateral undulatory velocity for µBots with linear regression (solid line). (b) Relationship between swimming speed and average BCF lateral undulatory velocity for µBots, other robotic fishes and biological fishes surveyed from the literature. The solid lines were acquired from the regression clustering method, the slope of which corresponds to twice the values of St_BCF. The sketches illustrate the body shapes of some fishes and robots as examples (not to scale). (c) Histogram of the distribution of St_BCF with each column representing the density of the data falling within the range. The two vertical lines mark the values of St_BCF from the regression clustering.

We then calculated St_BCF for a collection of biological and bioinspired robotic swimmers in the literature (figure 3b and electronic supplementary material, table S3). Distribution of the St_BCF is presented in figure 3c, where two clusters were clearly seen. Using regression clustering method [32] (electronic supplementary material), we identified two regression lines in figure 3b, corresponding to two classes of undulatory swimming (with St_BCF = 0.186 and St_BCF = 0.066) among all biological and robotic swimmers examined here (with clustering loss L_c = 0.08, electronic supplementary material). The line with St_BCF = 0.186 closely matched the linear regression of µBots’ data which gave St_BCF = 0.182. Then we measured the silhouette score for the clustering (electronic supplementary material). The average score (AS) of all data points was 0.669. Note that the clustering became weakened if Strouhal number with merely caudal-fin tip velocity was used (electronic supplementary material, figure S7, which gives L_c = 0.249 and AS = 0.612). We referred to the class of St_BCF = 0.186 as slow speed-to-undulation (swimming speed to BCF lateral undulation) swimmers and the class of St_BCF = 0.066 as fast speed-to-undulation swimmers. In general, the examined biological thunniform and carangiform swimmers [10,33–36] belonged to the fast speed-to-undulation swimmers (dark blue, figure 3b), and the examined anguilliform biological swimmers [37–43] belonged to the slow speed-to-undulation swimmers (light blue) except needlefish [44], a fast anguilliform swimmer with lunate caudal-fin (similar to those of typical carangiform or thunniform swimmers), capable of swimming at 2 BL s⁻¹. Surprisingly, all the examined bioinspired robotic swimmers, including µBots, belonged to the class of slow speed-to-undulation swimmers (grey), regardless of their body d.f. and morphological designs (e.g. anguilliform [45], carangiform [46–49] or thunniform [27,50]). Since Strouhal number is a dimensionless number that characterizes the hydrodynamics of undulatory swimmers [12], our results showed that the FSI in µBots' swimming possessed invariant hydrodynamic properties that could be independent of morphological design, motor control and swimming gaits. These properties might also be applied to other fish or robots that share the same St_BCF.

3.4. Robustness of gaits to motor control disruptions

Next, we tested whether the emergent gaits were robust against internal disruptions applied to motor control. This was done by applying disruptions (figure 4a) to the motor control (figure 4b), measuring the corresponding changes in the swimming speed and gaits (figure 4c), and quantifying their sensitivities (figure 4d) to the disruptions (see electronic supplementary material for details). Examples are illustrated in electronic supplementary material, movie S5.

Figure 4.

(a) Characteristics of the motor signals for µBot-6. µBot-2 and µBot-4 use only the first two and four signals, respectively. in1 to in6 represent the amplitudes of the motor signals. p1 to p5 represent the intersegmental phases between motor signals applied to neighbouring actuators. f+ and f− represent the increase and decrease of the frequency, respectively. (b) Nominal motor signal inputs for µBot-2, µBot-4, µBot-6. (c) Gait parameters to be measured. ip1 to ip5 represent the intersegmental phases between the neighbouring joint angles. m1 to m6 represent the amplitude of joint angles. spd and cv are abbreviations for swimming speed and caudal-fin tip velocity, respectively. (d) Examples for presenting the result of sensitivity analysis and legends for the following plots. Sensitivity was quantified as the absolute value of the slope derived from a linear regression analysis, which correlates the per cent change in measured gait parameters with the per cent disruptions. Details and examples are provided in electronic supplementary material and electronic supplementary material, figure S9. The sensitivity magnitudes are indicated by the colour bar. Note that the magnitudes above 1.5 are all considered maximum sensitivity. (e and f) Sensitivity of swimming speed (e) and gaits (f) to various disruptions applied to the motor control of SW ‘Res’ Gaits for µBot-2, µBot-4, µBot-6, respectively. (g and h) Sensitivity of swimming speed (g) and gaits (h) to various disruptions applied to the motor control for TW ‘FL’ Gaits for µBot-2, µBot-4, µBot-6, respectively. Error bars indicate the standard deviation in (e) and (g).

The results for SW ‘Res’ Gaits are shown in figure 4e,f. For all µBots, the sensitivities of swimming speed to motor frequency were the highest (brown lines were steeper, figure 4e), while those to other parameters were generally low (the lines were flatter). The only exception was for the amplitude of the second actuator of µBot-2, which had mild effects on swimming speed (in2, green line with mild slope, figure 4ei). The sensitivities of all gait parameters to motor frequency were also the highest (purple boxes, figure 4f; more darker red squares), while those (except the head) to other motor control parameters were generally low (cyan boxes; less darker red squares). Note that the head motion (including m1 and ip1 in figure 4c) was sensitive to both frequency and other motor control parameters (figure 4f), although it had no significant effects on the swimming speed.

The results for TW ‘FL’ Gaits are shown in figure 4g,h. Compared with the results for SW ‘Res’ Gaits, the swimming speed now became more sensitive to amplitudes and intersegmental phases, although the frequency still played a dominating role. The swimming gaits remained sensitive to frequency in most cases, but with several exceptions (e.g. gait sensitivity to increasing frequency was low for µBot-6, f+, figure 4hiii). The gaits also became more sensitive to the motor intersegmental phases and amplitudes in general (e.g. more darker red squares in the cyan boxes were observed in figure 4h).

Taken together, these results showed that the gait generation in μBots via motor-controlled FSI was robust against non-frequency disruptions in motor control, while the robustness was probably enhanced by the resonance effects in the SW ‘Res’ Gaits.

3.5. Frequency response—muscle power characteristics

We further examined the robot power characteristics in the frequency response by calculating actuators' work loops and CoT. The work loop is commonly used to evaluate the muscle physiology in terms of its mechanical work and power, and can be obtained by plotting the time course of the muscle force and length [51]. Here, we applied this method to µBots' actuators (see electronic supplementary material). The work loops of µBots' actuators showed strong frequency dependency (figure 5a), as their shapes evolved substantially over the frequencies (figure 5b for examples), which is in agreement with biological measurements [52] and modelling [53]. At low frequencies (highlighted in green), all the actuators behaved as motors (large enclosed area for power output) combined with virtual masses (rightward inclined work loop for overcoming extrinsic elastic effects). At high frequencies (highlighted in yellow), the posterior actuators of µBot-4 (last two) and µBot-6 (last four) behaved as virtual springs (leftward inclined work loop) with low power output (small enclosed area); together with the rubber suit elasticity, they effectively stiffened the posterior body of the robots. For all three µBots, while the head actuators did not have a consistent function, the actuators behind the head exhibited the highest power output at high frequencies. These functions are also observed in biological muscles, where different types of muscle fibres or sarcomeres are activated at different frequencies [54].

Figure 5.

(a) Changes in work loop shapes according to motor control frequency. The work loops of SW ‘Res’ Gaits and TW ‘FL’ Gaits are highlighted in yellow and green, respectively. (b) Illustration of work loops corresponding to mechanical springs, virtual springs and virtual masses. (c), (d) and (e) Dependencies of total actuator power output, CoT_net, and CoT_w-net on motor frequency, respectively. The frequencies for optimized speed are marked with stars.

The total power output of a µBot exhibited an inverted U-shaped dependency on the frequency (figure 5c). The optimized swimming speed occurred at frequencies higher than those for the maximum power output. We calculated two types of CoT: (i) CoT_net, defined as the ratio of power output of a µBot's actuators to the swimming speed, and (ii) CoT_w-net, defined as CoT_net divided by the weight of the µBot (which was analogous to the net metabolic CoT of fishes). The CoT_net and CoT_w-net curves also demonstrated inverted U-shapes (figure 5d,e). CoT_net at the maximum speed for all µBots were similar (around 0.09 J m⁻¹). However, the CoT_w-net at the maximum speed decreased substantially as the d.f. increased, with µBot-6 demonstrating the lowest CoT_w-net (1.3 J m⁻¹ kg⁻¹). The CoT_w-net of µBots is comparable to those reported for fishes with similar size (2.5 J m⁻¹ kg⁻¹ for bluegill sunfish [55]; 1.9 J m⁻¹ kg⁻¹ for mackerel and kawakawa [56]). For all the robots, there existed a trade-off between the optimized swimming speed and efficiency, which is in agreement with a recent anguilliform robot work [22].

The above results suggested that the capability to transport mass was enhanced by adding body segments and d.f. Although the current study is only limited to µBot with six body segments, it will be interesting to see if the CoT_w-net at the maximum speed can be further reduced by adding more body segments, analogous to the improved fuel efficiency in a long rail freight train with large number of freight cars.

It is also worth noting that the CoT for backward swimming was higher than that for forward swimming. For μBot-6, when the swimming speeds were the same (55 mm s⁻¹), the CoT_w-net for forward swimming was 1.61 J m⁻¹ kg⁻¹, and the CoT_w-net for backward swimming was 2.53 J m⁻¹ kg⁻¹. For μBot-4, as we did not have the data for two modes with the same speed, we compared them at the same undulating frequency (4.65 Hz). The CoT_w-net was 1.48 J m⁻¹ kg⁻¹ for forward swimming and 3.54 J m⁻¹ kg⁻¹ for backward swimming.

4. Discussion

Our robophysical models (μBots) provided an opportunity to quantitatively study fish-inspired swimming problems. The results confirmed a number of embodied traits. First, the FSI-embodiment promotes fast convergence of motor control frequency, and optimal swimming can be achieved by frequency modulation. Second, the swimming gaits are robust against internal disruptions that are applied to the intersegmental phases and amplitudes of motor signals. Therefore, motor-controlled FSI adds another layer of robustness to gait generation in addition to those in the rhythmogenesis provided by combination of feedforward control and peripheral sensory feedback [6]. The study in [6] also underscores the importance of rhythm and rhythmogenesis in gait generation, consistent with our results. However, the motor control disruptions of our work were not as radical as those in [6]. For example, our results showed that the intersegmental phases cannot be completely reversed (otherwise the robot swims backward). Third, swimming speeds of all µBots with varied d.f. and at all motor frequencies can be captured by a newly defined Strouhal number (St_BCF), indicating that µBots' swimming possesses invariant hydrodynamic properties that could be independent of morphological design, motor control and swimming gaits. Fourth, the homogeneous actuators of µBots and the associated work loops demonstrate different functions in terms of power output, virtual stiffness and mass. Although these traits were explored with three µBots, it would be interesting to see if they can be applicable to a broader context, which is a direction of our future work.

4.1. Central pattern generators prioritize frequency modulation over intersegmental phases and amplitudes

In the experiments, we noticed that there are optimal frequencies for maximizing the swimming speed (µBot-2: 7.3 Hz, µBot-4: 8.2 Hz, µBot-6: 8.8 Hz). These frequencies falls into the range of biological swimmers with comparable sizes [54]. It is also observed that adjusting motor control frequency, rather than intersegmental phases or amplitudes, is the primary method for altering swimming speed. This mirrors potential similarities with biological fish, where frequency modulation is a key mechanism in regulating swimming speed. Notably, studies on dogfish CPGs have shown a direct correlation between swimming speed and frequency changes, while the intersegmental phases remain consistent [57]. Additionally, research suggests that muscle forces in fish, analogous to the torques produced by μBot actuators and thus comparable to motor control amplitudes, do not vary significantly across different speeds and frequencies of undulation [52,58,59]. These parallels indicate that in both robotic and biological systems, CPGs probably prioritize frequency modulation over intersegmental phases and amplitudes.

4.2. Embodiment yields simplicity for control and high efferent predictability

Collectively, the embodied traits suggest simplicity in both gaits and speed control, as well as high predictability of motor efference (i.e. predicting movement [57]). Interestingly, the simplicity and predictability reside in a motor-controlled FSI system that has highly complex dynamics, which is difficult to be fully understood and accurately modelled [60]. Although a complex system is usually associated with unpredictability [61], our results suggest that there potentially exist simple properties embodied within the complex FSI underlying fish-inspired swimming. More importantly, these properties are easily ‘discoverable’ by an external agent (e.g. via reinforcement learning), from which a simple predictive model from motor control to swimming performance can be established [62], although substantive work will be needed to extend the work to more robots design and animals data.

4.3. Fluid–structure interaction resonance in µBots probably enhances the embodied properties

The aforementioned robustness and predictability in fish-inspired swimming are also probably enhanced by resonant effects [63], which are clearly visible in the µBot caudal-fin—there are clear peaks of caudal-fin tip velocity at optimized frequencies (figure 2). In addition, the SW ‘Res’ Gaits are standing waves, which are commonly produced by resonant effects [31]. Notably, similar gaits of the standing-wave type have been reported previously in a simulation work modelling fish swimming [64] and experimental studies with soft robots [65,66]. As the resonance came from the interaction between the robot and fluids (FSI), we defined it as FSI resonance.

In sensitivity analysis (figure 4), it is evident that SW ‘Res’ Gaits show a higher degree of robustness and stronger frequency modulation than those of TW ‘FL’ Gaits. Since it is apparent that the former has stronger resonant effects than the latter, one can conclude that resonant effect enhances the robustness and frequency-modulation in the µBot gait generation.

The work loops of µBots (figure 5a) indicate that the resonance effects probably occur at their posterior body parts which show increasingly higher amount of virtual elastic storage with SW ‘Res’ Gaits. Specifically, with SW ‘Res’ Gaits, the actuators at the anterior parts behave as motors, while those at the posterior parts behave approximately as virtual torsional springs. Note that unlike muscle fibres or sarcomeres that can store energy [67], an actuator cannot physically store elastic energy, which instead will be dissipated as heat. For µBot-2, the posterior actuator does not show spring effect since the elastic caudal-fin probably functions as the main source of elastic energy storage.

In addition, the FSI resonance is probably determined primarily by effective body mass (including both robot physical mass and fluid added-mass) and effective body stiffness (including rubber suit physical stiffness and actuator virtual stiffness). For example, our previous simulation showed that the fluid added-mass torque and effective spring torque can balance each other in µBots' swimming [68]. In a separate study [69], it was shown, using analytical modelling and simulation, that resonance can still occur even when the entire physical body mass and stiffness were removed, and the resonant frequency was probably determined by actuator virtual stiffness and fluid added-mass. Nevertheless, the mechanisms that enable FSI resonance and corresponding hydrodynamic phenomena may demand more examinations, which are beyond the scope of this work.

4.4. Body and/or caudal-fin Strouhal number and two fundamental classes of undulatory swimming

In this study, we show that the forward swimming of μBots can be characterized by a constant St_BCF. The St_BCF also enables us to categorize many biological and robotic swimmers into only two classes of undulatory swimming (figure 3). These results suggest that body undulation may also contribute to propulsion, in addition to caudal-fin. Note that Lighthill concluded, based on mathematical modelling, that thrust is only related to caudal-fin movement [70,71]. In addition, we also experimentally demonstrated that body undulation alone can generate thrust, for example, in the backwards swimming and swimming without caudal-fin.

The two classes of undulatory swimming revealed by St_BCF are slow speed-to-undulation and fast speed-to-undulation (figure 3). One common morphological trait that separates the two classes in biological fishes is the shape of the caudal-fins; the fast speed-to-undulation fishes have lunate-shaped caudal-fins, while slow speed-to-undulation fishes have pointed or rounded caudal-fins. However, all robotic swimmers examined in this study, regardless of the caudal-fin shape, fall into the slow speed-to-undulation class. This discrepancy suggests that there exist other design factors that influence the BCF undulatory propulsion in biological fishes that the existing robotic designs are unable to replicate.

So, what are these factors? Although our work does not provide direct evidence, we can reject the following possibilities. First, within a species, varying swimming kinematics or speed does not affect the class that the species belongs to. Second, the number of body segments of µBots does not affect its class. Third, since fishes in both classes have dorsal/pectoral fins, their presence is also unlikely to affect the classes. Therefore, the most likely factor for robots to achieve fast speed-to-undulation class, as we speculate, could be tuning of spatial stiffness distribution along the body. It is known that swimming performance is highly correlated to body stiffness [72]. Compared with our previous work [19], the swimming performance of µBots can be improved substantially via body stiffness tuning. A carefully designed body and caudal-fin stiffness might also produce swimming gaits that can exploit the vorticity mechanisms [1]. Nevertheless, future studies are required to fully address this question.

Ethics

This work did not require ethical approval from a human subject or animal welfare committee.

Data accessibility

All data generated are available in the paper or its electronic supplementary material [73].

Declaration of AI use

We have not used AI-assisted technologies in creating this article.

Authors' contributions

H.D.: conceptualization, data curation, formal analysis, investigation, methodology, software, validation, visualization, writing—original draft, writing—review and editing; D.L.: formal analysis, methodology, software, validation, visualization, writing—review and editing; C.N.: formal analysis, methodology, validation, writing—review and editing; A.W.: formal analysis, methodology, validation, writing—review and editing; S.P.: funding acquisition, supervision, validation, writing—review and editing; B.C.: conceptualization, funding acquisition, methodology, project administration, resources, supervision, validation, writing—original draft, writing—review and editing.

All authors gave final approval for publication and agreed to be held accountable for the work performed therein.

Conflict of interest declaration

We declare we have no competing interests.

Funding

Funding for this research includes National Science Foundation (grant no. CNS-1932130 (B.C.)), Army Research Office (grant no. W911NF-20-1-0226 (B.C.)) and United States Department of Agriculture (grant no. NIFA-2019-67021-28991 (S.P.)).