Model-based feedback control of live zebrafish behavior via interaction with a robotic replica

Abstract

The possibility of regulating the behavior of live animals using biologically-inspired robots has attracted the interest of biologists and engineers for over twenty-five years. From early work on insects to recent endeavors on mammals, we have witnessed fascinating applications that have pushed forward our understanding of animal behavior along new directions. Despite significant progress, most of the research has focused on open-loop control systems, in which robots execute predetermined actions, independent of the animal behavior. We integrate mathematical modeling of social behavior toward the design of realistic feedback laws for robots to interact with a live animal. In particular, we leverage recent advancements in data-driven modeling of zebrafish behavior. Ultimately, we establish a novel robotic platform that allows real-time actuation of a biologically-inspired 3D-printed zebrafish replica to implement model-based control of animal behavior. We demonstrate our approach through a series of experiments, designed to elucidate the appraisal of the replica by live subjects with respect to conspecifics and to quantify the biological value of closed-loop control.

I. Introduction

Interaction of robots with living creatures is of increasing importance in a number of domains, from human-robot cooperation [1] to interactive agents for animal enrichment [2]. Particularly promising is the application to the field of animal behavior, where robotics offers unprecedented tools to conduct hypothesis-driven experiments that could help unveil the influential cues triggering animal interactions – the celebrated “social releasers” in the work of the Nobel Laureate Nikolaas Tinbergen [3]. Through robots, we now have access to a new array of technological probes that can be utilized to induce standardized, repeatable, and consistent response in live animals in laboratory assays or field studies [4]-[8].

For example, should we desire to confirm the intuition that body size is a determinant of leadership in a given species [9], we could embark on the ambitious task of building a biologically-inspired robot that mimics its live counterpart in all its aspects except for the size. Then, we would fabricate several of these robots, systematically changing their size, and test how live animals will respond to them to ultimately determine the effect of body size. One may ask, why not simply choose animals of different size as stimuli rather than working with robots? The answer is that size may correlate with other factors, like fitness and age, thereby challenging the ability to tease out the individual role of body size on leadership. Replacing live with robotic stimuli, we guarantee that only size is manipulated, while keeping other factors constant across trials.

To the best of our knowledge, the first application of robotics in the study of animal behavior is the seminal work of Michelsen [10], who evaluated the effect of sound on honeybees’ dance. Building upon this endeavor, the last twenty five years have seen a surge of ambitious studies which capitalize on the potential of robotics to improve our understanding of the emotions, perception, and cognition of animals [11]-[14]. From insects to mammals, researchers have examined the interactions of animals with biologically-inspired robots coming from manifold perspectives and scientific premises. For example, Patricelli et al. [15] investigated the courtship of male satin bowerbirds using custom-made robotic females, Takanishi et al. [16] studied the behavioral response of rats to custom-made rat-like robots, and Kubinyi et al. [17] addressed the interactions between dogs and commercially available doglike robots.

A particularly fertile area of investigation is the study of interactions between fish and biologically-inspired robots. Several reasons have contributed to the increased interest towards this biological niche, resulting in a large number of technical publications, see the overview in Table I. First, several fish species constitute powerful animal models that carry translational value to more complex and sentient organisms. For example, zebrafish are quickly emerging as the species of choice for the study of functional and dysfunctional processes in humans due to their surprising neurobiological similarities with them [53], [54]. Second, fish and fisheries have a significant impact on local and global economies, thereby offering a compelling rationale for the need to devise strategies that can improve animal production [55], [56]. Third, several fish species are currently under significant threat, calling for advanced solutions in conservation and control [57].

TABLE I:

List of recent studies on fish-robot interaction.

Study	Species	Interaction
[18]-[32]	Zebrafish	Open-loop
[33]	Guppies	Open-loop
[34]	Poecilia mexicana	Open-loop
[35]	Three-spined stickleback	Open-loop
[36]	Bluefin killfish	Open-loop
[37], [38]	Golden shiners	Open-loop
[39]	Mosquitofish	Open-loop
[40]	Giant danios	Open-loop
[41]	Siamese fighting fish	Open-loop
[42]–[44]	Weakly electric fishes	Open-loop
[45], [46]	Guppies	Closed-loop
[47]–[50]	Zebrafish	Closed-loop
[51]	Three-spined stickleback/guppies	Closed-loop
[52]	Golden shiners	Closed-loop

As summarized in Table I, the literature on fish-robot interactions can be partitioned into two classes, depending on the degree of interactivity of the robot. Most of the research has focused on open-loop instances, in which the motion of the robot is preprogrammed, irrespectively of the live subject. Despite significant breakthroughs in the mechanical design and visual appearance of these robots, the lack of a bidirectional communication pathway with live subjects limits the range of their application. A smaller number of papers of the literature has tackled the more challenging task of closing the loop between the animal and the robot, such that the robot could adjust its behavior as a function of the response of the animal. Implementing a closed-loop control system mandates the solution of several technical hurdles from real-time scoring of animal behavior to implementing biologically-salient feedback laws onboard the robot. While the acquisition of animal behavior in real-time has seen considerable progress, our ability to implement closed-loop strategies that are reminiscent of real interactions between animals remains limited.

Few research groups have worked in this area. Swain et al. [52] developed a platform in which a wheeled robot maneuvered a magnetically-connected replica of a koi, a predator on golden shiners. Computer vision techniques were used to acquire the position of a group of golden shiners and create a feedback for the predator to attack the group. Using a similar setup, Krause and colleagues explored social interactions in guppy and three-spined stickleback [46], [51]. The motion of the robotic replica was programmed to approach the center of the group or implement some basic social rules found in fish [58]. Previous work by one of the authors examined closed-loop control in a binary choice test, where either the tail beat frequency of a stationary robotic fish was adjusted according to the position of a focal zebrafish [49], or a replica was maneuvered via a three-dimensional manipulator that dynamically actuated the replica towards the focal fish [47]. Halloy and colleagues sought to enhance the degree of biomimicry of a zebrafish replica via a probability-based feedback mechanism [48]. Specifically, the instantaneous orientation of the replica was drawn from a probability function that accounted for the presence of tank walls and four live fish [59], [60]. To dynamically adapt the behavior of the replica, this approach requires tracking the targets in real-time and periodically scoring the distribution of the group in the tank. The simultaneous execution of these tasks makes this approach computationally intensive, such that its implementation required three 32-core computers.

Here, we propose a different approach to close the loop between fish and robots, and implement for the first time a model-based feedback control strategy to drive the motion of the replica in real-time. To achieve this goal, we leverage recent advancements in data-driven modeling of zebrafish swimming which allow for accurately capturing the complex behavior of this species [61]-[63]. The model captures burst- and-coast swimming style of this species, together with behavioral rules of alignment, attraction, and repulsion that shape its social response. Toward demonstrating our approach, we built a novel robotic platform to maneuver a biologically-inspired zebrafish replica. The platform is based on a custom Cartesian manipulator that enables accurate turn rate and speed regulation, combined with a fast reaction time. In a series of experiments, we evaluated how zebrafish appraise the replica both in open- and closed-loop dynamics.

In open-loop, the replica moved along model-based trajectories, independent of the behavior of the live subject. In closed-loop, the replica utilized real-time data on the behavior of the live subject to adjust its turn rate and swimming speed based on the underlying model. Experimental observations on fish-robot interactions were compared with data collected on pairs of zebrafish. To offer a comprehensive assessment of zebrafish response to the replica, we computed measures of locomotor activity and social behavior. With respect to locomotion, we studied the distributions of the swimming speed and turn rate. With respect to social response, we combined classic observables of schooling and shoaling tendencies with information-theoretic measures of influence.

The main contributions of this work are: i) to present a novel robotic platform to study zebrafish behavior, and ii) to implement and validate a closed-loop strategy for controlling a biologically-inspired replica via a stochastic model of zebrafish behavior. The use of model-based feedback to control the replica contributes a number of technical advantages. First, grounding the implementation of the robotic platform in a dynamic model supports and fosters hypothesis-driven studies on social behavior, by allowing to independently manipulate locomotor activity and behavioral rules. Second, the generality of the model allows for examining a wide range of social interactions, involving regulation of both turn rate and speed – for example, speed modulation has been proposed as a key mechanism for explaining collective behaviour in teleosts [64], [65], but experimental data are lacking. Third, the availability of a mathematical model mitigates the need for on-line calibration that would be required when pursuing more data-hungry probabilistic-based approaches.

The rest of the paper is organized as follows. A detailed description of the robotic platform is provided in Section II. In Section III, the model used for controlling the motion of the replica is presented. The experimental scheme is summarized in Section IV, while our approach to data analysis is articulated in Section V. Experimental results are presented and discussed in Section VI. Finally, conclusions are drawn in Section VII.

II. Design and implementation of the robotic platform

We start by describing the design of our interactive robotics-based platform that allows for the implementation of model-based trajectories on zebrafish replicas. The platform enables: i) high-speed two-dimensional motion and body oscillations of a flexible replica, and ii) simultaneous target tracking of the replica and zebrafish. Differently from previous approaches, the platform makes explicit use of an experimentally-calibrated stochastic model of zebrafish behavior to control the replica motion in real-time. In what follows, we describe the key elements of this platform.

A. Experimental apparatus

The apparatus consisted of a glass water tank measuring 74 × 30 × 30 cm (length, width, and height), see Figure 1. The tank was placed on a holding structure made of T-slotted bars at 30 cm from the floor to allow sufficient space for the Cartesian manipulator. The level of the water was kept at 10 cm to minimize diving motion by experimental subjects. Two transparent acrylic panels covered by white contact paper were used to isolate a focal region of 44 × 30 × 30 cm, selected in accordance with the working area of the manipulator. Two 25 W white fluorescent lights were mounted at 110 cm from the floor to obtain a homogeneous illumination of the environment. Black curtains were placed on the four sides of the structure to further reduce external stimuli.

Fig. 1:

Schematic of the experimental apparatus, showing the replica (R) in the tank, along with the Cartesian manipulator, overhead cameras, lights, and surrounding curtains.

Two overhead cameras were installed at 140 cm above the tank floor for real-time closed-loop control and off-line behavioral scoring. A Logitech C920 Pro (Logitech, Lausanne, Switzerland) with a resolution of 640 × 480 pixels and frame rate of 20 fps was used for real-time tracking. A Flea3 FL3-U3-13E4C USB camera (Point Grey Research Inc., Richmond, British Columbia, Canada), with a higher resolution of 1280 × 1024 pixels and acquisition rate of 30 fps, was used to gather finer data for behavioral scoring.

B. Replica

The replica was fabricated in silicone using DragonSkin 10 Medium (Smooth-On Inc., Macungie, PA, USA) poured into a 3D-printed real size fish mold¹, see Figure 2. Unlike previous studies where plastic or plaster replicas were deployed [18], [35], we used silicone to allow passive body undulations of the replica. To mimic the morphology of a zebrafish, the replica included caudal, dorsal, and ventral fins. The replica was hand-painted with silicone paint (Psycho Paint 8-oz unit, Smooth-On, PA, USA-Silc Pig 9 pack color sampler tube, Smooth-On, PA, USA-Novocs gloss pint, Smoot-On, PA, USA- Pearl Ex Pigments 662 Antique Silver, Jacquard Products, CA, USA) to mimic the coloration of a zebrafish. Glass eyes (Van Dyke Supply Co., Granite Quarry, NC, USA) were glued, following [32].

Fig. 2:

Computer-aided design of the mold used to fabricate the replica (left panel), which was then hand-painted to resemble a live zebrafish (right panel).

A transparent plastic stick (0.3 cm diameter, McMaster, Robbinsville, NJ, USA) was used to rigidly connect the replica with a 3D-printed base, see Figure 3, painted in white acrylic paint (Basics, Liquitex, NJ, USA). The base contained two neodymium magnets (McMaster, Robbinsville, NJ, USA) with opposite polarization. A white Teflon patch (McMaster, Robbinsville, NJ, USA) was glued on the contact surface between the base and the tank bottom to facilitate tracking and reduce friction between the base and bottom of the tank.

Fig. 3:

The zebrafish replica was connected via transparent stick (left panel) to a base (right panel: top and bottom views of the computer-aided design).

C. Robotic manipulator

The robotic platform maneuvered the replica using a two-dimensional Cartesian manipulator (XY Plotter Robot Kit, Makeblock Co., Ltd, Shenzhen, China), which allowed accurate positioning and fast reaction time. The manipulator was placed below the tank to minimize visual and acoustic stimuli, which could act as experimental confounds. Two stepper motors (included in the XY Plotter Robot Kit) were used for in-plane actuation, and a third stepper motor (NEMA 14, Pololu Corp., Las Vegas, NV, USA) was mounted on the end-effector to regulate the orientation of the replica.

The manipulator was connected to a custom 3D-printed terminal (at 0.5 cm from the tank bottom) which encased two neodymium magnets (diameter 1.7 cm) with opposite polarization, mirroring the arrangement of the base. The working area for the manipulator was 39 × 31 cm (length and width), but for the experiments it was reduced to 35 × 23 cm to avoid collisions between the replica and the tank walls. The platform was controlled via a computer interface that could maneuver the replica in real-time with a maximum speed of 35.36 cm s⁻¹ and a maximum turn rate of 25 rad s⁻¹, which is higher than the typical peak motion data of a live zebrafish [66]. In addition, swift of actuation was guaranteed with a maximum linear acceleration of 282 cm s⁻², which is a significant improvement compared to [27]. Beyond the specific experiment presented herein, the manipulator might be used for mimicking the behavior of a zebrafish across a wider range of experimental conditions, including assays on startle response where large accelerations are achieved [67].

D. Real-time tracking system

A vision-based algorithm for real-time tracking was developed using the vision library of Matlab R2018a (Mathworks, Natick, Massachusetts, USA) to automatically track the position of two targets, simultaneously. The software implemented a blob detection algorithm, which consisted of the following steps. First, candidate targets are identified by comparing adjacent frames. Specifically, the previous frame is subtracted from the current one to isolate the potential location of the targets from their motion. This step is implemented using a binary filter with an empirical threshold to reduce noise, while limiting the search to a small region close to the previous positions to lower the computational load. To refine the location of the candidate targets, a binary filter is applied to the current frame and the centroids of the blobs are determined.

To help distinguish live fish from the replica, the nominal position of the replica commanded to the manipulator was taken into consideration during real-time tracking. Specifically, we implemented a Kalman filter to predict the current position of the two targets. The cost matrix of the algorithm that accounts for: i) the distance between the predictions and blob centers, and ii) the difference between the predictions and the commanded position of the replica. On the cost matrices, we apply the Hungarian assignment algorithm [68] to differentiate between the replica and the fish. A screenshot of the live tracking system is shown in Figure 4.

Fig. 4:

A screenshot of the live tracking system. The position of the live subject and of the replica are identified by green crosses. The red and blue curves identify trajectory segments for the fish and replica, respectively. The yellow cross corresponds to the input trajectory from the model of the replica. The circles in white and the magenta refer to the interaction radii r_ω and r_v, respectively – marked in the software legend on the top left corner.

E. Control implementation

An Arduino Uno microcontroller (Arduino Srl, Italy), equipped with three A4988 drivers (Kuman Trading Shenzhen Co Ltd, Shenzhen, China) and a shield (CNC V3 Shield, Kuman Trading Shenzhen Co Ltd, Shenzhen, China) were used to drive the stepper motors of the platform. The microcontroller was commanded via serial communication by a PC running a discretized version of the stochastic model of the replica implemented in Matlab 2018a.

III. Mathematical modeling of zebrafish behavior

Here, we summarize the stochastic model we use to generate the reference signals of position and heading that are fed to the experimental platform for controlling the replica. The model builds on our previous work [61]-[63] towards capturing mathematically biomimetic motion that encapsulates the burst- and-coast swimming style of zebrafish and behavioral rules underlying their social response. Specifically, the model combines the speed modulation and social interactions introduced in [63] with a refined description of turn rate dynamics [61] and wall interaction [62].

With reference to Figure 5, the equations describing the evolution of the heading, ϕ(t), and position vector, [x, y], in time, t, are

$$\mathrm{d}\varphi (t)=\omega (t)\mathrm{d}t,$$ (1a)

$$\mathrm{d}x(t)=v(t)\cos (\varphi (t))\mathrm{d}t,$$ (1b)

$$\mathrm{d}y(t)=v(t)\sin (\varphi (t))\mathrm{d}t,$$ (1c)

$$x_{\text{min}}\le x\le x_{\max },y_{\text{min}}\le y\le y_{\max }.$$ (1d)

Fig. 5:

Schematic of the replica (R) swimming alone with superimposed notation.

Here, ω and v are the stochastic processes describing the turn rate and forward speed, respectively, and constraints (1d) ensure that the position is confined in the rectangular domain [x_min, x_max] × [y_min, y_max]. In what follows, we summarize the stochastic equations describing the evolution of ω and v. We first illustrate the open-loop dynamics, and then articulate the modifications that are necessary to close the loop and simulate social interaction with the live subject.

A. Open-loop dynamics

When swimming alone, the dynamics of the replica are modeled by two coupled stochastic differential equations, associated with the evolution of the turn rate and forward speed.

The turn rate is modeled as a mean reverting stochastic equation with jumps, namely,

$$\mathrm{d}\omega (t)={\theta }_{\omega }[\omega (t)-f_w(t)]\mathrm{d}t+f_c(v(t))\mathrm{d}B(t)+\mathrm{d}J(t).$$ (2)

On the right hand side of (2), we identify four summands:

• θ_ωωdt represents the mean reversion term that drives the turn rate to its mean; θ_ω(s⁻¹) is the mean reversion coefficient (or relaxation rate), which characterizes the rapidity to recover straight swimming in the absence of interactions with walls or other fish.

• θ_ωf_wdt summarizes fish tendency to avoid the tank walls; the function f_w introduces a bias in the mean turn rate as a function of the incident angle ϕ_w to the boundary. Following [62], we set

  $$f_w(t)=k_w\left[\frac{1}{t_w(v(t))}\text{sgn}({\varphi }_w(t))-w_a\cos ({\varphi }_w(t))\right],$$ (3)

  where k_w > 0 determines the intensity of the wall interaction, w_a > 0 (s⁻¹) quantifies the tendency to follow the wall, and t_w(v) is the time to collision, that is, the time that the fish needs to reach the wall at the current speed.
• f_c(v)dB is additive noise, where B is a standard Wiener process. The function f_c(v) is used to couple ω and v, whereby the variance of fish turn rate decays exponentially as a function of speed. Following [63], we set

  $$f_c(v)={\alpha }_c{e}^{-{\beta }_{c}v},$$ (4)

  where α_c > 0 measures the strength of the coupling, and β_c > 0 (m⁻¹ s) its decay with the speed.
• dJ is the jump term, where J is a compound Poisson process [69], which is used to describe extreme changes in the turn rate. Following [61], we set

  $$J(t)=\sum _{i=1}^{j(t)}{G}_i,$$ (5)

  where G_i, i = 1, … , j(t), are independent and identically distributed normal random variables with variance γ² (s⁻²) and zero mean. j(t) indicates the number of sudden changes experienced by the fish turn up to time t, and is defined so that, for any t₁ and t₂, t₁ ≤ t₂, j(t₂) − j(t₁) is a Poisson random variable with parameter λ(t₂ − t₁). Therefore, λ (s⁻¹) represents the rate of occurrence of C- and U-turns in fish swimming.

The forward speed of the replica is also modeled through a mean reversing stochastic differential equation, that is,

$$\mathrm{d}v(t)=-{\theta }_v[v(t)-{\mu }_v]\mathrm{d}t+{\sigma }_v\mathrm{d}Z(t),$$ (6a)

$$v(t)\ge v_{\text{min}}.$$ (6b)

In (6a), we identify

• a mean reversion term −θ_v [v − μ_v] dt, with θ_v (s⁻¹) being the relaxation rate and μ_v the average forward speed, and

• an additive noise term σ_vdZ, with Z(t) being a standard Wiener process independent from B(t) and σ_v a positive parameter that quantifies the strength of the noise.

To ensure that the forward speed is always positive, in (6b) we assume v_min > 0.

B. Closed-loop dynamics

When the replica is swimming in the presence of a live subject, equations (2) and (6a) have to be updated to include feedback control actions that account for possible social interactions [63]. Namely, they become

$$\begin{gathered} \mathrm{d}\omega (t)={\theta }_{\omega }[\omega (t)-f_w(t)-u_{\omega }(t)]\mathrm{d}t+f_c(v(t))\mathrm{d}B(t) \\ +\mathrm{d}J(t), \end{gathered}$$ (7a)

$$\mathrm{d}v(t)=-{\theta }_v[v(t)-{\mu }_v-u_v(t)]\mathrm{d}t+{\sigma }_v\mathrm{d}Z(t),$$ (7b)

where u_ω and u_v constitute the feedback from the interaction with the live fish. Following [63], we set

$$u_{\omega }(t)=f_{d,\omega }(d(t))\left[k_p\frac{(d(t)-r_{\omega })\sin \theta (t)}{v(t)}+k_v\frac{\sin \varphi (t)}{{\theta }_{\omega }}\right],$$ (8a)

$$u_v(t)=f_{d,v}(d(t))k_s\cos \theta (t)[d(t)-r_v(v(t))].$$ (8b)

With reference to Figure 6, d and θ are the distance and angle between the replica and the live fish; ϕ(t) is the relative orientation; k_p (rad s⁻²), k_v (rad s⁻²), and k_s (s⁻¹) are a set of tunable gains associated with fish social behavior; r_ω and r_v(v) are the radii of the repulsion regions, with

$$r_v(v)=r_{v0}\frac{v(t)}{{\mu }_v}.$$ (9)

Fig. 6:

Schematic of the replica (R) swimming with a zebrafish (F) with superimposed notation.

Following [63], we assume a linear increase of the repulsion radius r_v with the speed, where r_v0 > 0 (m) is the nominal radius at the nominal mean speed. Finally, f_d,ω(d) and f_d,v (d) are two decay functions that attenuate the long-range interactions based on the distance d:

$$f_{d,i}(d)=\{\begin{array}{cc}1-e^{(d-{\delta }_i)∕{\nu }_i},\hfill & \text{if}d(t)<{\delta }_i,\hfill \\ 0,\hfill & \text{otherwise},\hfill \end{array}\phantom{\}}$$ (10)

where δ_i and ν_i are tunable parameters, i ∈ {ω, v}.

Equation (8) describes how the replica adjusts its turn rate and speed to react to the movements of the live fish. In particular, (8a) regulates the variations in the turn rate and is composed of two terms. The first steers the replica towards (away from) the live fish if their distance is greater (lower) than the repulsion radius r_ω, while the second term aligns the heading of the replica with that of the live fish. Equation (8b) regulates the variation in the speed, such that the replica is attracted (repulsed) by the live fish at distances higher (lower) than the repulsion radius r_v.

C. Model implementation

We used the standard Euler-Maruyama method [70] to discretize equations (1), (8), thus yielding

$$\begin{gathered} \omega (t+\Delta t)=\omega (t)+{\theta }_{\omega }[u_{\omega }(t)+f_w(t)-\omega (t)]\Delta t \\ +f_c(v(t))\sqrt{\Delta t}\Delta B(t)+\gamma \beta (t)\eta (t), \end{gathered}$$ (11a)

$$\begin{gathered} v(t+\Delta t)=v(t)+{\theta }_v[u_v(t)+{\mu }_v-v(t)]\Delta t \\ +{\sigma }_v\sqrt{\Delta t}\Delta Z(t), \end{gathered}$$ (11b)

$$\varphi (t+\Delta t)=\varphi (t)+\omega (t)\Delta t,$$ (11c)

$$x(t+\Delta t)=x(t)+v(t)\cos (\varphi (t))\Delta t,$$ (11d)

$$y(t+\Delta t)=y(t)+v(t)\sin (\varphi (t))\Delta t,$$ (11e)

$$\mid \omega (t)\mid \le {\omega }_{\max },v_{\text{min}}\le v(t)\le v_{\max }.$$ (11f)

Here, Δt is the time step, and ΔB, ΔZ are realizations of standard normal independent variables. Following [61], we assumed that only one jump may occur in the small time step Δt. Hence, the process j in (5) is such that j(t+Δt)−j(t) can be approximated by a Bernoulli variable β with mean λΔt. Accordingly, the jump term in (11a) is modeled as γβη, where η is a standard normal variable. Constraints (11f) are used to limit the stress on the actuators.

Most of the parameter values were selected from [62], [63]. The time step Δt was chosen to provide sufficient computational time, while allowing smooth motion of the replica and minimizing undesired mechanical vibration. The parameter k_w for wall interaction was re-calibrated in separate simulations to account for the different tank size compared to [62] and avoid unnatural behavior in the wall proximity. When operating in closed-loop, we used real-time data on the position of the live subject to generate the requisite social interactions, whose parameters were set as in [63]. Table II summarizes all the parameters selected for the experiment.

TABLE II:

Parameter values for the control of the replica in open- and closed-loop conditions.

Parameter	Value	Unit	Selection
Δt	0.05	s	ad hoc
θω	3.58	s−1	[63]
kw	0.10	nondimensional	ad hoc
wa	0.88	s−1	[62]
αc	11.81	${\text{rad s}}^{-\frac{3}{2}}$	[63]
βc	11.00	sm−1	[63]
γ	3.27	rad s−1	[62]
λ	0.42	s−1	[62]
θv	0.21	s−1	[63]
σv	2.59	${\text{cm s}}^{-\frac{3}{2}}$	[63]
μv	11.42	cm s−1	[63]
ωmax	20.00	rad s−1	[63]
vmin	0.01	m s−1	ad hoc
vmax	0.20	m s−1	ad hoc
kp	6.00	rad s−2	[63]
kv	12.00	rad s−2	[63]
ks	4.00	s−1	[63]
rω	1.80	cm	[63]
rv0	3.60	cm	[63]
δω	18.00	cm	[63]
δv	21.00	cm	[63]
νω	6.00	cm	[63]
νv	22.50	cm	[63]

IV. Experimental scheme

A. Animal and housing

The experimental procedure was approved by the University Animal Welfare Committee of New York University, under protocol number 13-1424. Zebrafish were acquired from the online aquarium supplier Carolina Biological Supply Co. (Burlington, NC) and were wild-type. The average body length was 3 cm. The population had a nominal female-to-male ratio of one. The animals were stored in a local vivarium for a minimum of 15 days before being used in the experiments. The water temperature was maintained at 26 °C, and its acidity at 7.0 pH. The stocking density was lower than one fish per 2 liters, and the illumination was controlled according to a 14 hours light/10 hours dark circadian rhythm.

At the beginning of each day of experiments, the tank was filled with new water and a heater was positioned in the tank to maintain a temperature of 27 °C. Coating (AcquaSafe Plus, Tetra, Blacksburg, VA, USA) was added to the water to neutralize potentially harmful chemicals present in tap water.

B. Experimental conditions

Three experimental conditions were performed to evaluate how zebrafish appraises the robotic replica in both open- and closed-loop conditions. More specifically, we considered the following conditions. In the control condition (2Live), two live fish swam together in the experimental tank to create a baseline on which to assess the degree of biomimicry of the replica. In the open-loop (OL) condition, the replica swam with a live fish and its motion was driven by the open-loop model in equations (1), (2), and (6). In the closed-loop (CL) condition, the replica swam with the fish implementing the closed-loop model in equations (1), (7), and (8) to simulate social interaction.

C. Experimental procedure

Experiments were performed from January to June 2018. Every condition consisted of 10 trials, each of them of 22 minutes. A total of 40 experimentally naive subjects were considered in the study, 20 in condition 2Live and 10 each for the two conditions with the replica. In each trial, the live fish were gently transferred from the vivarium to the experimental tank using a handnet. In 2Live condition, fish were randomly picked from two different housing tanks in the vivarium to avoid experimental biases. A habituation period of 12 minutes was given to the subjects such that only the last 10 minutes of each trial were considered for the subsequent behavioral analysis. The robotic platform was activated at the beginning of the habituation period in both OL and CL conditions to facilitate habituation to the robotic stimulus and mitigate novelty effects.

Each day, no more than 10 trials were performed: a maximum of five trials in the morning (from 9:00 AM to 1:30 PM) and five in the afternoon (from 2:00 PM to 6:30 PM). In each experimental condition, the schedule of the trials was equally distributed between morning and afternoon. The frames from the higher resolution camera were then fed to an in-house-developed automated software, PEREGRINE, which was used to accurately track the targets’ trajectories [71]-[74].

V. Data analysis

To assess the experimental results, we introduce appropriate metrics to characterize: i) the performance of the platform in terms of trajectory, speed and turn rate for the replica, as well as the computational time and uncertainties in real-time tracking; ii) the locomotor activity of live subjects; and iii) the interaction between the zebrafish and the replica.

For all the behavioral analyses, we selected a p-value equal to 0.05. In case of significant ANOVA comparisons, post-hoc tests were performed using the Tukey-Kramer method.

A. Metrics for evaluating the platform performance

As a first step in the analysis, we evaluated the performance of the platform in terms of its ability to reproduce the desired trajectory, speed, and turn rate for the replica. Specifically, we computed the average distance between the tracked trajectory of the replica and the input provided by the discrete model (11). With respect to speed and turn rate, we compared the distributions of experimental acquisitions for the replica with the model input. To quantify the similarity between probability distributions, say A and B, we used the score

$$S(A,B)=1-H(A,B),$$ (12)

where H (A, B) is the Hellinger distance, given by

$$H(A,B)=\frac{1}{\sqrt{2}}\sqrt{\sum _{i=1}^{n_b}{\left(\sqrt{a_i}-\sqrt{b_i}\right)}^2},$$ (13)

where a_i and b_i are the frequencies associated to the ith bin of A and B, respectively, and n_b is the number of bins. Notice that 0 ≤ H(A, B) ≤ 1 and that H(A, B) = 0 (S(A, B) = 1) only if the two histograms are identical.

To delve into the factors that may determine discrepancies between the desired and measured responses of the replica, we examined the computational time of the platform and uncertainties in the real-time tracking system. The computational time was a Matlab output, comprising the time required by the real-time tracking software to acquire the position of the live subject and the time needed to integrate the discrete model for calculating the position and heading of the replica at the next time step. Such a computational time should be sufficiently smaller than the desired time step Δt whenever experimenting with the replica to ensure proper functioning of the platform. The accuracy of the real-time tracking system was assessed from the distance between on-line and off-line tracked trajectories of two randomly selected trials from OL and CL conditions, including the habituation period.

B. Metrics for scoring fish locomotor activity

Upon clarifying the performance of the platform in replicating desired motions for the replica, we analyzed the locomotor activity of focal subjects in the presence of a live conspecific or the replica. For each fish in OL and CL conditions, we computed similarity indices in the distribution of speed, turn rate, linear and angular acceleration with respect to randomly selected subjects from 2Live trials. Should the fish appraise the replica as a conspecific, we would obtain similarity indices close to unity. To offer a baseline for ascertaining the degree of biomimicry of the replica, we compared these values with surrogate data generated by pairing subjects in condition 2Live from different trials using one-way ANOVA.

C. Metrics for measuring fish-replica interactions

Interactions between fish and replica were scored in terms of shoaling and schooling tendencies [75], social influence [76], and stress-related response [77]. Shoaling tendency [75] was measured by computing for each trial the average distance $\overline{d}$ between the two targets. Schooling tendency was measured in terms of the average polarization $\overline{P}$, which quantifies the extent to which the two targets align [75]. At any given time step, their instantaneous polarization was computed as

$$P(t)=\frac{1}{2}\Vert e^{\mathrm{i}{\varphi }_1(t)}+e^{\mathrm{i}{\varphi }_2(t)}\Vert ,$$ (14)

where ϕ₁(t) and ϕ₂(t) were the targets’ headings at time t, and i is the imaginary unit. To evaluate whether shoaling and schooling were different from chance, we computed the expected distance of two points with uniformly distributed positions in the tank, and the expected polarization of two vectors with heading uniformly distributed in [0, 2π]. Then, we used t-tests to compare these baselines with the observed values of distance and polarization. To ascertain differences across conditions, we compared average distance and polarization using one-way ANOVA.

To examine social influence underlying the motion of the two targets, we employed the information-theoretic notion of transfer entropy [78]. This quantity can be associated with information flow between two discrete stationary processes, say Ξ and Υ. More specifically, it quantifies the uncertainty in predicting the future of a process from its present given additional knowledge about the past of the other process. In formulas, this can be written as

$${\text{TE}}_{{\rm Y}\to \mathrm{\Xi }}(\tau )=\sum _{{\xi }_{t+1},{\xi }_t,{\upsilon }_t}p({\xi }_{t+1},{\xi }_t,{\upsilon }_{t-\tau }){\log }_2\frac{p({\xi }_{t+1}\mid {\xi }_t,{\upsilon }_{t-\tau })}{p({\xi }_{t+1}\mid {\xi }_t)},$$ (15)

where p(·) denotes a probability mass function; ξ_t and ξ_t+1 are possible values of Ξ at time t and t + 1, respectively; and v_t−τ are possible values of Υ at time t − τ, with τ being the interaction delay [79]. At a given interaction delay τ, the asymmetry in the information flow between Ξ and Υ is measured by the net transfer entropy, Net_TE(τ) = TE_Υ→Ξ(τ) − TE_Ξ→Υ(τ). As reviewed in [76], net transfer entropy can be used to infer influences underlying collective behavior. In particular positive Net_TE(τ) implies that Υ influences Ξ, while the opposite influence is implied by negative Net_TE(τ). For each trial, we estimated the interaction delay between the targets as

$${\tau }^{\ast }=\text{arg}\underset{0\le \tau \le {\tau }_{\max }}{\max }{\text{Net}}_{\text{TE}}(\tau ),$$

where τ_max = 1 s.

In order to mitigate the effects of measurement noise associated with tracking, we evaluated transfer entropy on symbolic sequences rather than raw positional data. Towards this aim, we first partitioned the original spatial domain into a square lattice of c_x × c_y cells. Then, we computed the discretized positions ($x_i^{\mathfrak{d}}(t),y_i^{\mathfrak{d}}(t)$), i = 1, 2, for each time step t = 1, … , T, with T being the total number of frames. Building on the symbolic approach proposed in [80], rather than considering the exact magnitudes of the discretized position, we focused on their relative change. To this aim, we transformed the time series $\{(x_i^{\mathfrak{d}}(t),y_i^{\mathfrak{d}}(t)){\}}_{t=1}^T$ into a simpler sequence, composed of pairs of ternary symbols $\{s_{xi}(t),s_{yi}(t){\}}_{t=1}^{T-1}$. Each symbol takes value −1, 0, or 1 depending on whether the target decreases, maintains, or increases its corresponding coordinate on the grid between two adjacent time steps. For example, a symbol (−1, 1) implies that from t to t + 1 the target has swam toward the left-top corner, such that both its coordinates in the grid have changed (the horizontal has decreased and the vertical has increased).

To evaluate the sensitivity of the results with respect to the spatial discretization, the computation was repeated for different resolutions by varying the number of cells. Specifically,

$$c_x(r)=120+40(r-1),r=1,\dots ,13,$$

where r identifies the spatial resolution of the discretization – c_y(r) was consistently chosen to preserve square-like cells. For r = 1, the cell size was 44/120 cm (roughly one tenth of a body length) and it reached 44/600 cm for r = 13. To assess whether net transfer entropy was significantly different than zero, we used a t-test.

In addition, we computed the percentage, say n_inspections, of discrete time steps in which the two targets became closer than one body length over each trial, that is,

$$n_{\text{inspections}}=\frac{100}{T}\mid \{t:d(t-\Delta t)>3\text{cm}\wedge d(t)\le 3\text{cm}\}\mid .$$ (16)

This quantity was used to complement the mean distance and indirectly assess the possibility of predator inspection by live fish [81]. One-way ANOVA was used to compare conditions.

Thigmotactic and freezing behaviors [77] were measured to study stress-related response of the animal that could be triggered by the exposure to a potential predator. Thigmotaxis or wall-following was computed as the percentage t_wall of time spent within one body length (3 cm) from the walls of the tank [66]. Time spent freezing, defined as complete cessation of movements except for eyes and gills [77], was computed by partitioning each trial in 2 s time-windows, and then counting the number of windows in which the fish position did not change more than 2 cm [49]. One-way ANOVA was employed to establish differences across conditions.

VI. Experimental results

A. Assessment of the robotic platform

As a first step in assessing the performance of the platform, we examined the degree of similarity between the distributions of the speed and turn rate of live fish with model implementation by the platform. From comparison, we determined similarity scores of 0.84±0.02 and 0.86±0.02 (average ± standard deviation) in OL and CL conditions, respectively. Even higher similarity scores were registered when comparing the turn rate distributions (OL: 0.90±0.01 and CL: 0.94±0.01), supporting the use of the proposed platform to generate biologically-inspired locomotory patterns for zebrafish experiments.

A similarity score on speed and turn rate above 0.80 offers compelling evidence for the biomimicry of the movement of the replica, but does not help clarify whether the platform implemented the sought model input in real-time. Toward this aim, we measured the distance between the tracked trajectories and input generated through (11). From the analysis of OL trajectories, we determined a mean distance of 2.49 ± 0.12 cm, which is less than the average body length of an animal, associated with the characteristic length of the experiment.

A direct comparison with respect to existing platforms is challenging since most of the current endeavors do not seek to address the problem of maneuvering a desired target according to a dynamically-varying model input. Perhaps the closest platform is the one presented in [33], in which a replica of a guppy is maneuvered along pre-programmed trajectories in two dimensions using a wheeled ground robot. Performance is therein assessed by scoring the 90th percentile of the distribution of the distance, which in our case is 4.35 cm against 1.30 cm reported in [33].

The reduced performance of our platform should be ascribed to the combination of methodological differences between the setups, together with uncertainties in real-time tracking. With respect to methodological differences, the authors of [33] considered a reference zigzag trajectory as opposed to the model-based trajectories implemented herein, and distance measurements were only taken at five waypoints rather than at every frame. With respect to the second aspect, we compared real-time tracked data with off-line tracked trajectories on a randomly selected open-loop experiment, identifying a 90th percentile on the distance distribution of 1.04 cm. We comment that this estimate is less forgiving than those based on the use of static objects [45], which exclude challenges associated with real-time estimation of fast moving targets. Also, estimating the distance between real-time and off-line trajectories is more informative than performing assessments based on the success rate in blob identification [47], which does not provide a direct quantification of the tracking accuracy.

Extending the assessment of the platform to the CL condition, we observed that the mean distance distribution between tracked trajectories and model input moderately increased to 3.18±0.21 cm. Such an increase with respect to condition OL should be attributed to two concurrent factors. First, in closed-loop the controller had to process real-time data on the position of the live subject to compute the social interaction terms in (8). Consequently, the computational time increased from 0.03 ± 0.01 s in OL condition to 0.04 ± 0.11 s when closing the loop between the replica and the live subject. Remarkably, this increase only marginally affected the functionality of the platform, as the computational time remained smaller than the desired time step Δt in 97.34% of the cases. Second, in closed-loop, the more frequent overlaps between the planar positions of the replica and the live subject represented a further challenge for the real-time tracking system, whereby the 90th percentile of the distance between off-line and real-time tracking increased to 4.55 cm.

The challenges posed by real-time tracking and by the rapid variations in speed and velocity of the live subject are efficiently counterbalanced by the swift of actuation of the platform. Figure 8 illustrates in a sample CL trial how the linear and angular distributions of the replica accurately match those of the live fish, thus enabling the replica to capture the complex locomotory repertoire of zebrafish swimming in shallow water.

Fig. 8:

Time-trace of the speed and turn rate of the replica and live fish in a sample CL trial (top panels), and the corresponding sample distributions of linear and angular accelerations (bottom panels).

B. Evaluation of fish response to the replica

The primary goal in designing our robotic platform was to lend the replica a biologically-inspired appearance and motion, which would favor its biomimicry. The analysis of the similarity scores for the speed (F_(2,27) = 2.70, p = 0.08), turn rate (F_(2,27) = 1.79, p = 0.19), linear (F_(2,27) = 1.01, p = 0.38) and angular (F_(2,27) = 0.34, p = 0.72) acceleration distributions does not suggest that fish altered their locomotory activity across conditions, see Figure 9. Overall, this comparison indicates that fish did not change their swimming speed and style as they interacted with the replica. In particular, these comparisons seem to exclude the possibility that fish perceived the replica as a predator or fear-eliciting stimulus in general, which should have likely resulted in erratic movements, increased freezing response, and higher activity [77], [82].

Fig. 9:

Similarity score between (from top to bottom) the speed, turn rate, linear and angular acceleration distributions of 10 zebrafish swimming in 2Live and those of other 10 live subjects swimming in 2Live, OL, and CL conditions, respectively.

This proposition is in agreement with our previous studies on the interactions between biologically-inspired 3D-printed replicas and live zebrafish in binary choice tests [25], [32], [47], which do not offer evidence of a fear-related emotional response to the replica. However, several differences exist between the present setup and our earlier studies, which, in principle, could have evoked a fear response in the focal subject. Differently from the present setting, in our previous work the replica was separated from the live subject by a transparent panel, which did not permit physical interaction between them. A priori, such physical interactions could have constituted a fear stimulus. Also, the replica considered in our previous work was neither actuated along model-based trajectories nor was manufactured in a soft material to elicit body undulations. If not properly implemented, both these experimental manipulations could have resulted in an unnatural swimming style and social interaction by the replica, thereby potentially triggering fear.

Just as fish did not alter their swimming activity due to the replica, we did not record a variation in their schooling tendency across conditions (F_(2,27) = 0.03, p = 0.98), see Figure 10. In particular, fish did not tend to align their swimming with a conspecific or a replica, whereby polarization data were not different from chance (OL: t₉ = −0.21, p = 0.84, and CL: t₉ = −0.52, p = 0.61). Likely, this is due to the reduced size of the tank, which could have limited the ability of fish to coordinate their swimming with a conspecific or the replica and favored a thigmotatic [77] response with a stronger wall interaction. Compared with our previous experiments [63] where polarization values above 0.9 were recorded, the size of the tank was reduced by about 80%.

Fig. 10:

Mean polarization as a function of the experimental condition. Error bars show one standard error. The red dotted line corresponds to the expected polarization of two vectors with random orientation.

Although locomotor activity and schooling tendency did not vary across conditions, shoaling tendency exhibited a marked dependency on the condition (F_(2,27) = 44.99, p < 0.01), see Figure 11. Specifically, the mean distance between the focal subject and the replica was considerably larger than the mean distance between two conspecifics in both open- and closed-loop conditions. While fish maintained a distance of about two body lengths when swimming in pairs, the distance between the replica and the fish was approximately six body lengths in OL condition and five body lengths in CL condition – the improvement associated with closing the loop on the response of the focal subject was close to significance (p = 0.06). Despite the reduction in shoaling tendency, the mean distance in OL and CL conditions was different from chance (OL: t₉ = −6.72, p < 0.01, and CL: t₉ = −6.02, p < 0.01), suggesting that the replica constituted a salient stimulus for the focal subjects, although less attractive than a live fish.

Fig. 11:

Mean distance as a function of the experimental condition. Error bars show one standard error. The red dotted line corresponds to the expected distance of two particles with random position in the tank. Significance from Tukey-Kramer post-hoc tests are indicated when relevant. A star denotes differences from chance (t-test, significance level 0.05). Post-hoc comparison between OL and CL conditions fail to reach statistical significance, yielding a p-value of 0.06.

The saliency of the replica is also supported by the transfer entropy analysis, which confirms that focal subjects were influenced by the presence of the replica, see Figure 12. More specifically, across a wide range of spatial resolutions, we identified a robust information flow from the replica to the live fish, which suggests that predicting the future behavior of a live fish from its present is improved upon additional knowledge about the past of the replica (OL: t₉ < −8.20, p < 0.01, and CL: t₉ < −2.62, p < 0.03 for r from 1 to 12, t₉ = −2.23, p = 0.05 for r = 13). Predictably, a net information flow is not observed in condition 2Live, in which the two fish mutually influence each other (t-tests, ∣t₉∣ < 1.85, p > 0.10 for r from 2 to 13, t₉ = −3.25, p = 0.01 for r = 1).

Fig. 12:

Net transfer entropy (TE_{fish→replica} − TE_{replica→fish}) for different values of the spatial resolution r_i. Error bars show one standard error. A subscript star denotes differences from chance (t-test, significance level 0.05).

To exclude the possibility that the observed attraction was due to predator inspection [81], we scored the fraction of the frames in which the focal fish approached the conspecific or the replica by crossing a circle of one body length in radius centered about them. As a function of the experimental condition, we registered a variation in the number of inspections (F_(2,27) = 26.46, p < 0.01), see Figure 13. In agreement with our expectations, fish did not inspect the replica more than a live conspecific. Instead, we registered a decrease in the number of inspections in both OL and CL conditions. Should the attraction towards the replica be explained by predator inspection, we would have likely observed an increase in the number of attempts by the fish to inspect the replica from close proximity. Based on similarities in locomotor activity across conditions, reduced number of inspection of the replica with respect to a conspecific, and our prior work on similar replicas [32], we can therefore discount predator inspection as the key mechanism underlying the observed attraction. This thesis is further supported by the analysis of stress-related behaviors, such as thigmotaxis and freezing. Indeed, we did not record a variation in neither the time spent close to the walls nor freezing (F_(2,27) = 1.62, p = 0.22, and F_(2,27), = 0.98, p = 0.39, respectively), suggesting that the replica did not evoke a stress-related response, see Figure 14.

Fig. 13:

Number of inspections over the total number of frames (18000) as a function of the experimental condition. Error bars show one standard error. Significance from Tukey-Kramer post-hoc tests are indicated when relevant.

Fig. 14:

Percentage of time spent at less than one body length from the wall (top panel) and freezing (bottom panel) as a function of the experimental condition. Error bars show one standard error.

Although the replica constituted an attractive, salient stimulus, it is unlikely focal fish perceived it as a conspecific. If they had appraised it as a conspecific, we would have found a closer correspondence between the number of inspections and the mean distance between 2Live condition and the two conditions with the replica. Also, we would have likely observed a bidirectional influence between the replica and the live fish in CL condition, similar to 2Live condition. We acknowledge this as a negative result, whereby we had initially anticipated that the technical improvements of our platform would have beget a replica highly comparable to a live stimulus. However, a number of experimental limitations could have hindered the biomimicry of the replica. First, the morphology and coloration of the replica were chosen to mimic a live fish, but practical limitations in the fabrication and painting process resulted in a prototype that may not have been appraised as a conspecific by a focal subject. Second, maneuvering the replica required both a base and a transparent stick, which could have reduced the attractiveness of the replica. Third, the presence of the stick might have contributed to generate unnatural flow cues from shedding of vortical structures in its wake, potentially causing a repulsive effect towards live subjects. Finally, the motion of the replica was constrained to a two-dimensional work space, such that the replica could not dive along the water depth similar to live zebrafish. Future research endeavors will seek to quantify the severity of these potential limitations through control experiments and new engineering solutions.

VII. Conclusions

The success of biologically-inspired robotics relies on the ability of robots to mimic social, bidirectional interactions through closed-loop control systems, where robots could adjust their behavior in real-time as a function of their live counterparts. Here, we have sought to contribute to this field of investigation by integrating a stochastic mathematical model in the design of realistic feedback laws. Leveraging recent advances in data-driven modeling of zebrafish swimming, we built a novel robotic platform that actuates a biologically-inspired replica in real-time through model-based feedback control. We demonstrated that the replica constitutes a salient stimulus for zebrafish through comparative, empirical analysis.

These findings support the use of closed-loop model-based control strategies in robotics-based studies on animal behavior, as a less computationally expensive alternative to probability based approaches, see, for instance, [48]. This could be useful, for example, to study the emergence and role of leadership in animal groups [83]. We envision that the combination of these interactive control strategies, along with further enhancements of the biomimetic appearance of the replica, will open the door for novel applications in preclinical research, where zebrafish is emerging as the species of choice.

Fig. 7:

Pairs of symbols (in red) associated with each region of the discretized domain as a function of the next position [$x_i^{\mathfrak{d}}(t+1),y_i^{\mathfrak{d}}(t+1)$] given the current one [$x_i^{\mathfrak{d}}(t),y_i^{\mathfrak{d}}(t)$].