Insect-machine Hybrid System: Remote Radio Control of a Freely Flying Beetle (Mercynorrhina torquata)

Abstract

The rise of radio-enabled digital electronic devices has prompted the use of small wireless neuromuscular recorders and stimulators for studying in-flight insect behavior. This technology enables the development of an insect-machine hybrid system using a living insect platform described in this protocol. Moreover, this protocol presents the system configuration and free flight experimental procedures for evaluating the function of the flight muscles in an untethered insect. For demonstration, we targeted the third axillary sclerite (3Ax) muscle to control and achieve left or right turning of a flying beetle. A thin silver wire electrode was implanted on the 3Ax muscle on each side of the beetle. These were connected to the outputs of a wireless backpack (i.e., a neuromuscular electrical stimulator) mounted on the pronotum of the beetle. The muscle was stimulated in free flight by alternating the stimulation side (left or right) or varying the stimulation frequency. The beetle turned to the ipsilateral side when the muscle was stimulated and exhibited a graded response to an increasing frequency. The implantation process and volume calibration of the 3 dimensional motion capture camera system need to be carried out with care to avoid damaging the muscle and losing track of the marker, respectively. This method is highly beneficial to study insect flight, as it helps to reveal the functions of the flight muscle of interest in free flight.

Download video file ^{(29.4MB, mp4)}

Introduction

An insect-machine hybrid system, often referred to as a cyborg insect or biobot, is the fusion of a living insect platform with a miniature mounted electronic device. The electronic device, which is wirelessly commanded by a remote user, outputs an electrical signal to electrically stimulate neuromuscular sites in the insect via implanted wire electrodes to induce user desired motor actions and behaviors. In the early stages of this research field, researchers were limited to conducting wireless recording of the muscular action of an insect, using simple analog circuits comprised of surface-mounted components^1-3. The development of system-on-a-chip technology with radio frequency functionality enabled not only the wireless recording of neuromuscular signals but also the electrical stimulation of the neuromuscular sites in living insects. At present, a built-in radio microcontroller is small enough to be mounted on living insects without causing any obstructions to their locomotion^4-13.

The development of the built-in radio microcontroller allows researchers to determine electrical stimulation protocols to induce desired motor actions to control the locomotion of the insect of interest. On the ground, researchers have demonstrated walking control by stimulating the neuromuscular sites of cockroaches^4,12,14, spiders¹⁵, and beetles^16,17. In the air, the initiation and cessation of flight were achieved using different methods such as the stimulation of the optic lobes (the massive neural cluster of a compound eye) in beetles^7,9 and brain sub-regions in bees¹⁸, whereas turning control has been demonstrated by stimulating the antennae muscles and nervous system of the abdomens in moths^11,19 and the flight muscles of beetles^7,9,13. In most cases, a built-in radio microcontroller was integrated on a custom-designed printed circuit board to produce a miniature wireless stimulator (backpack), which was mounted on the insect of interest. This allows wireless electrical stimulation to be applied to a freely walking or flying insect. Such a microcontroller-mounted insect is what is referred to as an insect-machine hybrid system.

This study describes the experimental protocols for building an insect-machine hybrid system, wherein a living beetle is employed as the insect platform, and instructs on how to operate the robot and test its flight control systems. The third axillary sclerite (3Ax) muscle was chosen as the muscle of interest for electrical stimulation and demonstration of left or right turning control¹³. A pair of thin silver wire electrodes was implanted in both the left and right 3Ax muscles. Moreover, a backpack was mounted on the living beetle. The other ends of the wire electrode were connected to the output pins of the microcontroller. The backpack was small enough for the beetle to carry in flight. Thus, this allows an experimentalist to remotely stimulate the muscle of interest of an insect in free flight and investigate its reactions to the stimulations.

Protocol

1. Study Animal

1. Rear individual Mecynorrhina torquata beetles (6 cm, 8 g) in separate plastic containers with wood pellet bedding.

2. Feed each beetle a cup of sugar jelly (12 ml) every 3 days.

3. Keep the temperature and humidity of the rearing room at 25 °C and 60%, respectively.

4. Test the flight capability of each beetle before implanting thin wire electrodes.

   1. Gently throw a beetle into the air. If the beetle can fly for longer than 10 sec for 5 consecutive trials, conclude that the beetle has regular flight capabilities and employ it for subsequent flight experiments. To recapture the beetle, switch off all the lights in the room to make it dark. This causes the beetle to terminate flight. Note: A beetle spontaneously begins to fly away when released into the air. It is better to conduct the flight experiments in a large closed room such as the one shown in Figure 1 (16 x 8 x 4 m³ with a motion capture space of 12.5 x 8 x 4 m³), as a flying beetle moves very fast (approximately 3-5 m/sec) and draws large arcs when turning in the air.

2. Electrode Implantation

1. Anesthetize the beetle by placing it in a plastic container filled with CO₂ for 1 min^13,16,20-24.

2. Soften dental wax by dipping it in hot water for 10 sec. Place the anesthetized beetle on a wooden block and immobilize it with the softened dental wax. The dental wax naturally cools and solidifies within a few minutes.

3. Cut insulated silver wires (127 µm bare diameter, 178 µm diameter when coated with perfluoroalkoxy) into lengths of 25 mm to use as thin wire electrodes for implantation.

4. Expose 3 mm of bare silver by flaming the insulator at both ends of each wire.

5. Dissect the top surface of the beetle's cuticle using a fine-tipped scissor to create a small window of approximately 4 x 4 mm on the metepisternum (Figure 2c). Note: A soft brown-colored cuticle is then exposed, as shown in Figures 2c-e. The 3Ax muscle is located underneath the soft cuticle.

6. Pierce two holes on the exposed brown cuticle using an insect pin (size 00) with a distance of 2 mm between the two holes (Figure 2d).

7. Insert two wire electrodes (including one active and one return electrodes prepared in step 2.4) carefully through the holes and implant them into each 3Ax muscle at a depth of 3 mm.

8. Secure the implanted electrodes and hold them in place to avoid contact and short-circuits by dropping melted beeswax on the holes. If needed, reflow the beeswax over the cuticle by touching the beeswax with the tip of a hot soldering iron. The beeswax quickly solidifies and reinforces the implantation. Note: To check if the implantation is correct, the elytra of the beetle can be lifted to observe the movement of the 3Ax muscle during electrical stimulation.

3. Wireless Backpack Assembly

Note: The backpack consisted of a built-in radio microcontroller on a 4 layered FR-4 board (1.6 x 1.6 cm²). The backpack was driven by a lithium polymer microbattery (3.7 V, 350 mg, 10 mAh). The total mass of the backpack including the battery was 1.2 ± 0.26 g which is less than the payload capacity of the beetle (30% of 10 g body weight). The backpack was pre-programmed to receive wireless communications and had two output channels.

1. Clean the pronotum surface (remove the wax layer on the cuticle) using double-sided tape. Then, attach the backpack on the pronotum of the beetle with a piece of double-sided tape.

2. Connect the ends of the implanted electrodes to the outputs of the backpack.

3. Wrap retro-reflective tape around the microbattery to produce a marker for motion capture cameras to detect.

4. Attach the microbattery to the top of the backpack using a piece of double-sided tape so that the retro-reflective tape can be detected by motion capture cameras.

4. Wireless Control System

Note: In this case, the term wireless control system includes a receiver for the remote controller, a laptop computer to run the custom flight control software, a base station, the backpack, and the motion capture system.

1. Connect the base station and receiver of the remote controller to the laptop computer via USB ports.

2. Switch on the motion capture system and connect it to the laptop computer via an Ethernet port.

3. Perform volume calibration by waving the calibration wand (provided by the vendor company of the motion capture system) to fully cover the motion capture space.

   1. Open the motion capture software from the desktop of the laptop. Click and drag to select all the cameras on the "System" menu of the "Resources" panel.
   2. Click on the "3D Perspective" menu and select "Camera" to change to the camera view. Click on the "Camera" tab on the "Tools" panel to show the calibration setup. Click "Start" on the "Create Camera Masks" menu to eliminate noise from the cameras and then "Stop" after the noise is masked in blue.
   3. Click and select "5 Marker Wand & L-Frame" from the "Wand" menu and the "L-Frame" menu on the "Camera" tab. Set the "Wand Count" to 2,500, click "Start" on the "Calibrate Cameras" menu, and wave the calibration wand through the entire motion capture space. The calibration process stops when the wand count reaches 2,500.
   4. Repeat the calibration process if image error (at the bottom of the "Camera" tab of the "Tools" panel) is higher than 0.3 for any camera. After calibrating, put the wand on the floor in the middle of the motion capture space and click "Start" on the "Set Volume Origin" menu to set the origin of the motion capture space.
4. Check the coverage of the motion capture system using a dummy test to record the motion path of a marker waved by a user in the motion capture space and confirm whether the marker is detected and tracked. If the marker is frequently lost during detection, repeat volume calibration until the dummy test succeeds.

   1. Click on the "Capture" tab on the "Tools" panel and then "Start" on the "Capture" menu before waving the sample marker through the entire motion capture space to record its trajectory.
   2. After recording, click on "Runs the Reconstruct pipeline" to reconstruct the positions of the marker and check the quality of the recording.

5. Connect the terminals of the microbattery (attached to the backpack in step 3.4) to the power pins of the backpack.

6. Test the wireless communication between the laptop and the backpack using the custom flight control software. Click the "Start" command on the software and check the displayed connection status.

5. Free Flight Experiment

1. Carry out the free flight experiment in a flight arena measuring 16 x 8 x 4 m³.

2. Input the appropriate parameters to the flight control software (voltage, pulse width, frequency, and stimulation duration). Note: For demonstration, we fixed the voltage to 3 V, pulse width to 3 msec, and stimulation duration to 1 sec and varied the frequency from 60 to 100 Hz.

   1. On the software screen, type 3 for 3 V in the "Voltage" box, 1,000 for 1,000 msec in the "Stimulation Duration" box, 3 for 3 ms in the "Pulse Width" box, and a desired frequency in Hz in the "Frequency" box on the command window.

3. Release the backpack-mounted beetle into the air allowing it to fly freely within the flight arena. Manually trigger the stimulation when the beetle enters the motion capture space. Press the appropriate command button (Left or Right) on the remote to stimulate the target muscle on the left or right side of the beetle. Note: Once the button is pressed, the flight control software running on the laptop generates the command and sends it to the backpack. The backpack then outputs the electrical stimulus to the muscle of interest (on the left or right side).

4. Observe the beetle's reaction in real time during the stimulation and reconstruct the data using 3D graphing software.

   1. Select one of the trials recorded in the data list of the "Beetle Display" window and click "Export Panda" to copy the data of that trial to the analysis folder and run the 3D graphing module.
   2. Press "N" on the keyboard to combine the stimulus signal with the recorded trajectory. Press I to show the trajectory of the beetle with the highlighted stimulation periods.

Representative Results

The electrode implantation procedure is presented in Figure 2. Thin silver wire electrodes were implanted into the 3Ax muscle of the beetle through small holes pierced on the soft cuticle on the muscle (Figures 2d-e). This soft cuticle is found just above the apodema of the basalar muscle after removing the anterior part of the metepisternum (Figures 2d-c). The electrodes were then secured using beeswax (Figure 2f).

Figure 3 shows the procedures for constructing an insect-machine hybrid system using an intact beetle. Figures 2 and 3b show the methods for implanting thin metal wires (stimulation electrodes) into the muscle of interest (for example, in Figure 2, the 3Ax muscle was used in this study) and mounting a backpack on the pronotum of a beetle. The free ends of the wires were inserted into the holes in the jumper connector on the backpack, which were electrically connected to the input/output pins of the microcontroller integrated on the backpack (Figure 3c). Lastly, a microbattery was mounted and the power cable of the microbattery was connected to the holes in the jumper connector leading to the ground and positive supply terminals of the microcontroller.

The wireless control system is shown in Figure 4. When the user presses a command button on the remote controller (Figure 4c), the flight control software in the laptop computer (Figure 4d) generates and wirelessly sends the command to the backpack via the base station (Figure 4b). The motion capture system (Figure 4e) detects the position (X, Y, and Z) of the beetle and marks it with a timestamp. This data is then fed to the laptop computer, and the flight control software synchronizes the data with the stimulation signals.

Representative turn control results are shown in Figure 5. The activation of the 3Ax muscle was found to cause a reduction in the wing beat amplitude of the ipsilateral side¹³, thus resulting in the beetle performing an ipsilateral turn in free flight. The electrical stimulation of the 3Ax muscle showed a similar effect as the beetle turned to the ipsilateral side when the left or right 3Ax muscle was stimulated¹³. The turning rate of the beetle was graded as a function of the stimulation frequency.

Figure 1: Free flight arena arrangement. The free flight arena was arranged into two parts: the control space (3.5 x 8 x 4 m³) was used for setting up the implantation kit (microscope and dissection tools) and control booth (computer, wireless base station, and camera controller), while the motion capture space (12.5 x 8 x 4 m³) was covered with 20 near-infrared cameras to record the position (X, Y, and Z) of the beetle. The flight arena was equipped with 30 lighting panels (60 x 60 cm², 48 W) to make it as bright as daytime conditions during the experiment. Please click here to view a larger version of this figure.

Figure 2: Procedure for electrode implantation. The beetle was anesthetized and immobilized with dental wax on a wooden block for the implantation procedure. (a-c) A small window was opened on the metepisternum of the beetle to access the 3Ax muscle. (d) Using an insect pin, two holes with a distance of 2 mm were pierced on the inner cuticle that bears the 3Ax muscle. (e) The electrodes were inserted into the muscles via these holes and kept in place with tweezers to ensure that no crosstalk occurred between the tips. (f-g) The electrodes were then fixed to the beetle using beeswax. Please click here to view a larger version of this figure.

Figure 3:Procedure to produce an insect-machine hybrid system using an intact beetle. (a) The muscle of interest on a living beetle was implanted with a pair of silver wire electrodes. (b) After fixing the electrodes with beeswax, we mounted the backpack on the pronotum of the beetle using double-sided tape. (c) The free ends of the electrodes were inserted into the outputs of the backpack and secured with micropin headers. (d) A microbattery, which was covered with retro-reflective tape, was mounted on the backpack using double-sided tape and connected to the power pins of the backpack. Please click here to view a larger version of this figure.

Figure 4:Wireless system for free flight experiment. The wireless system consists of (a) a cyborg beetle, (b) a wireless base station, (c) a remote, (d) an operating laptop with a Bluetooth receiver plugged in, and (e) a 3D motion capture system. When the user presses the command button on the remote, the custom flight control software on the laptop sends the stimulation command wirelessly to the cyborg beetle via a base station that is plugged into the laptop via a USB port. Once the backpack receives the command, it generates an electrical stimulus signal that stimulates the muscle. Simultaneously, the motion capture system records the 3D coordinates of the beetle and feeds them to the laptop for synchronization with the stimulation data. Please click here to view a larger version of this figure.

Figure 5:The behavior of the beetle owing to the electrical stimulation of the 3Ax muscle in free flight. (a) The beetle turned to the ipsilateral side when the left or right 3Ax muscle was stimulated, and the turning motion was graded as a function of the stimulation frequency. (b) The zigzag path of the flying beetle when the left or right 3Ax muscle was stimulated in sequence. The stimulus parameters were an amplitude of 3 V, a pulse width of 3 msec, and a frequency of 60-100 Hz. Please click here to view a larger version of this figure.

Figure 6:Proposed marker sets for tracking the 3D orientation (roll, pitch, and yaw) of the beetle. Configuration using (a) three markers, (b) four markers, and (c) five markers. Please click here to view a larger version of this figure.

Discussion

The implantation process is important, as it affects the reliability of the experiment. The electrodes should be inserted into the muscle at a depth of 3 mm or less depending on the size of the beetle (avoiding contact with nearby muscles). If the electrodes touch the nearby muscles, undesirable motor actions and behaviors may occur owing to the contraction of nearby muscles. The two electrodes should be well aligned to ensure that no short circuits occur. When melting and reflowing beeswax using a soldering iron, the experimentalist has to be careful and solder as quickly as possible, since the muscle can be burnt by prolonged contact with high temperatures, leading to a malfunction of the muscle. Although removing the cuticle is required to access the 3Ax muscle, the insertion and sealing process takes less than one minute and was managed to minimize damage to the muscle. The insects were returned to the rearing room after the experiments and could survive for up to 3 more months (end of their life time). To maintain good performance of the beetle, the beetle should be fed and allowed to rest for 3 to 4 hr after every 20 consecutive trials as the insect can become fatigued after many consecutive (40 to 50) flight trials and may not be able to open its wings.

As for the free flight experiment, volume calibration for the motion capture system is necessary, as it affects the trajectory tracking accuracy. It is important to fill the cameras' view full of the waves of the calibration wand with an image error of less than 0.3 for all of the cameras to maintain the accuracy of the motion tracking system. In addition, the surface of the marker should be clean, or the 3D motion capture system may frequently miss the marker. After calibration, a dummy test should be carried out by waving the battery wrapped with retro-reflective tape in the defined volume to check the coverage of the motion capture system. For testing the motion tracking accuracy, we measured the distance of two markers moving in the flight arena. The markers were fixed on a carton board with a distance of 200 mm to each other. The board was moved in the entire flight arena to obtain various positions of the two markers. The standard deviation was then calculated to be 1.3 mm (n = 3,000).

The free flight test facility (Figures 1 and 4) allows us to track the position (X, Y, and Z) of a flying insect along with a timestamp. Since only a single marker is attached to the beetle and the 3D motion capture system only detects that marker, the beetle is treated as a particle or a mass point. As such, data from the flying beetle has positional information but lacks orientation. Therefore, kinematic analysis from the positional data of the beetle provides only the translational velocity and acceleration along the X, Y, and Z axes without angular velocity or angular acceleration in rotations about the yaw, pitch, and roll axes. Multiple markers fixed on a beetle (such as the one shown in Figure 6) must be used for the 3D motion capture system to treat the flying insect as a rigid body and record rotation and translation data. However, the experimentalist must take note of the contribution of these markers to the kinetics of a flying beetle, because the marker is not a small piece of tape but needs to be large enough to be detected by the camera system with minimum tracking loss. Such an arrangement and the attachment of multiple markers may significantly increase its mass and moment of inertia²⁵. Besides, the size of the flight arena can be set as large as possible within the coverage range of the motion tracking system to reduce the constraints to the free flight behavior of the beetle. For this paper, the size of the flight arena is defined based on the maximum coverage of motion capture system (12.5 x 8 x 4 m³).

Various possibilities exist to modifying this technique along with increasing the number of markers to record the orientation of the insect as mentioned above. The stimulation of different muscles in free flight can produce various behaviors, e.g., the basalar muscle for a contralateral turn⁷ and 3Ax muscle for an ipsilateral turn¹³. In addition, certain parts of the nervous system of an insect can induce various reactions. Optic lobe stimulation can induce flight initiation⁷, whereas the stimulation of antennae can induce contralateral turning in a walking insect¹². Furthermore, we can change the function of the backpack from being an electrical stimulator to an electromyography recorder to record the activities of an insect during its natural behavior^3,26.

The free flight stimulation of the beetle helped to reveal and confirm the natural function of the 3Ax muscle by enabling observations of the instantaneous reaction of the insect freely moving in air. Such information is not available under tethered conditions^11,13,27-30. The behavior of an insect is constrained under tethered conditions and may be different from that in free flight, possibly leading to an incorrect understanding of insect behavior. Thus, free flight stimulation using this technique is a strong tool for validating the hypotheses drawn from tethered experiments. Furthermore, an insect-machine hybrid system is superior to current artificial flapping robots in terms of locomotive capabilities and power consumption^13,17,31,32.

Insect-machine hybrid systems may replace artificial robots in the future as they inherit the complex and flexible structure and locomotive capabilities of a living insect and reduce the production time of the fabrication process. Various locomotive capabilities can help an insect-machine hybrid system to operate more efficiently in constrained spaces that involve the combination of walking and flying, e.g., in rescue missions. In addition, insect-machine hybrid systems can potentially be used as a tool for insect control in agriculture as it may be able to blend into natural insect colonies and help to control their activities.

Disclosures

The authors declare that there are no conflicts of interest.