Distant touch hydrodynamic imaging with an artificial lateral line

Abstract

Nearly all underwater vehicles and surface ships today use sonar and vision for imaging and navigation. However, sonar and vision systems face various limitations, e.g., sonar blind zones, dark or murky environments, etc. Evolved over millions of years, fish use the lateral line, a distributed linear array of flow sensing organs, for underwater hydrodynamic imaging and information extraction. We demonstrate here a proof-of-concept artificial lateral line system. It enables a distant touch hydrodynamic imaging capability to critically augment sonar and vision systems. We show that the artificial lateral line can successfully perform dipole source localization and hydrodynamic wake detection. The development of the artificial lateral line is aimed at fundamentally enhancing human ability to detect, navigate, and survive in the underwater environment.

A lateral line is a spatially distributed system of flow sensors found on the body surface of fish (1) and aquatic amphibians (2). It is comprised of arrays of neuromasts, which can be classified into two types: superficial neuromasts and canal neuromasts (Fig. 1). A superficial neuromast is situated on the surface of the fish and responds in proportion to fluid velocity (3, 4). In contrast, a canal neuromast is packaged in fluid-filled canals located beneath the surface of the skin and are commonly described as a detector of outside water acceleration that is proportional to the pressure gradient (1, 3–6). As an integrated flow sensing system, such lateral lines form spatial-temporal images of nearby sources based on their hydrodynamic signatures (1, 3, 5) and provide mechanosensory guidance for many different behaviors, including synchronized swimming in schools, predator and obstacle avoidance, prey detection and tracking, rheotaxis, and holding station behind immersed obstacles in streams (4, 7, 8). This “distant touch” sense complements other sensory modalities, including vision and hearing, to increase survivability in unstructured environments. To date, there has never been an engineering equivalent of the fish lateral line system for underwater vehicles and platforms. The goal of the present research is to build an artificial lateral line that mimics the functional organization and imaging capabilities of the biological one. The artificial lateral line can facilitate fundamental studies of biological systems and provide unprecedented sensing and control functions to underwater vehicles and platforms. Specifically, we envision that the distant touch hydrodynamic imaging capability of the artificial lateral line can provide a new sense in addition to sonar and vision. In this article, we demonstrate the functions of an artificial lateral line under two biologically relevant scenarios: (i) localizing a moving target with flapping part (7, 9, 10) and (ii) imaging a hydrodynamic trail for prey capture (11, 12).

Fig. 1.

Schematics of lateral line neuromasts. (A) A superficial neuromast of a clawed frog (6). (B) A lateral line neuromast in a canal of ruffe (6). (C) A lateral line periphery of a teleost fish (4). Superficial neuromasts are represented by dots, and canal neuromasts are represented by dots inside shaded strips.

Development of the Artificial Lateral Line

The artificial lateral line consists of a monolithically integrated array of microfabricated flow sensors (13), with the sizes of individual sensors and spacings between them matching those of the biological counterpart. The sizes of individual sensors and intersensor spacings are ≈50 μm to 2 mm across biological species (14). The small dimensions of the sensors are important to ensure that they pose minimum interference with the flow field and with each other. An engineering equivalence of the lateral line sensing organ has not been made before. The making of miniaturized sensors in dense arrays is beyond the capabilities of conventional machining and commercial flow sensors. Fortuitously, advancements made in the area of microengineering in the past two decades now allow us to build sensors that match the sizes and mimic the functions and organization of the lateral line. Our artificial lateral line consists of a linear array of flow sensors, much like the array of neuromasts along the trunk of many fish. In the present version, each individual sensor within the array is based on the thermal hot wire anemometry (HWA) principle (15). The HWA sensors exhibit high sensitivity, small dimensions, and consequently, reduced interference to flow field.

We used a recently developed surface micromachining technique to fabricate arrays of miniaturized HWAs (13) that have dimensions on the same order of magnitude (tens to hundreds of microns tall) as biological neuromasts (Fig. 2). The hot wire does not reside in the substrate plane (i.e., the bottom of the boundary layer) but rather is elevated above the substrate by two prongs, just as some superficial neuromasts in fish are elevated above the skin surface by papillae (14). The sensor is first made in-plane by using photolithography. It is then assembled out-of-plane by using a 3D magnetic assembly method (16). The resultant elevation corresponds to the design length of the prongs defined by photolithography. Because of the use of a photolithography process, micromachined HWA sensors can have a prong length from 50 μm to 2 mm. For the present application, we focused on a wire length of 400 μm and an elevation of ≈600 μm. The hot wire consists of a nickel filament that is sandwiched by two layers of polyimide, which serve as passivation and structural support. The nickel hot wire exhibits a temperature coefficient of resistance (α) of 4,100 ppm/°C.

Fig. 2.

Schematic representation artificial lateral line. (A) Schematic of an individual microfabricated, out-of-plane HWA sensor used to build artificial neuromasts. The hot wire is elevated above the substrate surface by a prescribed distance. (B) Scanning electron micrograph of the artificial lateral line with 16 HWA sensors spaced 1 mm apart.

The largest array we have constructed so far consists of 16 HWA sensors with 1-mm spacing (Fig. 2B). Each sensor is monolithically integrated with dedicated complementary metal–oxide–semiconductor circuitry for on-chip signal conditioning, noise-floor reduction, and parallel data acquisition. Upon completion of this sensor array, the velocity sensitivities of the individual sensors were characterized by moving the array in quiescent water at various speeds. Under constant temperature mode with overheat ratio of 0.1, our measurements indicate a threshold of 200 μm/s and a bandwidth of 1 KHz (17). The sensor array was also calibrated in a water channel under the same conditions with inflow velocity up to 0.25 m/s, and a typical nonlinear trend for each sensor was observed (17). Further characterization showed that the drifts of zero outputs for all of the sensors were negligibly small over a testing period of 2 h.

Application on Dipole Source Localization

The developed artificial lateral line was used for hydrodynamic testing under biologically relevant scenarios. We first established the performance and functionality of our artificial lateral line by recording the spatial-temporal response to a nearby dipole source. The dipole is simple and yet ubiquitous in the underwater world. When a fish swims, its tail beat causes a dipolar near-field flow in addition to a wake behind it (10, 18, 19). Certain predators can accurately localize and attack a prey that is nearby (i.e., at a distance equivalent to one or two fish body lengths away) (9, 10, 20) solely by using the lateral line system to measure the dipole field associated with the prey.

A vibrating sphere was used to function as a dipole source (Fig. 3A). Theoretical prediction of pressure gradient felt by a canal lateral line in response to a nearby dipole has been made in the past (21, 22) and verified by neuro-physiological studies (9, 20). The spatial distribution of pressure gradient amplitude resembles a “Mexican hat” profile; the magnitude is the highest at the projected center of the dipole and diminishes gradually with increasing lateral distance from the center. Our artificial lateral line was able to record a profile well matched to the theoretical prediction (Fig. 3B), which was accomplished despite the differences between transfer functions of the biological canal neuromasts and HWAs. It should be noted that HWA sensors subjected to periodic oscillating flow sense water particle displacement (23), not velocity or pressure gradient. However, after normalization they all have the same spatial-distribution profile (21, 22). The HWA sensor poses another challenge; namely, it cannot discern the flow polarity. The amount of heat convected from the sensing filament of a HWA depends on the magnitude of water particle displacement for oscillating flows, not on its polarity. As a result, the HWA sensor provides a rectified reading of a complex oscillatory flow field (24, 25). To recover the signal rectification, we developed a derectification technique. It is based on reconstruction of Fourier series expansion of a signal, with initial phase angles among terms obeying analytically derived relationships, and amplitudes recovered by signal squaring method.

Fig. 3.

Characterization of dipolar near field and localization of the dipole source. (A) Analytical model (21, 22) of pressure contours (blue lines) and a linear array of lateral line canal neuromasts (in orange). (B) Comparison of experimental (green lines) and analytical (21, 22) (red line) results on displacement amplitude of water particles. (C) Time-elapsed spatial profiles of displacement amplitude with step-by-step translation of the dipole source along the artificial lateral line following path 1 diagramed in E. (D) Displacement profiles under step-by-step translation following path 2 indicated in E. (E) Comparison of actual paths (solid line with filled circles) and predicted ones (dashed line with empty circles).

Inspired by fish behaviors (9, 26, 27), we demonstrate that the spatially distributed response from the lateral-line array can be used to identify the exact location of a moving dipole source. We find that the location of a dipole source is encoded in the location and amplitude of the apex. A signal-processing algorithm based on maximum-likelihood analysis was developed (28). It compares the pattern of the signal received by the array with the expected pattern at all positions and selects the best match as the estimate of the actual dipole location. The algorithm can predict the location of the dipole even when the apex lies outside of the length of the lateral line. We illustrate the capability of this dipole tracking ability by using three representative cases. In all tests, the strength of the dipole source was maintained constant. The strength level does not affect the testing results much, as long as a proper signal-to-noise ratio at measurement locations is achieved. The dipole was confined to move within an area that covers two body lengths along the artificial lateral line, and one body length away from it, where body length represents the length of the sensor array. In the first case, the dipole source is translated stepwise parallel to the artificial lateral line (path 1 of Fig. 3E). The lateral output recorded at each step has identical amplitude with shifting apex (Fig. 3C). In the second case, the dipole source is moved perpendicularly away from the lateral line. The profiles of lateral line output flatten out when the dipole source fades into the distance (Fig. 3D). In a third case, the dipole traverses in a complex path in the plane of the sensor array (path 3, Fig. 3E). It is evident that for the three representative paths (Fig. 3E) predictions are accurate in most locations and generally become less accurate with increased distance from the sensor array.

Application on Hydrodynamic Wake Characterization

In addition to localizing a swimming prey in a dipolar near field, following the wake behind the prey can allow a predator to track and eventually localize the prey, starting at a much greater distance, i.e., a few to a few tens of prey-body length (11, 12). A functional lateral line is indispensable for following wake (11, 12).

The wake behind a swimming fish contains organized vortices (10, 29–31). Following this inspiration, we generated a turbulent wake by using a vertically mounted circular cylinder placed in water flow (Reynolds number = 5,000) (Fig. 4). The wake consists of alternately shed large-scale vortices known as a Kármán Street (32) (Fig. 4A). We found that by using an artificial lateral line one can identify the signature of a wake and the general direction of the source. Our artificial literal line was exposed to the wake to record the spatial distribution of local velocity fluctuations. To cover the desired size of the field of view (3.5D wide and 6D deep, with D being the diameter of the cylinder), we traversed the sensor array across the wake and stitched multiple images (Fig. 4A).

Fig. 4.

Wake signatures for tracking a source. (A) Schematic showing experimental set-up. (B) The pattern of rms water velocity in the wake of a cylinder. (C) The pattern of peak water velocity at vortex shedding frequency in the wake of a cylinder. Both rms and peak water velocities were normalized by free-stream inflow velocity.

The distribution of rms velocity fluctuation (Fig. 4B) shows the that lateral line array is capable of capturing the main feature of the wake (33): dramatic dual peaks with a valley in between in the near wake, and decreasing intensity of the peaks further down stream.

The velocity fluctuation was determined by using a second method that accentuates peak features with even greater contrast. This method uses fast Fourier transform to obtain spectral distribution of a signal. Then the flow velocity amplitude at a characteristic frequency associated with the wake-generating source, e.g., the vortex shedding frequency from the cylinder, is extracted (Fig. 4C). As a result, two clearly defined peaks occurred along the entire field of view. We conjecture that this algorithm lowers the background signal by rejecting broadband noise in the fluid.

Discussion

The current study illustrates the potential of biomimetic sensing of artificial lateral line-equipped underwater vehicles. In a near distance, for example, the capability of a lateral line on dipole source localization enables a fish to capture a prey or evade a predator by solely discerning the dipolar near field generated by others. By demonstrating the same capability of our artificial lateral line, it is convincing that with further engineering advancement, manmade underwater vehicles will be able to image hydrodynamic events from surroundings, like fish do. Such hydrodynamic events can be caused by aquatic animals or other vehicles. With advanced algorithms and training methods, this capability can even be extended to general water disturbances other than an idealized dipole source.

From a distance away, fish can still rely on their lateral lines to track a target by detecting the target's hydrodynamic trail. Our artificial lateral line demonstrates this capability, and the trail is not limited to a wake generated by a cylinder only. Instead, it can be any kinds of trails, e.g., a wake behind a propeller-driven submarine. However, different trails might have different features, thus they may require different data processing techniques to obtain a sharp definition of the features, based on measurements of the artificial lateral line. For example, for a complex wake with no dominant feature frequencies, a rms evaluation of the fluctuating velocities might be a good measure (see Fig. 4B). Whereas with a dominant frequency, the wake might be better defined by extracting this frequency component based on spectral analysis (see Fig. 4C), which is especially essential when the aquatic environment is contaminated with hydrodynamic noises.

However, the artificial lateral line discussed in the foregoing differs from its biological counterpart in many aspects. Structurally, a real lateral line consists of numerous superficial and canal neuromasts, with each neuromast having a bundle of hair cells capsulated in a cupula (34) to function as a flow sensor. The artificial lateral line, on the other hand, has only a limited number of superficially placed HWAs to serve this purpose. For information collection, aquatic animals acquire firing frequencies from each neuromast (3) to represent the strength of local flow; whereas the artificial lateral line records magnitude of voltage output from individual HWAs. For decision making, aquatic animals use a back-propagation learning algorithm (35); comparatively, the artificial lateral line uses the minimum mean-squared error method (28). There is no doubt that in many ways a real lateral line, after millions of years of evolution, is superior to the artificial lateral line presented herein. Nonetheless, this artificial lateral line enables biological behaviors to be realized, such as localizing a vibration source and detecting a hydrodynamic wake.

Future work should involve both close studies of biology and further development of engineering. Biological studies are needed at different levels, including the organism level, sensor level, and cellular level. These studies may involve different aspects, including behavior studies, neurophysiology studies, fluid mechanics, and morphological studies. Significant engineering efforts are also necessary to improve the performance and prove functions in a noisy environment. Engineering development in the areas of advanced materials, sensor development and integration, signal processing, control, and robotics is needed in the future.

In summary, the proof-of-concept biomimetic study shows that the developed artificial lateral line is able to localize an underwater vibrating source in a near distance and is able to detect a hydrodynamic wake for long-distance tracking. The accomplishments are of significant importance in various underwater applications. Being equipped with such an artificial lateral line system, a submarine is expected to be capable of detecting and tracking other moving underwater targets and to be capable of collision avoidance. An artificial lateral line is especially indispensable when vision and sonar are limited, such as in dark or murky environments, in sonar blind-zones, or when the sonar system is in passive mode for concealment purposes. Similarly, by applying the technique to other autonomous underwater vehicles, this augmented hydrodynamic-imaging capability will enable safer and more flexible navigation performance.

Methods

Fabrication.

The fabrication process of the artificial lateral line involves surface micromachining, a technique commonly used in microelectromechanical systems to create freestanding cantilevers (36). The micromachining process is performed on a silicon wafer with preexisting analog integrated circuitry. Surface micromachining is used to define the HWA, including a nickel-iron alloy support prong and a nickel-polymer composite hot-wire sensing element. The planar surface micromachining process is followed by a magnetically assisted assembly step that rotates the cantilevers out of plane. With this process, many devices can be bent out of plane in parallel with high yield (13). Further, the sensor elements can be monolithically integrated with signal processing electronics, as the process is performed at room temperature and would not cause incompatibility with elements of electronics circuits. At the end of the process, the hot wire may be encapsulated by a conformally deposited, 2-μm-thick Parylene film for waterproofing and further structure strengthening.

Dipole Experiments.

A minishaker (model 4010; B&K, Norcross, GA) was fixed to a motorized three-axis linear stage system (model 8MT175; Standa, Vilnius, Lithuania) equipped with computerized motion control through step motors. A sphere of 3 mm in diameter was attached to the minishaker through a 12-gauge needle that served as the dipole source. An accelerometer (model 352B10; PCB, Depew, NY) was attached to the base of the needle to measure the acceleration of the dipole. The artificial lateral line, the packaged HWA array, was mounted on a test fixture rigidly attached to the base of the water tank through a suction cup. For all experiments, arrayed HWA sensors were aligned parallel to the vibration axis of the sphere (i.e., the dipole source), which was operated in sinusoidal mode by the minishaker at a frequency of 75 Hz with displacement amplitude of 0.4 mm.

Wake Experiments.

A desktop water tunnel (model 501; ELD Inc., Lake City, MN) with a test section of 150 × 150 mm was used. A cylinder of 25 mm in diameter was vertically fixed in a uniform current at a speed of 0.2 m/s. The corresponding Reynolds number was ≈5,000. The artificial lateral line was exposed in the wake behind the cylinder, with arrayed HWA sensors perpendicular to the inflow and to the axis of the cylinder. The printed circuit board holding the substrate where sensors resided was tilted at an angle of attack of 5° to suppress flow separation from the leading edge on the sensor side. To correct sensitivity difference among sensors and the nonlinear response of each individual sensor, the sensor array was calibrated under the same condition without the cylinder.

Glossary

Abbreviation

HWA

hot wire anemometry.