ECOPAMPA: A new tool for automatic fish schools detection and assessment from echo data

Abstract

Accurate identification of aquatic organisms and their numerical abundance calculation using echo detection techniques remains a great challenge for marine researchers. A software architecture for echo data processing is presented in this article. Within it, it is discussed how to obtain energetic, morphometric and bathymetric fish school descriptors to accurately identify different fish-species. To accomplish this task it was necessary to have a development platform that allowed reading echo data from a particular echosounder, to detect fish aggregations and then to calculate fish school descriptors that would be used for fish-species identification, in an automatic way. This article also describes thoroughly the digital processing algorithms for this automatic detection and classification, as well as the automatic process required for surface and bottom line detection, which is necessary to determine the exploration range. These algorithms are implemented within the ECOPAMPA software, which is the first Argentinean system for marine species identification. Finally, a comparative result over experimental data of ECOPAMPA against Echoview^TM Software Pty Ltd (formerly Myriax Software Pty Ltd), is carefully examined.

Hydroacoustics, Echo data, Fish schools, Digital image processing.

1. Introduction

Since hydroacoustics can provide a continuous high resolution sampling of a large volumes of water over a short period of time, compared to other sampling technologies such as electromagnetics and optics, it is currently the most efficient tool to remotely study the aquatic environment (Simmonds and MacLennan, 2006). Today it is possible to generate a great amount of data during an acoustic survey, usually several GBytes per day. For this reason the extraction of useful information is delayed for a post-processing analysis stage. In addition, it is necessary to design new tools to speed up the scrutinizing process of these acoustic data. During a regular fish stock assessment it is possible to detect hundreds or thousands of fish schools. Therefore automatic detection of echotraces becomes a very valuable request. In a recent past, a profuse literature has been published describing the results of applying image processing to recognize fish schools silhouette's automatically, allowing the assessment of interesting information about size, shape, structure and position in the water column of the aggregations (Coetzee, 2000; Diner, 2001; Lefort et al., 2012; Scalabrin and Massé, 1993).

The ECOPAMPA, presented in this article, is a software tool to scrutinize synthetic echograms packed into different proprietary formats (Blondel, 2009; Blondel and Murton, 1997). ECOPAMPA has been promoted by the “National Institute for Fishery Research and Development” (INIDEP) and designed at INTELYMEC – CIFICEN Research Group, Faculty of Engineering at Universidad Nacional del Centro de la Provincia de Buenos Aires (UNCPBA). This software incorporates innovative algorithms for automatic detection of the sea bottom, surface and fish schools echo data based on Digital Image Processing (DIP). This suite of tools has been designed to extract and export fish school descriptors into a database for further analysis. Besides, this software allows classifying fish schools using Artificial Neural Network (ANN) (Cabreira et al., 2009).

This article is organized as follows: in Section 2 we describe materials and utilized methods. In Section 3 we detail in the methods for automatic detection of fish schools, surface and bottom lines, extraction of descriptors and classification for fish-species identification. In Section 4, we show experimental results, and compare them against those obtained using Echoview^TM Software Pty Ltd (formerly Myriax Software Pty Ltd). Finally, in Section 5 we discuss and conclude on expected perspectives from this work.

2. Material and methods

The proposed architecture for ECOPAMPA software is shown in Figure 1. This approach consists of a group of sequential procedures based on Digital Image Processing (DIP), including image segmentation, morphological operations, edge detection, image representation, Binary Large Object (BLOB) analysis and classification with ANN (Gonzalez and Woods, 2001; Haykin, 2009; Pratt, 2001; Qiusheng et al., 2014; Russ, 2000). As seen in Figure 1, these sequential procedures are applied to files containing acoustic data recorded by a digital echosounder. These data within each file are arranged as a set of telegrams with detailed survey information, such as echosounder settings, acquisition date and time, vessel location (latitude and longitude) and scatter recordings, among others.

Figure 1.

Flowchart of proposed architecture for image processing in ECOPAMPA software.

As a first step, several acoustic data files are uploaded to recover observations collected in sampling missions. They were obtained by INIDEP along several surveys carried out for fish stock assessment over an area above 1.000.000 km², between latitudes 35ºS and 56ºS, and from the coastal zone to the continental slope. The echo data that include backscattering strength $S_v$, were obtained using several SIMRAD (EK500 system) digital echosounders (Simrad AS, 2001, 1996), each one on board of different fishing vessels, and saved by means of a Bergen Echo Integrator (BEI) software (Foote et al., 1991; Korneliussen, 1993, 2004) in two formats, BEI file sets (∗.ping) and raw data files (∗.dgx files). Once the acoustic files were obtained, in a second step, an echo strength transformation (see Figure 1) is performed. This process transforms an echo data sequence (stored in a matrix) into digital images or synthetic echograms for visualization and for the processing described in next Section 3. Visualization will be explained at the end of this section. From the synthetic echogram, we define the echo signal strength$f\left(p\right)$, as a function of the echo depth $p$, for a fixed echosounder position. On survey campaign navigation, successive echo data are stored along the track. Consequently, it is possible to define a two-dimensional function $f(d,p)$, where $d$ stands for distance (traveled by the vessel) between consecutive echo data samples. The variables $d$ (distance along the track in centimeters, represented by an integer), $p$ (echo depth in centimeters, represented by an integer) and $f$ (echo strength, represented in dB) at each point $(d,p)$ are in this way discretized to be processed as follows.

From $f(d,p)$ the digital image is represented as a two-dimensional intensity function $I(x,y)$, where $x$ and $y$ are the spatial discrete coordinates in the plane defined along the survey track downwards. The intensity level $I$ at each point $(x,y)$ gets discrete values in the range [0, 255] and represents a pixel color (Gonzalez and Woods, 2001). Thus, digital echo data $f(d,p)$ are linearly transformed into a digital image $I(x,y)$ mapping from echo intensity (in decibels - dB) values, to pixel intensities at different positions and depths. This mapping process can be either binary or multiple. On one hand, it is binary when the intensity is separated into two different value ranges, below and beyond a single threshold (see Section 3.A). On the other hand, it is multiple when the intensity is divided into $n+1$ value ranges, using $n$ thresholds.

In this work, binary transformation is used during echo data processing for contours determination. In other words, a single threshold was used for surface line, bottom line and fish schools contour detections. Conversely, multiple transformations are used to generate a digital image for visualization. In particular, to obtain the synthetic echogram for final identification and classification, we divided the echo signal strength $f(d,p)$ into twelve color categories, from grey corresponding to the weakest signal to brown equivalent to the strongest (see Figure 2). The scale is logarithmic with a 3 dB step between each color, with a color scale range of 36 dB, mapping from the weakest to the strongest echo signal (Simrad AS, 2001, 1996).

Figure 2.

Processing of a typical Anchovy school echogram: (a) raw synthetic echogram; (b) processed synthetic echogram with surface line, bottom line and fish school contour detection in blue.

3. Digital echo data processing

This software architecture of Figure 1, was employed to on-line analyze echo data for the detection of the study objects. Surface line, bottom line and fish schools were automatically recognized, and displayed into the echogram, as will be described in Section 3.A and Section 3.B. Finally, a detailed digital image was produced, like the one in Figure 2 showing a typical echogram for an anchovy school (Engraulis anchoita). Figure 2 (a) plots an example of synthetic echogram before applying any object recognition process. This echogram was acquired with a SIMRAD EK500 echosounder. Figure 2 (b) shows the recognition results for surface line, bottom line and fish school contour. The detected objects contours are outlined in blue color.

3.1. Surface and bottom line detection

The shapes found in the echograms are highly irregular in general. Then, effective and robust detection processes are required to achieve reasonable classification performance. Water surface, sometimes referred as first echo line, and sea bottom, also referred as second echo line, are needed to determine the range to explore and then to detect the fish schools. In addition, efficient detection processes lead to computational time optimization. Therefore, accurate surface and bottom line detection constitutes a fundamental task for fish school descriptors assessment required for precise fish-species identification. Surface and bottom line detection (as presented in Figure 1) requires similar DIP chains (binary thresholding, opening, substraction and chain code), some of them with different parameter settings. Even though they are standard image processing procedures, they are briefly described in what follows for clarity.

3.1.1. Binary thresholding process

In the context of DIP, binary thresholding process is used to extract study objects from background by selecting a threshold $T$ (Gonzalez and Woods, 2001; Pratt, 2001; Russ, 2000). In this work we used $T=70dB$, as recommended by the experts from INIDEP. This fixed value has been used across this study wherever significant signal has to be distinguished from background. It has been used to identify bottom and surface lines as well as fish schools contours. In this way, echo data intensities $f(d,p)$, arranged in the Echo Data Matrix, are compared against this threshold and separated in two intensity values 0 (below) and 1 (above), according to Eq. (1).

$$g\left(d,p\right)=\{\begin{array}{l}1iff(d,p)>T\hfill \\ 0ifI(d,p)\le T\hfill \end{array}$$ (1)

These intensities are used to make the following classification: the group with intensity 1 is considered as object and the group with intensity 0 is considered as background. Note that the parameter p in Eq. (1) is constraint to a certain range to detect the surface and it is constraint to another range to detect the bottom. These ranges are predefined by expert users, according to the case study. Applying Eq. (1) to the $dxp$ elements of the Echo Data Matrix yields the thresholded echo data image $g\left(d,p\right),$ of the same size.

3.1.2. Opening process

In DIP the opening process consists of two binary morphological sequential operations called Binary Erosion and Binary Dilatation (Gonzalez and Woods, 2001; Pratt, 2001; Russ, 2000). Both morphological operations are defined in set theory (Heijmans Henk, 1994).

Binary Erosion operation “shrinks” or “thins” objects in binary image. In order to control this shrinking process on a digital image $A$, a structuring element $B$ is necessary. The mathematical definition of erosion of $A$ by$B$, denoted $A\ominus B$, is defined as:

$$A\ominus B=\left\{Z|{\left(B\right)}_Z\cap A^c\ne \varnothing \right\}$$ (2)

where $A$ is the binary digital image in $Z$ (set of integers), $B$ is the structuring element, $A^c$ is the set of all values coordinates that do not belong to set $A$, and $\varnothing$ is the empty set. In other words, erosion of $A$ by $B$ is the set of all structuring element origin locations where the translated $B$ has no overlap with the background of $A$.

Binary Dilatation operation “grows” or “thickens” objects in a binary image. The manner and extent of thickening is also controlled by a structuring element. The mathematical definition of dilatation of $A$ by $B$, denoted $A\oplus B$, is defined as:

$$A\oplus B=\left\{Z|{\left(\hat{B}\right)}_Z\cap A\ne \varnothing \right\}$$ (3)

where $A$ is the binary digital image in $Z$, $B$ is the structuring element and $\hat{B}$ is the reflection of the structuring element B. Hence the dilatation of $A$ by $B$ is the set consisting of all the structuring element origin locations where the reflected and translated $B$ overlaps at least some portion of $A$. The translation of the structuring element in erosion and dilatation is known as the mechanics of spatial convolution (Heijmans Henk, 1994).

The morphological opening operation of $A$ by$B$, denoted$A\circ B$, is simply an erosion operation of $A$ by$B$, followed by a dilatation operation of the result by$B$:

$$A\circ B=(A\ominus B)\oplus B$$ (4)

Morphological opening removes completely regions of an object that cannot contain the structuring element, smoothes object contours and removes small irregularities.

Hence, if $g\left(d,p\right)$ is the pixeled image, equivalent to $A$ in Eq. (4), and $B$ is a structuring element, the opening operation leads to the digitalized image $h(d,p)$. Generally, the structuring element consists of a digital image of size $3x3$ with all its values equal to 1.

3.1.3. Substraction process

In DIP the subtraction operation (Gonzalez and Woods, 2001; Pratt, 2001; Russ, 2000) consist of subtract (pixel by pixel) two digital images of the same size. This operation provides the edge or contour of the objects under study. The substraction operation is defined as:

$$s\left(d,p\right)=\left|g\left(d,p\right)-h(d,p)\right|$$ (5)

where $g\left(d,p\right)$ is the thresholded echo data image and $h(d,p)$ is the resulting digitalized image after morphological opening process (see Figure 1, boxes 5 and 9). As a result, the digital image $s(d,p)$ is obtained by applying substraction operation.

3.1.4. Chain code process

In DIP the image representation consists of generating the chain code of exhaustive description of either surface or bottom edges. Chain codes are one of the shape representations which are used to stand for an edge by a connected sequence of straight line segments of specified length and direction. This representation is based on 4-connectivity or 8-connectivity of the segments (Gonzalez and Woods, 2001). These numerical representations are known by Moore neighborhood or Freeman code (Freeman, 1961). A coding scheme for line structure must satisfy three objectives: to preserve the information of interest; to permit compact storage and convenient display; and to facilitate any further processing.

Therefore, the chain code process (with 8-connectivity in our case) is applied to get mentioned edges, and then stored in the database shown in Figure 1.

3.2. Fish schools detection

The software design to accurately detect fish schools consists of applying processes chain in a similar way as for surface and bottom line detection (see Section 3.A). Note that some processes applied in Section 3.A are reused.

3.2.1. Inferior surface and superior bottom process

This process calculates the inferior point on surface echo line and superior point on bottom echo line. Both points delimit space to fish schools range search, mainly to reduce computational effort. This process is shown as a new stage in our proposed architecture (see Figure 1, box 11). Therefore, using these two points as echo data boundaries allows defining a useful Region Of Interest (ROI) to apply the following processes.

3.2.2. Binary thresholding process

This process, already described in Section 3.A, is applied to $f\left(d,p\right)$ with parameter$p$ varying in the ROI, generating again a thresholded echo data image $g\left(d,p\right)$.

3.2.3. Opening process

This morphologic procedure is used to remove small fish schools irregularities (detailed in Section 3.A), and hence leading to smooth fish schools edges.

3.2.4. Binary Large Object (BLOB) analysis

The BLOB analysis (Qiusheng et al., 2014) is intended to extract objects from a binary digital image. BLOB refers in general to a group of connected target pixels in a binary image. In this work we input BLOB process with the digitalized image $h\left(d,p\right)$ (opening process output) and obtain an output list of fish schools$b_1,\dots ,b_k$, where $k$ represents the number of fish schools detected in an echo data matrix.

There are three commonly used alternatives for applying BLOB analysis: multi-scan algorithms (Suzuki et al., 2003), two-scan algorithms (He et al, 2008, 2009; Wu et al., 2009) and one-scan algorithms (Chang et al., 2004; Qiusheng et al., 2014; Zenzo et al., 1996). In this work we use a BLOB analysis algorithm based on RLE (Run Length Encoding) (Qiusheng et al., 2014) due to its evident computational efficiency. The algorithm consists of three steps: (1) transform the image using the RLE; (2) judge the connectivity between two adjacency runs and (3) organize the runs using a number of RLE-linked lists, manage all the RLE-linked lists with a single BLOB-linked list, and extract connected region features. After finishing these three steps, the following data relationships can be constructed. First, a single BLOB-linked list which represents all the connected regions in a binary image. Second, the runs of a connected region are allocated to a unique data field which represents the connected region. Finally, all the runs are organized by the RLE-linked lists, and all the RLE-linked lists are regulated by a single BLOB-linked list. This method reduces the time and computational resources as detailed in (Qiusheng et al., 2014).

The area of a BLOB is defined as the total amount of pixels in the BLOB. Hence, in order to recognize BLOBs as fish schools, BLOB's area should be extracted and judged whether it falls within a given range $[Min,Max]$ or not. BLOBs, whose area is out of range, will be ignored. The fish schools can be then identified by the following expression:

$$BLOBStatus=\{\begin{array}{cc}SCHOOL,& Min\le Area\le Max\\ ignored,& otherwise\end{array}$$ (6)

where $Min$ and $Max$ (minimum value and maximum value) are the appropriated range area, which are selected by the user.

The center of target-connected region is returned as BLOBs location. Center coordinates are calculated as the average $\overline{X}$ and $\overline{Y}$ for object location inside binary image:

$$\overline{X}=\frac{1}{m}\sum _{i=1}^m{x}_i$$ (7)

$$\overline{Y}=\frac{1}{m}\sum _{i=1}^m{y}_i$$ (8)

where $m$ is the number of pixels in the BLOB, and $x_i$ and $y_i$are their respective coordinates.

3.2.5. Binary erosion process

Erosion process is applied to the list of images provided by the BLOB analysis. This process, detailed in Section 3.A, aims to reduce the outline of each fish school. Note that the erosion process is applied only to the specific ROI of each fish school, reducing then the use of computational resources. Hence, we applied Eq. (2) for the binary erosion process, setting $A$ as each image in the list provided by previous BLOB analysis $b_1,\dots ,b_k$ and setting $B$ as the structuring element (again with size $3x3$ and elements’ values equal to 1). We obtained as a result, the eroded image list $e_1,\dots ,e_k$.

3.2.6. Subtraction process

This process subtracts (pixel by pixel) two digital images of the same size (as was detailed in Section 3.A). We applied Eq. (5) to subtract $e_i$ (the erosion process output) from $b_i$ (the BLOB analysis output) to obtain a list of digital images (schools)$s_1,\dots ,s_k$.

3.2.7. Chain code process

chain code process (detailed in Section 3.A) has been also applied to the images in the list$s_1,\dots ,s_k$ provided as an outcome of the substraction process. This process scans echo data to generate a border chain code for each fish school. Then, the resulting chain code and its central position (assessed using Eqs. (7) and (8) on the corresponding ROI) for each fish school are stored into a database. This chain code is also used to display the edges of each fish school on the digital image (all the steps described in the above paragraphs, are illustrated in Figure 4 in the results section).

Figure 4.

Digital echo data processing using ECOPAMPA software: (a) digital echo data displayed as a synthetic echogram; (b) binary thresholding; (c) binary erosion; (d) binary dilatation; (e) edge detection; (f) display of fish school detection (Villar et al., 2014).

3.2.8. Extraction of fish school descriptors

This process consists of calculating different measurements using the outline of each fish school detected together with its original echo data and location in the water column. This method is closely related to BLOB analysis and edge detection processes since accurate fish school contours lead to precise descriptors assessment. Finally, fish school descriptors are also stored in a database and used afterwards to precisely determine associated species.

In Table 1 we show a quantitative list of fish school descriptors determined using the ECOPAMPA software. These descriptors were grouped into three main categories: energetic, bathymetric and morphological. Morphological measurements length ($L$), area ($A$), height ($H$) and perimeter ($P$) were corrected for beam width effects as explained in (Diner, 2001), considering the pixel resolution. In Table 1 we show corrected descriptors ($L_c$, $A_c$, $H_c$ and $P_c$) and their dependencies on raw measurements. The remaining parameters in Table 1 are also defined in (Diner, 2001).

Table 1.

The main fish school descriptors computation for the ECOPAMPA software.

Fish school descriptors	Symbol	Computations	Units
Energetic
Volume-backscattering strength	$S_v$	$-$	$dB$
Maximum volume-backscattering strength	$S_{vmax}$	$-$	$dB$
Vertical roughness	$VR$	$-$(7.7) and (7.8) of (Zar, 1984)	$dB$
Horizontal roughness	$HR$	(7.13) and (7.15) of (Zar, 1984)	$dB$
Skewness	$Skew$		$-$
Kurtosis	$Kur$		$-$
Morphometric
Length	$L_c$	$L_c=\left[L-2D\text{tan}(f/2)\right]$	$m$
Height	$H_c$	$H_c=H-ct/2$	$m$
Perimeter	$P_c$	$P_c=P-2\left[\left(L-L_c\right)+\left(H-H_c\right)\right]$	$m$
Area	$A_c$	$A_c=A\left(L_c{H}_c\right)/\left(LH\right)$	$m^2$
Volume	$V_c$	$V_c=L_c{(H_c/2)}^2$	$m^3$
Fractal dimension	$FD$	$FD=2\text{ln}(P_c/4)/\text{ln}\left(A_c\right)$	$-$
Elongation	$EL$	$EL=L_c/H_c$	$-$
Image compactness	$IC$	$IC=P^2/\left(4\pi A_c\right)$	$-$
Rectangularity	$Rec$	$Rec=\left(LH\right)/A$	$-$
Circularity	$Cir$	$Cir=P^2/\left(\pi A\right)$	$-$
Bathymetric
School depth	$Z_c$	$-$	$m$
Bottom depth	$Z_f$	$-$	$m$
Altitude index 1	$AL{T}_1$	$AL{T}_1=(Z_c+H_c/2)/Z_f$	$m$
Altitude index 2	$AL{T}_2$	$AL{T}_2=Z_f-(Z_c+H_c/2)$	$m$

In order to calculate energetic descriptors for detected fish schools, the original Echo Data Matrix is also taken into account (see the information flow to box 18 in Figure 1). Finally, for bathymetric descriptors we locate fish school's center to place its position in the water column (estimated using Eqs. (7) and (8)), with respect to surface and bottom lines.

In the next Figure 3, we illustrate some of these morphological and bathymetric descriptors calculated using the ECOPAMPA software, such as, length ($L_c$), height ($H_c$), area ($A_c$), perimeter ($P_c$), school depth ($Z_c$), bottom depth ($Z_f$) and altitude index ($AL{T}_2$).

Figure 3.

Morphological and bathymetric descriptors calculated by the ECOPAMPA software.

3.3. Fish schools classification

The selection of descriptors and classification methods employed in ECOPAMPA was based on a previous work (Cabreira et al., 2009). In that work, three types of ANN models were tested: Self-Organizing Mapping (SOM), Multilayer Perceptrons (MLP), and Radial Basis Network (RBN) (Haykin, 2009; Kohonen, 2001). The fish school descriptors, presented in the previous section, were used as inputs to these ANNs to classify five types of species: 1. Argentine anchovy (Engraulis anchoita), which are divided into two stocks: northern and southern stocks; 2. rough scad (Trachurus lathami); 3. sprat (Sprattus fuegensis); 4. longtail hoki (Macruronus magellanicus) and 5. blue whiting (Malacosteus australis).

The dataset used for training and testing of ANN contains 6400 records, one record per fish school, from the total of the six analyzed species/stocks. There were 4100 records of Argentine anchovy (Engraulis anchoita), which were divided between two stocks: 2999 and 1101 for the northern and southern stocks respectively; 168 records of rough scad (Trachurus lathami), 279 of sprat (Sprattus fuegensis), 1768 of longtail hoki (Macruronus magellanicus), and 124 of blue whiting (Malacosteus australis). The dataset was partitioned into two parts; one is used for training and the remainder is used for testing the ANN. The best performance was achieved with a partition of 70:30% for training vs. testing. Three different trials were done: (a) all available fish school descriptors, including temporal and geographical, were used as inputs; (b) all energetic, morphometric, and bathymetric descriptors were used as inputs; and (c) only selected energetic and morphometric descriptors were used as input (the mean volume-backscattering strength $S_v$ and school height $H$). Each classification trial was done for the three types of ANNs (SOM, MLP, and RBN). In each trial, the input vector was the same.

4. Experimental results

ECOPAMPA was implemented in C# using Visual Studio 2010. Data structure within OpenCV 2.3 (Bradski and Kaehler, 2008) have been used to optimize processing. The relational database system used was MySQL version 5.5.16. Besides, in the Appendix we show pseudocode algorithms for surface, bottom line and fish schools detection and classification.

A subset of available echograms has been used to test the developed software. Figure 4 shows a synthetic echogram, shown as a digital image in 12 colours, to which we applied the sequence of processes detailed in Section 3. In Figure 4 (a) we can appreciate the synthetic echogram of a typical anchovy school (Engraulis anchoita). The processing chain leading to these images comprises: thresholding (Figure 4 (b)), opening, i.e. erosion (Figure 4 (c)) and dilatation (Figure 4 (d)), object edge detection using subtraction process (Figure 4(e)) and final exposure (Figure 4 (f)). In Figure 5 we show a comparison between results obtained by means of Echoview^TM school detection (Anon, 1993, 1999; Barange, 1994; Nero and Magnuson, 1989; Scalabrin et al., 1996; SonarData, 2005; Weill et al., 1993) and ECOPAMPA module. Several echo data have been used to prove detection algorithms for both software methodologies. Echo-records used in the analysis have been previously corrected applying $20\text{ log }\left(\text{R}\right)$ time varied gain procedure. A binary threshold ($T$) of $-70dB$, as recommended by INIDEP staff and the sonar-devices manufacturers themselves, has been used to separate useful information from background noise. Since all the images have been collected on similar meteorological conditions, by means of the same sonar-devices, the same $T$ value has been proved valid for all the images in this study. EchoView^TM software has been set up to use “school detection” algorithm, with the same threshold. As can be clearly appreciated in Figure 5 (a and c), despite the same threshold ($T$) has been applied, there is a small deviation between the schools detected by ECOPAMPA and EchoviewTM. The mentioned difference comes from the digital image processing used in our approach, which leads to a more accurate definition of schools. This methodological difference impacts mainly in the resulting perimeter, as can be appreciated in Table 2.

Figure 5.

Comparison of detection algorithms between EchoView™ and ECOPAMPA software.

Table 2.

Comparison of calculated fish school descriptors between Echoview™ and ECOPAMPA software using fish schools shown in Figure 5.

Fish School Descriptors	(a)		(b)		(c)
	ECOPAMPA	Echoview	ECOPAMPA	Echoview	ECOPAMPA	Echoview
Energetic
Volume-backscattering strength ($S_v$)	-37,529	-37,350	-37,259	-37,150	-35,481	-34.490
Maximum volume-backscattering strength ($S_{vmax}$)	-22,248	-22,247	-22,059	-22,059	-20,319	-20,319
Vertical roughness ($VR$)	0,00035	0,00035	0,00035	0,00035	0,00036	0,00036
Horizontal roughness ($HR$)	0,00135	0,00137	0,00035	0,00035	0,00179	0,00180
Skewness ($Skew$)	6,13104	6,07242	5,29112	5,16035	4,17147	4,16427
Kurtosis ($Kur$)	51,9423	52,6381	50,2412	49,9377	25,3253	26,4168
Morphometric
Length ($L_c$)	70,251	61,951	95,1758	95,091	64,803	58,170
Height ($H_c$)	17,3	18,850	32,7	33,650	30,5	26,050
Perimeter ($P_c$)	374,674	319,678	513,852	549,973	495,079	349,147
Area ($A_c$)	778,316	778,262	1610,21	1684,835	1196,05	1047,876
Volume ($V_c$)	5226,043		25364,9		15021,56
Fractal dimension ($FD$)	1,363		1,315158		1,359
Elongation ($EL$)	4,072		2,915033		2,128
Image compactness ($IC$)	15,445		13,54684		17,101
Rectangularity ($Rec$)	1,556		1,929872		1,649
Circularity ($Cir$)	54,370		50,78139		61,798
Bathymetric
School depth ($Z_c$)	28,4	28,692	18,6	17,469	22,8	22,178
Bottom depth ($Z_f$)	46,95		48,54
Altitude index 1 ($AL{T}_1$)	0,788485		0,719473
Altitude index 2 ($AL{T}_2$)	9,93215		13,6175

In Table 2 we show a list of quantified fish school descriptors used in both software. The descriptors and assessments of EchoView^TM software are detailed in (Lawson et al., 2001).

In terms of the quantitative assessment of the results shown in Table 2, when the three samples were compared regarding the energetic descriptors, they exhibit a small variation between EchoView and ECOPAMPA software. This is in the range of a minimum of 0% and a maximum of 2% in average values.

Otherwise, when compared with the morphometric fish school descriptors, these samples present a greater variation (6% and 21% minimum and maximum values on average, respectively). These variations are due to the application of different school detection algorithms. Finally, when they were compared with the bathymetric fish school descriptor, again a small variation in the range [0%, 6%] of average values was observed. The computation of these values depends on the calculation of the center point of the school. Keep in mind that the energy, morphometric and bathymetric descriptors that cannot be calculated with the EchoView software are shown without value in Table 2.

A global success percentage rate measurement, considering one percentage for each ANN and each trial, is summarized in Table 3. It is important to note that the ANN gave the same classification output regardless the input descriptors came from one software or the other.

Table 3.

Summary of the ANN performance in each conducted trial (Cabreira et al., 2009).

Trial	ANN	Average	Minimum	Maximum
(a)	FF	97.99	97.05	98.50
	SOM	96.63	96.32	96.93
	RBN	98.67	97.05	99.58
(b)	FF	84.10	83.27	84.98
	SOM	82.01	81.05	83.32
	RBN	84.43	83.64	84.88
(c)	FF	67.88	66.81	69.54
	SOM	68.73	67.74	69.60
	RBN	66.41	65.87	67.68

As it may be seen in Table 3, the best performance was obtained using all available fish school descriptors as input for the ANNs, in trial (a). Taking into account that most of the considered species inhabit non-overlapping areas and that surveys targeting a particular species are always done in the same season, it is expected that a good proportion of the successful species identification will be provided by the geographical and temporal input rather than by the fish school characteristics themselves. In order to investigate this further, and to classify species independently of the geographical and seasonal information, a trial (b) was done considering all energetic, morphometric, and bathymetric descriptors, but intentionally excluding the geographic position, time, and date. That improvement would allow a world-wide application of this tool. As expected, the overall performance decreased to some extent but it is still satisfactory for classification purposes.

As a final challenge, it would be interesting to classify the schools species even ignoring depth detection knowledge. This further relaxation would allow to detect moving schools towards deeper or shallower waters due to climate change impacts or due to water temperature changes. Hence, the trial (c) comprised only selected energetic and morphometric descriptors as the input for the ANNs and resulted in a further decrease of the overall performance. The average performance obtained for trials (a) and (b) may be considered as satisfactory for the simultaneous identification of six different kind of fish schools. In addition, training geographically delocalized and season independent ANNs (trial b) also enables us to improve school classification in overlapping species regions.

Another conclusion from these trials is that most of the failures in identifying a given species result from false identifications of other species more frequently found in the input dataset, i.e. those having more records in the database and thus, used the most for the ANNs training. In our case, this is the Bonaerensis anchovy stock.

5. Conclusion

The ECOPAMPA software presented in this work constitutes a useful and verified application that is aided to identify and to classify fish schools of commercial interest. In this way, it is an important tool to decide whether or not it is worth to fish at a certain place in a certain time, in order to prevent overfishing, since ECOPAMPA is prepared and able to operate on-line in real time. Therefore it contributes to elaborate politics aimed to achieve sustainable fishing industry, as well as to the monitoring and the evaluation of aquatic ecosystems. This software allows users to easily display, explore and process acoustic data. Furthermore, resorting to key descriptors and classification methodologies with artificial neural networks, it leads to automatic identification of different fish-species and its population assessment. Hence, it constitutes a novel assembly of image recognition, features assessment and category classification to make available a very useful support tool and pave the way to human decision-making regarding the fishing industry.

This work is also a good demonstration for both image and acoustic data processing to enhance image quality and to produce intelligible pictures. Effectively, ECOPAMPA allows obtaining high quality images from echosounders outputs. In this sense, the work presented here may be seen as a general framework and architecture for real time processing of acoustic data. Different combinations of processing algorithms, fish school descriptors, and classification approaches also may be used for new species recognition since the developed general framework will remain the same. In this sense, it will be very easy to extend its applicability to new seas and species just feeding the developed system with new training data sets. In addition, and considering the huge amount of acoustic data available after every campaign, the developed software is of great value, mainly for the fishing industry and the ecosystems research institutes, in the processing and the evaluation of terabytes of information in real time.

A drawback found in EchoView^TM that could not be overcome for ECOPAMPA is the need of empirically set the thresholds to discriminate the target under study from the background. This is a critical point due to the final classification may strongly depends on this decision of threshold selection. However, once a first choice is done and validated, the threshold value can remain fixed all along the campaign. In a near future we are planning to work in an adaptive tuning of the threshold resorting to machine learning techniques like reinforcement learning or deep learning.

Another future work to be carried out is related to next vessel campaigns to classify fish schools in the Argentinean Sea. It is expected that with new echo data, a significant statistical number of classification and sampling trials, the current ECOPAMPA performance could even be improved. From the standpoint of future research, we will explore new inputs to the ANN from multi-frequency analysis. In addition, we expect to improve online classification applying deep learning paradigm and integrating multiple convolutional neural networks (LeCun et al., 2015).

Declarations

Author contribution statement

S. A. Villar: Conceived and designed the experiments; Performed the experiments; Analyzed and interpreted the data; Contributed reagents, materials, analysis tools or data; Wrote the paper.

A. Madirolas, A. G. Cabreira: Conceived and designed the experiments; Performed the experiments; Analyzed and interpreted the data.

G. Acosta, A. Rozenfeld: Analyzed and interpreted the data; Wrote the paper.

Funding statement

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Data availability statement

Data included in article/supplementary material/referenced in article.

Declaration of interests statement

The authors declare no conflict of interest.

Additional information

No additional information is available for this paper.

Appendix A. Supplementary data

The following is the supplementary data related to this article: