A look at . . . . Moment-based approaches in imaging part 4: Some applications

This last paper of this series provides a survey of the topics the moments have addressed in image processing at large. Our introductive papers dealt with the theoretical expressions of orthogonal and non-orthogonal basis functions [1], with important properties like geometric invariance, robustness to noise or blurring [2] and the computational accuracy and complexity [3]. Here, most of the steps going from image reconstruction, segmentation, motion tracking to pattern recognition are examined through moment-based techniques. Though they have not an equal importance in this wide spectrum of applications, they can offer sound solutions. Rather than entering into methodological details and examples, this paper is intended to identify the main contributions brought by moments.

Featuring Image analysis

The multiple applications of digital images lead to segment the field into many areas, each having their own journals and conferences. This diversity makes more difficult to have a full view on the on-going research and may hide the general interest of a given method. It must be first pointed out that the physical process underlying the image formation is of major importance for the all subsequent processing. If not properly dealt with, most of the information later extracted can be biased. The reconstruction problem, so important in medical imaging, can be dealt with purely mathematical views but the capability to model and control the acquisition device should open new pathways. Conversely, the search for generic algorithms, a major trend in image analysis, is attractive: the efficiency of such approach has been demonstrated through many examples. Assumptions are always required however and they must be adapted to the situations under study. Thus, a full apprehension of the field has to take into account the 4-tuple, “observations (or data), objects (properties), goals (analysis objectives), methods (algorithms)”.

Multidimensional observations are more and more available in medicine (volume (3-D) images, 2-D+t and 3-D+t image sequences, with scalar or vectorial descriptions). The spatial and temporal sampling is here critical according to the task to be carried out. The huge data sets they provide (hundreds of slices or 2-D projections) call for challenging time computations. The objects of interest are varying in shape among individuals and with disorders, may change and deform over time, have poor contrast. Moreover, they are not reducible to simple geometric patterns. These difficulties may be compensated by introducing some, but limited, interactive pointing or delineation. The ultimate goal (omitting the image formation) is to provide the user with relevant features for clinical decision and therapeutic guidance. It includes parameters for quantifying the lesion extent and discriminating them, locating the abnormal patterns in anatomical space, or characterizing the behavior in space and time. The methods that may bring answers to these issues are many. The classical paradigms like low vs high level, data-driven vs model-driven, salient feature and primitive grouping, edge and region cooperation, local vs global analysis, image understanding through semantic labeling and spatial relation modeling, etc. are still of value in most of the problems to solve. Non-stationary noise and artifacts, low contrast, object occlusions and deformations, variability in observation conditions, etc. lead to recurrent challenges in computer vision when robustness, accuracy, completeness must be achieved under real-time constraints.

The main applications

Reconstruction

This issue has received much attention in the medical imaging literature. A number of imaging modalities are concerned from X-ray computerized tomography and emission tomography up to acoustic and optical techniques. They all bring different insights in the human body either morphological or functional. Recent works deal with reconstruction from only limited range projections with the aim to estimate the projections at unknown views such as the moment-based approaches reported by Salzman [4] and Goncharev [5]. Milanfar et al. [6] presented a variational framework for the tomographic reconstruction of an image from the maximum likelihood estimates of its orthogonal moments. Basu and Bresler [7, 8] investigated the problem of recovering the view angles from the projection data by means of moment method. Rouze et al. [9] derived a relation between Zernike moments and the Radon transform, and then applied it to reconstruction. By establishing a relationship between the image geometric moments and projection moments, Wang and Sze [10] presented an approach to reconstruct the CT images from limited range projections. Shu et al. [11] extended Wang’s method using the orthogonal Legendre moments to improve the quality of the reconstructed image. The advantage of Shu’s method is that the orthogonal moments have simple inverse transform, thus, the image can be more easily obtained from the orthogonal moments. Moreover, the geometric moments, especially at high order, are sensitive to noise and digitization error. However, both methods were based on the use of continuous moments. When applied to 2-D digital images, the double integrals are usually approximated by discrete summations that lead to numerical errors in the computed moments. The discrete orthogonal moments, in particular Tchebichef moments, do not require any discrete approximation for numerical implementation and moreover the discrete orthogonal Tchebichef polynomials are the simplest among all the discrete orthogonal polynomials with a definition domain ideally suited for square images [12]. However, the traditional Radon transform is no longer applicable, and a discrete version of Radon transform is required. Beylkin [13] described a discrete Radon transform (DRT) to map a set of sampled image points onto a set of discrete projections. The algebraic mapping for this transformation can be computed exactly, since no interpolation of the data is required. Matus and Flusser [14] developed a group theoretic and Fourier based approach to the DRT. Svalbe [15] and Kingston [16] derived improved versions of DRT to handle both continuous projections and discrete projections in Fourier and Radon space. The advantage of the techniques reported in [15] and [16] is that they require a straightforward 1-D linear interpolation, a simple sorting of projection samples and no pre-processing of the projection data. The purely mathematical views provided by moments for the reconstruction problem have a drawback: the difficulties they have to deal with the physical characteristics inherent to any image capture.

Pattern Recognition

This topic represents certainly the major applications of moments and a survey of the all literature is out of reach in a short paper. The classical views regarding pattern recognition rely on the ability to extract the objects of interest, on the choice of their representation and the final decision step, e.g. allocation to classes. These classes may be formed beforehand through a learning phase. Many techniques are available to manage that either by contour-based or region-based features with local or global descriptions. They can operate in the image or in the transform domains. They include kernel methods (like Support Vector Machine), neural networks, rigid or elastic matching, string coding and matching, etc. They all require a distance measure to estimate the similarity between two shapes. The flexibility of moments in dealing with binary or grey level images, polygonal and region representations on one hand and, in the other hand, the invariance properties discussed in [2] make them particularly attractive for a number of applications. They offer compact and robust descriptions and bring statistical cues on pixel (or voxel) value distributions: in some cases (the low orders) they have physical interpretations.

Invariance properties are central for pattern recognition as discussed in [2] [17]. Many attempts have been reported using moments for aircraft recognition [18], identification of building and bridge [19], partially occluded object localization [20], flower shape analysis [21], image retrieval [22], character recognition [23–27]. Bailey and Srinath [28] investigated invariant character recognition using Legendre, Zernike and pseudo-Zernike moments with different classifiers. Khotanzad and Hong [29] has shown that a neural network classifier using Zernike moments has a strong class separability power. The evolution towards images with higher dimensions (3-D and n-D) has also motivated a lot of theoretical works [30–36].

Beyond shape features, texture patterns are also of interest in order to recognize objects. Less moment-based works have been devoted to texture analysis [37, 38]. The attempts made for tissue characterization did not however show a significant breakthrough: this can be due to the texture complexity, resulting from mixture of structural tissue properties, partial volume effects, noise and reconstruction artefacts.

Miscellaneous

We briefly review here some additional topics where the moments have been called for but with limited impact. As pointed out above, nearly every system is initialized by detection and segmentation tasks. Despite the recent achievements provided by deformable models, level set and graph-cut methods, these problems remain open in medical applications and require some user interaction. Edge detection is mainly carried out by derivative operators while integrative schemes may offer a sound alternative as shown in 2-D with geometric moments [39] or Zernike moments [40]. Geometric and Legendre moments have also been proposed for 3-D surface detection assuming a local planar patch [41] or a parabolic model [42]. Image registration or alignment is another active area. Most approaches rely on establishing the feature correspondence between a reference image and the target image, a task that can be dealt with moments [43–50]. Motion estimation is another major area in image analysis [51]. Optical flow methods based on differential-geometric descriptors and matching techniques (or correlation) have been widely explored. Joint motion-tracking approaches combining different types of information (boundary-region) and different sources (gradient, intensity, motion) in scenes with multiple objects should provide new perspectives. Moments have not received much attention in this area and the few attempts made were essentially focused on optical flow methods using Zernike moments [52, 53].

Conclusion

Moment-based methods are still marginal in image processing and it is our hope that this series will motivate new research on this topic and open new applications especially in medical imaging. These techniques cover, as we pointed out, a wide spectrum of applications and deserve to be confronted to other attractive and competitive methods. They should also be combined in order to take advantage of their basic features, in particular the invariance properties and the continuously improvements in computational complexity.