Hyperspectral remote sensing image destriping via spectral-spatial factorization

Abstract

Hyperspectral images (HSIs) are gradually playing an important role in many fields because of their ability to obtain spectral information. However, sensor response differences and other reasons may lead to the generation of stripe noise in HSIs, which will greatly degrade the image quality. To solve the problem of HSIs destriping, a new iterative method via spectral-spatial factorization is proposed. We first rearrange the HSI data to get a new two-dimensional matrix. Then the original noise-free HSI is decomposed into a spectral information matrix and a spatial information matrix. The sparsity of stripe noise, the group sparsity of spatial information matrix, the smoothness of spectral information matrix can be used to achieve sufficient removal of stripe noise while effectively retaining spectral information and spatial details of the original HSI. Numerical tests on simulated datasets show that our method achieves an average PSNR growth above 4dB and a better SSIM result. The proposed method also obtains good results when processing real datasets polluted by Gaussian noise and stripe noise.

Introduction

The advantage of hyperspectral images (HSIs) over ordinary natural images is that they contain much finer spectral information. To capture these information, remote sensing HSI systems often take advantage of the platform movements along the orbit, which results in a common push-broom imaging manner. The push-broom hyperspectral remote sensing system has been widely used in anomaly detection¹, environmental monitoring², agriculture³, and so on. However, images obtained by push-broom hyperspectral remote sensing systems may be affected by severe stripe noise, resulting in a significant decrease in image quality. Moreover, the morphological structure of stripe noise is similar to that of geological faults and landslides, which will interfere with their recognition and adversely affect practical applications such as dynamic monitoring of geological faults and landslides using remote sensing images^4,5.

Stripe noise is caused by many reasons, such as calibration errors⁶ and sensor response differences⁷, etc. This also leads to great differences in the structure of stripe noise. In the paper⁸, researchers claimed that stripe noise has six features, including direction, degradation, distribution, pattern, persistence and periodicity. According to the features of stripe noise, many different approaches for destriping have been proposed. Statistics-based methods such as histogram matching⁹ were among the first methods proposed for destriping. However, these methods generally have poor performance when dealing with complex stripes cases in real scenes. Filtering-based methods^10–12 are another type of destriping methods. However, filtering-based methods often suffer from insufficient denoising or over-smoothness.

In addition, some iterative methods based on different image priors have also been proposed. Bouali et al.¹³ developed a classical destriping method based on the directional features of stripe noise. They selected the Moderate Resolution Imaging Spectroradiometer (MODIS) image which contains horizontal stripe noise in experiments. The horizontal variation of the image is almost unaffected by stripe noise and is used as a fidelity term in the model. The vertical variation of the image will be affected, which is used as a regularization term. Along with this iterative approach, the reweighted block sparsity and unidirectional total variation regularization (RBSUTV)¹⁴, unidirectional hybrid total variation and nonconvex low-rank regularization¹⁵, multimode structural nonconvex tensor low-rank (M2SNTLR) regularization¹⁶ and other models^17–20 have also been proposed. These models use the features of the original image or mixed noise as priors and improve performance by adding or improving regular terms. However, they only use the spatial information. If they are used for HSIs destriping, only one band of HSI data can be processed at a time, and the spectral correlation between adjacent bands cannot be exploited.

Using the low-rank matrix recovery theory, a denoising model named LRMR²¹ was proposed, which can effectively remove a variety of mixed noises. Combined with the low-rank representation technology in image processing, the graph-regularized low-rank representation (GLRR) algorithm²² was proposed. Then, the low-rank-based single-image decomposition (LRSID) method was proposed²³. LRSID treats the stripe components and image equally. The stripe component is depicted via the nuclear norm term, and the main structure of image is exploited by the TV regularization. LRSID can not only be used for 2-D single images, but also be extended to 3-D multispectral images. The model is also robust to various stripe noises. Furthermore, Liu et al.²⁴ applied the low-rank stripe models to specific subbands of 3-D wavelet transform and proposed the wavelet-domain low-rank/group-sparse destriping algorithm (WDLRGS), which can retain image details and texture while effectively removing stripe noise. In addition, fast graph Laplacian regularizer (FGLR) destriping model²⁵, double low-rank (DLR) matrix decomposition model²⁶, and other models^27,28 have also been proposed. Due to the high correlation between adjacent bands of clean HSI, the 2-D matrix obtained by rearranging the original 3-D HSI is low-rank^26,29,30. Therefore, the above destriping methods considering the low-rank property of HSI achieve good results.

Recently, deep learning methods have shown practical advantages in destriping compared to traditional methods. HSI-DeNet³¹ can effectively remove stripe noise, random noise and deadlines, and the model uses a convolutional neural network architecture. The satellite-ground integrated destriping network (SGIDN)³² can capture various stripe noise characteristics in spaceborne images from synthetic hyperspectral image pairs. The spatial and spectral information inherent in HSIs are extracted by the integrated 2-D convolution, 3-D convolution and residual learning modules in SGIDN. Since stripe noise has directionality, researchers proposed the multi-scale dilated unidirectional convolution (MsDUC)³³ network, in which the directional characteristics of stripe noise are obtained through unidirectional convolution. Then a progressive method for HSI destriping based on adaptive frequency focusing was proposed³⁴, which achieved destriping and fine image recovery. From the above methods and other destriping networks, the trend of deep learning destriping methods is to continuously improve network modules to better use the noise feature information, frequency domain information and spatial structure information of images^35–40. However, most deep learning methods lack of interpretability, they are purely data-driven which means their performance is closely related to the quality and quantity of training data. For practical applications where data acquisition is difficult, such as hyperspectral remote sensing, the application of deep learning methods will be greatly limited.

In contrast to deep learning methods, iterative methods use prior information to design physical models, which are interpretable and do not require large amounts of data for pre-training. We propose an effective iterative method for HSI destriping via spectral-spatial factorization (DeSSF) and apply the DeSSF method to push-broom hyperspectral remote sensing image destriping. Numerical tests on simulated datasets and real datasets show that DeSSF achieves sufficient removal of stripe noise. The main contributions are:

• We rearrange the HSI data and explore the new characteristics of stripe noise in a 2-D matrix. To be specifically, the smoothness of spectral information and the sparsity of spatial information.

• We design a new iteration algorithm based on ADMM, which selects the sparsity constraint of stripe noise, the group sparsity constraint of spatial information matrix and the smoothness constraint of spectral information matrix as different regularization terms to make better use of prior information, thereby effectively removing stripe noise while retaining spectral information and spatial details of the original HSI.

• Based on the fact that the gain of the detector in practical applications has a certain range and is greater than zero, we propose a new stripe degradation model to better simulate the generation process of stripe noise. And we define the gain range of the detector according to the magnitude of stripe noise in real data.

Next, the proposed destriping method via spectral-spatial factorization is introduced in Sect. 2. The algorithmic details of DeSSF are presented in Sect. 3. The numerical results of synthetic and real datasets are given in Sect. 4. The conclusion is given in Sect. 5.

Destriping via spectral-spatial factorization

HSI degradation model

In this paper, the spatial size of a HSI is , and its spectral band is . As can be seen from Fig. 1 (the HSI used in the figure is from⁴¹), after rearrangement of all columns of the HSI datacube, a two-dimensional matrix of size can be obtained, in which each column represents a spectral of one pixel. The stripe noise is regarded as additive noise¹³. After rearrangement, the noise-contaminated HSI is modeled as:

1

where and represent the original noise-free data and the noise-contaminated data of HSI respectively. and represent the stripe noise and the Gaussian noise.

Fig. 1.

Rearrange the 3-D HSI data cube into a 2-D matrix.

Typically, stripe noise in HSI has obvious directionality (horizontal stripes or vertical stripes, as shown in Fig. 2(a)) due to the movement direction of the imaging platform. Some methods, such as RBSUTV¹⁴, have achieved good results by using unidirectional total variation regularization in the model to characterize the directionality of the stripe layer. However, this directionality may be lost after the rearrangement. In Fig. 2(c), the horizontal stripe noise shows a shape similar to salt and pepper noise in the two-dimensional matrix obtained after rearrangement, and is randomly distributed without any direction characteristics. Although the directionality is lost, the stripe noise in Fig. 2(c) has turned into a small amount of sparsely distributed spot-like noise compared to the clean HSI in Fig. 2(b). Therefore, we consider using this sparsity in our method and no longer consider the directionality of stripe noise.

Fig. 2.

Rearrange the 3-D HSI with stripe noise into a 2-D matrix. (a) HSI of a certain band contaminated by stripe noise. (b) Rearranged 2-D matrix of the clean HSI. (c) Rearranged 2-D matrix of noisy image.

Spectral-spatial factorization model

Inspired by the model⁴², DeSSF tends to address the stripe noise in a familiar way, which decomposes the clean HSI into spectral and spatial matrix:

2

where is called the spectral information matrix, and is called the spatial information matrix. Since the correlation between adjacent spectral bands of a clean HSI is high, it is considered to be a low-rank matrix²¹. Therefore, the parameter is set as the rank of in DeSSF. We will continue to discuss the properties of U and V.

The premise of (2) is that generally the types of objects within a HSI area are limited, and the spectral characteristics of the same substance in adjacent spectral bands remain the same. Most of the pixels in the decomposed matrix only contain background information, and the target information we need only takes up for a small part of all the pixels in . We consider the spatial information matrix as a sparse matrix. Therefore, a sparsity constraint can be applied to in (2). As shown in Fig. 3, after transformation using (2), the information of an object in the original HSI data is still concentrated in certain positions of the spatial information matrix . In other words, the useful information in is not randomly distributed, but has an obvious group distribution, so we finally apply group sparsity constraint on .

Fig. 3.

Simplified diagram of spectral-spatial factorization model.

Combined with (2), we formulate the DeSSF noise degradation model as:

3

where Y, U, V, S and N have the same meaning as in (1) and (2). The goal is to remove noise from Y and obtain a clean HSI . To solve the above equation, the destriping optimization model is formulated as:

4

The first term in (4) is the data fidelity term and denotes the Frobenious norm. ensures the smoothness of the estimated spectral information matrix U. is designed to model the group sparsity prior of the spatial information matrix V, where the -norm of V is defined as:

5

According to Fig. 2(c), since the stripe noise is sparsely distributed in the two-dimensional matrix obtained after rearrangement, is used to model the sparsity prior of stripe noise. , and are regularization parameters.

Optimization algorithm

The alternating direction method of multipliers (ADMM) is used to solve (4). To make the variables separable when designing our iterative algorithm, we introduce an auxiliary variable , (4) is equivalent as:

6

The augmented Lagrangian function of (6) is:

7

where is the Lagrange multiplier and is another regularization term. Let and convert (7) to a simpler form:

8

An alternating minimization scheme can be adopted to solve (8). We can transform (8) into several minimization subproblems.

The U-subproblem is:

9

which can be solved in a closed-form:

10

The V-subproblem is:

11

and the solution of (11) is:

12

The S-subproblem is:

13

which can be calculated as:

14

where

15

The W-subproblem is:

16

Let , if is the optimal solution to:

17

the i-th column of can be calculated as:

18

Thus, the solution of (16) is:

19

where .

Finally, the variable is updated as follows:

20

where is a regularization parameter.

Algorithm 1 summarizes the iterative procedure.

Finally, the computational complexity of DeSSF can be calculated as follows. In one iteration, the computational complexity of updating and are both . The updating of requires flops. Updating requires matrix multiplication and the soft-thresholding shrinkage operator, with a computational complexity of . In addition, the computational complexity of updating the Lagrange multiplier is . The final part is updating with a computational complexity of . Therefore, after iterations, the overall computational complexity of DeSSF is . We also compared the algorithm running time with several different destriping methods using numerical tests in the next section.

Numerical experiments

Experimental setting

• 1. Datasets

The experiments are carried out on simulated and real noisy images respectively. All the images are normalized to [0, 1].

Simulated datasets: We use Washington DC Mall image⁴³ and Pavia Centre image⁴⁴ to conduct simulation experiments. The Washington DC Mall image contains pixels and its spectral number is 191. To improve the operation speed, a sub-image with the size of is randomly intercepted for experiments, and the spectral bands should be kept continuous after the interception. Pavia Centre is a pixels image and its spectral number is 102. To verify that our method can process more spectral bands, we crop a sub-image of size for experiments.

Real datasets: We use Gaofen 5 (GF-5) Baoqing dataset and Wuhan dataset to validate our method in real remote sensing scenarios. These two hyperspectral remote sensing datasets were taken in Baoqing and Wuhan, China⁴⁵. GF-5 images contain real stripe noise and Gaussian noise. The image size is . In order to make the effect more obvious, we select some spectral bands with severe stripe noise for experiments, which are band 87 to band 96 of Baoqing dataset and the last ten bands of Wuhan dataset.

• 2. Comparison methods

The DeSSF method is compared with six classic or newly proposed destriping methods, where anisotropic spectral-spatial total variation (ASSTV)⁴⁶ is a variation-based method, LRMR²¹ and LRSID²³ are classical low-rank-based methods, FGLR²⁵, stripe estimation and image denoising framework (SEID)⁴⁷ are the two latest proposed destriping methods, HSI single denoising convolutional neural network (HSI-SDeCNN, hereafter referred to as SDeCNN)⁴⁸ is a deep learning method.

• 3. Evaluation indices

The performance of different algorithms is evaluated not only by qualitative indices, but also by several quantitative indices. Visual inspection is the simplest qualitative evaluation method. In addition, the mean cross-track profile is used as another qualitative evaluation indicator.

As for the quantitative evaluation, structural similarity index (SSIM)⁴⁹, peak signal-to-noise ratio (PSNR), inverse coefficient of variation (ICV)⁵⁰ and mean relative deviation (MRD)⁵¹ are chosen to conduct a fair comparison. SSIM and PSNR are full-reference evaluation indices, which require corresponding noise-free images as reference, and are not applicable in real data experiments. ICV and MRD are no-reference evaluation indices. ICV can be calculated as:

21

where denotes the mean of pixels in the selected range and denotes the standard deviation of those pixels. ICV can evaluate the ability of different methods to remove stripe noise. It is calculated in homogeneous striped regions. MRD is defined as:

22

where represents the pixel value of the image after removing stripe noise, represents the pixel value of the original image. MRD is used to evaluate the ability of the method to retain useful information in image regions that do not contain stripe noise. Contrary to PSNR, SSIM and ICV, a smaller MRD value means better algorithm performance.

• 4. Noise setting

Traditional stripe noise simulation methods, such as the method used in RBSUTV¹⁴, take into account the variation of stripe intensity, but the intensity setting is arbitrary, which is not conform to the reality. In practical applications, the gain of the detector has a certain range without negative value, and the maximum value that the detector can detect is also certain. To make the stripe noise added in our simulation experiments more in line with these practical conditions, we propose a new stripe degradation model. For a noise-free HSI with pixel values in the range of [0, 1], we perform stripe degradation according to (23):

23

is the j-th line (column) data in the i-th band of original HSI, is used to simulate the gain of the sensor, and Y represents the HSI polluted by stripe noise.

We refer to the real data (Baoqing data) to calculate . As shown in Fig. 4(a), (b), (c), stripe noise significantly changes the pixel average of the image column in which it occurs. And can be obtained by the following calculation:

24

where represents Baoqing data, represents the calculation of pixel average of Baoqing data. The maximum gain and minimum gain of each band are shown in Fig. 4(d). Since the last few bands are greatly affected by other factors, we discard them. The range of gains is [0.36, 1.35]. Finally, is randomly selected within the range of minimum gain and maximum gain of the remaining bands.

• 5. Parameters setting

Fig. 4.

Mean cross-track profile, minimum gain and maximum gain of Baoqing data. (a) Mean cross-track profile of band 16. (b) Mean cross-track profile of band 42. (c) Mean cross-track profile of band 89. (d) Minimum gain and maximum gain of each band.

The proposed method has five regularization parameters. In the rest two subsection, we set , , and to get a satisfactory destriping result. Then in the last subsection, a comprehensive analysis on these parameters is conducted. The stopping criterion is set to 0.0001 in our method.

Simulated data experiments

In these experiments, no more than 50% of the rows are randomly selected, then in each band a horizontal stripe is added according to (23). Note that only horizontal stripes are taken into consideration in our simulated experiments. Because, if vertical stripes occur, the vertical stripes can be turned into horizontal stripes by simply transposing the image.

The DeSSF method is first tested on Washington DC Mall image. The stripe noise removal results of band 10 can be seen from Fig. 5. The figure clearly shows that DeSSF can achieve sufficient removal of stripe noise while effectively preserving the spatial details.

Fig. 5.

Destriped results of Washington DC Mall image band 10.

The average values of PSNR and SSIM across different spectral bands are shown in Table 1. As shown in Table 1, DeSSF achieves a PSNR improvement of more than 5 dB and a better SSIM result compared with the best results of the most advanced methods. As for why the PSNR indicator of DeSSF is significantly better than other methods, it is mainly because the targets in Washington DC Mall image are relatively simple, and we did not add Gaussian noise in it, that is, the noise situation is also simple. In this case, DeSSF can achieve good results.

Table 1.

The average PSNR and SSIM for Washington DC Mall image.

	ASSTV	LRMR	LRSID	FGLR	SEID	SDeCNN	DeSSF
PSNR	28.52	37.22	29.19	34.69	31.11	32.50	45.48
SSIM	0.89	0.98	0.92	0.98	0.90	0.89	0.99

Figure 6 shows the mean cross-track profiles of band 10. The curve of the original image has a rapid upward and then slightly downward trend around the last 50 lines, and the curve of DeSSF is significantly more consistent with this trend.

Fig. 6.

Mean cross-track profiles of Washington DC Mall image band 10.

To further compare the spectral recovery capabilities of different methods, we select four points in the image and plot their corresponding spectrum curves. The four representative points are, building 1 (50, 50), road (20, 210), building 2 (180, 130), tree (200, 270). As shown in Fig. 7, the spectral curves recovered by LRMR, LRSID and SEID have abnormal fluctuations. These abnormal fluctuations make the spectral curve inconsistent with the original spectral curve, which brings difficulties to target detection and recognition of HSIs. The spectral curves recovered by FGLR are over-smoothed and almost the same for different objects, which indicates that the spectral information of HSIs recovered by FGLR has been lost. The spectral curve recovered by DeSSF is closest to the real curve, indicating that this method will not cause the loss of spectral information.

Fig. 7.

Spectrum of representative points in Washington DC Mall image. (a) Building1. (b) Road. (c) Building2. (d) Tree.

The proposed method can remove Gaussian noise and stripe noise at the same time. We added Gaussian noise with standard variance to the Washington DC Mall image to explore how the Gaussian noise affects our proposed method. According to Table 2, as the intensity of Gaussian noise increases, the values of both PSNR and SSIM are gradually decreasing. However, the denoising results obtained by DeSSF when processing HSIs containing Gaussian noise are consistently better than those of the comparison methods. Our method gets better results with Gaussian noise level smaller than when compared to the noiseless scenarios of others.

Table 2.

The average PSNR and SSIM for Washington DC Mall image under different Gaussian noise intensities.

Noise level	Index	ASSTV	LRMR	LRSID	FGLR	SEID	SDeCNN	DeSSF
	PSNR	28.52	37.22	29.19	34.69	31.11	32.50	45.48
	SSIM	0.89	0.98	0.92	0.98	0.90	0.89	0.99
	PSNR	28.27	36.76	28.42	34.66	30.72	32.21	42.33
	SSIM	0.87	0.97	0.90	0.97	0.89	0.88	0.99
	PSNR	27.38	36.25	26.78	34.18	29.53	29.24	39.02
	SSIM	0.81	0.97	0.84	0.97	0.85	0.85	0.98
	PSNR	25.63	35.33	24.48	33.25	27.04	25.01	37.43
	SSIM	0.74	0.96	0.76	0.96	0.75	0.80	0.96
	PSNR	22.96	34.12	22.75	33.33	25.69	22.37	34.92
	SSIM	0.65	0.94	0.68	0.95	0.66	0.75	0.94
	PSNR	20.69	33.75	20.47	31.22	23.92	20.30	32.87
	SSIM	0.58	0.93	0.59	0.93	0.59	0.71	0.93

To verify the consistency of DeSSF, another test is conducted using Pavia Centre image. In this experiment, more spectral bands are taken into consideration and we add random Gaussian noise with standard variance to every band. Figure 8 shows the experimental results of band 30. We find that DeSSF can also recover very small targets such as cars on the road with high accuracy.

Fig. 8.

Destriped results of Pavia Centre image band 30.

The average values of PSNR and SSIM are shown in Table 3. Our proposed method still achieves an average PSNR growth above 3 dB and better SSIM values when processing Pavia Centre image with mixed noise, which indicates that DeSSF has great advantages over the comparative methods.

Table 3.

The average PSNR and SSIM for Pavia Centre image.

	ASSTV	LRMR	LRSID	FGLR	SEID	SDeCNN	DeSSF
PSNR	27.57	36.32	26.24	25.24	28.78	31.18	40.31
SSIM	0.86	0.98	0.89	0.88	0.89	0.88	0.99

The mean cross-track profiles of band 30 are shown in Fig. 9. After adding mixed noise, the original mean cross-track profile changes greatly, but it can still be recovered well after denoising by DeSSF.

Fig. 9.

Mean cross-track profiles of Pavia Centre image band 30.

Furthermore, the spectrum curves of building 1 (17, 35), tree (70, 460), grassland (200, 300) and building 2 (400, 480) in Pavia Centre image are shown in Fig. 10.

Fig. 10.

Spectrum of representative points in Pavia Centre image. (a) Building1. (b) Tree. (c) Grassland. (d) Building2.

We also test the performance of each method using ICV and MRD. In the MRD evaluation, the sharp regions with about 100 pixels are selected. Since the Pavia Centre image is affected by Gaussian noise in every band, we only calculate ICV on this image. According to Table 4, DeSSF can effectively remove stripe noise, so ICV values of this method are large in most cases. The ICV values of SEID and SDeCNN far exceed the other methods in some scenarios, mainly because the results of SEID and SDeCNN are over-smoothed, which can also be verified from Fig. 5 and Fig. 8. According to Table 5, since the brightness of healthy pixels will not be changed by DeSSF, the corresponding MRD values are small.

Table 4.

The values of ICV for Washington DC Mall image and Pavia Centre image.

	Washington DC Mall				Pavia Centre
	Band 7	Band 14	Band 21	Band 28	Band 12	Band 24	Band 36	Band 48
Original	2.95	15.06	4.53	10.46	4.24	3.95	15.53	11.10
ASSTV	3.14	7.14	8.98	24.84	9.25	14.65	18.80	21.10
LRMR	1.84	6.58	6.55	14.94	7.90	12.78	16.38	17.14
LRSID	2.91	8.04	11.43	17.99	6.48	18.16	17.43	19.24
FGLR	1.82	7.65	7.64	16.93	8.08	15.47	19.35	20.37
SEID	1.67	12.95	30.19	30.14	5.32	67.80	103.95	155.89
SDeCNN	49.94	36.06	41.24	50.96	10.98	39.42	47.20	46.95
DeSSF	20.84	20.59	21.51	31.35	8.21	16.14	19.90	21.47

Table 5.

The values of MRD for Washington DC Mall image.

	ASSTV	LRMR	LRSID	FGLR	SEID	SDeCNN	DeSSF
Band 7	14.17%	3.98%	26.28%	6.82%	3.11%	12.17%	4.86%
Band 14	7.53%	6.93%	19.13%	5.69%	9.92%	14.04%	1.57%
Band 21	17.05%	2.76%	3.79%	2.79%	0.78%	5.46%	0.65%
Band 28	7.85%	6.02%	8.22%	9.01%	9.40%	10.47	1.10%

Finally, the time costs of different methods are given in Table 6. All these numerical experiments are conducted on a personal computer with Intel Xeon E3-1505M v5 (2.80 GHz) processor. Among the comparative methods, FGLR has a lower time cost than DeSSF, but the cost of fast calculation of FGLR is a large loss of spectral information. Table 6 only lists the time cost of the SDeCNN testing process, but the time cost of the SDeCNN should also include the time spent on the training process. The time cost of the training process of SDeCNN is closely related to the amount of training data, the number of training rounds and other factors, and the training process usually takes a longer time.

Table 6.

Running time (seconds) of simulated data experiments.

	ASSTV	LRMR	LRSID	FGLR	SEID	SDeCNN	DeSSF
Washington DC Mall image	140.56	274.68	127.61	2.71	10,132.25	21.45	72.55
Pavia Centre image	484.78	1006.33	555.82	8.76	42,438.95	86.70	255.45

The size of Washington DC Mall image is , the size of Pavia Centre image is .

Real data experiments

Next, we will conduct numerical tests on real data to illustrate the performance of DeSSF. Since we chose the images of the last several bands of Wuhan dataset, the noise is more serious than that of Baoqing dataset. Especially in the image of Wuhan dataset band 155, it is almost impossible to distinguish specific targets from it.

After calculation, Fig. 11(a) and Fig. 12 show the destriping results of Baoqing dataset and their corresponding mean cross-track profiles respectively. There is obvious noise in LRMR denoising results. The results of ASSTV, SEID and SDeCNN tend to over-smooth the image. FGLR and DeSSF yield good visual results. However, the FGLR method still causes the loss of spectral information according to Fig. 11(a). The experimental results of Wuhan dataset are obtained and shown in Fig. 11(b) and Fig. 13. LRSID can remove stripe noise, but the denoising performance of this method reduces rapidly when severe Gaussian noise exists in the original HSI. In addition, from the perspective of the mean cross-track profiles, ASSTV, SEID and SDeCNN will cause a very serious over-smoothing phenomenon and lose a large amount of spatial details of the original image.

Fig. 11.

Destriped results of real datasets. (a) Baoqing dataset. (b) Wuhan dataset.

Fig. 12.

Mean cross-track profiles of Baoqing dataset band 89.

Fig. 13.

Mean cross-track profiles of Wuhan dataset band 152.

Besides the qualitative performance, we also quantitatively compare denoising results using ICV and MRD. DeSSF achieves sufficient removal of stripe noise, so its ICV values are large in Table 7. The ICV values of SEID and SDeCNN even exceed the proposed method in some scenarios, mainly because the results of these two methods are over-smoothed, which can also be verified from Fig. 11 to Fig. 13. According to Table 8, the MRD values of LRMR are better, mainly because it has no ability to remove Gaussian noise and severe stripe noise at the same time. As shown in Fig. 11, the images denoised by LRMR have almost no changes compared with the original images.

Table 7.

The values of ICV for Baoqing dataset and Wuhan dataset.

	Baoqing Band 89		Baoqing Band 96		Wuhan Band 152		Wuhan Band 154
	Sample 1	Sample 2	Sample 1	Sample 2	Sample 1	Sample 2	Sample 1	Sample 2
Original	6.72	6.17	11.86	13.55	4.44	3.32	3.73	4.46
ASSTV	21.98	22.13	23.18	31.35	18.86	7.22	6.17	15.65
LRMR	7.16	7.03	13.10	14.75	5.10	3.85	3.98	4.71
LRSID	19.48	10.63	17.03	19.45	12.53	5.30	5.96	11.98
FGLR	23.37	16.30	23.84	27.17	55.37	8.88	6.69	47.25
SEID	75.23	24.47	60.03	65.06	104.89	41.25	26.99	134.42
SDeCNN	56.19	65.62	56.34	72.47	77.53	20.39	17.15	36.08
DeSSF	23.71	16.66	24.28	27.21	110.16	10.24	6.88	136.17

Table 8.

The values of MRD for Baoqing dataset and Wuhan dataset.

	Baoqing		Wuhan
	Band 87	Band 90	Band 147	Band 149
ASSTV	6.59%	4.18%	6.04%	18.61%
LRMR	0.41%	1.23%	5.61%	4.75%
LRSID	1.51%	2.63%	6.22%	8.04%
FGLR	11.86%	1.78%	6.93%	4.64%
SEID	2.29%	3.35%	14.62%	10.93%
SDeCNN	3.98	5.41	22.92%	11.95%
DeSSF	0.36%	1.29%	5.56%	6.44%

Table 9 shows the time cost of running on Baoqing dataset and Wuhan dataset. The time cost of FGLR is lower than that of DeSSF, but the cost of fast calculation of FGLR is a large loss of spectral information, which can also be verified by Fig. 11. Table 9 only lists the time cost of the SDeCNN testing process. If the time spent on the training process is added, the time cost of SDeCNN is much greater than that of DeSSF. Combining the destriping performance and time cost, DeSSF will have greater advantages than other methods in practical applications.

Table 9.

Running time (seconds) of real data experiments.

	ASSTV	LRMR	LRSID	FGLR	SEID	SDeCNN	DeSSF
Baoqing dataset	24.28	58.72	47.42	0.86	3716.09	21.18	24.38
Wuhan dataset	40.89	90.65	49.86	0.84	3646.64	20.71	24.69

The size of the Baoqing dataset and the Wuhan dataset is .

Discussion

• 1. Performance of DeSSF in full bands

There may be dozens or even hundreds of spectral bands in HSIs. In the above simulated data experiments, we only selected a part of the spectral bands for processing. In this section, we first test the effectiveness of DeSSF in full bands. As shown in Fig. 14 and Table 10, our proposed method achieves good results in the full bands. Compared with the best results of other methods, DeSSF achieves an average PSNR improvement of more than 4dB and the better SSIM results.

Fig. 14.

The values of PSNR and SSIM for different methods in full bands. (a) Washington DC Mall image, PSNR. (b) Washington DC Mall image, SSIM. (c) Pavia Centre image, PSNR. (d) Pavia Centre image, SSIM.

Table 10.

The average PSNR and SSIM for different methods in full bands.

	Washington DC Mall image		Pavia Centre image
	PSNR	SSIM	PSNR	SSIM
ASSTV	26.48	0.84	29.35	0.88
LRMR	32.50	0.96	35.99	0.97
LRSID	25.41	0.87	26.98	0.88
FGLR	29.47	0.89	31.00	0.92
SEID	28.27	0.84	30.16	0.89
SDeCNN	31.50	0.88	33.17	0.88
DeSSF	36.62	0.98	40.82	0.98

Then, we test the effect of the number of bands on DeSSF. We use the first 50, 100, 150 bands and all 191 bands of the Washington DC Mall image and the first 30, 60, 90 and all 102 bands of the Pavia Centre image. The spectrum curves of representative points are shown in Fig. 15. As shown in Fig. 15, the number of spectral bands used in the experiment has little effect on the results. However, when a large number of spectral bands are used, DeSSF has relatively poor recovery for the first and last few bands. And the recovery is relatively poor when the spectrum curve of the target changes sharply.

• 2. Performance of DeSSF at different stripe noise levels

Fig. 15.

Spectrum of representative points recovered using different numbers of bands. (a) Washington DC Mall image, road. (b) Washington DC Mall image, tree. (c) Pavia Centre image, building 1. (d) Pavia Centre image, tree.

Different levels of stripe noise will have an impact on the performance of our method. In this subsection, we set up eight scenes using the Washington DC Mall image. We randomly select 0–10% of the rows in scene 1 and add horizontal stripes according to (23). In other scenes, add 10% of the rows in turn and add horizontal stripes to those rows. Then we use DeSSF in these eight scenes and the destriping results are shown in Table 11. We find that DeSSF method can deal with different levels of stripe noise, but with the increase of stripe noise, the performance of this method will gradually deteriorate. This is mainly because with the increase of stripe noise, the proportion of stripe noise in the image becomes larger and larger, and even the sparsity prior of stripe noise in (4) is no longer satisfied. Moreover, the spatial and spectral information of the original image is destroyed more and more seriously. All of these lead to a degradation in the performance of the DeSSF method.

• 3. Parameter analysis

Table 11.

The average PSNR and SSIM of the DeSSF method at different stripe noise levels.

	Scene 1	Scene 2	Scene 3	Scene 4	Scene 5	Scene 6	Scene 7	Scene 8
Stripe noise level	0–10%	10%−20%	20%−30%	30%−40%	40%−50%	50%−60%	60%−70%	70%−80%
PSNR	47.32	46.78	46.07	45.21	43.77	41.92	38.77	37.96
SSIM	0.99	0.99	0.99	0.99	0.99	0.98	0.98	0.97

There are five hyperparameters in the proposed method, including , , , , , and their values will affect the experimental results. In this part, we conduct simulation experiments using Washington DC Mall image. Table 12 lists the changes in PSNR when various parameters in the simulation experiments take different values. Since the changes of SSIM are very small, we do not show them in Table 12. Among them, has a greater influence on the effect of the proposed method, which is mainly used to control the relative weight of the stripe regularization term. When is small, the denoising effect of DeSSF is better. The other parameters, especially , have relatively little impact on the model, and the value of can be arbitrarily taken within the range of 0 to 1. Parameters and have similar effects on the proposed method, so we set and consider different values of to explore the impact of changing these three parameters at the same time on the model. As shown in Fig. 16, DeSSF can obtain the best performance when the values of and are in the range of [0.1, 0.6] and is taken in the range of [0.004, 0.01]. The parameter can be set within [0.01, 0.025] in experiments.

Table 12.

The average PSNR of DeSSF under different values of hyperparameters.

Hyperparameter	Value
	0.1	0.2	0.3	0.4	0.5	0.6	0.7	0.8	0.9
	45.42	45.44	45.47	45.49	45.48	45.45	45.42	45.39	45.36
	45.28	45.41	45.47	45.50	45.48	45.44	45.42	45.26	45.30
	45.22	45.52	45.48	45.49	45.48	45.48	45.48	45.48	45.48

Hyperparameter	Value
	0.005	0.01	0.015	0.02	0.025	0.03	0.035	0.04	0.045
	46.30	45.48	44.80	43.98	43.29	42.66	42.12	41.62	41.18
	45.38	45.48	45.48	45.48	45.48	45.47	45.47	45.47	45.47

Fig. 16.

Change in the average PSNR and SSIM values of DeSSF, with respect to parameters , and . (a) PSNR. (b) SSIM.

Conclusion

We propose a new method for HSI destriping via spectral-spatial factorization in this paper. In our method, we not only use the prior information of stripe noise, but also fully exploit the prior information of the original HSI in two-dimensional pixel space and spectral space. The experimental results show that DeSSF achieves an average PSNR growth above 4dB and a better SSIM result compared to these existing methods. In addition, DeSSF has a clear advantage in the ability of spectral information preservation and a relatively lower computational cost. As a consequence, our method can provide a better starting line for the subsequent application of HSIs.

However, DeSSF also has some limitations. When processing multiple spectral bands, the spectral information of the first and last several bands may have a greater loss. Secondly, when the spectrum curve of the target changes sharply, the recovery is relatively poor. In addition, when the stripe noise or Gaussian noise is severe, the performance of DeSSF will deteriorate.

Our method can still be improved in many aspects in the future. For example, the adaptive determination of regularization parameters needs to be addressed in the future work. Secondly, remote sensing images may be obscured by clouds, so it is necessary to study the influence of atmospheric conditions (such as haze and cloud cover) on destriping performance. In addition, exploring alternative optimization techniques beyond ADMM to improve computational efficiency will be another added value.

Author contributions

Conceptualization, Y. Z. and Q. Y.; software, Y. Z.; writing—original draft preparation, Y. Z. and Z. Y.; writing—review and editing, J. L. and Z. W.; All authors have read and agreed to the published version of the manuscript.

Funding

This work was supported by the National Natural Science Foundation of China under Grant 62101572; the National Key Research and Development Program of China under Grant 2020YFA0713504; and the Research Program of National University of Defense Technology under Grant ZK21-16.

Data availability

The datasets generated during and/or analyzed during the current study are available from the corresponding author on reasonable request.

Declarations

Competing interests

The authors declare no competing interests.