Enhanced dataset synthesis using conditional generative adversarial networks

Abstract

Biomedical data acquisition, and reaching sufficient samples of participants are difficult and time ans effort consuming processes. On the other hand, the success rates of computer aided diagnosis (CAD) algorithms are sample and feature space depended. In this paper, conditional generative adversarial network (CGAN) based enhanced feature generation is proposed to synthesize large sample datasets having higher class separability. Twenty five percent of five medical datasets are used to train CGAN, and the synthetic datasets with any sample size are evaluated and compared to originals. Thus, new datasets can be generated with the help of the CGAN model and lower sample collection. It helps physicians decreasing sample collection processes, and it increases accuracy rates of the CAD systems using generated enhanced data with enhanced feature vectors. The synthesized datasets are classified using nearest neighbor, radial basis function support vector machine and artificial neural network to analyze the effectiveness of the proposed CGAN model.

Introduction

In recent years, computer aided diagnosis (CAD) has been widely studied and supported by advanced machine learning algorithms and signal processing methods. Researchers have reached to high success rates using the machine learning (ML) and feature extractions methods together in 1D and 2D signal classification tasks [1, 2]. In the meantime, deep learning (DL) approaches have gained importance due to superior performance when compared to conventional ML algorithms [3]. Some feature extraction methods (e.g., spectrogram) were adopted to 1D and 2D convolutional neural networks (CNN) [4–6], or long-short term memory neural networks (LSTM) for recognition [7].

The physicians can improve diagnostic accuracy with the help of the CAD systems using ML or DL methods for classification, detection, 1D or 2D signal enhancement purposes [8]. However, the key points of these algorithms are based on the data acquired from the patients, and the feature extraction. Especially, DL methods require large training datasets to avoid overfitting issue. Thus, experienced physicians have to contact to more patients or participants for sample collection, but it can be limited to find sufficient number of annotated samples in every class [3]. Data augmentation approaches have been studied to overcome unbalanced and small data problems [9].

Recently, adversarial learning strategies and the generative adversarial networks (GAN) have been widely performed for data augmentation and synthesis [10]. In 2014, the GAN structure was proposed by Goodfellow [11] based on generator and discriminator networks learning in adversarial manner. Thus, random noise input generator can produce fake signals, the loss function of the discrimination due to real and fake data determines the training process, shortly. After successfully training duration, any data(signal or image) can be obtained using generator network.

Radar signal synthesis was studied in 2019 [12]. The GAN was trained by finite-difference time-domain based signal for three class objects, and the generated signal was reported as indistinguishable after expert analysis. 1D deep convolutional GAN (DCGAN) architecture was performed on electrical machine signal generation for fault diagnosis [13]. The Frechet inception distance showed that the proposed DCGAN model was capable of converging to real signals. Pan et al. [14] investigated the GAN models on mechanical fault diagnosis under small sample. In 2021 [15], speech synthesis using MelGAN and WaveGAN was evaluated to generate raw audio from two or three letter Hindi words for helping dyslexic children. In addition, different GAN models were successfully applied to several recognition scenarios such as unseen fall detection [16], pedestrian classification using DCGAN [17], activity recognition [18] and text generation [19].

Retina image generation using medical imaging-GAN (MI-GAN) model was studied in [20]. MI-GAN yields higher dice coefficients on retina stare and drive datasets. SpeckleGAN was proposed in [3] to analyze the speckle layer modification compared to traditional GAN model. Synthetic ultrasound images were produced using small training set, and they have been compared according to Frechet inception distance and lumen segmentation. In another ultrasonic image enhancement study, drawback of the gathering large dataset was introduced, and the SpadeGAN model generated ultrasonic images were described nearly indistinguishable from real ones. Cervical lesion image enhancement using GAN was introduced in [21]. Wound healing time [22] was predicted using electronic medical records of the patients and GAN model. Real and synthetic time-series combination was reported to be have significant prediction improvement. EEG signals were also synthesized using GAN for improved steady state visual evoked potential (SSVEP) classification [23]. Multichannel EGC generations were some important applications of the adversarial learning approaches [24, 25].

In this paper, aforementioned large amount of data collection and feature extraction processes are tried to be simplified using conditional GAN (CGAN) synthesized feature space. Five biomedical datasets from UCI machine learning repository with different sample and feature sizes are applied to adversarial learning technique to generate enhanced feature space. Thus, increased accuracy rates are aimed to be obtained using small sample data. Twenty five percent of the original samples are used as training the discriminator and generator with latent noise to perform small data training process, and then the generator is fed with noise to obtain original dimensional samples. The statistics and the classification accuracies of the new datasets are compared to original datasets. Briefly, the effect of quarter sample collection on CAD system accuracy will be analyzed in next sections. The remainder is organized as follows: In Sects. 2.1 and 2.2, the short description of the GAN, and datasets are given. The proposed CGAN based feature space synthesis is presented in Sect. 2.3. Consequently, simulation results of the CGAN recognition are examined in Sect. 3, and the conclusions are drawn in Sect. 5.

Methods

Conditional generative adversarial network

The basic principle is the competition between the discriminator (D) and the generator (G) networks. (G) with random noise input tries to confuse the (D) while distinguishing real samples from the database and fake samples from (G). Formally, the $\mathrm{dZ}$ dimensional noise space Z $\mathrm{G}:\mathrm{Z}\to X$ that converts to data space capturing the distribution. D computes the probability from data and the G $D:X\to \left[0,1\right]$ with min-max value function described by [26].

$$\begin{array}{c}\hfill \underset{G}{min}\underset{D}{max}V(D,G)=E_D+E_G\end{array}$$ 1

where $E_D$ = $E_{x\sim p_{\mathit{data}}\left(x\right)}\left[logD\left(x\right)\right]$

$E_G$ = $E_{z\sim p_z\left(z\right)}\left[log\left(1-D\left(G\left(z\right)\right)\right)\right]$

$x\in X$ and $z\in Z$ values are sampled from data $p_{\mathit{data}}\left(x\right)$ and noise $p_z\left(z\right)$ distributions, respectively. During training, iteration steps are applied to D and G sequentially [11].

CGAN is the projection-based extension of the GAN model [27]. An extra Y space is applied to include label information from training data. The modified D and G can be described as

$$\begin{array}{c}\hfill G:Z\times Y\to XD:X\times Y\to \left[0,1\right]\end{array}$$ 2

and the loss function is changed to

$$\begin{array}{c}\hfill \underset{G}{min}\underset{D}{max}V(D,G)=E_D+E_G\end{array}$$ 3

where

$E_D$ = $E_{x,y\sim p_{\mathit{data}}\left(x,y\right)}\left[logD\left(x,y\right)\right]$

$E_G$ = $E_{z\sim p_z\left(z\right),p\left(y\right)}\left[log\left(1-D\left(G\left(z,y\right),y\right)\right)\right]$

Basic graphical descriptions of the GAN and CGAN are given in Fig. 1.

Fig. 1.

Architectures of The GAN and CGAN

Thus, with the help of the class embedded latent space in CGAN architecture, it is preferred for synthesis applications [27].

Datasets

Publicly available biomedical datasets from UCI machine learning repository [28] have been used to evaluate the proposed CGAN based enhancement. The five sets with different sample and feature sizes: Wisconsin Diagnostic breast cancer (WDBC) [29], Wisconsin breast cancer (WBC) [30], Liver [31], Parkinsons [32] and Heart-Statlog (Heart) [28] are applied to CGAN to extract enhanced feature space. These are summarized in Table 1.

Table 1.

Information about the datasets

Dataset	Sample	Ratio	Attribute
WDBC	569	62.9% Benign	30 (cell features)
WBC	699	62.5% Benign	9 (cell features)
Liver	583	28.5% Healthy	10 (Bilirubin,SGOT,etc.)
Parkinsons	197	24.6% Healthy	22 (fundamental frequency,etc.)
Heart	270	55.5% Healthy	13 (age,sugar,cholesterol,etc.)

WDBC and WBC datasets are basically microscopic investigation of the aspirated cells from the 569 and 699 participants, respectively. Liver sets consists of 583 patients data of age and gender with blood analysis results (Bilirubin, Sgpt, Sgot, albumin etc.). Parkinsons data has basically frequency characteristics of the participants voice. The last dataset is the Heart, which consists of age, sex, cholesterol, pressure, sugar analysis.

In this study, 25 % of the samples of each data will be used as training data for the proposed CGAN model, and then synthetic dataset having the same sample, feature size and class balance. Thus, the original dataset will be compared to synthesized feature space.

Proposed enhanced feature generation method

The feature space enhancement aims for two developments. (1) Generate the synthetic database with the original dimensions using 25 % of them as training. Thus, it can simply sample collection processes by physicians and practitioners. (2) CGAN synthesized datasets with original dimensional sample and feature space (generated from 25%) can outperform classification using original datasets. Thus, it is aimed to increase the performance of the classifiers using synthetic data. The proposed approach consists of a CGAN model and a performance evaluation processes given in Fig. 2.

Fig. 2.

The proposed CGAN based feature enhancement and evaluation

Twenty five percent of the samples in the dataset are used as training for CGAN shown in Fig. 3. After successful loss value (nearly 0.5 for D & G), normally distributed random noise equal sized to original is applied to G, considering the class balance. Thus, the same sized synthetic dataset will be obtained. The final stage is to evaluate and compare the generated enhanced set with original set.

Fig. 3.

The CGAN architecture

The generator consists of a projection, embedding, three transposed convolution (Tconv), two normalization, and two rectified linear unit (Relu). The 100 dimensional latent input(normally distributed random noise) is upsampled with the help of projection and Tconv computations, and then generated fake signal is applied to discriminator consisting of three convolution (Conv) and two leaky Relu (Lrelu) blocks. The main issue to construct CGAN model is the different feature dimensions of the datasets (from 9 to 30). For this reason, the kernel sizes and the stride values of the Tconv and Conv blocks are chanced to be make equal sized G output to D input. The details of the G and D for each dataset are given in Table 2 as MATLAB 2021 parameters.

Table 2.

The detailed CGAN architecture for each dataset

CGAN Blocks		WDBC	WBC	Liver	Parkinsons	Heart
G	Latent	1 $\times$1 $\times$100	1 $\times$1 $\times$100	1 $\times$1$\times$100	1 $\times$1 $\times$100	1 $\times$1 $\times$100
	Label	1 $\times$1 $\times$1	1$\times$1 $\times$1	1$\times$1 $\times$1	1 $\times$1 $\times$1	1 $\times$1$\times$1
	Proj.	2 $\times$1 $\times$1024	2$\times$1$\times$1024	2$\times$1$\times$1024	2$\times$1$\times$1024	2 $\times$1 $\times$1024
	Embed	$2\times$1	$2\times$1	$2\times$1	$2\times$1	$2\times$1
	Concat	$3\times$2	$3\times$2	$3\times$2	$3\times$2	$3\times$2
	Tconv1	5 $\times$2@512,s2$\times$1	3 $\times$1@512,s2$\times$1	5 $\times$1@512,s2$\times$1	3 $\times$1@512,s2$\times$1	5 $\times$1@512,s2$\times$1
	Norm.	decay=0.1,$\epsilon ={510}^{-5}$	decay=0.1,$\epsilon ={510}^{-5}$	decay=0.1,$\epsilon ={510}^{-5}$	decay=0.1,$\epsilon ={510}^{-5}$	decay=0.1,$\epsilon ={510}^{-5}$
	Relu1	–	–	–	–	–
	Tconv2	3 $\times$1@256,s2$\times$1	2 $\times$1@256,s2$\times$1,c[1,0,0,0]	3 $\times$1@256,s1$\times$1	3 $\times$1@256,s2$\times$1	2 $\times$1@256,s2$\times$1,c[1,0,0,0]
	Norm.	decay=0.1,$\epsilon ={510}^{-5}$	decay=0.1,$\epsilon ={510}^{-5}$	decay=0.1,$\epsilon ={510}^{-5}$	decay=0.1,$\epsilon ={510}^{-5}$	decay=0.1,$\epsilon ={510}^{-5}$
	Relu2	–	–	–	–	–
	Tconv3	2 $\times$1@32,s2$\times$1	1 $\times$1@1,s1$\times$1	2 $\times$1@1,s1$\times$1	2 $\times$1@32,s2$\times$1	1 $\times$1@1,s1$\times$1
	Norm.	decay=0.1,$\epsilon ={510}^{-5}$	N/A	N/A	decay=0.1,$\epsilon ={510}^{-5}$	N/A
	Relu3	–	N/A	N/A	–	N/A
	Tconv4	1 $\times$ 1@1,s1$\times$ 1	N/A	N/A	1 $\times$ 1@1,s1$\times$1	N/A
D	Label	1 $\times$1 $\times$ 1	1 $\times$1 $\times$1	1 $\times$1 $\times$1	1 $\times$1 $\times$1	1 $\times$1 $\times$1
	RealData	30 $\times$1 $\times$1	9 $\times$1 $\times$1	10 $\times$1 $\times$1	22 $\times$1 $\times$1	13 $\times$1 $\times$1
	Embed	30 $\times$1 $\times$1	9 $\times$1 $\times$1	10 $\times$1 $\times$1	22 $\times$1 $\times$1	13 $\times$1 $\times$1
	Concat	$3\times$2	$3\times$2	$3\times$2	$3\times$2	$3\times$2
	Conv1	9 $\times$1@512,s2 $\times$2,d1 $\times$1	3 $\times$1@512,s1 $\times$1,d1 $\times$1	5 $\times$1@512,s1 $\times$1,d1 $\times$1	9 $\times$1@512,s1 $\times$1,d1 $\times$1	7 $\times$1@512,s1 $\times$1,d1 $\times$1
	Lrelu1	scale=0.2	scale=0.2	scale=0.2	scale=0.2	scale=0.2
	Conv2	7 $\times$1@256,s2 $\times$2,d1 $\times$1	3 $\times$1@256,s1 $\times$1,d1 $\times$1	3 $\times$1@256,s1 $\times$1,d1 $\times$1	7 $\times$1@256,s2 $\times$2,d1 $\times$1	3 $\times$1@256,s1 $\times$1,d1 $\times$1
	Lrelu2	scale=0.2	scale=0.2	scale=0.2	scale=0.2	scale=0.2
	Conv3	3 $\times$1@1,s2 $\times$2,d1 $\times$1	3 $\times$1@128,s1 $\times$1,d1 $\times$1	3 $\times$1@128,s1 $\times$1,d1 $\times$1	3 $\times$1@1,s2 $\times$2,d1 $\times$1	3 $\times$1@128,s1 $\times$1,d1 $\times$1
	Lrelu3	N/A	scale=0.2	scale=0.2	N/A	scale=0.2
	Conv4	N/A	3 $\times$1@1,s1 $\times$1,d1 $\times$1	2 $\times$1@1,s1 $\times$1,d1 $\times$1	N/A	3 $\times$1@1,s1 $\times$1,d1 $\times$1

Proj projection, Norm Normalization, Concat concenation, s stride, d dilation, c crop

Datasets have 30, 9, 10, 22, and 13 dimensional feature spaces. From 100 dimensional latent space to the original dimensions for each dataset, different kernel sizes, stride and dilation parameters are adjusted so that the fake space is same, which is applied to D with real data. The last convolution of the D should be 1 dimensional to be class output. The solver parameters are also given in Table 3.

Table 3.

The ADAM solver parameters

miniBatchSize	Half size of the data
learnRate	0.0005
gradientDecayFactor	0.3
squaredGradientDecayFactor	0.999
L2Regularization	0.0001

Finally, the synthesized datasets (using 25% of the original data as training) are evaluated applying to machine learning algorithms. 10-Fold cross validation (10-Fold CV) error rates of the nearest neighbor (NN) and radial basis function (RBF) support vector machine (SVM) are given. The dataset is divided into 10-folds, and each of them is used testing samples sequentially to obtain error rate (E) less biased than train/test splits. The error rate is then estimated using each fold’s ($f=1,2,3,\dots ,10$) true negative (TN), true positive (TP), false negative (FN) and false positive (FP)

$$\begin{array}{c}\hfill E=\sum _{f=1}^{10}\left(\frac{F{N}_f+F{P}_f}{T{N}_f+F{N}_f+T{P}_f+F{P}_f}\right).\end{array}$$ 4

Results and discussion

In this section, the classification performances of the generated feature spaces will be analyzed into two parts. First, the imbalanced original class distribution is evaluated using the k-NN, and RBF-SVM and ANN. Second, balanced class generation is synthesized and compared to imbalanced results.

Imbalanced original class distribution

Aforementioned datasets have imbalanced class distributions(62.9, 62.5, 28.5, 24.6 and 55.5%), and the synthesized datasets with enhanced feature space will have the same distributions. Twenty five percent of the original data are used to train CGAN, and the new data with original dimensions and class ratios. The first step of the proposed method is the training using the parameters in Table 3. The loss graph during WDBC training is shown in Fig. 4.

Fig. 4.

The training loss for WDBC data

After 2500 iterations, the training processes is stopped. 100-dimesional latent space with the same label order is applied to G for obtaining synthetic enhanced datasets ($\tilde{W}DBC$, $\tilde{W}BC$, $\tilde{L}iver$, $\tilde{P}arkinsons$, and $\tilde{H}eart$). In Fig. 5, the original and synthetic feature vector with 2D distribution are graphically presented.

Fig. 5.

The distributions of the WDBC and $\tilde{W}DBC$ ($x_1$ and $x_2$ denotes cell radius and texture, respectively. Feature graph shows the 1st sample of class 0)

The last step is the performance evaluation of the generated datasets. These are classified using k-NN, RBF-SVM and ANN for comparison. 10-Fold CV error rates of the original(e.g, WDBC) and the synthetic imbalanced (e.g, $\tilde{W}DBC$) and balanced (e.g, $\tilde{W}DB{C}_B$) sets are shown in Fig. 6.

Fig. 6.

10-Fold CV error rates of the original and the synthetic (imbalanced & balanced) datasets ($k=1,2\dots 5$, $\sigma =0.1,0.4,0.7\dots 5$ and $N=2,4,6\dots 20$)

Euclidean distanced k-NN up to k=5, RBF kernel SVM with $\sigma$ value up to 5, and one hidden layer ANN having up to 20 neurons are simulated. The worst and the best results in Fig. 6 are summarized in Table 4.

Table 4.

The worst &Best 10-Fold CV errors (%) of the original and synthetic datasets

Datasets	k-NN	RBF-SVM	ANN
WDBC	5.44–3.33	3.72–2.10	4.92–4.39
$\tilde{W}DBC$	5.62–4.39	3.72–1.23	2.81–1.93
$\tilde{W}DB{C}_B$	6.49–4.38	5.08–0.70	3.15–0.70
WBC	4.53–3.36	25.18–2.78	13.61–10.83
$\tilde{W}BC$	0.14–0.00	34.99–0.00	6.29–1.90
$\tilde{W}B{C}_B$	0.58–0.014	32.01–0.014	4.67–2.04
Liver	35.40–31.08	29.87–27.80	33.33–27.97
$\tilde{L}iver$	19.17–13.98	28.49–14.30	23.31–20.03
$\tilde{L}ive{r}_B$	31.37–26.37	50.0–17.41	30.34–25.86
Parkinsons	10.25–6.15	24.61–7.17	23.07–16.92
$\tilde{P}arkinsons$	9.74–6.15	24.61–1.53	10.25–6.15
$\tilde{P}arkinson{s}_B$	11.22–7.14	51.02–1.53	13.77–8.16
Heart	24.81–17.40	44.44–15.55	27.77–20.74
$\tilde{H}eart$	2.59–1.11	44.44–0.00	5.92–0.03
$\tilde{H}ear{t}_B$	3.57–2.55	51.02–0.51	5.10–1.02

The WDBC with k-NN and RBF-SVM outperforms $\tilde{W}DBC$ by rate of approximately 1.0%, but $\tilde{W}DBC$ increases by 2%. $\tilde{W}BC$ classifications have zero errors for k-NN and RBF-SVM, and 1.90% for ANN while the originals are 3.36%, 2.78% and 10.83%. All classifiers yield 17%, 13.5% and 7.94& error decreases for $\tilde{L}iver$. Error rates of RBF-SVM and ANN are decreased by 5.64% and 10.77%, respectively while k-NN are the same. Especially, higher decrease rates are existed as 16.29%, 15.55% and 20.71% for $\tilde{H}eart$. Generally, 15 tasks are performed (5 synthetic data and 3 classifiers) and 13 synthetic data classification tasks (up to 20.71% increase) outperforms the original ones except WDBC with k-NN and RBF-SVM (1–2% decrease).

Balanced distribution

Another aim is to eliminate imbalanced class disadvantage during data acquisition or reaching sufficient participants in clinics. For this reason, CGAN training will be same to steps in the previous section, but generation will have equal sized class distribution (e.g, $\tilde{W}DB{C}_B$ will have 285 class 0 and class 1 instead of existing 357 samples of class 0). From this point of view, k-NN, RBF-SVM and ANN are performed on balanced new data ($\tilde{W}DB{C}_B$, $\tilde{W}B{C}_B$, $\tilde{L}ive{r}_B$, $\tilde{P}arkinson{s}_B$ and $\tilde{H}ear{t}_B$), and 10-Fold CV error rates are given in previous section in Fig. 6 and Table 4.

Generally, the balanced synthetic datasets have similar trends to imbalanced synthetic datasets. $\tilde{W}DB{C}_B$ and $\tilde{W}DBC$ (class ratio is 62.9%) have the same best accuracy for k-NN , but the $\tilde{W}DB{C}_B$ with RBF-SVM and ANN outperforms by the rate of 0.63% and 0.83%. For $\tilde{W}BC$(ratio of 62.5%), the error rates are lower than $\tilde{W}B{C}_B$ (nearly 0.014%). $\tilde{L}iver$ (ratio of 28.5%) have error rates of 13.98%, 14.30% and 20.03% while $\tilde{L}ive{r}_B$ yields 26.37%, 17.41% and 25.86%. Balancing has similar effect on $\tilde{P}arkinson{s}_B$ and $\tilde{H}ear{t}_B$, yielding nearly 1–2% error increases when compared to the balanced synthetic datasets.

Discussion

The CGAN training process lasts up to 17.7 minutes on Intel i7 6500U@2.5GHz processor with 16GB RAM without GPU support, can be capable of enhancing feature space using only 25% of original datasets. The generated imbalanced and balanced datasets can have higher accuracy rates than originals. On the other hand, the balancing approach can increase the success rate or cannot chance when classes are nearly balanced. 10-Fold CV errors are affected by 1–2% when data have serious class imbalances (liver and parkinsons) due to the sensitivity(discrimination capability of class1) and specificity(discrimination capability of class0) values of the imbalanced original dataset. In other words, when class 0 or class 1 samples are hard to be distinguished, which is yielding imbalanced sensitivity and specificity scores, the increased size of hardly distinguished samples after balanced generation can decrease the accuracy rates. To demonstrate it, the sensitivity and the specificity scores (RBF-SVM) of the original datasets are computed as 91.51% and 97.48% for WDBC, 94.98% and 96.85% for WBC, 0 and 100% for Liver, 97.28% and 56.25% for Parkinsons, 78.33% and 89.33% for Heart.

Conclusion

The synthetic sample generation with enhanced feature space is proposed in this paper. Two strategies have been evaluated using conditional generative adversarial network (CGAN) models. Higher accuracy rates using small sample datasets are targeted. CGAN models use 25% of the original datasets(WDBC, WBC, Liver, Parkinsons and Heart) to generate original dimensional datasets ($\tilde{W}DBC$, $\tilde{W}BC$, $\tilde{L}iver$, $\tilde{P}arkinsons$, and $\tilde{H}eart$). The nearest neighbor (NN), radial basis function support vector machine (RBF-NN), and artificial neural network (ANN) classifiers are performed on the original and synthetic datasets to obtain 10-fold cross-validation (10-fold CV) error, and the results shows that synthetic datasets outperform classification with the original ones. The proposed CGAN based enhanced feature space model is capable of generating any sized datasets with required class distribution and increasing success rate of the classification algorithms.

Funding

The authors have not disclosed any funding.

Availability of data and materials

Data sharing not applicable to this article.

Code availability

Available upon request.

Declarations

Conflict of interest

The author has no relevant financial or non-financial interests to disclose.

Ethical approval

This article does not contain any studies with human participants or animals performed by the author.

Consent to participate

Not applicable.

Consent for publication

Not applicable