Use of Artificial Neural Networks and PCA to Predict Results of Infertility Treatment in the ICSI Method

Abstract

Background:

Intracytoplasmic sperm injection (ICSI) or microinjection is one of the most commonly used assisted reproductive technologies (ART) in the treatment of patients with infertility problems. At each stage of this treatment cycle, many dependent and independent variables may affect the results, according to which, estimating the accuracy of fertility rate for physicians will be difficult

Objective:

This study aims to evaluate the efficiency of artificial neural networks (ANN) and principal component analysis (PCA) to predict results of infertility treatment in the ICSI method

Material and Methods:

In the present research that is an analytical study, multilayer perceptron (MLP) artificial neural networks were designed and evaluated to predict results of infertility treatment using the ICSI method. In addition, the PCA method was used before the process of training the neural network for extracting information from data and improving the efficiency of generated models. The network has 11 to 17 inputs and 2 outputs.

Results:

The area under ROC curve (AUC) values were derived from modeling the results of the ICSI technique for the test data and the total data. The AUC for total data vary from 0.7670 to 0.9796 for two neurons, 0.9394 to 0.9990 for three neurons and 0.9540 to 0.9906 for four neurons in hidden layers

Conclusion:

The proposed MLP neural network can model the specialist performance in predicting treatment results with a high degree of accuracy and reliability

Introduction

Reproductive system disorders are known as a disease [ 1 ]. Infertility is a negative concept in ancient myths and civilizations [ 2 ] and one of the most important social problems creating structural effects in modern society [ 3 ]. According to the World Health Organization (WHO), in clinical terms, infertility is reproductive system disease, in which couples do not succeed in childbirth (clinical fertility) after twelve months or more of regular sex without using preventive methods [ 4 ]. On average, from every six to seven couples, one couple suffers from infertility [ 5 ]. At present, more than 80 million couples in the world and more than one and a half million Iranian couples are infertile. This estimate is based on the World Health Organization (WHO) report. According to these reports, on average, between 10% to 15% of couples in the world faces an infertility problem [ 6 , 7 ]. However, this has been reported in Iran up to 22%, which is worth considering [ 8 ]. Generally, male, female, and unknown factors are the causes of infertility. About 25%, 20%, 30% and 10% of the causes of infertility are related to ovulation disorders, damage in uterine tubes, male infertility factors, uterine disorders, respectively, and in 25% of cases, infertility occurs due to unknown causes. About 40% of infertility problems are related to both men and women. Uterine or endometrial factors, gamete or embryo defects, and pelvic conditions, such as endometriosis, may also be the cause of infertility [ 9 ].

Based on diagnosis of the infertility reasons by the physician, there are usually three types of treatments, including medical treatment to restore reproductive ability (such as taking medications to stimulate ovulation), treatment using surgery to restore fertility (such as laparoscopy for Endometriosis destruction) and assisted reproductive techniques (ART), which often involve tampering with gametes or embryos [ 9 ]. Since 1970, the use of assisted reproductive techniques has created a new perspective on the treatment of infertility problems, so that the approach of specialists changed from research to practice and the probability of fertility increased [ 1 , 10 , 11 ]. The goal of assisted reproductive techniques is to overcome unknown problems [ 12 ]. Using the techniques increase the chances of being pregnant in couples who have not had any children in spite of numerous surgical and drug therapies. Commonly used assisted reproductive techniques include intrauterine insemination (IUI), In-vitro fertilization (IVF) and intra-cytoplasmic sperm injection (ICSI) or microinjection. In the ICSI technique, the sperm is injected into the egg in the laboratory, followed by fertilization and cell division, and embryo is formed. This method involves steps such as ovarian stimulation, extracting the eggs, injection of sperm into the egg, fertilization and embryo transfer [ 13 ].

Over the past two decades, the use of knowledge and concepts of software and new technologies has grown in the field of medicine. With the growth of these technologies, generations of computer systems have been introduced that have been widely used in healthcare. Today, the use of decision support systems (DSS), because of these advances in the field of software and computing knowledge in medicine in order to reduce the diagnostic time and improve the accuracy of the diagnosis, has become an essential and necessary step [ 14 ]. Clinical decision support systems are interactive computer programs designed to assist physicians and other healthcare professionals in making decisions. In other words, these systems combine human resources with computer capabilities to enhance the quality of decision-making. The main purpose of decision support systems is to assist physicians at point of care, which means that a physician can interact with this system, and get help from the system in the analysis of patient data, diagnosis, prediction of treatment outcomes, and other clinical activities.

After decades, infertility treatment has been provided in Iran, physicians and patients are interested in predicting the results of treatment [ 15 ]. The two favorable events can be seen due to providing a reasonable prediction of the success rate of treatment in infertile couples. First, the couples will have a cost-benefit estimate and can decide on the treatment accordingly. Second, the physician can choose the best and most affordable treatment option depending on the patient’s condition. It is very important for the physician to reduce the duration of treatment by making the right decision and prevent the side effects that may physically and mentally damage the patient [ 16 ].

Due to the prominence of the human health in medical researches, proper modeling and predicting of the results will be more important; therefore, methods should be used which predictions based on them have the least error and the highest accuracy. One of the methods that have attracted the attention of many researchers is artificial neural networks (ANNs). Artificial neural networks have shown many potentialities in the fields of medicine, biology, engineering, robotics, and economics. In medical applications, these types of networks are commonly used to increase the accuracy of diagnosis and predict treatment results. The topic of Artificial Neural Networks is the simulation of learning power of the human brain and its implementation as computer algorithms. Neural networks are made up of simple operating elements (neurons) in parallel. These elements are inspired from biological nervous systems [ 5 ]. The neural network operates in a manner similar to a “living mind”; therefore, the neural network takes some time to learn and then is used operationally. The neural network maintains its observations in the form of its internal parameters. In fact, the repetition of each observation changes the internal parameters of the network to maintain the relations governing the observations. Therefore, it usually has the stability in practice, which in dealing with the general similar specimens, in addition to a proper function and negligible error [ 5 ]. Neural networks have different functions in dealing with different issues depending on its internal structure. Therefore, the choice of network structure, which is proportional to the problem under study, is very important. On the other hand, the proper selection of the network, which learns algorithm, will be very effective in its operation.

Material and Methods

In this study that is an analytical study, the demographic, clinical and laboratory data were extracted from the files of patients referred to the infertility treatment department of Ayatollah Taleghani hospital, educational and therapeutic center in Tehran for ICSI treatment and the database was formed. The database of the study includes 33 and 29 patients with unsuccessful and successful treatment; therefore, 62 samples form the study data. The sample size was selected qualitatively and according to the limitations in the number of cases and the problem requirements to achieve the desired results.

The patients under study had an average age of 31 years. Effective features on the treatment result were collected by studying specialized literature and related sources [ 15 , 17 - 19 ]. Then, these features were categorized in a checklist, evaluated and then prioritized by infertility specialists. Then, data was recorded on the computer for further analysis. MATLAB software version R2017a has also been used to process, modeling, and simulation of study. Each of the samples recorded in the database contains 18 variables, that 17 variables describe the ICSI cycle and a variable represents the output class. Tables 1 and 2 show, respectively, the variables describing the ICSI cycle and the statistical indices related to some of the most important variables.

Table1.

Variables describing the ICSI cycle.

Quantitative variables	Qualitative variables
Age of woman	Type of cycle
Body mass index (BMI)	Abortion history
Duration of infertility	Live birth history
Number of previous IUIs	Surgery history
Number of previous ICSIs	Endometriosis
FSH level	Medical Disease history
AMH level	Result of ICSI cycle
Average AFC
Thickness of endometrium
Number of embryos formed
Number of embryos transmitted

Table2.

Range and statistical indices related to some of the most important variables.

Variable	Min	Max	Mean	Variance
Age of woman	22	43	31.03	22.39
Body mass index (BMI)	18.5	34	24.62	9.73
Duration of infertility	1	15	4.72	13.82
FSH level	0	17.7	5.69	11.09
AMH level	0.1	20.1	4.86	22.93
Average AFC	1	12	8.34	11.73

Artificial Neural Networks

The computational core of the decision-making system in this study is the artificial neural network method. Figure 1 shows the neural network training procedures. This iterative method starts with data collection and preprocessing to make training process more efficient. Then, the collected data are divided into three sections, including training, validation and testing.

Figure1.

Flowchart of Neural Network training process [ 1 ].

After categorizing the data, the type of network such as multilayer, ‎competitive, dynamic, network architecture such as number of layers, number of neurons and training algorithm that are appropriate for solving our problem should be selected. After training of network, the network performance is analyzed to find problems with the data, the network architecture, or the training algorithm. Then the overall procedure is repeated until the network performance is acceptable [ 20 ].

Neural networks are divided into different methods, including topology, structure, and learning, and each one has a good performance in certain applications. Multilayer Perceptron (MLP) neural networks with backpropagation-based learning algorithms is one of the most commonly used networks in practical problems. It has been theoretically proved that a multilayer network is able to model and simulate any non-linear systems if the proper structure is selected. Figure 2 shows that the neural network used in this study consists of two layers of the neurons.

Figure2.

Multilayer Perceptron (MLP) Artificial Neural Network with two layers.

In this modeling, scaled conjugate gradient back-propagation (SCGBP) training algorithm is used to train networks. This algorithm has more interest in pattern recognition problems and also has advantages over other algorithms such as search algorithm and minimization algorithm. The high accuracy, reduction of the number of repetitions in the learning process and thus the reduction of learning time are its most important advantages.

The SCGBP algorithm produces generally faster convergence compared to steepest descent back-propagation techniques and even conventional conjugate gradient algorithms. This algorithm is fully automated and independent of user-defined parameters. In addition, the hyperbolic tangent sigmoid transfer functions in the hidden layer and the softmax transfer functions in the last layer of the networks and the cross entropy performance index have been used to evaluate the performance of the networks. The area under ROC curve (AUC) criterion was used to evaluate and compare the models. Out of all samples, randomly, about 70%, 15% and 15% were selected as training data, validation data and test data for training and evaluation of networks, respectively. The neural network training process has been repeated 15 times for each set of network inputs (to avoid being caught in the local minima and reaching the absolute minimum of the performance surface), and finally, the optimal neural network has been selected.

Principal Component Analysis

Principal component analysis (PCA) is a tool for finding patterns in high-dimensional data and a feature extraction method [ 21 ]. Moreover, the components of the transformed vector are arranged in such a way that the first and second ‎components have the greatest and next greatest variance, respectively. Commonly we preserve just the first few components of the transformed vector, which make up the largest variance in the original vector. This results in a large reduction in the dimension of the input vector, if the original components are highly correlated. The disadvantage of applying PCA is that it only considers the linear relationships between the input vector components. Therefore, if the dimension of a vector using a linear transform reduces, this could lead to the loss of some nonlinear information. As the main strength of neural networks is to obtain the nonlinear basis functions allowing to ‎handle nonlinear and multidimensional dependencies; therefore, it should be consided that‏ the use of ‎PCA to reduce the input dimensions must be implemented before applying the inputs for the neural network [ 20 ].

Results

In this section, the results of modeling the ICSI cycles with MLP neural network and the features extracted by the PCA method are expressed. In this modeling, an artificial neural network with two layer of neurons is used, in which the number of hidden layer neurons in the network varies between two to four. The network has 11 to 17 inputs (principle components obtained from the PCA method) and two outputs (representing the corresponding class). Table 3 shows the total variances associated with each set of principle components and Figure 3 shows percent of the variance explained by each principal component. In addition, Tables 4, 5 and 6 show, respectively, the AUC values derived from modeling the results of the ICSI technique for the test data and the total data for the two, three, and four neurons in the hidden layer of the network. As shown in tables, the AUC for total data vary from 0.7670 to 0.9796 for two neurons in hidden layer, 0.9394 to 0.9990 for three neurons in hidden layer 0.9540 to 0.9906 for four neurons in hidden layer.

Table3.

Total variances associated with each set of principle components.

Number of PCs	Percent of the Total Variance
11	87.05
12	90.14
13	93.09
14	95.47
15	97.49
16	99.18
17	100

Figure3.

Percent of the variance explained by each principal component.

Table4.

AUC values related to the test and total data for the two neurons in the hidden layer of the network.

Number of PCs	Test Data AUC	Total Data AUC
11	0.84	0.7670
12	0.84	0.8380
13	0.84	0.9018
14	0.96	0.9572
15	0.96	0.9791
16	1.00	0.9822
17	0.96	0.9796

Table5.

AUC values related to the test and total data for the three neurons in the hidden layer of the network.

Number of PCs	Test Data AUC	Total Data AUC
11	0.76	0.9394
12	0.64	0.9227
13	0.80	0.9289
14	1.00	1.0000
15	1.00	0.9801
16	1.00	0.9979
17	0.96	0.9990

Table6.

AUC values related to the test and total data for the four neurons in the hidden layer of the network.

Number of PCs	Test Data AUC	Total Data AUC
11	0.76	0.9540
12	0.72	0.9279
13	0.76	0.9812
14	0.96	0.9979
15	1.00	1.0000
16	1.00	0.9979
17	1.00	0.9906

Discussion

In this research, the MLP neural network trained by SCGBP algorithm with inputs obtained from the PCA method was used to predict result of the ICSI cycles. Reports in the literature reveal that preceding the neural network training process with use of the PCA method for inputting data can increase the predictive power of obtained models [ 22 ]. Ioele et al. suggest that the combined use of PCA and ANN usually improves the training speed and enhances the robustness and quality of created models [ 23 ]. Both ANN and PCA-ANN approaches were compared, with the second approach taking precedence over the first. The authors illustrated that both of the ANN models were reliable, but that the PCA-ANN model demonstrated a better prediction quality, showing lower residual errors [ 22 ]. In addition, orthogonalization of the initial space, dimensionality can also be reduced in the PCA method. This method is more helpful if the data contains correlated input variables. Chen et al. prove that the combination of PCA and ANN is a valid approach to use in classification and pattern recognition problems [ 24 ]. The PCA-ANN method also has many applications in medicine. Buciński et al. showed that PCA and ANN methods are able to predict recurrence in breast cancer using data analysis based on long-term records. They proved that PCA and ANN analyses are capable of practically testing an unlimited number of reciprocally or unrelated factors [ 25 ]. Jilani et al. use PCA-ANN technology for classification of hepatitis-C patients. The results of the process used by them are, including 99.1% and 100% for training and testing data, respectively [ 26 ].

In the current study, comparing the AUC of different models showed that the MLP network with the three neurons in its hidden layer and 14 principle components as its inputs could predict the results of the ICSI cycles with a higher level of reliability. Neural networks performance was measured using the Cross Entropy (CE) function and according to Figure 4, the best performance of the optimal network in the validation data obtained at epoch 19 was about 0.0197. The accuracy obtained from proposed optimal model is significant. In fact, obtained AUC for this model indicates that this model can predict treatment results with 100% accuracy.

Figure4.

Neural Network performance diagram during the training process.

Conclusion

Regarding the results of this study, it can be admitted that the multilayer perceptron neural network model using the principle components obtained by the PCA method can model the performance of a specialist with reasonable accuracy; therefore, it can be used to predict the results of treatment with the ICSI technique before actualize various steps of this cycle. On the other hand, this model helps physicians to recognize the outcome of the treatment better. It is possible to minimize the physical and psychological effects of the ICSI treatment cycles using the effective factors identified in this study and optimal model introduced. Furthermore, designing and implementing a clinical decision-making system and appropriate user interface based on designed artificial neural network and proposed topological structure are suggested.