A Combination of Artificial Intelligence with Genetic Algorithms on Static Time-Lapse Images Improves Consistency in Blastocyst Assessment, An Interpretable Tool to Automate Human Embryo Evaluation: A Retrospective Cohort Study

Abstract

Background:

In recent times, various algorithms have been developed to assist in the selection of embryos for transfer based on artificial intelligence (AI). Nevertheless, the majority of AI models employed in this context were characterized by a lack of transparency. To address these concerns, we aim to design an interpretable tool to automate human embryo evaluation by combining artificial neural networks (ANNs) and genetic algorithms (GA).

Materials and Methods:

This retrospective cohort study included 223 human blastocyst time-lapse (TL) images taken at 110 hours post-injection. All the images were evaluated by five embryologists from different clinics in terms of blastocyst expansion (BE), quality of the inner cell mass (ICM), and trophectoderm (TE). The embryo database was used to develop an AI system (70% training, 15% validation, and 15% test) for automate blastocyst assessment. The entire set of images underwent a standardization process, followed by processing and segmentation using Matlab software. The resulting quantified variables were utilized in AI techniques (ANN and GA). Finally, the accuracy and performance of the automation tool was assessed with the area under the receiver operating characteristic (ROC) curve (AUC). Then, the level of agreement among embryologists and between embryologists and the AI system was compared with Kappa Index.

Results:

The overall agreement among embryologists was low (Kappa: 0.4 for BE; and 0.3 for TE and ICM). The AI tool achieved higher consistency (Kappa 0.7 for BE and ICM; and 0.4 for TE). The AI exhibited high accuracy in classifying BE (test 81.5%), ICM (test 78.8%), and TE (test 78.3%) and better performance for BE (AUC 0.888-0.956) than for ICM (AUC 0.605-0.854) and TE (AUC 0.726-0.769) assessment.

Conclusion:

Our AI tool highlighted the superior consistency of AI compared to human operators in grading blastocyst morphology. This research represents an important step towards fully automating objective embryo evaluation.

Introduction

The field of assisted reproductive medicine has placed significant emphasis on embryology (1), with several advancements in time-lapse (TL) technology playing a crucial role (2, 3). While continuous embryo monitoring has proven advantageous in embryo selection, its applications beyond tracking embryo development remain uncertain. The standard practice for evaluating and selecting embryos on day five of embryo development is based on classical morphology at the blastocyst stage (4).

The scientific community in reproductive medicine has established criteria for evaluating embryo morphology, along with standardized systems for grading and selection (5). However, embryo evaluation still presents challenges due to its subjective and inconsistent nature. The assessment of blastocyst expansion (BE), inner cell mass (ICM), and trophectoderm (TE), which are considered in the morphological grading, can vary depending on the evaluator and the tools used (6, 7). To address these obstacles, the use of artificial intelligence (AI) with artificial neural networks (ANNs) has gained significant attention in research and fertility conferences worldwide (8).

Accurately assessing embryo viability based on morphology is crucial for optimizing in vitro fertilization (IVF) treatment outcomes. AI holds tremendous potential in processing complex datasets, detecting subtle yet crucial patterns, and providing a more objective assessment of embryos compared to humans. However, a comprehensive literature review reveals that significant progress is still required before ethically implementing AI for this purpose (9). Additionally, key challenges of AI include data privacy concerns, lack of human-like creativity, and the potential for job displacement (10, 11). Currently, most AI models exhibit opacity, making them challenging to interpret. This gives rise to various epistemic and ethical concerns, including issues of trust, potential generalization problems across different populations, adverse economic implications for IVF clinics, potential misalignment with patient values or broader societal implications. To address these concerns, the use of interpretable models, which are designed to be easily understandable and explainable by humans, could be a solution.

Over the past few years, there has been a growing interest in utilizing AI for supporting embryo quality assessment. Numerous algorithms have been developed to analyze static images or TL videos of embryos, aiming to assist in the selection of embryos for transfer (12-19). Digital image processing involves extracting information on size, color scale, and saturation using mathematical algorithms. Several key variables, such as circularity, radius, uniformity, texture, luminosity, and color scale, can be measured and analyzed from blastocyst images (20, 21) to design interpretable models. By leveraging AI, embryo evaluation can achieve greater accuracy and consistency, overcoming the inherent subjectivity associated with classical methods due to variations in embryologists' training, experience, and instruments (8, 22). AI tools have already demonstrated success in classifying human and non-human mammalian embryos, indicating their potential in assessing embryo quality, predicting outcomes, and establishing standardized grading practices (23-26).

In paralel, there has been a growing exploration of the potential of ANNs in conjunction with other programming techniques like GA, to enhance and optimize results. GAs are evolutionary algorithms inspired by both Darwin’s theory of evolution and Gregor Mendel’s laws of genetics. This approach involves generating a random population of individuals, which are then evaluated and selected for the next generation based on their fitness. This iterative process continues, creating new populations, until a satisfactory solution is achieved (27, 28). In the context of this study, the individuals referred to are the ANNs themselves (29).

The objective of the current study is to design an interpretable tool to automate human embryo evaluation by combining ANNs and GA. To justify its use, we first compared the embryologist-embryologist agreement in embryo evaluation. Secondly, we compared the match between the embryologist and the decision based on machine learning.

Materials and Methods

Study design

In this retrospective cohort study, we included 223 human blastocyst images (from 74 patients) obtained using the Embryoscope® (Vitrolife, Sweden) from January to June 2020. All the TL microscopy (TLM) images of human embryos captured at 110 hours post-injection (hpi) were evaluated and graded for expansion, inner cell mass, and TE by five experienced embryologists from three different countries (more than 5 years of experience). Then, the images were randomly sorted and designated for use as the input dataset for the AI system, with 70% allocated for training, 15% for validation, and 15% for testing purposes.

Blastocyst morphology evaluation and image analysis processing

The blastocysts were routinely evaluated and assigned scores based on the grade of expansion, inner cell mass, and TE appearance, following the widely accepted and adapted Gardner Grading System (GGS) (4). The cohort’s images were independently assessed by five experienced embryologists using the GGS criteria (Fig .S1, See Supplementary Online Information at www.ijfs.ir).

Fig 1.

Schematic structure of the Artificial Intelligence technique applied to blastocyst classification.

Then, all blastocyst images were captured using the EmbryoScope® (Vitrolife, Denmark) and the most frequently assigned value for BE, ICM, and TE among the five embryologists (i.e., the mode) was used to train the AI tool.

In this study, the AI model utilized an algorithm that automatically imports blastocyst images into MatLab® software and standardizes them. The standardization process involved converting the images into grayscale, adjusting the resolution and aspect ratio (30), and converting them to 8-bit grayscale (Fig .S2, See Supplementary Online Information at www.ijfs.ir). To ensure uniformity for analysis, images with different resolutions and proportions were adjusted accordingly. Additionally, 1% of the pixel information between light and dark pixels was saturated in each blastocyst image, facilitating the subsequent segmentation step (31). Following standardization, the blastocyst was isolated from the surrounding image by modifying the image gradient to define its boundaries. This was accomplished using Hough’s Transform (32), a method that detects the circumference of the blastocyst (Fig .S3, See Supplementary Online Information at www.ijfs.ir).

The image processing involved multiple algorithms that operate independently. The Gray Level Co-Occurrence Matrix (GLCM) was used for texture analysis, the Watershed transformation was employed for image segmentation, and the Gabor filter was utilized to differentiate various textures within the image. After applying these techniques, the TE and ICM regions were identified separately (Fig .S4, See Supplementary Online Information at www.ijfs.ir). Following the image segmentation process, the extracted information provided a numerical vector that describes the key features of a blastocyst image in quantities suitable for digital applications (i.e., to mathematize the quality of the inner cell mass, the trophectoderm, and the degree of blastocyst expansion).

Statistical analysis

Artificial Intelligenceand genetic algorithms

After analyzing and processing the images, the quantified variables of the embryos underwent the GA technique. This technique involved constructing a population of individuals that represent different architectures of ANNs. The ANNs were randomly generated to form the initial population, with 100 or 200 individuals in each population. Each individual in the population represented a specific ANN configuration. The parameters that defined the ANNs include the maximum and minimum number of neurons per layer, the number of layers, the learning rate, the transfer functions (i.e. models that are a frequency-domain representation of linear timeinvariant systems) and the learning functions (i.e. training functions that facilitate the training or learning process of artificial neural networks). The development of these networks was carried out on the Matlab® platform.

Once the ANNs were generated as individuals, the entire population underwent training, validation, and testing using the blastocyst image database. The most suitable individuals were selected based on the smallest error in the test set when applying the ANN technique. A schematic structure of the AI system used for embryo classification according to Gardner’s grading system is depicted in Figure 1. Finally, the performance of the predictions was compared using the area under the receiver operating characteristic (ROC) curve (AUC).

Cohen’s Kappa Coefficient

To assess the agreement between observers and within a single observer, the Cohen’s Kappa coefficient (k) was employed. The calculations were performed using SPSS statistical software (version 24.0.0.1, IBM Corp, USA). The kappa coefficient is a statistical measure that takes into account chance agreement and quantifies the level of agreement between two sets of data.

The interpretation of the kappa coefficient values is as follows: absolute agreement (k=1), excellent agreement (k ≥0.80), good agreement (k=0.60-0.79), moderate agreement (k=0.40-0.59), poor agreement (k=0.20-0.39), very poor agreement (k <0.20), and no agreement (k=0), as suggested. For clinical laboratory applications, a minimum acceptable classification was defined as excellent agreement (k ≥0.80).

To evaluate inter-observer variability, the classification among embryologists was compared for each embryo grading parameter (BE, ICM and TE). Intra-observer agreement was determined by assessing each embryologist's consistency when rating the images. Each embryologist rated all images three different times, and from these classifications, the Kappa value was calculated.

Ethical considerations

The procedure and protocol for analyzing embryos were approved by the Institutional Review Board (IRB reference 1902-VLC-018-MM), which monitors and approves database analyses and clinical IVF procedures for research at IVIRMA Global.

Results

Artificial intelligence tool for blastocyst assessment

The overall accuracy of the AI system in analyzing key blastocyst features was acceptably high (Table 1).

Table 1.

Overall accuracy of the artificial intelligence tool for blastocyst expan- sion, inner cell mass and trophectoderm quality on the test and training data

Morphological feature	Training	Test
Blastocyst expansion	93.9	81.5
Inner cell mass	93.0	78.8
Trophectoderm	78.8	78.3

Data are presented as the percentage of correct predictions between the AI tool and the conventional approach by embryologists.

The AI tool demonstrated higher performance in grading BE (AUC from 0.923 to 1) compared to ICM (AUC from 0.605 to 0.854) or TE quality (AUC from 0.726 to 0.769) on the test dataset (Table 2).

Table 2.

Performance of the model with AUC values for blastocyst expansion, ICM and trophectoderm quality on test and training data

Morphological feature	Training	Test
Blastocyst expansion
Class 2	0.956	1
Class 3	0.888	0.923
Class 4	0.946	1
Inner cell mass
Grade A	0.979	0.854
Grade B	0.967	0.691
Grade C	0.961	0.605
Trophectoderm
Grade A	0.900	0.726
Grade B	0.863	0.733
Grade C	0.914	0.769

Class 2; Early blastocyst, Class 3; Full blastocyst, Class 4; Expanded blastocyst, AUC; Area under the curve and ICM; Inner cell mass.

Degree of agreement among embryologists

Five embryologists from different clinics were involved in grading blastocysts based on their expansion, inner cell mass, and trophectoderm. There was considerable variation in the assigned grades, resulting in low agreement values measured by the Kappa coefficient (0.4 for expansion and 0.3 for ICM and TE) (Table 3).

Table 3.

Cohen’s Kappa coefficients from the evaluation by five embryologists of the three features of blastocysts (grade of expansion, inner cell mass and trophectoderm)

Morphological feature	Class 2	Class 3	Class 4	Average
Blastocyst expansion
Kappa	0.422 (0.381-0.464)	0.222 (0.181-0.264)	0.508 (0.466-0.549)	0.371(0.342-0.400)
P value	<0.001	<0.001	<0.001	<0.001
Inner cell mass	Grade A	Grade B	Grade C	Average
Kappa	0.404(0.363-0.446)	0.178 (0.137-0.220)	0.285 (0.243-0.326)	0.267 (0.236-0.298)
P value	<0.001	<0.001	<0.001	<0.001
Trophectoderm	Grade A	Grade B	Grade C	Average
Kappa	0.353 (0.312-0.395)	0.209 (0.167-0.250)	0.376(0.334-0.417)	0.299(0.268-0.331)
P value	<0.001	<0.001	<0.001	<0.001

Kappa values are expressed with confidence intervals (CI) 95% between brackets. Class 2; Early blastocyst, Class 3; Full blastocyst, and Class 4; Expanded blastocyst.

Degree of agreement between the automated system and the mode of embryologists

When comparing the results obtained by the AI technique with the modal value (the value calculated based on the mode of the results provided by the five embryologists), a significant improvement in blastocyst classification agreement was observed, as indicated by the assigned Kappa values (Table 4).

Table 4.

Cohen’s Kappa coefficients from the evaluation by artificial intelligence compared by the mode from 5 embryologist of the three features of blastocysts: grade of expansion, inner cell mass and trophectoderm

Morphological feature	Class 2	Class 3	Class 4	Average
Blastocyst expansion
Kappa	0.751 (0.620-0.883)	0.652 (0.521-0.783)	0.790(0.659-0.921)	0.729(0.632-0.826)
P value	<0.001	<0.001	<0.001	<0.001
Inner cell mass	Grade A	Grade B	Grade C	Average
Kappa	0.779 (0.648-0.910)	0.688 (0.558-0.818)	0.681(0.551-0.811)	0.705 (0.606-0.804)
P value	<0.001	<0.001	<0.001	<0.001
Trophectoderm	Grade A	Grade B	Grade C	Average
Kappa	0.382(0.256-0.507)	0.402 (0.273-0.531)	0.501 (0.370-0.631)	0.438(0.332-0.544)
P value	<0.001	<0.001	<0.001	<0.001

Kappa values are expressed with confidence intervals (CI) 95% between brackets. Class 2; Early blastocyst, Class 3; Full blastocyst, and Class 4; Expanded blastocyst.

Discussion

An automated tool for grading blastocyst morphology has been developed, utilizing temporal information from TL imaging and machine learning techniques. This study aligns with other research efforts aimed at enhancing the quality of assisted reproduction by leveraging the potential of AI to assist embryologists and physicians.

In early studies on the use of AI in the IVF laboratories, Bormann et al. (33) applied convolutional neural networks to assess the consistency of grading embryo quality and facilitate selection for biopsy and cryopreservation. They also used GA and single embryo images to achieve an accuracy of 90% in choosing the highest quality embryo available. Their results were very promising in distinguishing between viable and non-viable blastocysts and embryos with different implantation potential. However, they did not analyze the concordance in the morphological evaluation of blastocysts, where our results show the superior performance by image analysis. Then, several authors employed deep learning techniques to evaluate embryo quality and achieved superior results compared to embryologists. Life whisperer made a binary comparison of viable/nonviable embryo classification demonstrated an improvement of 24.7% over embryologists’ accuracy (34). Similarly, STORK performed a classification between good and poor embryos and compared the prediction to the majority vote of embryologists, achieving a Kappa coefficient of 0.63 (12). Our similar system achieved higher coefficients for two blastocyst features (BE and ICM), but not for the trophectoderm. Chavez-Badiola et al. (14) explored AI’s application in determining embryo viability and predicting ploidy, yielding promising outcomes. Several subsequent studies promote the standardization of embryo evaluation (15-19, 26, 35) and novel systems have been developed to predict clinical pregnancy by combining embryo images with clinical data (36). Their system (FiTTE) achieved higher accuracy than Gardner scoring assessment (59.8 vs. 65.2%). Given the large number of AI models that have recently emerged, it is the responsibility of embryologists to assess each of them to the best of their ability. For instance, Diakiw et al. (37) outlined methods that should be considered more widely for evaluating automated algorithms such as simulated cohort ranking analyses. However, AI methods used for embryo selection are often perceived as black box evaluations. To instill confidence in the predictions they generate, it is crucial to thoroughly examine and validate these models, ensuring their consistency with established clinical features of embryo quality.

Our study has prioritized the objective and consistent evaluation of blastocyst morphology, rather than aiming to create a new automated scoring system for embryo classification. To the best of our knowledge, the system proposed in this study is the only one that provides interpretable results for each part of the blastocyst, mimicking an embryologist’s evaluation but yielding more objective results. We acknowledge the challenge of standardizing the assessment of embryo morphology, considering its crucial connection with subsequent clinical outcomes. The pursuit of objectivity in embryo assessment plays a crucial role in the success and advancement of reproductive medicine. In line with our results, clinics from different countries, and even embryologists within the same clinic reported varying outcomes (6, 7, 38, 39). Discrepancies in embryo grading stem from factors such as differences in training, variations in quality control, and numerous other possibilities inherent in human work. According to our findings, AI may address these inadequacies and discrepancies that arise during embryo evaluation. Through the analysis of TL images, our results have demonstrated superior performance for AI compared to embryologists in grading human blastocysts. In addition, such technology may have some advantages: it is cost-effective, non-invasive, and provides greater consistency compared to assessments made by individual operators. AI can analyze thousands of images at a faster pace than humans, while continuously learning and integrating additional embryo information.

The main limitation of our study is its retrospective nature. Although AI technology has been proposed as a revolution in the assessment of embryo morphology, offering increased accuracy and consistency in grading across individuals and clinics, further independent studies are necessary to validate and ensure the reproducibility of these findings before implementing them in clinical settings. Also, the low inter-operator agreement was likely attributed to the use of single 2D images with a fixed central focal plane, although he EmbryoScope® software ensured consistent resolution and illumination across all images. We are also aware that our sample size is not very large. However, our AI technique (ANN with GA) analyzed thousands of ANN architectures to find the most optimal one. Typically, models developed with ANNs can be susceptible to overfitting. Nevertheless, in this case, we made efforts to mitigate this issue by carefully defining the input and output variables. Moreover, the validation of our tool should involve a comparison between the clinical outcomes based on embryo quality assigned by the AI and by the embryologist. This was not conducted in this study as not all embryos were transferred.

Conclusion

Our data demonstrate the superior ability of AI in accurately grading 3D embryos based on a single 2D image compared to human operators. A notable advantage of this approach is its universal applicability. Additional clinical investigations could explore the potential influence of AIbased blastocyst assessment on reproductive outcomes, with a particular focus on the potential enhancement of live birth rates. However, it would need to be validated beforehand on a dataset with known implantation embryos. This research serves as an initial stride towards achieving complete automation of an objective embryo evaluation. Enhancements in these processes hold the promise of augmenting the effectiveness and uniformity of routine laboratory work, leading to improved overall efficacy and consistency.

Authors’ Contributions

M.M., M.T., N.Z.; Project development and Supervision. L.B., C.H., J.M., Q.Z., Data collection, Review, and Editing. J.C.R., M.F.G.N., M.C.M., A.S.; Data curation and Formal analysis. M.T., M.M., L.B.; Manuscript editing and Submitting. All authors read and approved the final manuscript.