Artificial intelligence system for outcome evaluations of human in vitro fertilization-derived embryos

Abstract

Background:

In vitro fertilization (IVF) has emerged as a transformative solution for infertility. However, achieving favorable live-birth outcomes remains challenging. Current clinical IVF practices in IVF involve the collection of heterogeneous embryo data through diverse methods, including static images and temporal videos. However, traditional embryo selection methods, primarily reliant on visual inspection of morphology, exhibit variability and are contingent on the experience of practitioners. Therefore, an automated system that can evaluate heterogeneous embryo data to predict the final outcomes of live births is highly desirable.

Methods:

We employed artificial intelligence (AI) for embryo morphological grading, blastocyst embryo selection, aneuploidy prediction, and final live-birth outcome prediction. We developed and validated the AI models using multitask learning for embryo morphological assessment, including pronucleus type on day 1 and the number of blastomeres, asymmetry, and fragmentation of blastomeres on day 3, using 19,201 embryo photographs from 8271 patients. A neural network was trained on embryo and clinical metadata to identify good-quality embryos for implantation on day 3 or day 5, and predict live-birth outcomes. Additionally, a 3D convolutional neural network was trained on 418 time-lapse videos of preimplantation genetic testing (PGT)-based ploidy outcomes for the prediction of aneuploidy and consequent live-birth outcomes.

Results:

These two approaches enabled us to automatically assess the implantation potential. By combining embryo and maternal metrics in an ensemble AI model, we evaluated live-birth outcomes in a prospective cohort that achieved higher accuracy than experienced embryologists (46.1% vs. 30.7% on day 3, 55.0% vs. 40.7% on day 5). Our results demonstrate the potential for AI-based selection of embryos based on characteristics beyond the observational abilities of human clinicians (area under the curve: 0.769, 95% confidence interval: 0.709–0.820). These findings could potentially provide a noninvasive, high-throughput, and low-cost screening tool to facilitate embryo selection and achieve better outcomes.

Conclusions:

Our study underscores the AI model’s ability to provide interpretable evidence for clinicians in assisted reproduction, highlighting its potential as a noninvasive, efficient, and cost-effective tool for improved embryo selection and enhanced IVF outcomes. The convergence of cutting-edge technology and reproductive medicine has opened new avenues for addressing infertility challenges and optimizing IVF success rates.

Introduction

Globally, one in six couples suffers from infertility, and in vitro fertilization (IVF) has revolutionized the treatment of infertility, resulting in the birth of over 8 million babies.^[1] Nevertheless, achieving favorable live-birth outcomes remains challenging. The conventional approach to embryo selection relies on subjective visual inspection of embryo morphology and is experience-dependent and highly variable.^[2–4] The various assessments involved, including zona pellucida thickness variation, number of blastomeres, degree of cell symmetry and cytoplasmic fragmentation, aneuploidy status, and maternal conditions,^[5] demand an automated system capable of objectively evaluating these parameters to predict the final live-birth outcome.

Artificial intelligence (AI) holds significant promise for transforming healthcare and improving outcomes across various domains,^[6–9] including image-based diagnosis,^[10] voice recognition, and natural language processing.^[11] Convolutional neural networks (CNNs), especially those that incorporate transfer learning, have emerged as powerful tools for the development of accurate and efficient image diagnostic systems.^[10,12]

In the field of IVF, the application of deep learning has been explored for tasks such as classifying embryos based on morphological quality and transfer outcomes. Despite the challenges of accuracy and general applicability, recent efforts have been made to predict live births using blastocyst images, although real-world implementation remains limited.^[5,13–15] For example, Khosravil et al^[13] proposed an AI approach based on deep neural networks (DNNs) to predict blastocyst quality using embryo time-lapse images. Huang et al^[16,17] developed several AI models to predict euploidy and live-birth outcome. Currently, most models use static embryo images or time-lapse videos as inputs and focus on a specific prediction task that does not cover the entire IVF cycle. Our study addresses this gap in existing research by proposing a comprehensive AI system designed to address key challenges throughout the IVF cycle. This system comprises four main components: an embryo morphology grading module, blastocyst formation assessment module, aneuploidy detection module, and final live-birth occurrence prediction module.

Although recent reports have highlighted AI-based measurements of key morphological features in embryos,^[18] a comprehensive delineation of embryo selection for implantation is lacking. Our approach addresses the challenge of comprehensive embryo selection for implantation by introducing a morphological scoring system that assesses embryos at the pronuclear, cleavage, and blastocyst stages following the Istanbul consensus criteria. By employing multitask learning, our model evaluated various morphological features, including pronucleus type on day 1, number of blastomeres, and asymmetry and fragmentation of blastomeres on day 3. To predict development to the blastocyst stage, we combined results from models on day 1 or day 3 using a noisy-OR inference method, considering the potential influence of observed embryo factors. Embryos selected for transfer were primarily based on morphological scores on day 3 or day 5, with additional information from preimplantation genetic testing for aneuploidy (PGT-A) diagnosis reports. Previous studies have shown that embryonic aneuploidies are significant contributors to implantation failure.^[19] The detection of blastocyst ploidy (euploid vs. aneuploid) using static embryo images and clinical metadata holds significant promise for enhancing real-world outcomes. In addition, we leveraged 418 time-lapse videos with PGT-based ploidy outcomes to train a 3D CNN for noninvasive ploidy prediction, capturing more embryonic developmental information.

The detection of aneuploidy with preimplantation genetic testing (PGT) has improved the success rate of embryo transfer and pregnancy outcomes. However, the trophectoderm biopsy prerequisite faces challenges, including invasiveness, sequencing cost, variability in detecting mosaicism, and a limited selection of blastocysts. This study proposes an innovative noninvasive approach employing deep learning for the selection of euploid embryos, which significantly reduces costs and complications. Using time-lapse images and associated clinical parameters that encapsulate the genetic information crucial for proper embryo development, we integrated AI to predict embryo ploidy (euploid vs. aneuploid) without resorting to biopsy.

Leveraging embryo images, PGT results, and maternal clinical metadata, we developed a neural network to identify high-quality embryos for implantation on days 3–5 and predict subsequent live-birth outcomes. Our approach demonstrates the potential of AI as a noninvasive, efficient, and cost-effective tool for enhancing embryo selection and improving overall IVF outcomes.

Methods

Dataset characteristics

Data (embryo images and medical records) were collected at Guangzhou Women and Children’s Hospital (Guangdong, China) and Jiangmen Central Hospital (Guangdong, China). All procedures were performed as part of the patient’s standard care. Institutional Review Board (IRB)/Ethics Committee approvals were obtained in all locations (approval No. 2020-30401 in Guangzhou Women and Children’s Hospital) and all participants signed informed consent forms.

Overview of in vitro fertilization-embryo transfer (IVF–ET) cycles

The oocytes were inseminated by conventional IVF or intracytoplasmic sperm injection (ICSI) according to sperm parameters after retrieval. Then, all two-pronuclei embryos were cultured individually after the fertilization check and turned into cleavage-stage embryos after cell division. The embryos were observed daily up to day 5 or day 6, with each embryo having at least two photographs: at the fertilization check (16–18 h after insemination) and at day 3 embryo assessment (66 h after insemination) [Supplementary Tables 1 and 2, http://links.lww.com/CM9/C33].

On day 1 (16–18 h after embryo morphological evaluation), the embryologist scored the zygote according to the number, size, and location of the pronucleus and pronuclei. Scott et al^[20] classified zygotes into four groups, Z1–Z4, according to pronuclear morphology labeled with grades corresponding to their quality, including nuclear size, nuclear alignment, nucleoli alignment and distribution, and the position of the nuclei within the zygote.

Cleavage-stage embryos were evaluated for cell number, relative degree of fragmentation, and blastomere asymmetry according to the Istanbul consensus (2011).^[21]

If the embryo was cultured to the blastocyst stage, photographs taken on day 5 or day 6 were stored for analysis. Only available blastocysts that meet the criteria of stage ≥3, with at least one score of inner cell mass or trophectoderm ≥B, were selected for transfer or freezing for future use.

If the embryo was scheduled for PGT, a biopsy was performed on day 5 or day 6 according to the blastocyst grade, and next-generation sequencing (NGS) was used for euploidy assessment. In the PGT cycles, all embryos underwent blastocyst culture. The available blastocysts were biopsied, and NGS was performed for euploidy assessment. In most cases, embryos were transferred according to morphological scores either on day 3 or at the blastocyst stage, whereas in PGT cycles, embryos were selected according to PGT diagnosis reports.

All the patients were strictly followed up, and live birth was defined as the birth of a live infant at ≥28 weeks of gestation.

Time-lapse microscopy was used for a subset of patients to assist in embryo selection, and the videos were also utilized for analysis. We used images from the Primo Vision time lapse system (EVO; Vitrolife Kft, Szeged, Hungary), which takes an image of the embryos every 10 min at nine focal planes in 10 μm increments.

Embryo scoring

Eight senior embryologists from the two centers scored the embryos according to internal scoring rules. Viable blastocysts meet the criteria of stage ≥3 and at least one score of inner cell mass or trophectoderm ≥ B, according to Gardner scoring.

Live birth was defined as the birth of a live infant at ≥28 weeks of gestation with normal development parameters, and the live birth rate per embryo transfer was defined as the number of deliveries divided by the number of embryo transfers. According to these criteria, 3469 static images and 418 time-lapse videos of embryos were collected from 543 patients, along with medical record information.

Embryo image preprocessing

Embryo images were preprocessed using image segmentation and enhancement. Embryos were initially cropped from each image, and isolated embryos were used to train the embryo segmentation UNet to generate segmentation masks that assigned positive labels (foreground) to pixels within the embryo and negative labels (background) to others.^[22] These masks were used to locate the center of the box bounding the embryo in each image. To reduce bias in data collection and enhance focus on the embryo, all embryo images were aligned by cropping along the calculated bounding box.

Deep learning and transfer learning methods

CNNs were used to analyze embryo images. The transfer learning technique was used, where the ResNet-50 model^[23] pretrained with the ImageNet dataset^[24] was initialized as the backbone and fine-tuned for all the deep learning models. ResNet-50 is a five-stage network with residual-designed blocks that utilizes residual connections to overcome the degradation problem of deep learning models and enable very deep networks.

Blastocyst formation assessment and noisy-OR inference

We employed embryo images from day 1 or day 3 to train the two models to predict blastocyst formation. Embryos are expected to undergo blastocyst formation on day 5, developing an outer layer of cells (the trophectoderm) that encloses a smaller mass (the inner-cell mass). Under the assumption that embryo status independently impacts blastocyst formation, we subsequently combined the predicted results from both models using noisy-OR inference. Thus, the probability of blastocyst formation is composited by $p=1-{\displaystyle \prod }_{i\in \left\{1,3\right\}}\left(1-p_i\right)\partial$, where p_i is the predicted probability with the image on day i.

Prediction of live-birth occurrence with convolutional neural network-recurrent neural network (CNN-RNN) architecture

The live-birth occurrence prediction module maps a transfer, T, involving either single or multiple embryos to the probability of live-birth occurrence, where T is a sequence of n × m images from n embryos with m viewed images.

Statistical analysis

To assess the performance of the regression models for continuous value prediction, we utilized the mean absolute error (MAE), R-square (R²), and Pearson correlation coefficient (PCC). The evaluation was extended to employ the Bland-Altman plot, depicting the difference between the measured and predicted values against their average. The agreement between the predicted and actual values was assessed using 95% limits of agreement and the intraclass correlation coefficient (ICC). For the binary classification models, receiver operating characteristic (ROC) curves were employed to depict sensitivity versus 1−specificity. The area under the ROC curve (AUC), along with 95% confidence intervals (CIs), was reported. A non-parametric bootstrap method (1000 random resamplings with replacement) was used to estimate the 95% CIs of the AUC. The operating point of the AI system was adjusted to balance the true positive rate and false positive rate. Embryo-level models, generated from the average predictions of image-level models, utilized the Python package scikit-learn (version 0.22.1, https://pypi.org/project/scikit-learn/) for AUC calculations.

Results

Image datasets and patient characteristics

The AI system proposed in this study serves as a general embryo assessment platform throughout the IVF cycle, featuring modules for grading embryo morphology, assessing blastocyst formation, detecting aneuploidy, and predicting final live-birth occurrence [Figure 1]. The process involved retrieving oocytes, inseminating them using conventional IVF methods, and individually culturing two-pronucleus embryos with daily observations up to day 6. Each embryo underwent at least two photographic assessments: one to check fertilization on day 1 and the other to evaluate embryo morphology on day 3. The study’s developmental dataset and internal validation dataset were collected from March 2010 to December 31, 2018, and included 15,602 embryos from 5468 patients in Guangzhou Women and Children’s Hospital and Jiangmen Central Hospital. Demographic and clinical information is summarized in Table 1. The developmental dataset with random splits for training, and tuning of the AI algorithm at a 75%:25% ratio.

Figure 1.

Schematic illustration of the general artificial intelligence (AI) platform for embryo assessment and live-birth occurrence prediction during the whole in vitro fertilization (IVF) circle. The left panel: The AI models utilized images of human embryos captured at 17 ± 1 hours post-insemination (hpi, Day 1) or 68 ± 1 hpi (Day 3). Clinical metadata (e.g., maternal age, body mass index) are also included. The middle and right panel: An illustration of the explainable deep-learning system for embryo assessment during the whole IVF circle. The system consisted of four modules. The middle panel: a module for grading embryo morphological features using multitask learning; a module for blastocyst formation prediction using Day 1/Day 3 images with noisy-OR inference. The right panel: a module for predicting embryo ploidy (euploid vs. aneuploid) using embryo images or time-lapse videos; a final module for the live-birth occurrence prediction using images and clinical metadata. The models were tested on independent cohorts to ensure the generalizability. We also studied the AI versus embryologist comparison performance.

Table 1.

Basic characteristics of patients in the developmental dataset and validation cohorts for embryo morphological assessment.

Parameters	Developmental dataset		Internal validation	External validation set 1	External validation set 2
	Training set	Tuning set
Number of patients	3272	1071	1125	393	2410
Number of embryos	9141	3130	3331	407	3192
Number of images	19,006	6638	6940	843	6351
Age (years)	31.8 ± 4.4	31.9 ± 4.3	32.0 ± 4.6	31.5 ± 4.4	32.3 ± 4.8
BMI (kg/m2)	21.5 ± 2.9	21.4 ± 2.9	21.5 ± 2.9	21.4 ± 2.9	21.8 ± 3.1
FSH (mIU/mL)	6.0 ± 2.7	5.9 ± 2.1	5.9 ± 2.3	5.7 ± 1.9	6.8 ± 2.9
AMH (ng/mL)	4.0 (2.3, 7.4)	3.9 (1.9, 6.8)	3.4 (1.9, 5.8)	4.1 (2.6, 6.8)	3.2 (1.8, 5.2)
Number of oocytes retrieved	12.7 ± 7.1	12.8 ± 7.2	12.8 ± 6.9	13.1 ± 6.5	14.2 ± 8.7
Endometrial thickness (mm)	10.6 ± 2.8	10.4 ± 2.7	10.7 ± 2.6	10.7 ± 3.0	10.0 ± 1.8

Data are shown as mean ± standard deviation or median (Q1, Q3). AMH: Anti-Müllerian hormone; BMI: Body mass index; FSH: Follicle-stimulating hormone.

Our proposed AI system is a general embryo assessment platform covering the entire IVF cycle and consists of four modules: an embryo morphology grading module, blastocyst formation assessment module, aneuploidy detection module, and final live-birth occurrence prediction module [Figure 1]. We first developed AI models using multitask learning for embryo morphological assessment, including pronucleus type on day 1, number of blastomeres, asymmetry, and fragmentation rate of blastomeres on day 3. On the fifth day, the embryo forms a “blastocyst”, consisting of an outer layer of cells (the trophectoderm) enclosing a smaller mass (the inner-cell mass). To assess blastocyst formation, we used embryo images from day 1 or 3 to predict blastocyst formation using noise-OR inference. The aneuploidy detection module predicts the embryo ploidy (euploid vs. aneuploid) using embryo images and clinical metadata. For the aneuploidy detection module, we constructed a 3D CNN model using time-lapse image videos and further tested it on independent cohorts using 418 videos from patients to ensure generalizability. Finally, for the live-birth occurrence prediction module, we utilized embryo images and clinical metadata from 4537 patients with known live-birth outcomes to train our AI model. To evaluate the performance and generalizability of the module, we conducted an independent prospective study. This study utilized two external validation sets: the one from Jiangmen Central Hospital covering 393 patients (external validation set 1), and the other one from Guangzhou Women and Children’s Hospital including 2410 patients (external validation set 2). All validation datasets were collected from 2019 to 2023 [Table 1].

Explainable AI system for embryo morphological assessment

The selection of good-quality embryos typically involves parameters such as pronuclear morphology, number of blastomeres on a particular day of culture, and blastomere characteristics, including size, symmetry, and fragmentation. Using multitask learning, we developed AI models for assessing embryo morphology, including pronucleus type on day 1 and number of blastomeres, asymmetry, and fragmentation of blastomeres on day 3.

Zygote morphology at the pronuclear stage, which is assessed using the Z-score system based on nuclear size, alignment, nucleoli number, and distribution, is related to growth ability, blastocyst development, and implantation outcomes. The AI model demonstrated the ability to detect abnormal pronuclear morphology, with an area under the curve (AUC) of 0.805 (95% CI: 0.785–0.820) [Figure 2A].

Figure 2.

Performance in the evaluation of embryos’ morphokinetic features using the AI system. ROC curve showing performance of detecting abnormal pronucleus type of the Day 1 embryo (A). Morphological assessment of the Day 3 embryos (B–D). ROC curves showing performance of detecting blastomere asymmetry (B). The orange line represents detecting asymmetry (++ or +) from normal (–). The blue line represents detecting severe asymmetry (++) from good one (–). Correlation analysis of the predicted embryo fragmentation rate versus the actual embryo fragmentation rate (C). Correlation analysis of the predicted blastomere cell number versus the actual blastomere cell number (D). AI: Artificial intelligence; MAE: Mean absolute error; R²: Coefficient of determination; ROC: Receiver operating characteristic; PCC: Pearson’s correlation coefficient.

We evaluated the performance of the AI model in determining the asymmetry, fragmentation, and number of blastomeres at the cleavage stage. Blastomere symmetry criteria, as defined by Prados et al,^[25] relied on the diameter difference between blastomeres, determined by dividing the diameter of the smallest blastomere by that of the largest blastomere. This allowed the categorization of embryos into symmetrical (–) when blastomere diameter differences were <25%, severely asymmetrical (++) with ≥75% blastomere diameter differences, and mildly symmetrical (+) for blastomere diameter differences between 25% and 75%. The AI system achieved an AUC of 0.909 (95% CI: 0.896–0.921) for differentiating severely asymmetrical (++) blastomeres from symmetrical blastomeres (–) and an AUC of 0.854 (95% CI: 0.841–0.865) for detecting any degree of asymmetry (++ or +) from symmetrical blastomeres (–) on the held-out test set [Figure 2B]. Further comparisons also demonstrated a strong linear relationship between our AI-predicted fragmentation scoring system and the gold standard fragmentation scoring system [Figure 2C and Supplementary Figure 2A, http://links.lww.com/CM9/C33], with a PCC of 0.843, R² of 0.710, and MAE of 3.335 percent [Figure 2C]. Subsequently, AI models were trained to perform binary classification tasks to distinguish the pattern of fragmentation from normal and demonstrated an AUC of 0.916 (95% CI: 0.907–0.923) [Supplementary Figure 2A, http://links.lww.com/CM9/C33]. Finally, we investigated the performance of the AI model in predicting the cell numbers. Figure 2D shows that the cell numbers predicted by the AI algorithm achieved an excellent correlation with the actual number of blastomeres (PCC = 0.854, R² = 0.728, MAE = 0.625%).

Prediction of blastocyst development using embryo images

Next, we tested the ability of our AI models to predict the fate of the cleavage-stage embryos.

First, we investigated the ability of AI to predict an embryo’s day 5 status. The model was trained by incorporating information from day 1 or day 3 embryo images using end-to-end deep learning methods. The AI model was able to predict whether an embryo could develop to the blastocyst stage with an AUC of 0.857 (95% CI: 0.834–0.880) using day 1 embryos alone. The AI model achieved improved prediction accuracy with an AUC of 0.902 (95% CI: 0.885–0.919) using day 3 embryos. With the combined day 1 or day 3 images, our model achieved better performance, with an AUC of 0.918 (95% CI: 0.901–0.932) [Figure 3A].

Figure 3.

Performance in predicting the development to the blastocyst stage using the AI system. ROC curves showing performance of selecting embryos that developed to the blastocyst stage. The blue, orange, and green lines represent using images from Day 1, Day 3 and combined Day 1 & Day 3, respectively (A). Comparison of predicted fragmentation (B) and probability of asymmetry (C) between the Develop to Blastocyst and Fail to Blastocyst groups. Box plots showed median, upper quartile and lower quartile (by the box) and the upper adjacent and lower adjacent values (by the whiskers). Visualization for embryos’ morphokinetic characteristics that developed to the blastocyst stage or not at 40x magnification (D). In the upper-left corner, the embryo image depicts successful development into the blastocyst stage, showcasing excellent symmetry, minimal fragmentation, and the presence of 8 cells. Conversely, the remaining three embryos failed to progress to the blastocyst stage. The embryo in the upper-right exhibits cellular asymmetry, while the one in the lower-left displays a high fragmentation rate. Additionally, the embryo in the lower-right shows a considerable disparity from the expected 8-cell count. Comparison of predicted Fragmentation (B) and Probability of Asymmetry (C) between the Develop to Blastocyst and Fail to Blastocyst groups. AI: Artificial intelligence; AUC: Area under the curve; ROC: Receiver operating characteristic.

Subsequently, we assessed its ability to evaluate embryo viability using the output of the previous embryo morphology scoring module, which consisted of pronuclear morphology, asymmetry, fragmentation, and number of blastomeres. These studies demonstrated a comparable predictive ability for evaluating embryo viability when compared with embryologists’ traditional morphokinetic grading methods [Supplementary Figure 3, http://links.lww.com/CM9/C33]. Furthermore, failed blastocyst formation was significantly associated with an increased embryo fragmentation rate [Figure 3B]. Similarly, failed blastocyst formation was associated with a significantly increased embryo asymmetry [Figure 3C]. Human blastocyst morphological information on embryo fragmentation and asymmetry was correlated with blastocyst development outcomes [Figure 3D], which was the main impetus for obtaining an overall AI assessment. These results imply that human embryo morphology positively correlates with blastocyst development.

Detection of blastocyst ploidy using an embryo image-based AI system

We hypothesized that genome aneuploidy can affect cell morphology and migration patterns during embryonic development and is therefore amenable to detection by an AI algorithm. In this study, we developed three models for aneuploidy detection: a deep learning model using days 1 and 3 embryo images, a random forest model using clinical metadata, and a combined AI model using both input modalities. For all tasks, the combined and embryo image-only models performed better than the metadata-only model [Figure 4A]. The AUC for detecting embryo aneuploidies was 0.684 (95% CI: 0.664–0.707) for the metadata-only model, 0.739 (95% CI: 0.717–0.762) for the embryo image-only model and 0.761 (95% CI: 0.733–0.788) for the combined model [Figure 4A].

Figure 4.

Performance of our AI system in identifying blastocyst ploidy (euploid/aneuploid). The ROC curves for a binary classification using the clinical metadata-only model, the embryo image-only model and the combined model. PGT-A test results are available (A). The ROC curves for a binary classification using the clinical metadata-only model, the embryo video-only model and the combined model. The videos of embryo development is captured using time-lapse (B). Illustration of features contributing to progression to euploid blastocysts by SHAP values. Features on the right of the risk explanation bar pushed the risk higher and features on the left pushed the risk lower (C). Performance comparison between our AI model and eight practicing embryologists in embryos’ euploid ranking. The euploid rate of blastocysts selected for PGT-A test by AI versus average embryologists on different filtering rate scenarios. The baseline euploid rate was 46.1% (D). AI: Artificial intelligence; AMH: Anti-Müllerian hormone; AUC: Area under the curve; BMI: Body mass index; FSH: Follicle-stimulating hormone; PGT-A: Preimplantation genetic testing for aneuploidy; ROC: Receiver operating characteristic; SHAP: Shapley additive explanation.

Next, we trained a 3D CNN model to predict the ploidy status (euploid vs. aneuploid) of an embryo using 418 time-lapse images that presented both morphological and temporal information regarding embryonic development. This algorithm was validated in a prospective pilot study using a series of time-lapse videos of 145 embryos. When tested on the external test set using still embryo images, the AUCs for predicting the presence of embryo aneuploidies were 0.660 (95% CI: 0.608–0.719) using the clinical metadata model, 0.734 (95% CI: 0.688–0.779) using the embryo video model, and 0.805 (95% CI: 0.760–0.853) using the combined model [Figure 4B].

To elucidate the factors influencing embryo aneuploidy prediction, we employed the Shapley additive explanation (SHAP) explainer method.^[26] The analysis revealed robust contributions from both embryo image features and clinical parameters, such as maternal age, blastomere asymmetry, and day 3 blastomere cell number. The integrated insights depicted in Figure 4C highlight the significant impact of these elements on the accurate prediction of aneuploid embryos.

In our evaluation, we demonstrated that the performance of our model was superior to that of embryologists. In the context of euploidy screening, the eight embryologists from two different clinics ranked all embryos based on their likelihood of being euploid, with the top candidates progressing to PGT-A. The testing dataset, comprising 560 images from 110 patients, included 46.1% euploid embryos. The model achieved an AUC of 0.734, outperforming both junior and senior embryologists.

We then investigated whether our AI system could help embryologists improve their performance in aneuploidy prediction. The embryologists were also requested to rank the embryos based on the images and the accompanying information about maternal age and other clinical information. To assess the performance, we assessed the sensitivity of the model by changing its selection thresholds for further PGT-A testing, yielding different calculations of the euploidy rate, which we compared with those determined by the embryologists. The baseline euploidy rate in this population was 46.1% [Figure 4D]. By ranking embryos based on their likelihood of aneuploidy, the prediction of euploidy rates by both the embryologists and the model improved significantly, with the AI model performing better. Moreover, the removal of specific embryos further enhanced our AI-based prediction of the euploidy rate.

Predicting live birth using embryo images and clinical metadata

To expand the predictive ability of our AI system for live-birth occurrence, we introduced three distinct models: a baseline random forest model using clinical metadata, a deep learning model leveraging embryo images, and a hybrid AI model integrating both input modalities.

Embryo transfer adhered to the guidelines of the American Society for Reproductive Medicine (ASRM),^[27] transferring a maximum of two embryos on either day 3 or day 5 or day 6. Internal validation results showed that the clinical metadata and embryo image-only models attained AUCs of 0.713 (95% CI: 0.673–0.749) and 0.746 (95% CI: 0.709–0.784), respectively, whereas the hybrid model demonstrated superior performance with an AUC of 0.804 (95% CI: 0.773–0.834) [Figure 5A]. To underscore the generalizability of our models, we conducted external validation using an independent cohort, external validation set 1 [Table 1], revealing AUCs of 0.647 (95% CI: 0.586–0.712), 0.748 (95% CI: 0.679–0.799), and 0.769 (95% CI: 0.709–0.820) for the clinical metadata-only, embryo image-only, and hybrid models, respectively [Figure 5B].

Figure 5.

Performance in predicting live‑birth occurrence of the AI models. ROC curves showing performance on live‑birth occurrence prediction on internal test set (A) and external validation cohort (B). The green, orange and blue ROC curves represent using the metadata-only model, the embryo image-only model and the combined model. Illustration of features contributing to progression to live‑birth occurrence by SHAP values. Pink features pushed the risk higher (to the right) and blue features pushed the risk lower (to the left) (C). Comparison of our AI system with the PGT-A assisted approach for live-birth occurrence (D, E). The live birth rate by the AI system is associated with the proportion of embryos be selected for transfer. The orange line represents transplant on Day 3. The blue line represents transplant on Day 5/6 (D). Illustration of the baseline rate by Kamath et al, baseline rate on our external validation set 2, the PGT-A assisted live-birth rate and the AI-assisted live-birth rate. PGT-A was only performed for Day 5/6 transplant (E). AI: Artificial intelligence; AMH: Anti-Müllerian hormone; AUC: Area under the curve; BMI: Body mass index; FSH: Follicle-stimulating hormone; PGT-A: Preimplantation genetic testing for aneuploidy; ROC: Receiver operating characteristic; SHAP: Shapley additive explanation.

Employing SHAP, we affirmed the interpretability of the AI model by providing insights into the factors influencing live births. The SHAP method scored the relevant features used in IVF based on the impact of live birth rate transfer, highlighting image-based scores as the most significant, followed by maternal age, endometrial thickness, FSH, BMI, and AMH [Figure 5C]. These results not only validate the AI model but also underscore its potential practical implementation.

AI-assisted live-birth prediction performance in a prospective study

We conducted a prospective trial to evaluate the performance enhancement of our AI models over the current standard practice. We enrolled patients from two IVF centers from 2017 to 2020. Embryos were selected for implantation according to their morphological scores on either day 3 or days 5/6 based on the PGT-A diagnosis report. To validate the AI system’s clinical utility, we further studied its performance on external validation set 2, comprising 6351 embryo images from 2410 participants for the scenario of a single embryo transfer.

The performances of AI against embryologists in the live-birth rate on day 3 or against live-birth results assisted by PGT-A on day 5 or day 6 have been summarized in Figures 5D, 5E. For different clinical applications, the operating point of the AI system can be set differently to achieve a compromise between the transfer rate and live birth rate outcomes [Figure 5D]. The average live birth rate using embryos selected by embryologists in our study was 30.7% (86/280) on day 3 and 40.7% (114/280) on day 5. When evaluated on day 3 of transfer, our AI model achieved superior performance compared to the baseline, with a live birth rate of 46.1% (129/280). Furthermore, for the day 5 transplants, the success rate of individual embryos by our AI model alone was 55.0% (154/280), which was superior to that of the PGT-A-assisted performance [Figure 5E]. The results demonstrated that AI-assisted evaluation could help optimize embryo selection and maximize the likelihood of a viable pregnancy with an accuracy comparable to that of the PGT-A test.

As live-birth occurrence is correlated with maternal age, we further analyzed our AI’s performance in live-birth occurrence stratified by median age (32 years). As shown in Supplementary Figure 4, http://links.lww.com/CM9/C33, the AI model had a significant 13.4% and 13.5% improvement compared to the baseline in the older group (age >32 years), which is superior to that in the younger group (age ≤32 years).

Visualization of evidence for AI prediction

Finally, to improve the interpretability of the AI model and shed light on its prediction mechanism, integrated gradients (IG) were used to generate saliency maps, which helped highlight the areas of the images that were important in determining the AI model’s predictions. The saliency maps from the explanation techniques suggested that the model tended to focus on the pronuclei when evaluating day 1 embryo morphology [Figure 6A].

Figure 6.

Visualization of evidence for embryo morphological assessment at 40x magnification using integrated gradients method. Left: The original embryo images; Right: Explanation method generated saliency heatmaps. (A) Normal pronuclear type of Day1 (good one); (B) blastomere symmetry of Day3 (good one); (C) fragmentation rate of Day3 embryo (normal); (D) Day3 blastomere cell number (normal); (E) Day 1 embryo failed to develop to the blastocyst stage; (F) Day 3 embryo failed to develop to the blastocyst stage.

To predict the degree of cell symmetry [Figure 6B] and the number of blastomeres [Figure 6D], the model tended to focus on the spatial features around the center of day 3 embryos. In addition, the saliency maps suggested that the AI model focused on fragments around the cells of day 3 embryos for cytoplasmic fragmentation [Figure 6C] and the fate of cleavage-stage embryos [Figure 6F], which failed to develop to the blastocyst stage. Finally, in Figure 6E, the highlighted “points of interest” map appears more scattered over the day 1 embryo that failed to develop to the cleavage stage when compared to the successful day 1 embryo in Figure 6A.

Discussion

In this study, we introduced a general AI platform for embryo evaluation and live-birth occurrence prediction across the entire IVF cycle. The platform integrates modules for embryo morphology grading, blastocyst formation, aneuploidy, and live-birth prediction. Our primary aim was to maximize IVF success rates, with the overarching goals of expediting the time to conception and minimizing the risk of multiple pregnancies.

This study differs from previous AI-assisted morphological grading^[18] and blastocyst prediction^[28] models in a few ways. The evaluation of cleavage-stage embryos currently relies on morphological grading methods employing descriptive parameters; however, our model’s incorporation of modules enables the selection of embryos based on subtle visual features beyond the clinicians’ observational power. This departure from traditional descriptive parameters introduces a more objective and data-driven approach to embryo evaluation, thereby enhancing the precision and efficiency of predictions. Its noninvasive nature is particularly noteworthy, as it eliminates the need for invasive procedures such as biopsies or genetic testing. The conventional approach to oocyte^[29] and embryo aneuploidies, which are major contributors to implantation failure and miscarriage in the IVF cycle, typically involves costly IVF PGT-A tests. However, this invasive procedure can potentially lead to embryo damage and wastage owing to biopsy and vitrification. Our model not only eliminates the risks associated with invasive procedures but also reduces the overall cost of IVF. Such AI algorithms offer a noninvasive, high-throughput, and cost-effective screening tool that can potentially standardize embryo selection methods across clinical environments.

Time-lapse microscopy (TLM) is a newly emerged tool that can capture morpho-kinetic data (timing and duration of cell division) and differentiate embryo ploidy based on differences in morphokinetic patterns that were previously undetectable by human observers. This enables our model to consider morphokinetic features in the prediction of embryo ploidy prior to implantation and to replace invasive PGT-A procedures.

We addressed the limitations of previous research on the application of deep learning to live-birth prediction.^[30] Prior studies have lacked practical clinical implementation because of the difficulty in interpreting predictions and the absence of prospective clinical trials. To overcome these challenges, we utilized embryo images and clinical metadata to develop a neural network that identified high-quality embryos for implantation on days 3–5 and the consequent live-birth outcomes. The prediction of birth outcome considered various factors, including maternal age; menstrual, uterine, and cervical status; previous pregnancy; and fertility history.^[25–29,31] Employing interpretable methods, we aimed to gain insights into the factors influencing prediction, which is essential for determining targeted interventions in clinical settings.

Our study also demonstrates that our model outperforms current clinical practitioners in terms of enhancing live birth rates. Using embryo images and clinical metadata, automated AI algorithms contribute to a more accurate and objective assessment of embryo viability, improving the chances of selecting embryos with a greater potential for successful implantation in both single embryo transfers (SET) and double embryo transfers (DET). Unlike PGT-A, which exhibits comparable performance but is constrained to blastocyst transplantation on day 5, the AI model offers continuous scoring flexibility. Additionally, the operational point of the AI system can be adjusted to achieve a flexible balance between the blastocyst transfer rate and live birth rate in real-world clinical applications. Despite these promising findings, this study acknowledges certain limitations, such as the training and testing of the AI model being confined to the Chinese population. Recognizing the need for broader generalizability, further validation across diverse racial populations is required. In addition, the experimental exposure of mothers might also influence the quality of embryos, which was not explicitly considered in this study because of the scarcity of data. The impacts of these factors also need to be validated in future studies.

Data availability statement

Data access can be requested by writing to the corresponding authors. All data and code access requests will be reviewed and (if successful) granted by the Data Access Committee.

Code availability statement

The deep learning models were developed and deployed using standard model libraries and the PyTorch framework. Custom codes were specific to our development environment and used primarily for data input/output and parallelization across computers and graphics processors. We will make our codes available upon request and approved by a Data Access Committee.

Acknowledgements

This study was funded by Guangzhou Women and Childrens’s Medical Center, Guangzhou National Laboratory, Wenzhou Medical University. We thank many volunteers and physicians for grading embryo photographs.

Conflicts of interest

None.