Inferring simple but precise quantitative models of human oocyte and early embryo development

Abstract

Macroscopic, phenomenological models are useful as concise framings of our understandings in fields from statistical physics to finance to biology. Constructing a phenomenological model for development would provide a framework for understanding the complicated, regulatory nature of oogenesis and embryogenesis. Here, we use a data-driven approach to infer quantitative, precise models of human oocyte maturation and pre-implantation embryo development, by analysing clinical in-vitro fertilization (IVF) data on 7399 IVF cycles resulting in 57 827 embryos. Surprisingly, we find that both oocyte maturation and early embryo development are quantitatively described by simple models with minimal interactions. This simplicity suggests that oogenesis and embryogenesis are composed of modular processes that are relatively siloed from one another. In particular, our analysis provides strong evidence that (i) pre-antral follicles produce anti-Müllerian hormone independently of effects from other follicles, (ii) oocytes mature to metaphase-II independently of the woman’s age, her BMI and other factors, (iii) early embryo development is memoryless for the variables assessed here, in that the probability of an embryo transitioning from its current developmental stage to the next is independent of its previous stage. Our results both provide insight into the fundamentals of oogenesis and embryogenesis and have implications for the clinical IVF.

1. Introduction

Understanding the manner by which a multicellular organism develops from a single cell is one of the grand challenges of biology. In mammals, this process begins with oogenesis inside the female, which results in an egg that becomes an embryo after fertilization. Early embryo development in mammals, including humans, is self-organized [1,2]: the course of events that unfold are governed by the embryo’s internal dynamics and can proceed without external signals. Oogenesis and early embryogenesis have been studied from diverse perspectives, including molecular genetic, cell biological, chemical and mechanical [3–13]. Despite the vast amount of knowledge that has been obtained, many basic questions remain, including: what determines which oocytes are selected for ovulation? How is the timing of embryonic events regulated? How are oogenesis and embryogenesis negatively impacted by age and disease? Answering these will provide fundamental insight and have strong implications for evolution and for medical treatments of infertility. However, these issues are difficult to study using the molecular approaches that are the mainstay of current research, because of the integrated nature of the problems they pose concerning the overall trajectory of development.

An alternative to the microscopic, molecular perspective is to develop a macroscopic, phenomenological understanding. Such an approach has been productive in diverse areas from statistical physics [14] to finance [15] to some fields of biology [16–18], including protein evolution [19] and cell-size control in bacteria [20–22]. These phenomenological models focus on describing a few key variables that subsume the detailed descriptions of the component parts, e.g. temperature rather than the motions of individual molecules, or market volatility rather than the financial decisions of individual companies or investors. One significant concern is that the great complexity of oogenesis and embryogenesis might make simple, phenomenological descriptions inapplicable. Furthermore, the validity of the phenomenological approach can only be determined by developing models and rigorously testing them. This requires a large amount of quantitative data, which is difficult to obtain from oocytes and embryos in model organisms.

Here, we overcome this challenge by leveraging a large dataset from 7399 routine clinical in-vitro fertilization (IVF) treatment cycles resulting in 98 264 oocytes and 57 827 embryos; the treatments were performed from 2012 to 2017 at the Brigham and Women’s Hospital in Boston, MA. We show that the data can be quantitatively described using simple, phenomenological models. The large variance in the data allows testing these models over a wide range of physiological conditions. The models we develop are Bayesian networks, a form of probabilistic graphical models which represents conditional dependencies by a directed, acyclic graph. We infer models directly from the data, making little use of prior knowledge. The resulting models recapitulate well-established aspects of oocyte and embryo development. Moreover, the resultant models are sparse: only one or two factors directly impact physiological processes. This implies that human oogenesis and embryogenesis are highly modular. Our analysis leads to a number of additional, surprising conclusions. We present strong evidence that:

• (i) Each pre-antral follicle produces anti-Müllerian hormone (AMH) independently of effects from other follicles. This argues that AMH is a faithful indicator of the number of pre-antral follicles, consistent with its physiological role in regulating follicle recruitment and supporting its clinical use as a measure of ovarian reserve [23–26].
• (ii) While the number of oocytes released from follicles depends on many factors, the probability that a released oocyte matures to metaphase-II is independent of the patient’s age, BMI and other, external factors. This argues that physiological processes that are correlated with these external factors, such as mitochondrial metabolism and aneuploidy [27–29], do not significantly impact meiosis resumption.
• (iii) After oocytes are fertilized, the probability of successfully transitioning from one embryonic developmental stage to the next depends on the embryo’s present state, but not on its earlier state. Thus, embryo development is memoryless, at least for the variables examined here. This argues that clinical embryo selection procedures need only consider the state of the embryos immediately before transfer, as the state of the embryo at earlier times provides no additional information.

Taken together, our results show that the development of oocytes and embryos emerges as a simple process, despite the underlying molecular complexities of the biology and despite the plethora of disease aetiologies and treatment protocols presenting in a clinic. More broadly, this work validates the use of phenomenological models of oogenesis and embryogenesis by demonstrating that simple models can be constructed without sacrificing quantitative accuracy. Although we infer the models using data drawn from controlled ovarian stimulation and not from natural menstrual cycles, the models provide insight into the principles that govern oogenesis and embryogenesis, and may be useful in guiding clinical IVF treatments.

2. Results and discussion

2.1. A model of oocyte development

The formation of a healthy embryo begins with the successful progression of an oocyte from prophase-I arrest to metaphase-II. Which clinical factors affect the number of metaphase-II oocytes retrieved during an IVF ovarian stimulation?

The clinical data contain 69 variables that describe either the patient or the ovarian stimulation. We start by examining four variables that are strongly correlated with the number of metaphase-II oocytes: (1) the number of total eggs retrieved during an ovarian stimulation cycle (Eggs), (2) the number of eggs in metaphase-II arrest (MII), (3) the patient’s maximum serum oestradiol concentration during the cycle (E2) and (4) the patient’s serum AMH before the cycle. Oestradiol is a hormone produced by the ovaries during natural and stimulated ovulatory cycles [30]; AMH is considered a measure of the patient’s ovarian reserve [26]. Each of these variables varies widely across the 4910 cycles for which all four variables are recorded, with coefficients of variation of 0.5–1.2 (figure 1a, left). All four variables are strongly correlated with one another, with Pearson correlation coefficients between 0.26 and 0.91 and p-values between ${10}^{-300}\hspace{0.17em}\mathrm{and}\hspace{0.17em}{10}^{-155}$ (figure 1a, right).

Figure 1.

(a) Distributions and correlations of the four variables AMH (measured in International Units, IU), Eggs, MII and E2 (measured in pg ml⁻¹). (b) Left: the number of metaphase-II oocytes retrieved (MII) is strongly correlated with the patient’s serum AMH. Grey dots: raw data, red circles: raw data binned into 20 separate bins with equal counts, green line and shaded region: nonlinear regression and errors. Centre: that correlation disappears after regressing against the total number of retrieved oocytes (Eggs). Grey dots: residuals after regressing against Eggs, red circles: residuals binned into 20 separate bins with equal counts, green line and shaded region: linear fit to the residuals, with slope and standard error shown at top of plot. The axes range of the plots is cropped to show details of the trends. Right: this conditional independency suggests a graph of the form AMH–Eggs–MII. (c) Left: the patient’s serum oestradiol concentration (E2) is strongly correlated with AMH. Centre: that correlation disappears after regressing against Eggs. Right: this conditional independency suggests a graph of the form AMH–Eggs–MII. (d) Left: MII is strongly correlated with E2. Centre: while regressing against Eggs greatly weakens that correlation, E2 and MII remain correlated after conditioning on Eggs. This suggests a fully connected graph is needed to describe these three variables (right). (e) A graphical model that is consistent with the data. Edge labels show the conditional correlation coefficients after conditioning on all other incoming edges; the data are consistent with the arrow marked with a * oriented in either direction. (f) The graphical model expected from prior knowledge of ovarian stimulation.

To understand which factors quantitatively affect oocyte maturation, we first search for conditional independencies among the variables AMH, MII and Eggs. A conditional independency between two variables implies that one variable can be completely described without direct knowledge of the other, suggesting the existence of a simple phenomenological model. To search for conditional independencies, we nonlinearly regress both MII on Eggs and AMH on Eggs, by finding the best-fit polynomial that maximizes the Bayesian posterior evidence. This method allows for capturing complex dependencies without overfitting the data [31,32] (see electronic supplementary material, §1); we also split the data into separate train and test sets as a further check against overfitting. We then take the residuals from the two regressions and evaluate their correlation. We denote this procedure as Corr(AMH, MII|Eggs). We find that, although there is a strong correlation between AMH and MII (figure 1b, left), that correlation disappears after conditioning on Eggs: Corr(AMH, MII, |Eggs) = −0.02 (p = 0.19; figure 1b, centre). This correlation is both consistent with zero and smaller than an effect size threshold of 0.05, suggesting that AMH and MII are conditionally independent given Eggs.

We encode this conditional independency using a class of graphical models known as Bayesian networks. These have found usage in causal inference [33–36]; here, we use them to construct phenomenological models that correspond to mechanistic descriptions of biology. Briefly, for a given factorization of a probability distribution, these graphs contain a directed edge from one variable to another if the probability of the second explicitly depends on the first. Two variables are conditionally independent if all paths from one variable to the other are ‘blocked’, by head-to-tail or tail-to-tail nodes meeting at a variable that is conditioned on, or by head-to-head nodes meeting at an edge that is not conditioned on [34,37]. Both the observed correlation between MII and AMH and their conditional independency given Eggs can be captured by any of the graphs AMH → Eggs → MII, AMH ← Eggs → MII, or AMH ← Eggs ← MII; we denote this ambiguity by AMH–Eggs–MII, with an as-yet undetermined orientation of the arrows (figure 1b, right). If the data are described by one of these graphs, then Eggs and AMH should remain correlated given MII, which is indeed the case: Corr(Eggs, AMH|MII) = 0.24 (p = 10⁻⁴⁷; electronic supplementary material, figure 2a). Likewise, MII and Eggs should remain correlated given AMH, which is the case: Corr(MII, Eggs|AMH) = 0.87 (p < 10⁻³⁰⁰; electronic supplementary material, figure S2b).

Second, we examine the variables AMH, E2 and Eggs. AMH and E2 are strongly correlated (figure 1c, left), but regressing AMH and E2 on Eggs shows that Corr(AMH, E2|Eggs) = 0.03, consistent with no conditional correlation (p = 0.11; figure 1c, centre). This suggests a graph of the form AMH–Eggs–E2 (figure 1c, right). Third, we examine Eggs, MII and E2. While regressing on Eggs greatly weakens the correlation between MII and E2 (compare figure 1d left and centre), the measured conditional correlation is still positive: Corr(E2, MII|Eggs) = 0.08, p ≈ 10⁻⁶. Thus, an edge must connect each of Eggs, E2 and MII (figure 1d, right). Finally, we examine the variables AMH, E2 and MII and find no additional conditional independencies (electronic supplementary material, figure S2).

Of the 543 graphical models that describe four variables, only eight models capture exactly the two conditional independencies described above. The data alone cannot distinguish between these graphs. However, the patient’s AMH is measured before the ovarian stimulation starts, whereas the other three variables are measured during the treatment. Thus, any graph with an edge pointing into AMH cannot correspond to a mechanistic description of the biology. Ruling out these graphs leaves only two graphs consistent with both the data and a mechanistic interpretation (figure 1e).

Physiologically, this quantitative, phenomenological model recapitulates our qualitative understanding of ovarian stimulation (figure 1f): (1) pre-antral follicles, containing immature oocytes and associated somatic cells, produce the hormone AMH [23,25,38]. Since pre-antral follicles can grow into large antral follicles with prophase-I-arrested eggs, AMH is a measure of the potential number of oocytes that could develop. This is captured by the inferred arrow in figure 1e from AMH to Eggs, which indicates that the patient’s AMH determines how many eggs she will produce. (2) Antral follicles produce oestradiol, captured by the inferred arrow from Eggs to E2. (3) Some, but not all, eggs progress from prophase-I arrest to metaphase-II arrest. This is captured by the inferred arrow from Eggs to MII. (4) During natural ovulation, the oestradiol produced by antral follicles signals the pituitary and hypothalamus to release hormones which modulate oocyte maturation. During an ovarian stimulation, clinicians attempt to temporarily disable this feedback between the hypothalamus and the pituitary [30], suggesting that oestradiol should not impact oocyte maturation during ovarian stimulation. However, the inferred arrow from E2 to MII suggests that a weak feedback between the hypothalamus, the pituitary and oestradiol is still present during an ovarian stimulation cycle.

This inferred phenomenological model (figure 1e) provides a quantitative representation of oocyte development that allows direct and indirect effects to be disentangled. For instance, a patient starting treatment with a higher AMH is likely to produce more oocytes, via the direct arrow AMH → Eggs. In addition, that patient is likely to have a higher oestradiol level during the cycle, as the additional eggs she is likely to produce will on average produce more oestradiol, via the path AMH → Eggs → E2. However, the graph states that this effect is indirect: the patient’s E2 increases only through the associated increase in the number of eggs for high-AMH patients. This is borne out by the data. Likewise, a patient with a larger number of MII oocytes retrieved is likely to have a higher AMH, since following the arrows backwards shows that higher MII implies that Eggs is higher, and higher Eggs implies a higher AMH. However, once again, this is an indirect effect; the patient’s AMH is more likely to be high only because of the associated increase in Eggs when many MII oocytes are retrieved.

2.2. Oocyte maturation is robust to other factors

A woman’s age and obesity are known to affect her fertility and IVF prognosis. Does age or obesity directly affect oocyte maturation after accounting for E2 and Eggs?

The data show that age has no direct effect on an oocyte’s ability to reach MII. The conditional correlation between MII and age is consistent with zero: Corr(Age, MII|Eggs, E2) = 0.02 (p = 0.35, figure 2a). In addition, the data constrain the magnitude of any effect to be tiny. Fitting a line to the residuals constrains the slope to be (1.1 ± 1.2) × 10⁻² MII oocytes/year (point estimate ± standard error). To place this in perspective, consider a treatment cycle for two women, aged 33 and 40 years old (the 25th and 75th percentile in the data). If the treatment results in the same number of total oocytes and the same max oestradiol for both women, then on average the number of retrieved MII oocytes should differ by no more than 0.2. Since the median MII oocytes retrieved per cycle is eight, the data constrain the direct effect of age on MII to be 2% or less.

Figure 2.

MII versus the patient’s age (a), BMI (b), and the doses of stimulation drugs FSH (c) and HMG (d), after regressing against Eggs and E2. Grey dots show the residuals from the regressions, red circles and error bars show the mean and standard error of the data binned into 20 bins with equal number of points, and green lines, shaded regions and labelled slopes show the mean and standard error of the best linear fit to the residuals. (e) The fraction of MII oocytes (MII/Eggs) versus Eggs. (f) Histogram of observed MII (red circles) versus that expected from independently triggering follicles (green line and shaded region show the expected counts and their standard deviation), for the 116 cycles that have 17 eggs retrieved, which is where the discrepancy between the two histograms is the largest as measured by a χ²-test (p < 10⁻³⁰⁰, primarily due to the cycles with MII of 1–5). On the scale of the plot, the expected histograms from a binomial distribution where the probability for an oocyte being metaphase-II is constant is indistinguishable from one where the probability varies with E2.

Likewise, the data show that the woman’s BMI has no direct effect on an oocyte’s ability to reach MII. The conditional correlation between MII and BMI is consistent with zero: Corr(BMI, MII|Eggs, E2) = −0.005 (p = 0.78; figure 2b). The data constrain the direct effect of BMI on MII to be 1% or less.

Ovarian stimulation drugs are necessary for multiple oocytes to reach metaphase-II in a single cycle. However, the data show that the dose of these drugs has no direct effect on an oocyte’s ability to reach MII. The conditional correlation between MII and the dose of FSH or HMG is consistent with zero: Corr(FSH, MII|Eggs, E2) = 0.01 (p = 0.49; figure 2c), Corr(HMG, MII|Eggs, E2) = −0.03 (p = 0.08; figure 2d). The data constrain the direct effect of these stimulation drugs to be 1% or less for FSH, and 5% or less for HMG. These observations show that oocytes develop to MII independently of a patient’s age, her BMI, or details of the ovarian stimulation procedure.

Perhaps only a fixed number of oocytes per cycle are capable of maturing to metaphase-II, and therefore the probability of an oocyte being metaphase-II depends on the total number of eggs retrieved. However, while Eggs is strongly predictive of MII, it provides no predictive power for the fraction of eggs in MII arrest (MII/Eggs): Corr(MII/Eggs, Eggs) = −0.01 (p = 0.48). Linearly regressing MII/Eggs on Eggs gives a slope tightly constrained near zero (figure 2e).

Combined, these observations suggest the following simple picture for oocyte maturation: each follicle independently triggers its oocyte to leave prophase-I, with a probability that depends only on E2. The oocyte then progresses to metaphase-II, with both processes independent of interactions with other follicles or the aggressiveness of the ovarian stimulation.

To check whether this simple picture completely describes oocyte maturation, we examine the distribution of MII for fixed Eggs. If each follicle independently triggers its egg to progress to metaphase-II with some probability p, then MII for each cycle will be binomially distributed, denoted as B(MII; Eggs, p). If that probability depends only on E2, then the measured distribution of MII across cycles with a given Eggs should be the average of many binomial distributions, each with a probability that depends on E2: 〈B(MII; Eggs, p(E2))〉_E2. Instead, the empirical distribution is much broader than the expected one (figure 2f). This discrepancy suggests that additional factors affect oocyte maturation, such as biochemical processes within the oocyte that are shared by multiple eggs from the same patient, interactions between follicles beyond serum oestradiol, or simply other clinical factors that we have not accounted for.

The data provide some insight into human meiosis, especially given prior knowledge of human ovulation. As a woman ages, the oocytes she ovulates become much more likely to be aneuploid, rising from an aneuploidy rate of roughly 25% at age 30–80% at age 42 [27,39,40]. Chromosomal signatures show that aneuploidy in human oocytes can arise both during the oocyte’s progression from prophase-I to metaphase-II and immediately after fertilization [11,41–43]. In mitotic cells, mis-segregation of chromosomes is reduced by the spindle-assembly checkpoint [44], which can arrest mitosis until chromosomes are properly aligned. If there were a strong spindle assembly checkpoint in meiosis I, then the typically aneuploid oocytes from older women would reach metaphase-II at a reduced incidence than those from younger women. Instead, oocytes from older and younger women reach metaphase-II arrest at the same incidence. This is consistent with experimental work that shows that human oocytes have a weak meiotic spindle assembly checkpoint [45–47].

2.3. Follicular recruitment is robust to patient factors

While the patient’s age, BMI and dose of ovarian stimulation drugs do not directly affect an oocyte’s progression to metaphase-II, they could affect the rest of follicular recruitment. To investigate this, we examine the joint distribution of all of Age, BMI, AMH, FSH, HMG, Eggs, E2 and MII, constructing a directed acyclic graph that is consistent with the data. The number of possible graphs grows rapidly with the number of variables: there are 543 possible graphs for four variables, but 783, 702, 329, 343 possible graphs with eight variables. To deal with this inordinately large number of possible models, we use prior knowledge to split the variables into three groups: prognostic variables measured before the treatment starts (Age, BMI and AMH), treatment variables (FSH and HMG), and response variables measured after the drugs have been applied (Eggs, MII, E2). We then search for graphs that are consistent with a mechanistic interpretation, by excluding graphs with edges directed from treatment to prognostic variables, from response to prognostic variables, or from response to treatment variables.

Among the prognostic variables, the data show that BMI does not directly affect AMH: Corr(AMH, BMI|Age) = −0.04 (p = 0.01; electronic supplementary material, figure S4a). Physiologically, this implies that the likelihood that a primordial follicle develops into a pre-antral follicles is independent of obesity. This is the only conditional independency among the prognostic variables.

The data show that clinicians customize the doses of FSH and HMG based on the patient, as expected. The FSH dose depends on all three prognostic variables, and the HMG dose depends on Age, AMH and FSH, but not BMI (electronic supplementary material, figure S4f). The lack of conditional independencies between the treatment and prognostic variables demonstrates that the conditional independencies we do see elsewhere are real and not an artefact of our analysis.

Among the response variables, the data show that follicular recruitment is simple, although follicular hormone production is not. The data are consistent with Age and BMI having no direct effect on Eggs: Corr(Age, Eggs|AMH), HMG = −0.04, p = 0.01; Corr(BMI, Eggs|AMH, HMG) = −0.01, p = 0.54 (electronic supplementary material, figure S4c,d). Physiologically, this suggests that the ability of a pre-antral follicle to be recruited does not worsen with age or obesity. Likewise, Eggs is conditionally independent of FSH: Corr(FSH, Eggs|AMH, HMG) = −0.04 (p = 0.02; electronic supplementary material, figure S4e). This suggests that, at this particular clinic, clinicians prescribe sufficient FSH to recruit all the follicles in the cohort activated from the primordial pool in that menstrual cycle. Interestingly, we observe a weak negative correlation between Eggs and the dose of HMG: Corr(HMG, Eggs|AMH) = −0.11 (p = 10⁻¹¹). Taken at face value, this seems to imply that HMG is typically supplied at more than the optimal dose at this clinic. While there is some evidence that excessive HMG can cause follicles to degrade [48,49], another possibility is that the negative correlation is due to clinicians prescribing more HMG to patients whom they know a priori to be poor responders even after accounting for their age, BMI and AMH—for example, patients who have had a poor response in previous treatments. However, the correlation remains negative even when excluding patients on repeat stimulation cycles: Corr(HMG, Eggs|AMH, first cycle) = −0.09, p ≈ 10⁻⁵. Combined, the data paint a simple picture for follicle recruitment: all available follicles are typically recruited, independently of effects from age or obesity but weakly affected by HMG. Finally, in contrast to the simplicity of Eggs and MII, all of Age, BMI, FSH, HMG and Eggs affect E2. These observed conditional independencies are captured by the graphical model in figure 3a.

Figure 3.

(a) A graphical model of oogenesis that is consistent with a mechanistic interpretation of the data. Labels show conditional correlation coefficients. For edges with two labels, the upper corresponds to FSH, the lower to HMG; dashes signify no dependence. The data are consistent with marked arrows (*) oriented in either direction. (b) Rank plots of p-values for the 99 conditional correlations corresponding to the conditional independencies predicted by the model. The green line shows that from the training data, the red line that from the test data. The black line and shaded regions show the median, 95% and 99.9% centred percentile of rank plots from 3000 datasets simulated according to the proposed model. (c) The same as (b), but showing the distribution of rank plots for the 99 conditional correlations from fully connected, linear Gaussian models.

The model in figure 3a predicts 99 conditional independencies among the eight variables shown. If the model completely describes the data, then the 99 corresponding conditional correlations must be consistent with zero, i.e. none of their p-values should be statistically significant. As a stringent check of the model, we measure the conditional correlations and associated p-values for each of these 99 independencies in both the train and test sets. We then check that the measured conditional correlations are consistent with zero by comparing their associated p-values to those calculated from datasets simulated according to the model in figure 3a. We find that the measured p-values are consistent with the simulated ones, although the lowest measured p-value is lower than the typical simulated one (figure 3b). By contrast, datasets generated according to more complicated models display a different distribution of these p-values (figure 3c). Moreover, anything missing from the model in figure 3a must correspond to a small effect with small explanatory power. Fitting the training data with a fully-connected model explains 0.6% or less of each variable’s variance in the test data, with the fully-connected model actually performing worse on the test set for most of the fits than the model in figure 3a does (electronic supplementary material, §4). Combined, these observations show that the graphical model accurately describes human ovarian physiology and oogenesis.

Viewed holistically, the probabilistic graphical model in figure 3 has a simple mechanistic interpretation. The patient’s age and BMI only act to determine the hormone levels. Hormones other than oestradiol determine how many antral follicles develop. Follicles then produce oestrogen and trigger eggs to progress to metaphase-II, with a slight feedback between these. Because of this simplicity, changes due to age or obesity manifest themselves in simple ways, after ignoring the physiologically irrelevant question of how clinicians choose drug doses for ovarian stimulation. In particular, the only direct effect of obesity on the oocyte maturation process is to decrease E2. This is consistent with work showing that obesity affects fertility by interfering with the hormonal regulation of ovulation [29]. The primary effect of age on the oocyte maturation process is to decrease the ovarian reserve, as measured by the patient’s AMH. Physiologically, this is consistent with the well-known decrease in a woman’s pool of primordial follicles as she ages [30].

While this model is a quantitative description of oocyte maturation, it is not a microscopic model. There are many intermediate factors that are not included in the model but are known to affect follicular recruitment and oocyte maturation, such as GnRH from the hypothalamus, FSH and LH from the pituitary and inhibin, androstenedione and cAMP from the follicles. However, the model’s accuracy suggests that these additional factors are intermediate and can be coarse-grained out to give a macroscopic, phenomenological description of oocyte maturation. Moreover, the model’s accuracy also shows that these additional factors do not cause additional dependencies between variables: the model places strong constraints on age-related sensitivity of oocytes to the maturation signal, for example.

AMH and E2 are produced by ovarian follicles at different stages in follicular growth. The data provide insight into both these processes. For a given number of retrieved oocytes, the mean of AMH is linear in the number of retrieved oocytes, with an intercept near zero (figure 4a, also electronic supplementary material, figure S5). For Eggs not too large, the variance of AMH is also linear in Eggs (figure 4b). The deviation from linearity at large Eggs is largely due to patients with polycystic ovary syndrome, who tend to have high AMH; excluding patients with a diagnosis of ovulatory dysfunction (i.e. polycystic ovary syndrome) brings most of the Var(AMH) measurements onto the linear fit (data not shown). Since means and variances add when summing independent random variables, the linearity of both the mean and the variance of AMH in the number of follicles suggests that each follicle produces AMH independently from interactions with other follicles. (The small but non-zero intercept could arise if some pre-antral follicles do not mature sufficiently to be retrieved during the IVF retrieval procedure.) By contrast, the mean E2 is not linear in Eggs, systematically deviating from linearity and having an intercept that is far from zero (figure 4c; variance in panel d). These variations from linearity show that the E2 is not produced independently by each follicle, perhaps due to additional production from outside follicles (such as by adipose tissue or the adrenal glands), due to inter-follicle feedback, or simply due to other factors that affect E2 production, such as those shown in figure 3a.

Figure 4.

(a) AMH versus Eggs. Grey dots show raw data, red circles show the mean and standard error of AMH binned at each value of Eggs and green line shows the best linear fit to the data, with slope of 0.20 ± 0.01 and intercept 0.36 ± 0.11. A linear fit is the best-fit polynomial to the data, as determined by the model evidence. (b) Variance and error estimate of AMH versus Eggs, trimmed to the central 95% for each value of Eggs (red dots, errors calculated using the variance of the k-statistic). The green line shows the best linear fit to the variance, with a slope of 0.28 ± 0.01 and intercept −0.22 ± 0.03. (c) E2 versus Eggs. The green line shows the best linear fit to the data. A quadratic model (not shown) provides the best fit to E2 versus Eggs. (d) Variance and error estimate of E2 versus Eggs, trimmed to the central 95% for each value of Eggs.

2.4. Pre-implantation development

Once the oocyte is fertilized, the resulting embryo starts to divide. By the third day after fertilization, a human embryo typically has eight cells. As its cells continue to divide, on the fifth day the embryo differentiates into a blastocyst, composed of two distinct cell lineages: the trophectoderm and the inner-cell mass. In natural development, the blastocyst then attaches to the woman’s endometrial epithelium and implants in her uterus [10,12,30,50]. In the IVF clinic, human embryos are typically cultured for 3 or 5 days after fertilization, at which point embryologists attempt to select the highest quality embryo(s), which is then transferred into the patient’s uterus.

We first examine the overall trajectory of development, as described by three variables: the number of cells in the embryo on Day 3 after fertilization (Day 3 Cells), its developmental stage on Day 5 (Day 5 Stage), and whether it resulted in a fetal heartbeat after transfer (FH; see electronic supplementary material, SI for details). Day 5 stage is scored from 1 to 9: (1) degenerate or arrested; (2) morula with incomplete compaction, (3) morula with complete compaction, (4) early blastocyst, (5) expanding blastocyst, (6) full blastocyst, (7) expanded blastocyst, (8) hatching blastocyst and (9) hatched blastocyst.

However, not all embryos are cultured to Day 5, and thus not all embryos have data on both Day 3 and Day 5: of the 55 350 embryos recorded on Day 3, only 41 932 are also recorded on Day 5. Since the decision to culture embryos to Day 5 is made based on patient prognosis and embryo quality, embryos that are assessed on Day 5 systematically differ from those that are not. To avoid biases due to this missing data, we treat missingness as an additional variable and model both the variable and its missingness [51]. The clinical data are in the ‘missing at random’ regime, where a variable’s missingness depends on other variables in the dataset, but not on the missing variable itself. In the language of probabilistic graphical models, there are edges from some of the normal variables to the missingness variables, but no edge from a variable to its own missingness. In this regime, valid inferences require conditioning on missingness and the variables on which the missingness depends. Multiple transfers cause an additional problem for the measurement of fetal heartbeat—if two embryos are transferred simultaneously and one fetal heartbeat is observed, it is not obvious which embryo formed the fetus. We solve this problem via generative modelling. Briefly, we construct a parameterized model that predicts the probability of one embryo implanting from properties of the embryo and the woman. We then fit the model to the data by finding the maximum a posteriori parameters, using a Poisson-binomial likelihood for multiple transfers. To check whether a variable is conditionally independent of fetal heartbeat, we fit two models, one with the additional variable and one without, and perform Bayesian model selection to see if the additional variable is necessary (electronic supplementary material, SI §1). We include all available cycles with four or fewer embryos transferred (95% of cycles).

With this approach, we construct a model of human pre-implantation development. The embryo’s number of cells on Day 3 is strongly correlated with its stage on Day 5: $\mathrm{Corr}(\mathrm{Day}\hspace{0.17em}3\hspace{0.17em}\mathrm{Cells},\mathrm{Day}\hspace{0.17em}5\hspace{0.17em}\mathrm{Stage}\hspace{0.17em}|\mathrm{both}\hspace{0.17em}\mathrm{measured})=0.44$ (p < 10⁻³⁰⁰, figure 5a). Both the embryo’s Day 3 Cells and its Day 5 Stage are individually predictive of fetal heartbeat (figure 5b,c), as appreciated in the literature [52–54]. However, when considered jointly, only Day 5 stage is predictive of fetal heartbeat; Day 3 Cells provides no additional information whether the embryo will develop (figure 5d). This conditional independence implies a model of the form $\mathrm{Day}\hspace{0.17em}3\hspace{0.17em}\mathrm{Cells}\to \mathrm{Day}\hspace{0.17em}5\hspace{0.17em}\mathrm{Stage}\to \mathrm{FH}$; this is the only graph with two or fewer edges that is consistent with the data and the fact that Day 3 happens before Day 5.

Figure 5.

(a) Day 5 Stage versus Day 3 Cells. Grey dots show the raw data, red circles show the mean Day 5 Stage for each separate value of Day 3 Cells, the green line and shaded region show the nonlinear model with the highest evidence and its uncertainty. The Day 5 Stages are: (1) degenerate or arrested, (2) morula with incomplete compaction, (3) morula with complete compaction, (4) early blastocyst, (5) expanding blastocyst, (6) full blastocyst, (7) expanded blastocyst, (8) hatching blastocyst, and (9) hatched blastocyst (see electronic supplementary material, §2 for details). (b) Estimated probability of an embryo resulting in a fetal heartbeat (FH) as a function of Day 3 Cells alone, for embryos recorded on Day 3 and transferred. The red circles and error bars show the probability estimated by a model that fits an independent probability of implantation for each number of cells; the green line and shaded region shows the nonlinear model with the highest model evidence and its uncertainty. (c) The estimated probability of FH as a function of Day 5 Stage alone, for embryos recorded on Day 5 and transferred. (d) The logit of the estimated probability of FH as a function of Day 3 Cells, after regressing against Day 5 Stage. Red circles and error bars show the additional log probability estimated from a model that fits an independent logit for each value of Day 3 Cells; green line shows the best linear model and uncertainty for the logit. The data are consistent with Day 3 Cells having no additional predictive power on FH once Day 5 Stage is known. (e) The graph with the minimal number of edges that is consistent with the data and a mechanistic interpretation. Black nodes and arrows show the measured data; grey nodes and arrows show the data’s missingness. Only the arrow Age → BMI can be re-oriented without breaking consistency with the data or a mechanistic interpretation. Edge labels for continuous variables are conditional correlation coefficients (roman typeface). Edge labels for discrete variables are coefficients from logistic regression (italic typeface), after treating the effects of other variables with edges into the discrete variable and after normalizing the input variable by its mean and standard deviation. The missingness variables D5 Rec. and Trans. are 1 if the embryo is recorded on Day 5 or transferred, respectively, and 0 otherwise. The distribution of Trans. changes depending on whether Day 5 Stage was recorded; the two labels on edges into Trans correspond to Day 5 Stage missing or recorded. (f) The data are consistent with a picture where processes which control pre-implantation development are largely different from those which control post-implantation development.

How do the woman’s age, her BMI and the aggressiveness of the ovarian stimulation additionally affect her embryos’ development? The woman’s age affects all stages of her embryos’ development, being weakly correlated with the embryo’s number of cells on Day 3 and its stage on Day 5: Corr(Day 3 Cells, Age) = −0.07 (p = 10⁻⁴⁷; electronic supplementary material, figure S6a), $\mathrm{Corr}(\mathrm{Day}\hspace{0.17em}5\hspace{0.17em}\mathrm{Stage},\mathrm{Age}\hspace{0.17em}|\mathrm{Day}\hspace{0.17em}3\hspace{0.17em}\mathrm{Cells},\mathrm{measured})=-0.11$ (p = 10⁻⁷⁸; electronic supplementary material, figure S6b), and having a strong effect on the probability that an embryo forms a fetal heartbeat (electronic supplementary material, figure S7c). Surprisingly, the woman’s BMI affects none of Day 3 Cells (electronic supplementary material, figure S6c), Day 5 Stage (electronic supplementary material, figure S6d) or FH (electronic supplementary material, figure S7e). Likewise, the conditional correlation of MII with all of D3 Cells, D5 Stage and FH is either consistent with zero or less than 0.05 in magnitude (electronic supplementary material, figures S6e,f and 7f). Combined, these observations yield the simple graphical model for development in figure 5e. The woman’s age determines the embryo’s cell number on Day 3; her age and the embryo’s cell number on Day 3 determine its stage on Day 5; and her age and the embryo’s stage on Day 5 determine whether it will continue to develop. Neither the patient’s BMI nor the number of retrieved MII eggs significantly affect the embryo’s development. By contrast, the data’s missingness shows a much more complex distribution, with almost all variables affecting the clinical decisions regarding embryo transfer and culture duration. Once again, the data paint a picture of simple biology but complex clinical decisions.

That Day 3 Cells has no direct effect on the fetal heartbeat probability is particularly striking. The time-varying morphology (morphokinetics) of human embryos is known to be predictive of developmental success [30,53]. In principle, morphokinetics up to Day 3 could provide a different set of information than morphokinetics between Days 3 and 5. For example, since the embryo’s genome activates on Day 3 [55], one might reasonably propose that the embryo’s progress before Day 3 provides information about the ooplasm, that the embryo’s progress from Day 3 to Day 5 provides information about aneuploidy, and that both of these factors independently determine the embryo’s prognosis. Instead, the data show that the embryo’s stage at Day 5 contains all the information about its viability that its cell number at Day 3 contains.

One physiological interpretation consistent with the data is that there are two distinct sets of mechanisms that influence an embryo’s development (figure 5f). One set of mechanisms influences pre-implantation development only through its overall rate, including the number of cells on Day 3 and the stage on Day 5. This set of mechanisms depends weakly on age. Another set of mechanisms influences an embryo’s post-implantation developmental potential and depends strongly on age. Perhaps surprisingly, meiotic aneuploidy of the oocyte cannot strongly affect pre-implantation development, since meiotic aneuploidy is strongly associated with the woman’s age. Instead, the data suggest that meiotic aneuploidy primarily affects post-implantation, rather than pre-implantation, development. Other works have provided mixed evidence regarding the extent to which aneuploidy affects pre-implantation development [56–59].

Is development genuinely this simple, or is this apparent simplicity an artefact of the variables we chose to describe the embryo? In addition to the number of cells, the dataset also describes the embryo on Day 3 with the presence of cytoplasmic fragments, the presence of multiple nuclei in individual cells, size asymmetries among cells within the embryo, the presence of large vacuoles, and the granularity of the cell cytoplasm. On Day 5, the dataset also includes grades of the inner-cell mass and the trophectoderm, for embryos that have formed blastocysts. Of the Day 3 variables, embryo fragmentation and cell symmetry are predictive of fetal heartbeat, along with Age and Day 3 Cells. Likewise, of the Day 5 variables, the trophectoderm grade is predictive of fetal heartbeat, along with Age and Day 5 Stage, in agreement with recent studies [60,61]. (The data are consistent with the other Day 3 variables providing no additional predictive power once Age, D3 Cells, symmetry, and fragmentation are known; and the data are consistent with the inner-cell mass grade providing no additional predictive power once Age, D5 stage and the trophectoderm grade are known.) Nevertheless, the Day 3 variables provide no additional predictive power for fetal heartbeat once the Day 5 variables are known (electronic supplementary material, §4). These observations suggest a memoryless model of pre-implantation development: provided the embryo makes it to the blastocyst stage, what happened before is irrelevant for its viability. Thus, despite the molecular complexities of early development and the complicated trajectory of human development before 12 weeks, a simple, phenomenological view of embryonic viability may be possible without sacrificing quantitative accuracy.

3. Conclusion

Here, we have used clinical IVF data and minimal prior knowledge to infer quantitative, phenomenological models of human oogenesis and embryogenesis. Not only does constructing these models with a data-driven approach give confidence in their validity, but the models recapitulate known aspects of oogenesis and embryogenesis. Surprisingly, the models that best describe the data are sparse, with only one or two factors affecting most physiological processes. This suggests that oogenesis and embryogenesis are modular processes. Our analysis leads to three additional, surprising conclusions which support this overall picture of modularity.

• (i) AMH production by one pre-antral follicle is independent of the amount produced by others. This is in stark contrast to other follicularly produced hormones. Hormones such as oestradiol and inhibin participate in feedback loops which regulate the formation of the dominant follicle in a natural cycle. As a result, these hormones are produced in a highly regulated manner, and not independently by each follicle [30,62]. Moreover, mathematical modelling suggests that, for these feedback loops to function, the hormone production and response needs to be highly nonlinear [63–65]. Thus, the amount of these hormones produced by one follicle depends strongly on the amount produced by other follicles. By contrast, AMH appears to be produced without regulatory feedback. This is particularly interesting because AMH regulates the follicle number, by regulating the growth of primordial follicles into primary (pre-antral) follicles. Thus, while feedback loops are needed to accurately control the number of mature follicles recruited during natural ovulation, feedback loops appear to be unnecessary to sufficiently control the number of primary follicles recruited.
• (ii) Neither age, obesity, the ovarian stimulation, nor even the number of recruited oocytes affects whether an individual oocyte progresses to metaphase-II once triggered to resume meiosis. These observations have several implications for the biology of the oocyte. Definitive evidence shows that age is strongly correlated with aneuploidy in the oocyte [27,39]. Since our analysis shows that the ability of an oocyte to progress to metaphase-II is independent of age, we conclude that this ability is the same for both euploid and aneuploid oocytes. Thus, the spindle assembly checkpoint in human oocytes must be weak, in agreement with recent experimental work [45–47]. A similar argument can be made regarding the effect of metabolism on meiosis. Evidence suggests that oocyte mitochondrial metabolism worsens with increasing obesity [29,66]. Since oocytes progress to metaphase-II independently of obesity, metabolic defects must not typically be enough to stop an oocyte from progressing from prophase-I to metaphase-II.
• (iii) Early embryonic development is memoryless, in that embryos with the same status on Day 5 develop the same, regardless of their status on Day 3. This memorylessness is reminiscent of the robustness of early mammalian embryos to damage to individual cells [67–69]; however, memorylessness is more than robustness. Robustness signifies that an embryo can recover from a setback. Memorylessness signifies that, once recovered, neither the setback nor what caused it has any impact on the rest of development. The memorylessness implies a modularity and robustness in development, similar to previous proposals for modularity in cellular biology [70].

The models we present also have implications for clinical IVF. For embryos transferred on Day 5, embryo selection can be based solely on how developed they are on Day 5, independent of their status on Day 3. While Day 3 information is correlated with implantation potential, the effect is completely captured by the embryo’s status on Day 5. For ovarian stimulation, the data provide no evidence that aggressive ovarian stimulation is detrimental to the oocyte, either in its ability to mature to metaphase II, to develop as an embryo, or, if transferred, to form a viable pregnancy (figures 3 and 5); moreover, the data constrain any of these effects to be small. Thus, a clinic should not be concerned about a potential trade-off between the quality and quantity of retrieved oocytes. Conversely, the data show that, at the clinic from which our data were derived, the ovarian stimulation drugs FSH and HMG are typically applied at saturating or slightly deleterious doses for follicular recruitment. Thus, the hormone dosage could presumably be slightly reduced here, to mitigate side effects such as ovarian hyper-stimulation syndrome or the high cost of stimulation drugs, without a large decrease in the number of retrieved oocytes.

The results we present here are inferred using data from only one clinic. As such, some aspects of our models reflect the practice at one particular clinic rather than general biology. The treatment portions of the models, such as FSH and HMG doses, transfer decisions and missingness, will change from clinic to clinic. Likewise, properties of the patient population, such as the joint distribution of patient age and BMI, will presumably change from clinic to clinic. By contrast, we suspect that the broader, structural relationships among hormones, oocytes and embryos reflect general biology that will be broadly applicable, although varying measurement standards across clinics may cause quantitative changes in these relationships. It will be interesting to test our models by looking at data both from other, non-clinical sources and from IVF clinics, especially from other countries.

More broadly, our results provide fundamental insights into the overall process of development. Unlike the sparseness that arises in theoretically motivated models simply as a way to manage complexity, the sparseness in our models is a property of the data. This sparseness implies that the biology itself is simple, consisting of modularized processes that are quantitatively siloed from one another. The simplicity is surprising given the many ways that oogenesis and embryogenesis are affected by diseases, including age-related infertility, endometriosis, sperm malfunction and polycystic ovary syndrome, all of which are present in our dataset. Overall, our results suggest that, despite their underlying complexities, oogenesis and embryogenesis are modular processes that result in simple, emergent behaviour and that a concise, quantitative understanding of the rest of development may be possible.

Data accessibility

Data are available at https://doi.org/10.6084/m9.figshare.15153525.

Authors' contributions

C.R. provided the data; B.L. performed the analysis; B.L., C.R. and D.N. identified relevant study questions; and B.L., C.R. and D.N. wrote the paper.

Competing interests

We declare we have no competing interests.

Funding

D.N., B.L. and C.R. acknowledge the National Institutes of Health for financial support (grant nos. R01HD092550-01 and R01HD104969-01).