Role of Markov Modeling Approaches to Understand the Impact of Infertility Treatments

Abstract

We are proposing to use Markov modeling type of analysis to understand data generated by treatments for infertility in women receiving ovarian stimulations. We describe the conceptual novelties, need for such an analysis, basics of the proposed methods, and theoretical constructions of various probabilities associated with practical level implementation of the Markov modeling procedures. This method can be adopted to infertility-related data visualizations whenever progression of outcome stages in infertility treatment is recorded. These methods if implemented should be able to enhance the understanding of treatment impacts of gonadotropins, clomiphene citrate, or an aromatase inhibitor at the beginning of treatment cycles of infertile women. This framework will be very useful for infertility treatment practitioners to compute the values of success rates of treatment for total population or population divided by demographic, clinical, and genetic factors. These methods can be continuously updated with newer data and translated into a mobile app to be used by clinical practitioners.

Motivation

In non-Markovian type of models, for example, standard regression models, which are very popular statistical tools for data analysis, are in general perfectly capable to understand the associations between several dependent variables that influence an independent variable, but such a mechanism has no capacity to predict the probability of occurrence of a future event of interest. One of the essential differences of the Markov modeling approach as compared to other leading modeling approaches (differential equations based or statistical models) is that Markov models consider probabilities of events of the recent past, which are distinct from the probability of occurrence of the event of primary interest. Situations amenable to the use of Markov modeling arise in biological, medical, and several other fields. Kimura’s model in population genetics understanding chromosome division is one of the earliest applications of Markov modeling ideas^1,2 that was published during the middle of the 20th century. In his model, a hypothetical chromosome consists of n subunits, which will duplicate to 2n subunits in each generation and split into a pair of chromosomes of each with n subunits. The number of mutant subunits in each generation could be any number in the set S, where $S=\left\{0,1,2,\dots ,n\right\}$. Suppose we are interested in the random variable, X_n that describes the number of mutant subunits in the nth generation. Let i and j be the 2 numbers from the set S. Given that X_n = i, we are interested in finding what will be the probability of $X_{n+1}=j$ (which in stochastic process terminology is called a transition probability from the state i to the state j, denoted by p_ij, and here X_n is a Markov chain). We will describe more on states and Markov properties later in the text. According to Kimura’s chromosome model²:

$$p_{ij}=\left(\begin{array}{c}2i\\ j\end{array}\right)\left(\begin{array}{c}2n-2i\\ n-j\end{array}\right)/\left(\begin{array}{c}2n\\ n\end{array}\right)\hspace{0.17em}\text{for }i,j=0,1,2,\dots ,n.$$ 1

Equation 1 is one of the original models using biological data to compute transition probabilities and has been frequently referred to for historical importance of Markov chain models in biology. Although mathematical models are widely used in biological and medical fields for measuring dynamics of variables of interest, sometimes these models could be complex and at times include unwanted past interactions between variables to understand future dynamics. When the probability of occurrence of future event depends only on the occurrence of the current event, rather than probabilities of all prior events (a feature sometimes referred to as memoryless property), the Markov modeling approach is often adopted.^3–6 We see this in the present context of infertility treatment data in the United States described in the recent publications,^7,8 where data on time durations between treatment initiation and other events such as conceiving and live birth were explored or between randomization of treatment categories and these other events mentioned. Markov modeling could give new insights on probabilities of pregnancy outcomes. More specifically, a Markov modeling framework can be used to predict the pregnancy outcomes of infertile women who are receiving fertility medications for ovarian stimulations, such as treatments like gonadotropins, clomiphene citrate, or an aromatase inhibitor. If we are not considering various stages of infertility treatment cycles, for example, in unexplained infertility data analysis, endometriosis type of data, male infertility factors in pregnancy difficulties, and so on, then such data also can be modeled by fitting other types of probability models where jumps in state space do not follow Markov property. The essential feature is whether infertility data under consideration have features that can be adopted by Markov modeling setup that we describe.

Markov Modeling Setup

In this article, we will explore a theoretical setting where Markov modeling ideas can be of immense help in clearly describing the infertility treatment data collected from a large number of women for quantifying the probability of live birth, given that the treated woman conceives, quantifying the probability of conceiving, given that the woman starts infertility treatment, and so on. Such ideas can be easily extendable for obtaining the probability of miscarriage or the probability of live births of pregnancy for a woman who has conceived. One can also compute the probability of not conceiving in a woman, given that the treatment was initiated and continued for a certain number of cycles. All of these quantifications of probabilities mentioned can be provided for the whole population of women who received treatment or can be provided for subpopulations of women, stratified by age, clinical presentations, genetic background, race, husband or partner’s clinical status, medications, and other demographic and clinical factors. One of the novelties of this work is that we develop the Markov modeling setup purely from the data visualization of various type of infertility data and build concepts as a consequence of real-world scenarios (without sacrificing technicalities basics of model-building). One can extend this setup to quantify probabilities to various appropriate infertility data analysis. Another novelty of our approach is that it can give very precise values of success or failure probabilities of treatments for women seeking treatment through developing a software code or even an app in doctors mobile.

In general, for a Markov modeling framework on sequence of clinical events, computing the probability of a future event depends only on an event just preceding the future event (say, the current event) and not at all on any events prior to the current event. (The model does not need anything prior to the current event so it does not memorize information prior to the current event.) In some sense, we compute probability of transition from a given clinical state at a point of time of an infertile woman to a different clinical state at a future point of time for this particular woman. Selection of a time unit for monitoring changes in the clinical state (eg, days, weeks, months, years, etc) depends upon the availability of data and the research questions posed. Time steps (ie, length between 2 time points of monitoring) are usually considered in uniform intervals. In our context, we can consider the probability that an infertile woman moving (or transitioning) from a state (say, initiation of infertility treatment) to a different state (say, conceiving) or to a higher state (say, live birth) with a weekly or monthly or some other prescribed time step. There could be some finite number of intermediate clinical states depending upon how we decide to model the information of treatment impact on infertility. A chain of events is considered in such a modeling and this chain obeys the Markov property of memoryless described above. When the time steps for observing the moves (or transitions) are continuous, we call this a continuous-time Markov chain. In contrast, if the time steps are discrete, this is called a discrete-time Markov chain. The decision to use a continuous-time or discrete-time framework depends on the availability of appropriate data. The set of all possible clinical states that an infertile woman after initiation of treatment moves can be taken as our state space. Since an infertile woman has to go through well-defined clinical states within the state space, such a Markov modeling consists of discrete state spaces over discrete time points.

State-Spaces Construction

Each infertile woman initiating a given treatment and adheres to some prescribed timing of intercourse is assumed either to become pregnant within some finite treatment cycles or to remain infertile following these cycles. To describe transitions of each infertile woman in a Markov modeling setup, let us define a few following successive clinical states. We define a woman to be in state 0 when she has started infertility treatment and is continuing the treatment, in state 1 when she conceives, in state 2 when she remains infertile after certain number of treatment cycles completion and treatment stopped, in state 3 when the conceived pregnancy ends in a miscarriage, and in state 4 when the conceived pregnancy results in delivery of a live birth. We can call the collection of all of these 4 states as a state space and let it be denoted by S₁ as given in Equation 2:

$$S_1=\left\{\text{state 0, state 1, state 2, state 3, state 4}\right\}.$$ 2

Suppose each woman who is under treatment (ie, in state 0) is monitored to determine whether she has moved to other states through monthly monitoring. A woman who has moved to state 1 from state 0 will take several months to reach state 4, if she does not reach state 3. After a certain number of monthly treatment cycles if a woman does not conceive, she will be classified as a move from state 0 to state 2. A woman who is in state 0 will have one of the following possible transitions:

$$\begin{array}{l} \left(\text{i}\right) \text{state }0\to \text{state }0\\ \left(\text{ii}\right) \text{state }0\to \text{state }1\to \text{state }4\\ \left(\text{iii}\right) \text{state }0\to \text{state }2\\ \text{(iv)} \text{state }0\to \text{state }1\to \text{state }3\end{array}$$

We could also add another state of pregnancy, say, state 1C for the ectopic pregnancy and also could divide state 4 into 2 states, namely, state 4A and state 4B, corresponding to pregnancy resulting in a preterm live birth and term live birth, respectively. These new states will create a different state space (say, S₂) as given in Equation 3:

$$S_2=\left\{\text{state 0, state 1C, state 1, state 2,}\text{ state 3, state 4A, state 4B}\right\}\text{.}$$ 3

These new states will lead to splitting of the above transition (ii) into 2 transitions:

$$\begin{array}{l}\left(\text{ii A}\right)\text{ state 0}\to \text{state 1}\to \text{state 4A}\hfill \\ \left(\text{ii B}\right)\text{ state 0}\to \text{state 1}\to \text{state 4B}\hfill \end{array}$$

The state space S₁ is flexible enough for the construction of other state spaces that are relevant to infertility treatments. For example, we could add intermediate states such as fertilization of oocytes in the reproductive tract, implantation of embryos in the uterus, and chemical pregnancy, which we denote as, say, state 1A, state 1B, and state 1D, respectively. The state space S₁ is modified into 2 state spaces S₃ and S₄ as given in Equations 4 and 5:

$$S_3=\left\{\text{state 0, state 1A, state 1B, state 1D,}\text{ state 2, state 3, state 4}\right\}.$$ 4

$$S_4=\left\{\text{state 0, state 1A, state 1B, state 1D, state 2,}\text{ state 3, state 4A, state 4B}\right\}.$$ 5

Some of the possible transitions for the state space S₃ are:

$$\begin{array}{l}\left(\text{ii C}\right)\text{ state 0}\to \text{state 1A}\to \text{state 1B}\to \text{state 4}\\ \left(\text{ii D}\right)\text{ state 0}\to \text{state 1A}\to \text{state 1D}\end{array}$$

and some of the possible transitions for the state space S₄ are:

$$\begin{array}{l}\left(\text{ii E}\right)\text{ state 0}\to \text{state 1A}\to \text{state 1B}\to \text{state 4A}\\ \left(\text{ii F}\right)\text{ state 0}\to \text{state 1A}\to \text{state 1B}\to \text{state 4B}\\ \left(\text{ii G}\right)\text{ state 0}\to \text{state 1A}\to \text{state 1B}\to \text{state 3}\end{array}$$

For a large number of women who were on treatment and have gone through one or more of the above type of transitions, it is possible to combine the information for all the women who have gone through similar types of treatment. This allows construction of probabilities associated with each type of transition. See Figure 1 for a schematic structure of transitions between various states of 4 different state spaces constructed. Suppose we retrieve the information on the number of women who were treated and followed until any of the above end states are observed, then we can obtain the probability of live birth as follows:

$$\text{Probability of live birth for thewomen who conceived withinfirst treatment cycle month =}\frac{\begin{array}{c}\text{Number of women in state 4 who conceived during firsttreatment cycle month}\end{array}}{\begin{array}{c}\text{Number of women in state 1 who conceived during firsttreatment cycle month}\end{array}}.$$ 6

Figure 1.

Schematic diagram for describing possible transitions within each state space for which Markov modeling is proposed. Each number within a circle from (A) to (D) indicates a state in the Markov chain whose description is given in the section on State-Spaces Constructions. Some of the women in a given state might remain in the same state within a short time period (eg, a woman who conceives remains pregnant for 40 weeks before delivery) or a woman in a given state might move to another state and return to the original state after some time (eg, a woman who has initiated a treatment might move to the state of ectopic pregnancy and then return back to a no-pregnancy state prior to initiation of another cycle of treatment).

Clearly, the above probability (Equation 6) did not consider the number of women who were in state 0, although the number of women in state 1 was a direct result of the number of women who were on treatment, a feature that obeys the Markov property. We call the probability of live birth that was computed from such type of data the transition probability from state 1 to state 4. Also note that, in computation of the above probability, we considered the time step as constant for all the women who delivered a live birth. Such transition probabilities can be computed for the other transitions mentioned above, for example, we can compute the probability of a chemical pregnancy given that there was a successful implantation as given in Equation 7:

$$\frac{\text{Number of women in state 1B who had successul implantation}}{\text{Number of women who had successful implantation}}.$$ 7

We can also add time units for computing probabilities of transitions between states. Suppose we collect weekly data of women from the time of initiation of treatment (state 0) and record the information until a woman drops out of the study (due to failure to conceive or due to miscarriage after conceiving) or until the woman delivers a live birth. The state space can be defined as states generated by weekly data, where state k could be for a woman who reached kth week of treatment or pregnancy or dropped out for k = 1, 2, … w, where w is the week of live birth. We do not have to form a cohort of women on treatment prospectively for generating transition probabilities. We can collect treatment responses of each woman retrospectively and formulate retrospective cohorts of all women with the same time step between state 0 and conceiving (some other suitable time step can be defined). The probability of transition from state j to state j + 1 (which we denote as ${\left(p_T\right)}_{jj+1}$ for a constant time step T) is computed as

$${\left(p_T\right)}_{jj+1}=\frac{{{\displaystyle \int }}_1^n{W}_T^{j\to j+1}\left(s\right)ds}{{{\displaystyle \int }}_1^m{W}_T^j\left(s\right)ds},$$ 8

for large n, m, and n ≥ m.

Here $\underset{1}{\overset{n}{{\displaystyle \int }}}{W}_T^{j\to j+1}\left(s\right)ds$ is the number of women moved from a state j to the state j + 1 who all have satisfied a defined time step T, and the number of woman who moved from state j to state j + 1 $\underset{1}{\overset{m}{{\displaystyle \int }}}{W}_T^j\left(s\right)ds$ is the number of women in a state j who have all satisfied the same time step T.

The constant time step rule helps in formulation of time homogenous cohorts of women. Markov property holds because the number of women who were at state j − 1 is not influencing the computation of transition probability from a state j to the state j + 1. These transition probabilities help us to formulate the transition probability matrix, ${\left[{\mathit{P}}_{\mathit{T}}\right]}_{\mathit{\omega }+1\times \mathit{\omega }+1}$, where P_T will be generated by the values of ${\left(p_T\right)}_{jj+1}$ for $j=0,1,\dots ,\mathit{\omega }-1$. Since treatment impact could vary by age, baseline clinical factors, genetic factors, and so on, we can formulate transition probability matrices for each subpopulation with a fixed time step. For sth subpopulation of women with time step T, we can have ${\left[{\mathit{P}}_{\mathit{T}}\left(\mathit{s}\right)\right]}_{\mathit{\omega }+1\times \mathit{\omega }+1}$ using ${\left(p_T\right)}_{jj+1}\left(s\right)$. Of note, the lesser the number of transitions between the states (a very smaller sample), the lesser efficient is the transition probability matrix.

Once the state space is decided and transition matrices are formulated, other properties of the states can be derived. Since our stochastic process begins from the initiation of a treatment and ends if a woman delivers a live birth or miscarriage happens or treatment remains ineffective after predetermined number of treatment cycles, these 3 ending states are called absorbing states. A woman who exited from a pregnancy process due to any of the 3 absorbing states might come back for the same treatment or for a different treatment to try to conceive will be considered in a new process, and all such women data will be used for building a new Markov modeling for predicting infertility treatments.

Transition Probability Matrices

A sample transition probability matrix, say, ${\mathit{P}}_{\mathit{T}}^{\text{'}}\left(\mathit{s}\right)$ using 5 basic states described in S₁ for a woman in sth subpopulation with a time step T can be represented as follows:

$${\mathit{P}}_{\mathit{T}}^{\text{'}}\left(\mathit{s}\right)=\begin{array}{c} \\ 0\\ 1\\ 2\\ 3\\ 4\end{array}\left[\begin{array}{ccccc}0& 1& 2& 3& 4\\ {(p_T)}_{00}(s)& {(p_T)}_{01}(s)& {(p_T)}_{02}(s)& 0& 0\\ 0& {(p_T)}_{11}(s)& 0& {(p_T)}_{13}(s)& {(p_T)}_{14}(s)\\ 0& 0& 1& 0& 0\\ 0& 0& 0& 1& 0\\ 0& 0& 0& 0& 1\end{array}\right].$$ 9

Suppose ${\mathit{P}}_{\mathit{T}}^{\text{'}}\left(\mathit{s}\right)$ is computed based upon the data available on 2 treatments: gonadotropin, say ${\mathit{P}}_{\mathit{T}}^{\text{'}}\left(\mathit{s}:\mathit{G}\right)$ and for letrozole, say ${\mathit{P}}_{\mathit{T}}^{\text{'}}\left(\mathit{s}:\mathit{L}\right)$, and we wanted to determine whether the treatment differences are significantly different. For this, we will consider each transition probability and corresponding sample sizes that led to the computation of ${\mathit{P}}_{\mathit{T}}^{\text{'}}\left(\mathit{s}:\mathit{G}\right)$ and ${\mathit{P}}_{\mathit{T}}^{\text{'}}\left(\mathit{s}:\mathit{L}\right).$ Let n ₀₀(G), n ₀₁(G), n ₀₂(G), n ₁₁(G), n ₁₃(G), n ₁₄(G) be the sample sizes that led to the computation of ${\left(p_T\right)}_{00}\left(s:G\right)$, ${\left(p_T\right)}_{01}\left(s:G\right)$, ${\left(p_T\right)}_{02}\left(s:G\right)$ ${\left(p_T\right)}_{11}\left(s:G\right)$, ${\left(p_T\right)}_{13}\left(s:G\right)$, ${\left(p_T\right)}_{14}\left(s:G\right)$ and let n ₀₀(G), n ₀₁(G), n ₀₂(G), n ₁₁(G), n ₁₃(G), n ₁₄(G) be the corresponding sample sizes that led to the computation of ${\left(p_T\right)}_{00}\left(s:L\right)$, ${\left(p_T\right)}_{01}\left(s:L\right)$, ${\left(p_T\right)}_{02}\left(s:L\right)$ ${\left(p_T\right)}_{11}\left(s:L\right)$, ${\left(p_T\right)}_{13}\left(s:L\right)$, ${\left(p_T\right)}_{14}\left(s:L\right)$ in the transition probability matrix (Equation 9). We can compute 6 sets of z scores using the following test statistics:

$$z_{ij}=\frac{{\left(p_T\right)}_{ij}\left(s:G\right)-{\left(p_T\right)}_{ij}\left(s:L\right)}{{\text{SE}}_{ij}},$$ 10

where the formulae for the standard error (SE) terms are:

$${\text{SE}}_{ij}=\sqrt{p_{ij}\left(1-p_{ij}\right)\left(\frac{1}{n_{ij}\left(G\right)}+\frac{1}{n_{ij}\left(L\right)}\right)},$$

$$\text{where }{p}_{ij}=\frac{{\left(p_T\right)}_{ij}\left(s:G\right)n_{ij}\left(G\right)+{\left(p_T\right)}_{ij}\left(s:L\right)n_{ij}\left(L\right)}{n_{ij}\left(G\right)+n_{ij}\left(L\right)}.$$

Here, $z_{ij},{\text{SE}}_{ij,}\text{ and }{p}_{ij}$ are computed for the 6 pairs of the following (i, j) values:

$$\left(i,j\right)=\left\{\left(0,0\right),\left(0,1\right),\left(0,2\right),\left(1,1\right),\left(1,2\right),\left(1,4\right)\right\}.$$

Using these test statistics, we can compute P values and draw conclusions on differences between the 2 treatments. The transition probability matrix using 5 basic states and test statistics computations can be extended to generalized transition probabilities ${\left(p_T\right)}_{jj+1}$ for a finite number of states that were introduced in other state spaces such as S ₂, S ₃, and S ₄. Of note, we can also extend this method of comparison to 3 treatments by comparing 3 pairs of transition probability matrices.

Numerical Example

In this section, we provide a numerical example of the transition probability matrix based on hypothetical data for treatment cycles and movement to various states by women on 2 treatments.

Suppose 560 and 723 infertility women initiate treatments of gonadotropin and aromatase inhibitor (state 0), the number of women among them who are at various clinical states (1A to 4), namely, fertilization (completion of penetration of sperms into an egg), implantation (attachment of fertilized egg to lining of uterus), ectopic pregnancy (implantation of egg outside the uterus), chemical pregnancy (improper implantation of egg within the uterus), becoming pregnant, miscarriage (termination of pregnancy), stopped treatment after failure to conceiving, preterm delivery (delivering before 37 weeks of pregnancy), and term delivery (delivery during or after 37th week of pregnancy) was recorded. The probability that a woman on gonadotropin who had successful fertilization will have oocytes implanted in the uterus is 162 of 510 = 0.32 and the probability that a woman on an aromatase inhibitor has remained infertile and discontinued the treatment over a time period is 51 of 723 = 0.071. In general, we can compute the transition matrix of Equation 8 for 2 treatments: gonadotropin (G) and aromatase inhibitor from the data in Table 1, which is below:

Table 1.

A Numerical Example of Various States in the Markov Chain for a Fixed Time Period for Women Who Are on Infertility Treatments—Gonadotropin and Aromatase Inhibitor.

Treatment	Stated Infertility Treatment (State 0)	Fertilization (State 1A)	Implantation (State 1B)	Ectopic Pregnancy (State 1C)	Chemical Pregnancy (State 1D)	Pregnant (State 1)	Remain Infertile (State 2)	Miscarriage (State 3)	Live Birth (State 4)	Preterm Delivery (State 4A)	Term Delivery (State 4B)
Gonadotropin	560	510	162	11	39	162	50	10	153	17	136
Aromatase inhibitor	723	672	174	16	51	174	51	7	166	18	148

$$\begin{array}{cc}{\mathit{P}}_{\mathit{T}}^{\text{'}}\left(\mathit{s}:\mathit{G}\right)=\begin{array}{c} \\ \begin{array}{c}0\\ \begin{array}{c}1\\ \begin{array}{c}2\\ \begin{array}{c}3\\ 4\end{array}\end{array}\end{array}\end{array}\end{array}& \left[\begin{array}{c}0\\ \begin{array}{c}0\\ \begin{array}{c}0\\ \begin{array}{c}0\\ \begin{array}{c}0\\ 0\end{array}\end{array}\end{array}\end{array}\end{array}\begin{array}{c}1\\ \begin{array}{c}0.29\\ \begin{array}{c}0\\ \begin{array}{c}0\\ \begin{array}{c}0\\ 0\end{array}\end{array}\end{array}\end{array}\end{array}\begin{array}{c}2\\ \begin{array}{c} 0.09\\ \begin{array}{c}0\\ \begin{array}{c}1\\ \begin{array}{c}0\\ 0\end{array}\end{array}\end{array}\end{array}\end{array}\begin{array}{c}3\\ \begin{array}{c}0\\ \begin{array}{c}0.06\\ \begin{array}{c}0\\ \begin{array}{c}1\\ 0\end{array}\end{array}\end{array}\end{array}\end{array}\begin{array}{c}4\\ \begin{array}{c}0\\ \begin{array}{c}0.94\\ \begin{array}{c}0\\ \begin{array}{c}0\\ 1\end{array}\end{array}\end{array}\end{array}\end{array}\right]\end{array}.$$ 11

$$\begin{array}{cc}{\mathit{P}}_{\mathit{T}}^{\text{'}}\left(\mathit{s}:\mathit{A}\mathit{I}\right)=\begin{array}{c} \\ \begin{array}{c}0\\ \begin{array}{c}1\\ \begin{array}{c}2\\ \begin{array}{c}3\\ 4\end{array}\end{array}\end{array}\end{array}\end{array}& \left[\begin{array}{c}0\\ \begin{array}{c}0\\ \begin{array}{c}0\\ \begin{array}{c}0\\ \begin{array}{c}0\\ 0\end{array}\end{array}\end{array}\end{array}\end{array}\begin{array}{c}1\\ \begin{array}{c}0.24\\ \begin{array}{c}0\\ \begin{array}{c}0\\ \begin{array}{c}0\\ 0\end{array}\end{array}\end{array}\end{array}\end{array}\begin{array}{c}2\\ \begin{array}{c} 0.07\\ \begin{array}{c}0\\ \begin{array}{c}1\\ \begin{array}{c}0\\ 0\end{array}\end{array}\end{array}\end{array}\end{array}\begin{array}{c}3\\ \begin{array}{c}0\\ \begin{array}{c}0.04\\ \begin{array}{c}0\\ \begin{array}{c}1\\ 0\end{array}\end{array}\end{array}\end{array}\end{array}\begin{array}{c}4\\ \begin{array}{c}0\\ \begin{array}{c} 0.96\\ \begin{array}{c}0\\ \begin{array}{c}0\\ 1\end{array}\end{array}\end{array}\end{array}\end{array}\right]\end{array}.$$ 12

These 2 matrices gives us various transition probabilities for a woman on gonadotropin (Equation 11) and woman on AI (Equation 12) to conceive, to deliver a live birth, miscarriage, and so on. In case we want to compare these probabilities, we can compute z scores using the test statistic in Equation 9. For example, to test the hypothesis that 2 transition probabilities from state 0 to state 1 under both the treatments are equal, the z ₀₁ score will be:

$$\frac{0.05}{{\text{SE}}_{01}}=2.024,$$

where ${\text{SE}}_{01}=\sqrt{0.26\left(1-0.26\right)\left(\frac{1}{560}+\frac{1}{723}\right)}=0.0247$. Since 2.024 > 1.96 (the critical value of at $z_{1-0.05/2}$), we conclude that the impact of gonadotropin treatment is significantly different for conceiving in women treated with an AI, and also we conclude that the AI has lesser impact than gonadotropin for a successful conceiving.

$${{\mathit{P}}^{\prime }}_{\mathit{T}}\begin{array}{cc}\left(\mathit{s}:\mathit{G}\right)=\begin{array}{c} \\ \begin{array}{c}0\\ \begin{array}{c}1\\ \begin{array}{c}2\\ \begin{array}{c}3\\ 4\end{array}\end{array}\end{array}\end{array}\end{array}& \left[\begin{array}{c}0\\ \begin{array}{c}0\\ \begin{array}{c}0\\ \begin{array}{c}0\\ \begin{array}{c}0\\ 0\end{array}\end{array}\end{array}\end{array}\end{array}\begin{array}{c}1\\ \begin{array}{c} 0.29\\ \begin{array}{c}0\\ \begin{array}{c}0\\ \begin{array}{c}0\\ 0\end{array}\end{array}\end{array}\end{array}\end{array}\begin{array}{c}2\\ \begin{array}{c} 0.09\\ \begin{array}{c}0\\ \begin{array}{c}1\\ \begin{array}{c}0\\ 0\end{array}\end{array}\end{array}\end{array}\end{array}\begin{array}{c}3\\ \begin{array}{c}0\\ \begin{array}{c}0.06\\ \begin{array}{c}0\\ \begin{array}{c}1\\ 0\end{array}\end{array}\end{array}\end{array}\end{array}\begin{array}{c}4\\ \begin{array}{c}0\\ \begin{array}{c} 0.94\\ \begin{array}{c}0\\ \begin{array}{c}0\\ 1\end{array}\end{array}\end{array}\end{array}\end{array}\right]\end{array}.$$ 13

$${{\mathit{P}}^{\prime }}_{\mathit{T}}\begin{array}{cc}\left(\mathit{s}:\mathit{A}\mathit{I}\right)=\begin{array}{c} \\ \begin{array}{c}0\\ \begin{array}{c}1\\ \begin{array}{c}2\\ \begin{array}{c}3\\ 4\end{array}\end{array}\end{array}\end{array}\end{array}& \left[\begin{array}{c}0\\ \begin{array}{c}0\\ \begin{array}{c}0\\ \begin{array}{c}0\\ \begin{array}{c}0\\ 0\end{array}\end{array}\end{array}\end{array}\end{array}\begin{array}{c}1\\ \begin{array}{c} 0.24\\ \begin{array}{c}0\\ \begin{array}{c}0\\ \begin{array}{c}0\\ 0\end{array}\end{array}\end{array}\end{array}\end{array}\begin{array}{c}2\\ \begin{array}{c} 0.07\\ \begin{array}{c}0\\ \begin{array}{c}1\\ \begin{array}{c}0\\ 0\end{array}\end{array}\end{array}\end{array}\end{array}\begin{array}{c}3\\ \begin{array}{c}0\\ \begin{array}{c}0.04\\ \begin{array}{c}0\\ \begin{array}{c}1\\ 0\end{array}\end{array}\end{array}\end{array}\end{array}\begin{array}{c}4\\ \begin{array}{c}0\\ \begin{array}{c} 0.96\\ \begin{array}{c}0\\ \begin{array}{c}0\\ 1\end{array}\end{array}\end{array}\end{array}\end{array}\right]\end{array}.$$ 14

$$Z_{01}{Z}_{01}=\frac{\text{0.05}}{{\text{SE}}_{\text{01}}}=2.024{\text{SE}}_{01}\sqrt{0.26(1-0.26)\left(\frac{1}{560}\text{+}\frac{1}{723}\right)}={0.0247}_{Z_{1-0.05/2}}.$$

Discussion

Of note, these Markov modeling analyses are distinguishable from infertility data analysis done through standard regression processes, which will provide associations between various outcomes of the treatment with clinical, biological, genetic, and demographic factors of women and does not provide any probabilities for these outcomes measures (or absorbing the 3 major states described above). It should be noted that there is a need for caution when implementing a Markov modeling approach as proposed. For example, there may be situations where the number of transitions between certain states are not observed in enough samples for computing probabilities representing populations by age or the memoryless property may not actually hold for women who are older or who have previously undergone multiple cycles of in vitro fertilization before starting the current treatment cycle.