. 2020 Oct 21;40(2):369–381. doi: 10.1002/sim.8779

TABLE 1.

Estimands describing the uncertainty of model estimation incurred by variable selection, their approximation by simulation, and their estimation by resampling

Estimand

Definition

Approximationby simulation

Resampling‐based estimator

VIF_j

E [I ({\hat{β}}_{j} \neq 0)]

\sum_{q = 1}^{Q} I ({\hat{β}}_{j}^{q} \neq 0) / Q

\sum_{b = 1}^{B} I ({\hat{β}}_{j}^{b} \neq 0) / B

MSF(J)

E [\prod_{j \in J} I ({\hat{β}}_{j} \neq 0) \cdot \prod_{j^{'} \in J^{'}} I ({\hat{β}}_{j^{'}} = 0)]

\sum_{q = 1}^{Q} \prod_{j \in J} I ({\hat{β}}_{j}^{q} \neq 0) \cdot \prod_{j^{'} \in J^{'}} I ({\hat{β}}_{j^{'}}^{q} = 0) / Q

\sum_{b = 1}^{B} \prod_{j \in J} I ({\hat{β}}_{j}^{b} \neq 0) \cdot \prod_{j^{'} \in J^{'}} I ({\hat{β}}_{j^{'}}^{b} = 0) / B

RCB_j

(\frac{E ({\hat{β}}_{j})}{β_{j} \cdot VI F_{j}} - 1)

(\frac{\sum_{q = 1}^{Q} {\hat{β}}_{j}^{q}}{β_{j} \cdot VI F_{j} \cdot Q} - 1)

(\frac{\sum_{b = 1}^{B} {\hat{β}}_{j}^{b}}{{\tilde{β}}_{j} \cdot \hat{VI F_{j}} \cdot B} - 1)

RMSDR_j

\sqrt{\frac{E {({\hat{β}}_{j} - β_{j})}^{2}}{E {({\tilde{β}}_{j} - β_{j})}^{2}}}

\sqrt{\frac{\sum_{q = 1}^{Q} {({\hat{β}}_{j}^{q} - β_{j})}^{2} / Q}{\sum_{q = 1}^{Q} {({\tilde{β}}_{j}^{q} - β_{j})}^{2} / Q}}

\sqrt{\frac{\sum_{b = 1}^{B} {({\hat{β}}_{j}^{b} - {\tilde{β}}_{j})}^{2} / B}{{\tilde{σ}}_{j}^{2}}}

Note: Superscripts q or b indicate estimates obtained in the qth simulated dataset or the bth resample, respectively. Estimates ${\hat{β}}_{j}$ and ${\hat{β}}_{j}^{b}$ are set to 0 if they are not selected by the variable selection algorithm in the corresponding model. For the sets J and J′ of indices, J ∪ J^′ = {1, …, k}, and J ∩ J^′ = { }. I(·) is the indicator function, that is, it is 1 if the expression (·) is true and 0 otherwise.

Abbreviations: MSF, model selection frequency; RCB, relative conditional bias; RMSDR, root mean squared difference ratio; VIF, variable inclusion frequency.