Table 1.
Summary of validity appraisal checklist results obtained by the two raters after resolution of discrepancies by consensus.
| Studies | ||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| A | B | C | D | E | F | G | H | I | J | K | L | M | N | O | P | Q | R | S | T | U | V | W | X | Y | Z | |
| Face validity | ||||||||||||||||||||||||||
| The model aims to describe a real-world phenomenon (i.e., a target state/condition vis a vis comparators) | 1 | 1 | 1 | 1 | 1 | 0.5 | 1 | 1 | 0.5 | 1 | 1 | 0.5 | 0.5 | 1 | 0.5 | 1 | 1 | 0.5 | 1 | 0.5 | 1 | 0.5 | 1 | 1 | 0.5 | 0.5 |
| The target condition being modeled is identifiable according to observable features | 0.6 | 0.2 | 0.8 | 0.2 | 0.2 | 0.6 | 0.8 | 0 | 0.4 | 0 | 1 | 0 | 0.8 | 1 | 0.8 | 0.2 | 0 | 0.2 | 0.2 | 1 | 1 | 0.8 | 0.2 | 0 | 1 | 1 |
| The comparator condition being modeled is identifiable according to observable features | 0.7 | 0.2 | 0.7 | 0.3 | 0 | 0 | 0.8 | 0 | 0 | 0 | 1 | 0 | 0 | 1 | 0 | 0.2 | 0 | 0 | 0.3 | 0 | 1 | 0 | 0.2 | 0 | 0 | 0 |
| The model actually explains or predicts the target condition vis a vis the comparator | 0.1 | 0.1 | 0.4 | 0.1 | 0.1 | 0.1 | 0.4 | 0.1 | 0.3 | 0.1 | 0.4 | 0.1 | 0.4 | 0.1 | 0.4 | 0.1 | 0.1 | 0.1 | 0.1 | 0.7 | 0.4 | 0.7 | 0.1 | 0.1 | 0.4 | 0.4 |
| Predictive validity | ||||||||||||||||||||||||||
| There are identifiable and meaningful transitions between conditions or states of interest in the real-world phenomenon | 1 | 0.6 | 0.6 | 0.6 | 0.6 | 1 | 0 | 0 | 0 | 0.4 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0.6 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Interventions or transitions in the model explain or predict corresponding transitions in the condition of interest | 0.6 | 0.4 | 0 | 0.4 | 0.8 | 0.8 | 0 | 0.4 | 0 | 0.2 | 0.8 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0.4 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Construct validity | ||||||||||||||||||||||||||
| There is a real and identifiable or plausible mechanism underlying the target condition | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0.5 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0.5 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0.5 | 0 | 0 |
| The model architecture is homologous to the mechanism of interest, at an appropriate level of abstraction | 0 | 0.3 | 0.3 | 0.3 | 0 | 0.3 | 0 | 0.7 | 0.3 | 0 | 0 | 0 | 0.3 | 0.3 | 0 | 0.3 | 0.3 | 0.3 | 0 | 0.3 | 0 | 0 | 0 | 0 | 0.3 | 0 |
Results are split into sections concerning face validity, predictive validity, and construct validity. Each row indicates a specific subscale for the validity type being scored. Values depicted here are the proportions of affirmative responses across individual items within each subscale (the possible range is 0 to 1). Full results, including specific subscale items are shown in Supplemental Table 1. Legend: [A] Huber, Braun, and Krieg 1999; [B] Mohan 2007; [C] Conte et al. 2009; [D] Daugherty et al. 2009; [E] Nana 2009; [F] Goldbeter 2011; [G] Bonsall et al. 2012; [H] Frank 2013; [I] Goldbeter 2013; [J] Hadaeghi, Hashemi Golpayegani, and Gharibzadeh 2013; [K] Steinacher and Wright 2013; [L] Koutsoukos and Angelopoulos 2014; [M] Bonsall et al. 2015; [N] Ortiz et al. 2015; [O] Cochran, McInnis, and Forger 2016; [P] Hadaeghi et al. 2016; [Q] Bayani et al. 2017; [R] Cochran et al. 2017; [S] Chang and Chou 2018; [T] Cochran et al. 2018; [U] Ortiz et al. 2019; [V] Prisciandaro, Tolliver, and DeSantis 2019; [W] Doho et al. 2020; [X] Nobukawa et al. 2020; [Y] Moore et al. 2012; [Z] Moore et al. 2014.