Table A2.
Quality appraisal form
| Component Ratings of Study: | Score | Justification/Comments |
|---|---|---|
| Strong = 3/Modest = 2/Weak = 1 | ||
| A) Design | ||
| Outcome of interest as main (3) or control variable (2/1)? | ||
| Cross‐sectional (2/1) or longitudinal (3) | ||
| Prospective (3) or retrospective (2/1) | ||
| Is the method of analysis appropriate? (strong, modest, weak) | ||
| Is the method of analysis sufficiently rigorous? (strong, modest, weak) | ||
| B) Quality of reporting | ||
| Enough data have been presented to show how the authors arrived at their findings (Strong, Modest, Weak) | ||
| Enough information is given what the methodological design is? (Strong, Modest, Weak) | ||
| Enough information is given where the data comes from and what the characteristics are of the sample (ie, summary statistics and sample sizes). (Strong, Modest, Weak) | ||
| C) Selection bias | ||
| Strong: The selected individuals/hospitals are very likely to be representative of the target population | ||
| Moderate: The selected individuals/hospitals are at least somewhat likely to be representative of the target population | ||
| Weak: The selected individuals/hospitals are not likely to be representative of the target population | ||
| D) Confounders (ie, region, demographics) | ||
| Strong: will be assigned to those articles that controlled for most relevant confounders | ||
| Moderate: will be given to those studies that controlled for relevant confounders, but explicitly mentions that it missed some relevant confounders | ||
| Weak: will be assigned when the relevant confounders were not controlled for | ||
| E) Data collection methods | ||
| Strong: The data collection tools have been shown to be valid; and the data collection tools have been shown to be reliable | ||
| Moderate: The data collection tools have been shown to be valid; and the data collection tools have not been shown to be reliable or reliability is not described. | ||
| Weak: The data collection tools have not been shown to be valid or both reliability and validity are not described. | ||
| F) Outcome variable | ||
| The choice of measurement of the outcome variable (accessibility, quality of care efficiency) is valid? | ||
| Strong: Clear connection with 1 of the 3 concepts, and/or is generally accepted by scholars | ||
| Moderate: A couple of validity issues arise. The connection between the outcome variable and the concepts of interest is moderate (eg, only one disease is analyzed) | ||
| Weak: Serious concerns about how the outcome variable (1 of the 3 concepts) is measured | ||
| G) Number of hospitals | ||
| Strong: More than 10 hospitals are included in the analysis | ||
| Moderate: Between 3 and 10 hospitals are included in the analysis | ||
| Weak: Only 2 hospitals are compared | ||
| H) Context | ||
| Strong: Includes many different contexts/regions, high complexity in demographic characteristics | ||
| Moderate: Combines 2 or 3 different regions | ||
| Weak: One very specific region with specific characteristics | ||
| J) Independence | ||
| Is this an independent study? Yes (3) Debatable (2) No (1) | ||
| K) Drop‐outs—only if applicable | ||
| Strong: (If applicable: will be assigned when the follow‐up rate is 80% or greater). | ||
| Moderate (If applicable: will be assigned when the follow‐up rate is 60%‐79%). | ||
| Weak: (If applicable: will be assigned when a follow‐up rate is less than 60% or if the withdrawals and drop‐outs were not described). | ||
| Total score | ||
| Additional comments | Answers to comments | |
| Do the results seem to be valid? | ||
| Do the results seem to be reliable? | ||
| Are the results relevant? Does it fall within the scope of our research question? | ||
| Can the results be generalized? | ||
| In or out | If needed: justification | |
| Final judgment made based on the score and the additional comments | ||