Table 1.
Summary of Medical Education Research Study Quality Instrument (MERSQI) Domain and Item Scores for 31 Studies of Undergraduate Medical Education in Substance Abuse*
| Possible score | Actual mean (SD) scores | |||||
|---|---|---|---|---|---|---|
| Domain | MERSQI item | Studies, no. (%) | Item | Maximum domain |
Item | Domain |
| Study design | 1. Study design | 3 | 1.64 (0.52) | 1.64 (0.52) | ||
| Single group cross-sectional or single group posttest only | 8 (25.8) | 1 | ||||
| Single group pretest and posttest | 11 (35.5) | 1.5 | ||||
| Nonrandomized, 2 (or more) groups | 10 (32.3) | 2 | ||||
| Randomized controlled trial | 2 (6.5) | 3 | ||||
| Sampling | 2. No. of institutions studied | 3 | 0.56 (0.21) | 1.66 (0.58) | ||
| 1 | 26 (83.9) | 0.5 | ||||
| 2 | 2 (6.5) | 1 | ||||
| >2 | 3 (9.7) | 1.5 | ||||
| 3. Response rate, % (highest reported) | 1.10 (0.49) | |||||
| Not applicable | 0 | |||||
| < 50 or not reported | 10 (32.3) | 0.5 | ||||
| 50–74 | 1 (3.2) | 1 | ||||
| ≥ 75 | 20 (64.5) | 1.5 | ||||
| Type of data | 4. Type of data | 3 | 2.29 (0.97) | 2.29 (0.97) | ||
| Assessment by study participant (e.g., self-reported data) | 13 (42.0) | 1 | ||||
| Objective measurement (e.g., OSCE, written exam) | 18 (58.1) | 3 | ||||
| Validity of evaluation instrument† | 5. Internal structure | 3 | 0.26 (0.44) | 0.97 (0.87) | ||
| Not applicable | 0 | |||||
| Not reported | 22 (71.0) | 0 | ||||
| Reported | 9 (29.0) | 1 | ||||
| 6. Content (purposeful process to instrument development) | 0.65 (0.49) | |||||
| Not applicable | 0 | |||||
| Not reported | 23 (74.2) | 0 | ||||
| Reported | 8 (25.8) | 1 | ||||
| 7. Relationships to other variables (criterion, predictive or discriminate validity) | 0.06 (0.25) | |||||
| Not applicable | 0 | |||||
| Not reported | 25 (80.6) | 0 | ||||
| Reported | 6 (19.4) | 1 | ||||
| Data analysis | 8. Appropriateness of analysis | 3 | 0.84 (0.37) | 2.58 (0.72) | ||
| Data analysis inappropriate for study design or type of data | 4 (12.9) | 0 | ||||
| Data analysis appropriate for study design and type of data | 27 (87.1) | 1 | ||||
| 9. Complexity of analysis | 1.74 (0.44) | |||||
| Descriptive analysis only (means and variances) | 11 (35.5) | 1 | ||||
| Beyond descriptive analysis (any comparisons) | 20 (64.5) | 2 | ||||
| Outcomes | 10. Outcome | 3 | 1.27 (0.25) | 1.27 (0.25) | ||
| Satisfaction (“happiness”) ‡ Opinions and general facts ‡ | 3 (9.7) | 1 | ||||
| Attitudes/perceptions ‡ | 7 (22.6) | 1 | ||||
| Knowledge, skills (e.g. OSCEs, SPs as outcome measure) | 21 (67.7) | 1.5 | ||||
| Behaviors (e.g., physician actual practice) | 0 | 2 | ||||
| Patient/health care outcome | 0 | 3 | ||||
| Total score | 18 | 10.42 (2.59) | ||||
*
All comments in parentheses are clarifications. OSCE indicates objective structured clinical examination; SD, standard deviation; SP, standardized patient.
†
Each of 5, 6, and 7 can apply to different instruments.
‡
Items were separated out for data abstraction but analyzed as a single item.