Table 3.
Analysis results of subgroups
| Subgroup and domain | n | Effects | Heterogeneity | ||
|---|---|---|---|---|---|
| SMD (95% CI) | p-value | I2,% | p-value | ||
| Outcome measures | |||||
| Exam scores | 24 | 1.82 (1.71,1.93) | < 0.001 | 99.00% | < 0.001 |
| Course grade | 3 | 0.25 (0.13,0.38) | < 0.001 | 96.70% | < 0.001 |
| OCSE | 2 | 0.98 (0.77,1.20) | < 0.001 | 97.90% | < 0.001 |
| Research design | |||||
| RCT | 1 | 1.41 (1.03,1.79) | |||
| Historical control | 3 | 1.58 (1.49,1.68) | < 0.001 | 99.00% | < 0.001 |
| Quasi-experiment | 24 | 0.31 (0.04,0.58) | 0.023 | 89.60% | < 0.001 |
| Degree programs | |||||
| Doctor of Pharmacy | 9 | 0.24 (0.15,0.32) | < 0.001 | 97.60% | < 0.001 |
| Graduate | 1 | -0.31 (-0.65,0.04) | |||
| Undergraduate | 2 | 1.2 (0.9,1.51) | < 0.001 | 75.30% | 0.044 |
| Associate | 1 | 0.71 (0.29,1.13) | |||
| Country | |||||
| US | 22 | 1.63 (1.53,1.72) | < 0.001 | 99.10% | < 0.001 |
| Non-US | 6 | 0.74 (0.58,0.91) | < 0.001 | 90.40% | < 0.001 |
| Format of test | |||||
| MCQ | 16 | 2.57 (2.41,2.73) | < 0.001 | 99.40% | < 0.001 |
| Calculation questions | 1 | 0.17 (-0.11,0.44) | |||
| Short answer and fill-in-blanks | 1 | 0.29 (0.06,0.51) | |||
| Blend | 6 | 0.62 (0.49,0.75) | < 0.001 | 88.50% | < 0.001 |
| Course type | |||||
| Pharmacy course | 26 | 1.58 (1.48,1.67) | < 0.001 | 99.00% | < 0.001 |
| Interdisciplinary course | 2 | 0.57 (0.31,0.82) | < 0.001 | 97.70% | < 0.001 |
| Flipped classroom design | |||||
| Short recorded video pre-class (average duration < 18 min) | 6 | 0.63 (0.37,0.88) | 0.004 | 70.8 | < 0.001 |
| Quiz at start of in-class? | 9 | 3.91 (3.66,4.16) | < 0.001 | 99.70% | < 0.001 |
| Incorporating patient case discussion in-class? | 18 | 2.14 (2.02,2.27) | < 0.001 | 99.30 | < 0.001 |
SMD, Standardized Mean Difference; CI, Confidence Interval; OCSE, Objective Structured Clinical Examination; RCT, Randomized Controlled Trial; MCQ, Multiple Choice Questions