Table 4.
Features of the e-learning system, and characteristics of assessment and evaluation methods of 52 included studies on LMICs e-learning program evaluation (Jan 2007–June 2017).
|
Features of the e-learning system | ||
|
Characteristics |
Number of Participants, n (%) |
Number of E-learning interventions, n (%) |
| Stage of implementationa | ||
| pilot | 4,690 (34%) | 38 (72%) |
| implemented | 9,611 (66%) | 16 (28%) |
| E-Learning access | ||
| online | 9,458 (77%) | 38 (73%) |
| offline | 300 (2%) | 4 (8%) |
| online and offline | 2,536 (20%) | 10 (20%) |
| Format of e-learning delivery | ||
| mobile-based | 334 (3%) | 5 (9%) |
| computer-based | 9,301 (76%) | 39 (75%) |
| mobile- and computer-based | 2,236 (18%) | 3 (6%) |
| unspecified |
423 (3%) |
5 (10%) |
|
Assessment and evaluation | ||
|
Characteristics |
Number of Participants, n (%) |
Number of E-learning interventions, n (%) |
| Assessment and evaluation methoda,b | ||
| questionnaire | 12,140 (44,6%) | 46 (47%) |
| knowledge testing | 8,732 (32%) | 35 (35%) |
| document review | 43 (0,2%) | 1 (1%) |
| interviews | 110 (0,4%) | 3 (3%) |
| focus groups | 228 (0,8%) | 6 (6%) |
| system log data | 6,001 (22%) | 8 (8%) |
| Assessment typea | ||
| Diagnostic | 2,187 (14%) | 30 (35%) |
| Formative | 2,569 (17%) | 8 (9%) |
| Summative | 10,339 (69%) | 48 (56%) |
| Number of assessment and evaluation methods employed | ||
| 1 method type | 1,133 (9%) | 19 (37%) |
| 2 method types | 4,495 (37%) | 12 (23%) |
| 3 method types | 5,933 (48%) | 16 (31%) |
| 4 method types | 438 (4%) | 4 (8%) |
| 5 method types | 295 (2%) | 1 (2%) |
| Quality scorec | ||
| MERSQI ≥ 10.625 | 2,025 (17%) | 26 (50%) |
| MERSQI < 10.625 | 10,269 (83%) | 26 (50%) |
| NOS-E ≥ 2.5 | 6,550 (53%) | 33 (64%) |
| NOS-E < 2.5 | 5,774 (47%) | 19 (36%) |
| NOS ≥ 5 | 5,408 (44%) | 32 (62%) |
| NOS < 5 |
6,886 (56%) |
20 (38%) |
|
Learning outcomes (assessment) | ||
| Outcome measurementa,d | ||
| 1. knowledge | 6,618 | 31 (16%) |
| 2. satisfaction | 6,613 | 20 (10%) |
| 3. usage | 8,680 | 17 (9%) |
| 4. perceptions | 7,681 | 14 (7%) |
| 5. attitude | 963 | 10 (5%) |
| 6. opinion | 2,127 | 10 (5%) |
| 7. skills | 335 | 9 (5%) |
| 8. usefulness | 2,348 | 8 (4%) |
| 9. demographics | 2,513 | 7 (4%) |
| 10. ease of use | 333 | 4 (2%) |
| 11. usability | 156 | 4 (2%) |
| 12. acceptance | 289 | 4 (2%) |
| 13. experience | 330 | 3 (2%) |
| 14. self-perceived confidence | 48 | 3 (2%) |
| 15. learning relevance | 243 | 3 (2%) |
| Response rate (to assessment) | ||
| 100% | 665 (5%) | 14 (27%) |
| 80–99% | 2,809 (23%) | 16 (31%) |
| 50–79% | 2,672 (22%) | 7 (13%) |
| below 50% | 3,806 (31%) | 5 (10%) |
| NAe | 2,342 (19%) | 10 (19%) |
| Success reported | ||
| Yes | 10,349 (84%) | 45 (87%) |
| No | 1,184 (10%) | 5 (10%) |
| NAe | 761 (6%) | 2 (4%) |
Characteristics described in more than one study are indicated. The number of participants and the number of e-learning interventions are depicted in absolute numbers and the percentage is based on the absolute numbers of participants and studies for each individual characteristic.
a
More than one option possible for each study.
b
Participant numbers separately counted for each option.
c
Scores between raters averaged.
d
Measurement outcomes only included if study n ≥ 3, for a detailed list see Appendix I.
e
N/A: number of participants not explicitly stated in study.