Table 1. Effect of Intervention on knowledge level.
| Study number | Author, year & country | Outcome of interest | Summary statistics (Percentage of mark/ Score by using mobile application (Mean±SD, P Value)) | Sample size (I: Intervention; C: Control) | Duration | Effectiveness of mobile application |
|---|---|---|---|---|---|---|
| 1 | Bonabi et al, 2019, Iran [14] | Change in knowledge level | Mean knowledge score: Cont: Pre test: 8.17 ± 2.03; Post test: 10.43 ± 1.8 (P<0.001); Int: Pre test: 7.51 ± 1.7; Post test: 10.7 ± 2.1 (P<0.001) | 107 (I: 57; C:50); Final evaluation: 86 (I: 43; C:43) | 4 months | Effective |
| 2 | Velasco et al, 2015, Brazil [15] | Change in knowledge level |
Mean score in the intervention: Pre test: 4.8±3; Post test: 7.5 ± 2 (P = 0.000) Mean score in the control:Pre test: 5.9 ± 3; Post test: 7.5±3 (P = 0.005) |
66 (I: 33; C: 33) | 2 weeks | Effective |
| 3 | Clavier et al, 2019, France [16] | Change in knowledge level |
Post test scores: SCT: Int: 60±9%; Cont: 68±11%; (P = 0.006) MCQs: Int: 18±4; Cont: 16±4; (P = 0.22) |
62 (I: 32; C: 30); Final evaluation: 44 (I:22; C:22) | 3 weeks | Not effective |
| 4 | Noll et al, 2017, Germany [17] | Change in knowledge level |
Average improvement in score (Immediately after learning): Group A: 3.59±1.48; Group B: 3.86±1.51; (P = 0.1) After 14 days follow-up: Average decrease of the number of correct answers as follows; Group A: 0.33±1.62; Group B: 1.14±1.30 |
44 (Group A:22; Group B: 22) | 45 min | Effective |
| 5 | Samra et al, 2016, USA [18] | Change in knowledge level |
Range of score in intervention (Out of 16): Pre test: 0–1; Post test: 0–12 (P = 0.01) Range of score in control (Out of 16): Pre test: 0–2; Post test: 0–4 (P = 0.08); Average improvement in score: Int: 5.4 points (range, 0–12 points); Cont: 0.5 points (range, –1 to +1 points) (P = .0286) |
29 (I: 15; C: 14); Final evaluation 21 (I: 7, C:14) | 8 weeks | Effective |
| 6 | Albrecht V et al, 2013, Germany [19] | Change in knowledge level | Difference in Pre-post score: Int: 4.7±2.9; Cont: 3±1.5 (P = .03). | 10 (I: 6; C: 4) | 105 minutes | Effective |
| 7 | Stirling et al, 2014, Australia [20] | Change in knowledge level |
Pre test score: Int: 9.289 ± 2.265; Cont: 9.727 ±2.565; Post test score: Int: 10.737±1.996; Cont: 10.424 ±2.437; Difference in score: Int: 1.447, 0.384 (P = 0.001); Cont: 2.368, 0.436 (P = 0.12) |
71 (C: 33; I: 38) | One practical session | Effective |
| 8 | Amer et al, 2017, USA [21] | Change in knowledge level | The mean grade on the standardized test: Int: 89.3±6.0%; Cont: 75.6±8.7%; (P < .05) | 100 (C: 50; I: 50) | 3 times visualization | Effective |
| 9 | Kucuk et al, 2016, Turkey [22] | Change in knowledge level | Academic achievement score: Int: 78.14±16.19; cont: 68.34±12.83 (P<0.05) | 70 (I: 34; C: 36) | 5 hours | Effective |
| 10 | Brown et al, 2018, USA [23] | Change in knowledge level | The increment in mean score: Int: 34% to 81%; Cont: 33% to 63%; (P = 0.81) | 67 | 4 days | Not effective |
| 11 | Lacy et al, 2018, USA [24] | Change in knowledge level |
Mean Pre test score: Int: 75%; Cont: 74.7%; Mean post test score: Int: 86.3%; Cont: 77.5% |
36 | 1 hour | Effective |
| 12 | Fernandes Pereira et al, 2016, Brazil [25] | Change in knowledge level |
Mean score (Out of 10): Int: 8.14±1.67; Cont: 5.02±3.21 Error Average: Int: 1.83±0.5; Cont: 4.98±1.0 Average execution time (min): Int: 15.7±21; Cont: 38.9±4.3 |
100 (C: 50; I: 50) | 4 months | Effective |
| 13 | Putri et al, 2019, Indonesia [26] | Change in knowledge level | Mean difference in BLS knowledge: Int: 33.75±12.09; Cont: 25.41±10.93; P = 0.016 | 48 (I: 24; C: 24) | NA | Effective |
| 14 | Martínez et al, 2017, Chile* [9] | Change in knowledge level |
Increase in score: App group: 16.2 ± 8.3 (P < 0.001); Control: 10.6 ± 11.7 (P < 0.001) Difference in score between the groups: 3.5 (P = 0.22). |
80 (I: 40; C: 40) | 4 weeks | Effective |
| 15 | Naveed et al, 2018, England [52] | Change in knowledge level | Average MCQ Score (Percentage): Int: 62.95±5.37; Cont: 56.73±5.18; P = 0.0285 | 20 (C: 10, I: 10); Final evaluation: 15 (C: 7, I: 8) | Int: 1 hr; Cont: 2 hrs | Effective |
| 16 | Kim et al, 2018, South Korea [53] | Change in knowledge level |
Improvement in mean knowledge (Out of 23): Int: 21.24±1.74 to 22.18±0.76; Cont: 20.84±1.35 to 21.25±1.41 Mean Difference in Pre-post Knowledge: Int: 0.94±1.74; Cont: 0.41±1.04; P = 0.133 |
72 (C: 36; I: 36) Final evaluation: 66 (C: 32, I: 34) | 1 week | Not effective |
| 17 | Kang et al, 2020, South Korea [61] | Change in knowledge level | Mean Difference in Pre-post Knowledge: Cont: 0.02 ± 0.13; Exp 1: 0.15 ± 0.14; Exp 2: 0.11 ± 0.15; P = 0.004 | 86 (Exp 1: 26; Exp 2: 32; Cont: 28) | 2 weeks | Effective |
| 18 | Bayram et al, 2019, Turkey [54] | Change in knowledge level |
Median First Knowledge test scores: Int: 18 (9–22); Cont: 17 (12−23); P = 0.441 Median Last Knowledge test scores:Int:19 (13−23); Cont: 19 (8–23); P = 0.568 |
118 (C: 59; I:59) | 1 week | Not effective |
| 19 | Fernández-Lao et al, 2016, Spain [55] | Change in knowledge level | Knowledge test (out of 10 points): Int:7.21 ± 1.988; Cont: 8.09 ± .921; P = 0.089 | 49 (I: 25; C: 24) | 2 weeks | Not effective |
| 20 | Lozano-Lozano et al, 2020, Spain [56] | Change in knowledge level | Pass percentage (MCQ): Int: 86% (43/50); Cont: 27% (15/55); P<0.001 | 110 (C: 55, I: 55) Final Evaluation: 105 (C: 55, I: 50) | 2 weeks | Effective |
| 21 | Bunogerane et al, 2017, Rwanda [57] | Change in knowledge level |
Percentage change in knowledge: Tendon repair theory: Cont:13.0% (P = 0.535); Int: 39.1% (P = 0.056) Tendon repair technique: Cont:19.0% (P = 0.165); Int: 38.1% (P = 0.0254) |
27 (C: 13; I: 14) | Till post-test completion | Effective |
| 22 | Wang et al, 2017, China [27] | Change in knowledge level |
Base line: Int: 19.43±2.48; Cont: 19.04± 2.66; Post test (2nd week): Int:24.18 ± 3.51; Cont: 20.02 ±2.53; Post test (3rd month): Int: 23.61 ± 3.37; Cont:19.54 ± 2.67; |
115 (I: 61; C: 54) | 3 months | Effective |
| 23 | Ziabari et al, 2019, Iran [28] | Change in knowledge level |
Increment inmean awareness score: Int: 11.44 ± 2.37 to 14.88 ± 1.97, P < 0.0001; Cont: 11.38 ± 3.22 to 12.54 ± 3.04; P<0.0001; Mean difference in BLS awareness score: Int: 3.44±1.48; Cont:1.16±1.51; P<0.0001 |
100 (I: 50; C: 50) | 3 months | Effective |
| 24 | Briz-Ponce et al, 2016, Spain [29] | Change in knowledge level |
Pre–post test scores in intervention: Pre test: 2.2000±1.3732; Post test: 3.6000±1.12122 (P = 0.031) Pre–post test scores in control: Pre test: 2.6667±1.54303; Post test: 2.4000±1.40408 (P = 0.157) |
30 (I: 15; C: 15) | 3 Sessions | Effective |
| 25 | Golshah et al, 2020, Iran [30] | Change in knowledge level |
Mean grade point: Post test: Int: 15.57 ± 0.91; Cont: 15.39 ± 1.09; P = 0.503 |
53 (I: 27; C: 26) | 2 weeks | Not effective |
| 26 | Salameh et al, 2020, Palestine [31] | Change in knowledge level | Mean difference in pre-post test scores: Int: 3.86±1.65; Cont: 1.5±2.21; P<0.000 | 104 (I: 52; C: 52) | NA | Effective |
| 27 | Kang et al, 2018, South Korea# [58] | Change in knowledge level |
Mean HTN knowledge: Pre test:Int: 89.4±5.9; Cont: 87.4±7.5; Post test:Int: 93.7±4.4; Cont: 87.4±4.7; P = 0.001 Mean DM knowledge: Pre test:Int: 86.5±8.3; Cont: 84.9±6.8; Post test:Int: 91.0±5.7; Cont: 86.0±8.7; P = 0.009 |
92 [I: 49 (HTN: 21; DM: 28); C: 43 (HTN: 20; DM: 23)] | 1 week | Effective |
| 28 | Young Yoo et al, 2015, South Korea [59] | Change in knowledge level |
Mean Lung test: Pre test:Int: 6.7±1.0; Cont: 6.6±2.1 (P = 0.666);Post test:Int:7.4±0.8; Cont: 5.8±1.6 (P = 0.031) Mean Heart test: Pre test: Int:6.4 ± 1.2; Cont:5.7 ± 1.2 (P = 0.161);Post test: Int: 8.3±1.2; Cont: 8.7 ± 0.9 (P = 0.489) |
22 (11 each cross over) | 4 weeks | Effective in one condition (Out of 2) |
| 29 | Kim et al, 2017, South Korea [60] | Change in knowledge level |
Mean Knowledge: Pre test: Int: 8.69±1.62; Cont: 9.10±1.57 (P = 0.266); Post test: Int: 11.80±1.32; Cont: 11.84±1.48 (P = 0.899) Mean Difference in knowledge: Int: 3.11±1.78; Cont: 2.74±1.86 (P = 0.379) |
80 (I: 40; C: 40) Final evaluation: 73 (I: 35; C:38) | 1 month | Effective |
| 30 | Chung et al, 2018, USA [32] | Change in knowledge level |
Difference in mean pre-post score: Int: 2.4% [–3.1 to 8]; Cont: 4.8% [0.3–9.4]; P>0.05 Post score: Int: 73.8% [69.2–78.4]; Cont: 74.1% [70.3–78.0]; P>0.05 |
37 (I: 18; C: 19) | 4 weeks | Not effective |
| 31 | Shore et al, 2018, USA [62] | Change in knowledge level | Median score of Pre test: 87 (IQR, 81 to 94); Median score of Post test: 100 (IQR, 94 to 100). | 53 (single group) | 3 weeks | Effective |
| 32 | Deshpande et al, 2017, India [33] | Change in knowledge level |
Mean score in SCT: Pre test: 41.5±1.7; Post test: 63±2.4 (P < 0.005). |
92 (single group) | 1 year | Effective |
| 33 | Man et al, 2014, USA [34] | Change in knowledge level |
Mean total score (%): Percentage of correct answers increased after 12 weeks when compared to baseline (P = 0.205). Mean confidence level: Managing outpatient adults with major depression: Baseline: 4.214; After 12 Weeks: 5.364 (P = 0.048); Starting an antidepressant for newly diagnosed major depression: Baseline: 4.286; After 12 Weeks: 5.636 (P = 0.018); Choosing an antidepressant based on patient factors: Baseline: 3.642; After 12 Weeks: 5.273 (P = 0.010) |
N = 14 (Single group) | 12 weeks | Effective |
| 34 | Liu et al, 2018, Taiwan [35] | Change in knowledge level |
Average pre-course score: Overall: 27.50±15.83; Non-dermatology trainees: 15.38 ±8.03; Dermatology trainees: 39.62 ± 11.81. Average post course score: Overall: 91.44 ± 5.92; Non-dermatology trainees: 90.77 ± 5.98; Dermatology trainees: 92.12 ± 6.02 |
26 (Group 1:13; Group 2: 13) | 3 weeks | Effective |
| 35 | Fralick et al, 2017, Canada [36] | Change in knowledge level |
Improvement in knowledge score: Int: 6.2±2.1 vs 8.1±2.2 (P = 0.0001); Cont: 7.1±1.7 vs 7.5±2.0 (P = 0.23) Unadjusted linear regression analysis: P = 0.006 [95% CI: 0.46, 2.48]) Adjusted multivariable linear regression analysis: P = 0.04 [95% CI: 0.10, 2.1]) |
62 (I: 32; C: 30); Follow up: 53 (I: 27; C: 26) | 4 weeks | Effective |
| 36 | Weldon et al, 2019, UK [37] | Change in knowledge level |
Percentage of correct answers: Pre test: Q1: 40%; Q2: 100%; Q3: 60%; Q4: 20%; Q5: 20%; Q6: 20% Post test: Q1: 80%; Q2: 100%; Q3: 80%; Q4: 60%; Q5: 20%; Q6: 80% |
5 (Single group) | Till completion of post test | Effective |
| 37 | Smeds et al, 2016, USA [38] | Change in knowledge level | NBME score: Int: 77.5%; Cont: 68.8% (P< 0.01); USMLE scores: Int: 225.4; Cont: 209.8; (P < 0.001); Cumulative GPA: Int: 3.3; Cont: 2.9; (P < 0.001); Mean MCAT scores: Int: 9.6; Cont: 8.9; (P < 0.01). | 288 (I: 152; C: 136) | 1 year | Effective |
| 38 | Ebner et al, 2019, Germany [63] | Change in knowledge level | Mean Score of MCQs: Int: 30.2; Cont: 36.8 (P = 0.13) | 66 (I: 33; C: 33) | 1 week | Not effective |
| 39 | Hirunyanitiwattana et al, 2020, Thailand [39] | Change in knowledge level |
General asthma knowledge scores: Asthma knowledge in both groups improved significantly between pre & post test (WAAP: P = 0.135; ACA: P = 0.002) Asthma action plan knowledge scores: No statistical difference in score between, or within each group |
44 (I: 25; C: 19) | 3 hours | Effective |
| 40 | Baccin et al, 2020, Brazil [40] | Change in knowledge level | Mean grade: Pre test: 4.77±1.63; Post test: 8.49±1.27 (P<0.0001). | 161 Final evaluation: 150 | 7 weeks | Effective |
| 41 | Ameri et al, 2020, Iran [41] | Change in knowledge level | Difference in mean pre-post test score: Cont: 0.06 (7.69 to 7.75; P = 0.84); Group 1: 1.88 (7.71 to 9.59; P<0.0001); Group 2: 7.6 (7.5 to 15.1; P<0.0001) | 316 (C: 106 Group 1: 105 Group 2: 105) | 2 months | Effective |
| 42 | Hisam et al, 2019, Pakistan [8] | Change in knowledge level |
Score in the last professional examination: Int: 69±7%; Cont: 67±9%. Average usage of application VS academic performance (P<0.01) |
448 (I: 323; C: 125) | NA | Effective |
ACA: Asthma Care Application; AR: Augmented Reality; BLS: Basic Life Support; BSE: Breast Self Examination; CI: Confidence Interval; DM: Diabetes Mellitus; GPA: Grade Point Average; Hr: Hour; HTN: Hypertension; IQR: Inter Quartile Range; MCAT: Medical College Admissions Test; MCQ: Multiple Choice Questionnaire; MD: Mean Deviation; NA: Not Available; NBME: National Board of Medical Examiners; RCT: Randomized Controlled Trial; SCT: Script Concordance Test; SD: Standard Deviation; SEM: Standard Error of the Mean; UK: United Kingdom; USA: United States of America; USMLE: United States Medical Licensing Examination; WAAP: Written Asthma Action Plan
* Indicates that the study reported final score of EUNACOM (theoretical practical exam of general medicine) as combined knowledge and skill score. So the same result is repeated in skill domain also.
# indicates that the study considered as 2 studies in meta analysis, as it is evaluating 2 apps