Table 4.
Trainee & course evaluation
| First author Year Published |
Trainee feedback/survey Course evaluation, Scanning confidence, Training needs assessment, Patient feedback |
Pre/post course theoretical knowledge assessment | Practical/OSCE assessment | Expert image review | Frequency and/or application of PoCUS | Patient outcomes & management |
|---|---|---|---|---|---|---|
| Adler [66] 2008 |
X | X | Minimum 20 supervised US examinations. No structured OSCE or knowledge exam. | X | Frequency & application of PoCUS following training. | X |
| Baltarowich [61] 2009 |
X | Identical test at program entry, course completion & 6 months follow-up. | X | X | X | X |
| Bell [22] 2016 |
X | Pre course test- open book 90% pass mark for enrolment. | OSCE- assessing image interpretation & quality. | X | Follow-up survey of US use. | PoCUS impact on patient management. |
| Dalmacion [25] 2018 |
X | Pre & post Knowledge test. | X | Expert review of trainee images. | X | Estimate of maternal/fetal deaths averted following PoCUS. |
| Dornhofer [54] 2020 |
Post course survey to assess scanning confidence & provide course feedback. | Identical pre & post course knowledge test. | Post course practical test on image acquisition & interpretation. | X | X | X |
| Henwood [53] 2017 |
Pre-training needs assessment. | X | Image based assessment & post course OSCE. Regular practical assessment over 6 month follow-up. | Expert review of trainee images. | Frequency & application of PoCUS following training. | PoCUS impact on clinical decision making/patient management. |
| Kimberly [62] 2010 |
X | X | Practical assessment (14 item OSCE) of skills & sustainability over time (2 & 6 months). | Expert review of trainee images. | Frequency & application of PoCUS following training & teaching colleagues. | PoCUS impact on clinical decision making/patient management. |
| Kolbe [26] 2015 |
Pre-training needs assessment. | X | X | Expert review of trainee images. | Application of post training PoCUS. | Change in patient diagnosis & management following PoCUS. |
| Kotagal [56] 2015 |
Pre & post survey to measure trainee confidence. | X | X | X | X | X |
| Lathrop [58] 2011 |
X | X | X | Learner portfolio & images reviewed using rubric to evaluate progress, knowledge & skills prior to credentialing. | X | X |
| Lee [55] 2017 |
Post course evaluation survey. | Identical pre & post knowledge test. Follow-up testing scheduled in 12 months. |
OSCE practical assessment. | X | X | X |
| Lindgaard [52] 2017 |
X | X | Short practical assessment. | 25 specific US exams (video sequences & screen shots) uploaded for instructor review. | X | X |
| Mandavia [67] 2000 |
X | Identical knowledge assessment pre & post course & at 10 month follow-up (24 positive, negative, & nondiagnostic US images for interpretation). | X | Expert review of trainee images- sensitivity & specificity. | Frequency & application of PoCUS. | X |
| Nathan [49] 2017 |
X | Written exam at end of 2 week course for pilot eligibility. Monthly practical assessment for 3 months. |
Practical exam at end of 2 week course for pilot eligibility. | Expert review of trainee images for errors in scanning parameters & diagnosis, using predetermined criteria. | X | Patient outcome- High risk pregnancies. |
| Rominger[24] 2018 |
Pre-training needs assessment. | X | X | Case logs & images (35%) reviewed for quality assurance & feedback. | Frequency & application of PoCUS following training. Scanning frequency over time. | PoCUS impact on patient management & diagnosis. |
| Shah [45] 2020 |
Pre & post US confidence survey & at 3 month follow-up. Key informant interviews assessed trainees’ perception of training program. |
X | 25 proctored scans prior to final OSCE (80% pass mark). | 2 months of blinded expert image review & inter-rater reliability of trainee scans. | X | X |
| Shah [48] 2014 |
Post course confidence survey | Pre and post course knowledge assessment. | Time & accuracy of scanning recorded during practical training. | X | X | X |
| Shah [47] 2009 |
Pre-training needs assessment. | X | X | Expert review of trainee images for quality & accuracy of interpretation. | X | PoCUS directed change of patient management. |
| Shah [46] 2008 |
Pre-training needs assessment. Focus group discussion of barriers to US utilisation. |
X | X | X | X | X |
| Shaw-Battista [29] 2015 |
Post course evaluation immediately following practical training. | Pre & post online module knowledge test. Passing grade required before seminar & practical training. Post-practical training knowledge test (results not reported). | X | X | X | X |
| Shokoohi [51] 2019 |
Post training survey to evaluate course, trainee demographics, medical/PoCUS experience, US use, challenges/barriers & opinions. | X | X | X | Frequency & application of US (including teaching others) | X |
| Stolz [30] 2015 |
X | Invigilated examinations conducted throughout 2 year training period. | Invigilated examinations conducted throughout 2 year training period. | X | Frequency & application of PoCUS over duration of training course. | Patient outcome following PoCUS. |
| Swanson [85] 2014 |
X | Oral competency tests on return to clinic following training. | Practical competency tests on return to clinic following training. | Expert review of trainee images- Sensitivity & specificity for clinical indications. | X | Patient outcome/ Altered diagnosis following trainee PoCUS. |
| Vinayak [63] 2017 |
Patient survey of experience following trainee provided PoCUS. | Pre-test following e-module. Pass mark 100% within 5 attempts to progress to practical training. Post course written assessment. | Practical assessment throughout the course. Post course practical assessment. |
Expert review of trainee images for quality & accuracy of interpretation. | Time to complete PoCUS & improvement with time. | Patient outcomes (high risk pregnancies) following Trainee PoCUS. |
| Vyas [70] 2018 |
X | X | Post-training OSCE practical assessment. | Blinded expert review of trainee images over following 12 months for image quality ability to identify OB pathology. | X | Patient outcomes. |
| Wanjiku [21] 2018 |
3 month post course evaluation. | Online test following self-directed online course- 90% pass mark for eligibility for 1 day practical training. Post course practical assessment. |
Post course OSCE/practical assessment with image quality scores. Follow-up in-facility testing scheduled 3–4 months after initial training. | X | Frequency & application of PoCUS following training. | X |
| Westerway [50] 2019 |
Course evaluation- satisfaction, engagement, understanding & relevance of learning. |
Identical pre & post course knowledge test repeated at 6 & 11 months follow-up. |
Post training practical assessment, repeated at 6 & 11 month follow-up. |
X |
Scanning on returning to clinic/work (confidence and application). |
X |
|
Abbreviations *KEF – Kirkpatrick Evaluation Framework OSCE- Objective Structured Clinical Examination PoCUS- Point-of-Care UltrasoundUS- Ultrasound | ||||||