Table 7.

Summary of the 5 papers that assessed the effectiveness of the education program.

Programs and authors		Measure	Actual outcomes
Educational summit
	Barbour et al 2019, [62]	Conducted a 5-question before-and-after poll of those attending our educational summit	Level 2a: Baseline beliefs about how AIa will affect the future of health care careers and patient care were similarly positive before and after the event Level 2a: At arrival, 70% of the attendees felt that AI would make health care less humanistic; 50% left the summit feeling neutral Level 2a: We did not observe a meaningful shift in attitudes regarding the desire to take a leadership role in developing or implementing AI Level 2b: At arrival, 40% of the attendees believed that they had a poor baseline understanding of AI’s role in health care; 90% left the summit with an enhanced understanding of the topic
Workshops
	Kang et al 2017, [50]	A survey was designed to capture residents’ opinions after their minicourse, covering 5 major areas of interest: How helpful the minicourse was as an introduction to CERb and big data research (on a 5-point scale, with 5 indicating very helpful) Whether the residents would likely pursue further educational or research opportunities in CER Whether the residents had prior educational or research exposure to CER Whether a mentor was available for CER at their home institutions The importance of CER and big data research to the field of radiology (on a 5-point scale, with 5 indicating very important)	Level 1: 90% of the residents reported that the course was helpful or very helpful Level 1: 94% of the participants felt that the lectures were of high or very high quality Level 2a: 82% reported that they planned to pursue additional educational or research training in CER or big data analytics after the course Level 2a: 98% of the respondents felt that health services and big data research are important or very important for the future of radiology
	Kinnear et al 2019, [49]	Evaluations were conducted on a 5-point Likert scale	Level 1: The average weighted rating on a 5-point Likert scale over the 3 years for the prompt “Overall satisfaction with the session” was 4.32 out of 5 Level 2a: The participants reported an increase in confidence to use this knowledge to teach residents in the coming academic year
Biomedical informatics course within medical education
	Sanchez-Mendiola et al 2013, [39]	Administered a program-evaluation anonymous survey to the students at the end of the course, a 41-item questionnaire that explored several aspects of the program	Level 1: Overall opinion of the students regarding the different elements of the program was good to excellent for educational activities, course resources, and perception of clinical relevance
	Sybenga et al 2016, [35]	Competency of senior residents on the basis of their project results was evaluated by staff during a multidisciplinary conference	Level 2b: After introductory education in big data analysis concepts, the residents were able to rapidly analyze large sets of data to answer simple questions Level 2b: The senior residents were able to engage in complex problem solving requiring management and application of multiple seemingly unrelated resources and successfully present these results

^aAI: artificial intelligence.

^bCER: comparative effectiveness research.