Table 3.

Quality Assessment of Study Methods^a

Study		Study designb	Sampling (number of programs)c	Survey response rated	Types of outcomes measurese	Validity of evaluation toolsf	Data analysis appropriate?g	Data analysis strategyh	Reported outcome(s)i	Total quality scorea
“Short” ambulatory blocks (1 week)
Chaudry et al., 201324		Pre-post	1	> 75%	Objective	Providedj	Appropriate	Beyond descriptive	Satisfaction, general facts	11.5
Harrison et al., 201425		Non-randomized trial	1	> 75%	Self-assessed	Providedj	Appropriate	Beyond descriptive	Satisfaction, general facts	10
Heist et al., 201426		Non-randomized trial	1	NA	Objective	NAk	Appropriate	Beyond descriptive	Behaviors	10.5
Mariotti et al., 201027		Pre-post	1	50–74%	Self-assessed	Providedj	Appropriate	Beyond descriptive	Satisfaction, general facts	9
Other ambulatory block designs
Bates et al., 2016 23		Pre-post	1	> 75%	Self-assessed	Providedj	Appropriate	Descriptive	Satisfaction, general facts	8.5
Francis et al.	June 201417	Non-randomized trial	> 3	> 75%	Self-assessed	Provided	Appropriate	Beyond descriptive	Satisfaction, general facts	13
	Sept 201418	Non-randomized trial	> 3	> 75%	Objective	Provided	Appropriate	Beyond descriptive	Patient health care outcomes	17
	201519	Non-randomized trial	> 3	> 75%	Objective	Provided	Appropriate	Beyond descriptive	Satisfaction, general facts	13.5
	201620	Non-randomized trial	> 3	NA	Objective	Provided	Appropriate	Beyond descriptive	Patient health care outcomes	17
Warm et al., 200828		Pre-post	1	NR	Objective	Provided	Appropriate	Beyond descriptive	Patient health care outcomes	14.5
Wieland et al., 201329		Pre-post	1	50–74%	Objective	Provided	Appropriate	Beyond descriptive	Patient health care outcomes	15

NA not applicable, NR not reported

^aAssessed using the Medical Education Research Study Quality Instrument (MERSQI). The total possible score is 18. Green denotes higher methodologic quality, yellow denotes moderate quality, and orange denotes fair quality

^bStudy design defined as post-test only (1 point), “pre-post” (one group compared before and after intervention; 1.5 points); “non-randomized trial” (separate concurrent control and intervention groups; 2 points), randomized control trial (3 points)

^cPotential number of institutions is 1 (0.5 points), 2 (1 points), or ≥ 3 (1.5 points). Studies involving a greater number of institutions are considered higher quality

^dSurvey response rate on post-test data; < 50% or not reported (0.5 points), 50–74% (1 point), ≥ 75% (1.5 points)

^ePotential types of outcomes measures are assessment by a study participant (1 point), or objective (3 points)

^fValidity of evaluation tools is defined as content validity or using existing tools (1 point), internal structure validity (1 point), and correlation with other variables (1 point)

^gAppropriateness of data analysis is defined as analysis that was appropriate for the study design (1 point) or not appropriate for study design (0 point)

^hData analysis strategy is defined as descriptive only (1 point) or beyond descriptive analysis (2 points)

ⁱReported outcomes are defined as satisfaction, attitudes, perceptions, opinions, general facts (1 point), knowledge or skills (1.5 points), behaviors (2 points), and patient health outcomes (3 points)

^jValidity was based on expert opinion only

^kNo survey instruments were used in this study