Table 4.

Modified QUESTS (Harden, Grant, Buckley, & Hart, 1999; Sullivan, 2011) Appraisal

QUESTS dimensions	Not stated=0	Poor=1	Good=2	High=3
Quality: how reliable is the evidence?	Not possible to identify	Evidence based on professional judgment, case studies, individual experience, or educational principles	Evidence based on consensus views built on experience (committees/task force), comparable studies in other areas, or well-designed non-experimental studies	Evidence based on well-designed quasi-experimental (pre/post, case/control) or controlled studies (treatment/control groups)
Utility: to what extent (time, cost, resources, flexibility, effort) can the method be transferred and adopted without modification?	Not possible to identify	Very difficult	Some difficulty	Easily
Extent: What is the magnitude of the evidence?	Not possible to identify	Single, isolated study	Interinstitutional, interprofessional study	Combination of several studies (meta-analysis, review)
Strength: How strong is the evidence?	Not possible to identify, no clear conclusions can be drawn	Most differences are of slight significance (p<.1), OR ambiguous, may represent a trend	Most differences are significant (p<.05) OR conclusions can be based on the results	Most differences are highly significant (p<.001) OR results are unequivocal
Target or outcomes measured: What educational outcomes are measured?	Not possible to identify. Reports only on participation in the intervention (number of participants)	Learners’ reactions to the intervention (self-reports of learning) – Kirkpatrick level 1	Learning gains (assessments of change in skills, knowledge, abilities, attitudes, behaviors) – Kirkpatrick levels 2 and 3	Impact of intervention on healthcare delivery (assessment of provider performance, patient satisfaction and outcomes) – Kirkpatrick level 4
Setting or context: How relevant or applicable is the evidence to healthcare-related practice	Not possible to identify	Not at all	A little	Very
Maximum total of points	0	6	12	18