TABLE 3.

The New World Kirkpatrick Model to Evaluate the Reviewer Academy

NWKM Evaluation Level	Criteria Related to Participants/Program	Criteria Related to the Journals
Level 1: Reaction
The degree to which participants find the training favorable, engaging, and relevant to their jobs	Postcourse evaluation of the content and the instructors.	N/A
Level 2: Learning
The degree to which participants acquire the intended knowledge, skills, attitudes, confidence, and commitment based on their participation in the training	Proportion of participants who completed all requirements of the program	N/A
	Successful completion of assessments for each module
	Successful completion of assignments for manuscript reviews
	Time required for completion of modules
Level 3: Behavior
The degree to which participants apply what they learned during training when they are back on the job	Online access to the Reviewer Academy toolkit	Quality of completed reviews, using journal-specific quality assessment measures
	Number of completed reviews
	Quality of completed reviews (using pre-defined criteria)
Level 4: Results
The degree to which targeted outcomes occurs as a result of the training and the support and accountability package	Reviewer for an SCCM journal	Number of reviewers recruited
	Appointed to the editorial board of an SCCM journal	Number of reviews completed by the trained reviewers
		Quality of the submitted reviews

N/A = not applicable, SCCM = Society of Critical Care Medicine.