Table 1.

Literature regarding the criteria for each CIPP model element

Item	Author	Category	Context		Input	Process	Product
Proposal by Stuffelbeam, founder of CIPP Model	Stufflebeam and Shinkfield [7] (1985)	Criteria	Define institution situation; learner identification and demand inspection; search for demand satisfaction opportunity; problem diagnosis and determination of objective appropriacy		System capability, solution program strategy, design procedure of strategy execution, budget, schedule check	Flaw check or prediction of procedures in progress or the execution process; information provision for preplanned sequential decisions; report and judgment of events and activities regarding the execution	Collection of technology/judgment regarding achievements; linkage with information on the objective, situation, input, process; value and advantage analysis
		Material collection method	Utilization of systems analysis, survey study, literature survey, public hearings, interviews, diagnostic assessment, Delphi technique		Available human and material resources, resolution strategy, design procedure, possibility and economic analysis, literature survey, pilot program survey, advocacy groups, pilot attempt	procedural disorder identification and accidental disorder awareness, detailed information acquisition for scheduled decision-making, describe the actual process, continuous interaction with the program operation staff, and observation or their activities	Operational definition and measurement of the performance standards and collection of the judgments by the interested parties, qualitative/quantitative analysis
		Purpose	Necessary for decision-making regarding education objective and purpose when education begins (provide standard of change plans, performance judgment): use in the decision-making for education planning		Necessary for support resource, resolution strategy, and design procedure selection (provide the basis for change activity composition, execution process judgment basis): use in the decision-making for education structuralization	Necessary for program planning, procedure, and improvement; necessary for actual situation basis provision in performance analysis: use in the decision-making for education execution	Operational definition and measurement of the performance standard; collection of judgments by interested parties regarding the performance; qualitative/quantitative analysis: use in the decision-making for recycling
Institution	Jung and Moon [26] (2013)	Criteria	Service demand and situation, service objective domain		Budget, human resource management, facility and resource environment, service operation and content	Service activity, service satisfaction, service evaluation	Service application, service performance
School	Shin et al. [19] (2018)	Criteria	Demand analysis, objective setting		Execution plan (human resource, procedure, support system, etc.), performance detail	Program activity, program management and evaluation	Program performance (achievement, satisfaction, effectiveness)
Nursing	Kim and Son [27] (2017)	Criteria	Intention and necessity		contents of hospital introduction, senior nurses’ working experience	Composition and facilitation	Usefulness of the program, feeling involvement through activities, global satisfaction
		Material collection method	Questionnaire		Questionnaire	Questionnaire	Questionnaire
Medical health professions	Ashghali-Farahani et al. [14] (2018)	Criteria	Inappropriate infrastructure; unknown duties		Biomedical approach; incomprehensive curriculum; lack of professional NICU nursing mentors; inappropriate admission process of NICU students; lack of NICU skill labs	More emphasize on theoretical education; the overlap of credits with each other and the inconsistency among the mentor; ineffective assessment	Preferring routine work instead of professional job; tendency to leave the job; clitical incompetency of graduates; dissatisfaction of graduates
		Material collection method	Semi-structured interview; open question
		Target of evaluation	NICU student, NICU graduate nurse, neonatologist, faculty member, nurse
	Neyazi et al. [10] (2016)	Criteria	Goals, organization and management area	Interest and understanding of students towards field and labor market; faculty members; research and educational spaces and equipment		Student research activity; educational courses and programs, teaching and learning process; student progress evaluation; evaluated factors for graduates	Efficiency of research and educational programs, teaching and learning process to increase knowledge and job performance of graduates
		Material collection method	Researcher-made questionnaires inspired from the CIPP model and internal evaluation literatures
		Target of evaluation	Students, graduates
	Al-Khathami [11] (2012)	Criteria	Achievement of program goals; barriers to achieve goals, objectives, and needs	Alternative procedural design for: contents, academic sessions, hospital sessions, half day release sessions		Process involved in to learning activities; trainers; theoretical sessions; clinical sessions	Overall impression about the program; barriers to achieve goals, objectives, and needs; assessment tools; enjoyment; satisfaction
		Material collection method	Questionnaire (quantitative, qualitative)
		Target of evaluation	Trainee
	Yarmohammadian and Mohebbi [15] (2015)	Criteria	Human specialists and scientific services for needs of the local community	Head of department, faculty,		Activities of group manager, students, administrators	-
				students, curriculum, funding, training facilities		of library; scientific research and teaching–learning activities of faculty
		Material collection method	Questionnaire
		Target of evaluation	Directorates, faculty members, students, and library staff
	Neyazi et al. [12] (2016)	Criteria	Goals, management, and organization area	Facility and spaces		Educational courses and programs, learning and teaching process; administration and financial; program evaluation	Graduates
		Material collection method	Questionnaire
		Target of evaluation	Department head, faculty members, and library staff
	Rooholamini et al. [9] (2017)	Criteria	Perceptions of learning; perceptions of teachers; academic self-perceptions; perceptions of the environment; social self-perceptions	Content of curriculum		The process of learning; process of teaching	Students’ performance; the process of teaching and learning
		Material collection method	Review of current evidence on integration; consultation with experts; Modified Dundee Ready Education Environment Measure (DREEM)	A researcher made questionnaire		1: researcher–made questionnaires for evaluating the quality of each integrated course; 2: researcher–made questionnaires for evaluating the quality of early clinical exposure	1: learner centered integrated basic science, portfolios; 2: brainstorming (students); 3: semi-structured interview (professors of basic sciences)
		Target of evaluation	Students, faculty and administrators	Faculties and curriculum committee		First and second year medical students	First and second year medical students; professors of basic sciences
	Lee et al. [8] (2019)	Criteria	Goals, necessity or needs	Available input resources (human and material resources); educational strategy		Implementation according to plan; evaluation of the program by students	Goal achievement; satisfaction of the curriculum
		Material collection method	Questionnaire, FGI, meeting minutes, syllabus, curriculum	Questionnaire, FGI, meeting minutes, time table		Questionnaire, FGI, meeting minutes, syllabus	Questionnaire, FGI, meeting minutes, grades
		Target of evaluation	Students, faculty

CIPP, Context, Input, Process, and Product; NICU, newborn intensive care unit; FGI, focus group interview.