Evaluation of Problem Based Learning Course at College of Medicine, Qassim University, Saudi Arabia

Abstract

Background:

The PBL approach to learning in medical education can be considered as the most significant educational innovation in the past four decades. PBL is by now a well established method of learning and instruction. Evaluating the success of PBL as compared to more traditional Lecture Based Learning requires more complex techniques.

Objective:

The objective of this study is to examine the conceptualization, design, implementation and usefulness of the PBL programme, and to determine its effectiveness.

Methods:

This study was conducted at a premier problem-based leaning medical school of Saudi Arabia. The Course Experience Questionnaire [CEQ], designed to measure the quality of learning experience, and the Student Course Experience Questionnaire (SCEQ) have been used in this study. The survey included the four aspects of learning environments known to relate to the quality of student learning.

Results:

The study reveals that the PBL system helps developing student skills particularly problem solving skills and help sharpening analytic skills However, majority of the students are not satisfied with the evaluation system in the college as they think that it is not student-centered as it does not reflect the improvement made by the student with the passage of time. Students are satisfied with many objectives of the Problem Based Learning. The majority of respondents agreed that PBL is better than the traditional system and consider it superior to the traditional Lecture-Based System in Medical Education.

Conclusion:

The growing popularity and increased application of PBL presents significant challenges. Results indicate that after 18 months students become bored with the repetitious routine of working through problems and the problems and the approach used in the tutorial groups cease to be challenging and motivating. The authors conclude that PBL is having an impact on the performance of students, their perspective on learning and teaching methods. Our strategy for evaluating the success of PBL is ongoing and the results represent only an initial stage in analysis as we are still in the process of standardizing the process of collecting the outcome data particularly from those who are pass outs from the college.

Introduction

The PBL approach to learning in medical education can be considered as the most significant educational innovation in the past 35 years. PBL is a powerful class room process, which uses real-world problems to motivate students to identify and apply research concepts and information, work collaboratively and communicate effectively. It is a strategy that promotes life-long habits of learning. Active learning is the most effective technique for students to learn, apply, integrate, and retain information. PBL is active and applied rather than passive and absorbed.

PBL is by now a well established method of learning and instruction. It is a teaching technique used in many medical schools to facilitate learning basic science concepts in the context of clinical cases. Students are assigned to groups of 8–10, and each group is assigned a faculty member who plays the role of a tutor or facilitator as the students work through a case or a problem. This model is very student-centred.

In the PBL approach, complex, real-world problems are used to motivate students to identify and research the concepts and principles they need to know to work through those problems. Students work in small learning teams, bringing together collective skills at acquiring, communicating, and integrated information.

In PBL curriculum the problem scenarios serve as central component, a set of problem situations that equip students to become independent inquirers, who see learning and epistemology as flexible entities and perceive that there are also other valid ways of seeing things besides their own perspective.

PBL instruction addresses several desirable outcomes of an undergraduate education, particularly critical thinking, research skills, communication skills, and other lifelong learning skills. PBL strategy is remarkably adept and adaptable vehicle to develop in students, core knowledge in a content area, cognitive skills (analysis, synthesis, application, evaluation, and critique) and action skills (organizing time, resources, coordination, negotiating, tolerating)

In PBL, students first encounter a problem, followed by a student centred inquiry process ^(1–4). PBL emphasizes active student-centred learning in which students are challenged to examine, inquire, reflect, make meaning, and understand the sciences basic to medicine as they develop approaches towards the solutions of defined problems in a context relevant to their future professional careers ⁽¹⁾ .

Both content and the process of learning are emphasized in PBL. Key elements of the PBL include the formulation of questions that can be explored and answered through systematic, self-directed inquiry and the testing and revision of hypothesis through the application of newly acquired knowledge. Active discussion and analysis of problems, hypothesis, mechanisms, and learning issues among students are essential to this process, enabling students to acquire and apply content knowledge and to learn and practice both individual and group communication skills critical to learning and teaching.

PBL curricula are often integrated across the sciences basic to medicine, as well as among departments and activities such as clinical skills and doctor-patient-society courses that have traditionally been restricted to particular years of the curriculum.

PBL provides an antidote to the increasing fragmentation of information and knowledge and promotes the connectedness of ideas, information and knowledge. PBL needs to be seen as an approach to learning that really does help students to engage with and live in a complex world. Assessment in PBL focuses on multiple skills and abilities, on process as well as product. PBL’s student-centred focus and emphasis on Self directed learning (SDL) create unique challenge for development of an effective assessment technique.

Evaluating the success of PBL as compared to more traditional Lecture Based Learning requires more complex techniques. The guiding principle to assessment includes content learning. An effective assessment and evaluation programme can ensure that students are deriving maximum benefits from PBL.

Teaching program evaluation in medical education presents a different set of challenges. Many methods (quantitative, survey, checklists, interviews, document reviews, observations, focus groups, Nominal Group technique, Case Studies) have been used to evaluate PBL programme. Most of the studies evaluate outcomes – knowledge, learning process and skills.

Evaluation can be conveniently structured in four levels including Level One: Reaction (participants’ immediate satisfaction, perception of usefulness, motivation), Level Two: Learning (acquisition of knowledge, skills and behaviour), Level Three: Transfer (transfer of knowledge, skills and behaviour), Level Four: Results (ultimate and intended outcome). Each of these levels evaluates specific elements of the program. An ideal program evaluation planning incorporates elements from each of these levels.

Evaluation of the PBL programme involves collecting information which can include programme goals and the structure of the curriculum, connections with the student needs, scholarly inquiry or other disciplines; teaching quality, advising, inclusiveness, institutional support and outcomes. The evaluation tends to examine the conceptualization, design, implementation and usefulness of the programme.

Research on student learning has shown that students adopt qualitatively different approaches to their studies, depending upon their prior experiences of studying and the particular context in which they find themselves. These different approaches lead to qualitatively different learning outcomes. Student approaches to study are not stable aspects, but are conceived of as relations between the student and the context ^(5).

The student learning perspective suggests that students’ experiences of teaching and learning contexts are a function of both their prior experiences and of the present context. It is in relation to these experiences that they approach their studies. In order to improve their learning outcomes, we need to be concerned about both the context and their experiences of that context. Institutional policies and practices of student evaluation of teaching would be expected to have substantial effects on the way staff approach their teaching and structure the teaching and learning context and institutions need to establish their policies and practices with an explicit view of student learning in mind.

Usually, traditional assessment tools focus on isolated facts and techniques to the detriment of student’s understanding of the larger integrated concepts involved. Narrow assessments allow students to pass through the system on rote memorization rather than true understanding. The concept principle emphasizes that assessments should never be trivialized for the convenience of assessment, but rather should emphasize problem solving, thinking and reasoning skills.

The learning principle emphasizes that assessments should continue the learning process and not be viewed as a disjoint activity. The idealistic goal of the learning principle is to make it indistinguishable to a visitor as to whether instruction or assessment is occurring in the classroom. These principles certainly lay out laudable goals. Besides the fundamental challenge of creating a good problem, educators are faced with the task of deciding how to evaluate the technique’s effectiveness and how to assess whether students have met the overall learning objectives for the course.

Although a major component of the assessment of students’ progress comes from self and peer assessment that occurs at the end of every problem, additional formal assessments must assess the students’ problem-solving skills, self-directed learning skills, clinical skills and ability to recall and apply an integrated knowledge base in work with a problem. The students must become proficient in assessing their individual learning progress and that of their peers.

Methods

The Course Experience Questionnaire (CEQ) has been developed in Australia as a performance indicator for measuring the quality of teaching on particular degree program. Students’ responses to the CEQ have been shown to vary systematically (a) across different institutions, (b) with students’ rated satisfaction with their degree program as a whole, and (c) with their approaches to studying in higher education. In the present study, responses obtained from students in one of the premier Colleges of Medicine in Saudi Arabia replicated the broad constituent structure of the CEQ.

The Course Experience Questionnaire [CEQ] developed by Ramden & colleagues (1991), designed to measure the quality of learning experience, and the Student Course Experience Questionnaire (SCEQ) have been used in this study. The CEQ is a standard instrument used to gain an indication of how students experience some aspects of their course. Its features include: being a research-grounded instrument, results that can be related to student learning quality, generic items which enable use, and some comparison, across contexts, and a system of scales related to five key aspects of the learning environment which capture more than individual items from which they are composed.

The CEQ yields a global index of perceived teaching quality that can be used in a Saudi setting. Where CEQ scores on the five key aspects being investigated are higher, students are more likely to be experiencing a learning context that fits their needs and from which they are more likely to learn effectively. However, A slightly modified version of the SCEQ was designed using the same scales as the CEQ with some modification to capture other aspects of PBL that are less likely to be a part of traditional learning environment.

The MQCES: Modified Qassim University Course Experience Scales is more related to PBL System in Saudi setting:

The authors identified four items to be more related to PBL System in Saudi setting: Item number 3 from The Good Teaching Scale (GTS), Items number 2 and Number 11 from The Generic Skills Scale (GSS), and Item number 24 under The Clear Goals & Standards Scale (CGSS). After deleting three Items - Number 7 from The Good Teaching Scale (GTS), number 13 from The Clear Goals & Standards Scale (CGSS) and number 23 from The Appropriate Workload Scale (AWS) we added three items to The Overall Satisfaction Item (OSI) to become constituent of 4 items in SCEQ instead of one in CEQ, which are item numbers 7, 13, and 26. Two items were added to The Appropriate Assessment Scale (AAS) that are related directly to evaluation system raising the number of items in CEQ to 5 instead 3.

Items and Scales

The Student questionnaire contains 30 items each with a five-point (1 = Strongly Disagree, 2 = Disagree, 3 = Neutral, 4 = Agree, 5 = Strongly Agree) Likert response scale. For each scale the item scores are averaged to form scale scores.

Item and scale score means are reported in the range + 2 to −2. The midpoint zero, corresponds to the neutral Likert scale rating of 0, +2 corresponds to five and −2 corresponds to 1.

The seven scales are:

1. Good Teaching Scale (GTS): Items Number 3, 15, 17, 18 and 20

2. Clear Goals Scale (CGS): Items 1, 6, 8, 24

3. Appropriate Workload Scale (AWS): 4, 14, 21.

4. Appropriate Assessment Scale (AAS): 12, 19, 23, 29

5. Generic Skills Scale (GSS): Items 2, 5, 9, 10, 11, 22

6. Overall Satisfaction Scale: 7, 13, 25, 26.

7. Workforce Relevance Scale (WRS), 7 Questions

In addition to SCEQ scores, the authors added to MQCEQ three different items to the questionnaire to capture some important aspects of PBL system and usefulness of utilizing the time provided for the SDL.

All the undergraduate students of the College of Medicine were asked to respond to a Questionnaire on their conceptions of PBL, their perceptions of learning environment and approaches to learning. Separate questionnaires ware administered to the basic sciences tutors and the clinical instructors.

The survey included the four aspects of learning environments known to relate to the quality of student learning (Good Teaching Scale, Appropriate Workload Scale, Clear Goals & Standards Scale, and the Appropriate Assessment Scale). A fifth aspect (students’ perception of learning) was included in this study under Overall Satisfaction Scale. Workforce Relevance Scale (WRS) was measured only for the 4^th and 5^th year students.

Subjects and procedure

The study was conducted during 2006–2007. Subjects were all undergraduate students of years 1 to 5 at College of Medicine, Qassim University 384, (M= 279) and (F= 105)

Response rate is 89.84 %. A total 341 out of 384 (88.17%) including 246 out of 279 male and 95 out of 105 female (94.28%) students responded.

Results

Response rate is 89.84 %. A total 341 out of 384 (88.17%) including 246 out of 279 male and 95 out of 105 female (94.28%) students responded to our Questionnaire.

The study reveals that the PBL system helps developing student skills including: problem solving skills in 81.4 %, sharpening analytic skills in 61.3%, helping them to work as team member in 69.5 %, feeling confident about tackling unfamiliar problems in 61%, improving expression skills in 70.4 %. 63.7% respondents agreed that PBL system help them to develop their ability to plan their own work. 85.3 % respondents agree that the workload was too heavy. 83.7% students revealed that there is a lot of pressure on them to do well in the course (Table 1).

Table (1).

Modified Qassim University Student Course Experience Questionnaire Items: Percentage Responses, Means and Standard Deviation.

Items	I do not know	5	4	3	2	1
1. It was always easy to know the standard of work expected.	3.2	7.6	32.8	22.9	27	6.5
2. The PBL course developed my problem-solving skills, so far.	0.9	28.3	53.1	10.2	5.5	2
5. The course sharpened my analytic skills.	1.2	17.0	54.3	18	7.7	1.8
6. I usually had a clear idea of where I was going and what was expected of me in this course.	3.9	5.6	30.6	23.7	27.0	9.2
7. I am satisfied with the facilities (books, internet, etc) in the PBL room.	2.6	7.9	11.1	6.4	16.0	56.0
11. The course improved my expression skills.	0.3	27.2	53.2	9.9	5.8	3.5
12. The staff seemed more interested in testing what I had memorized than what I had understood.	1.7	32.8	29.9	19.2	11.3	4.9
17. The teaching staff normally gave me helpful feedback on how I was going.	0.9	3.8	15.7	12.2	18.4	49.0
18. My lecturers were extremely good at explaining things.	1.2	7.0	36.8	27.2	21.6	6.1
19. Tutors asked me questions just about facts not concepts.	7.6	12.3	30.8	24.0	17.9	7.3
23. The evaluation system in the college is not student-centered as it dose not reflect the improvement made by the student with the passage of time.	2.6	42.7	24.9	10.5	9.1	10.2
25. Overall, I was satisfied with the quality of this course.	0.6	5.6	38.3	23.0	23.6	8.8
25. Overall, I was satisfied with the quality of the course.	1.2	6.4	53.1	23.9	11.4	4.1
26. I was satisfied about   - Tutors role in PBL session.	1.2	11.7	41.4	19.5	20.4	5.8
- Problem scenario.	17.8	2.4	16.0	18.3	21.0	24.6
- My Group dynamics.	1.5	20.7	43.7	14.6	13.7	5.8
- Tutors evaluation	2.6	42.7	24.9	10.5	9.1	10.2
29- I am satisfied about the method of evaluation of our course and I think it is fair	2.9	2.4	13.6	17.7	21.8	41.6

Majority (67.6 %) of respondents agreed that the evaluation system in the college is not student-centered as it does not reflect the improvement made by the student with the passage of time. 63.4% respondents disagreed about satisfaction of the method of evaluation of the course and its fairness Table (1).

Some (19.5%) respondents agreed that the tutors gave them helpful feedback on how they are going while 67.4% disagreed, 29.1% agreed that tutors motivate them to self directed learning, 45% disagreed that the staff made a real effort to understand difficulties. 43.8% agreed that tutors were extremely good at explaining things. 28.1 % agreed that the tutors made block objectives clear right from the start and also what they expected from students while 44.2 disagreed. Moreover, 29.2 agreed that the teaching staff worked hard to make their subjects interesting while 39.7 disagreed Table (1).

The authors measured the student preference regarding PBL system. 68.8% of respondents agreed that it is better than the traditional system. 62.5 % disagreed that they are generally given enough time to understand the things that they had to learn while 25.1% agreed.

Regarding teaching facilities (PBL room), the researchers observed that the students are satisfied about tutor’s role in PBL session (29.2%), problem scenario (59.5%), and tutor evaluation (18.4%). Respondents expressed their dissatisfaction about tutors role (45.9%), problem scenario (15.5%), tutor evaluation (45.6%) and facilities (books, internet, etc) in the PBL room (72%). 57.6% respondents agreed that the course was more theoretical and less clinical while 20.4 disagreed. 54.7% like the idea of self evaluation and evaluation of their group members while 27.8% disagreed. 31% respondents agreed that they used SDL time in the college for self study while 52.6% disagreed Table (1).

Comparison of the scores between boys and girls is given in Table 2. Comparison of the scores by year (present all numbers with 2 decimal places) is depicted in Table (3).

Table (2).

Comparison of the scores between boys and girls.

Score	Mean (SE of Mean)
	Boys	Girls
WRS	2.7 (0.60)	1.81 (1.17)
GTS*	–2.17 (0.23)	0.27 (0.40)
CGS*	0.18 (0.20)	1.32 (0>33)
AWS	1.96 (0.11)	1.82 (0.17)
AAS*	–0.50 (0.17)	–1.32 (0.31)
GSS	4.7 (0.24)	4.90 (0.40)
OSI*	– 1.67 (0.31)	0.60 (0.55)

^*P < 0.05

Table (3).

Comparison of the scores by year (present all numbers with 2 decimal places).

Score	1st year	2nd year	3rd year	4th year	5th year
WRS	-----	-----	-----	2.1 (0.7)	2.9 (0.9)
GTS*	–0.3 (0.5)	–1.8 (0.5)	–2.7 (0.4)	– 0.7 (0.4)	–1.8 (0.5)
CGS*	1.0 (0.4)	1.5 (0.3)	0.3 (0.4)	0.3 (0.3)	–0.8 (0.4)
AWS	2.3 (0.21)	2.1 (0.2)	2 (0.2)	1.6 (0.2)	2 (0.3)
AAS*	1.22 (0.42)	–1.2 (0.3)	–0.3 (0.4)	–0.7 (0.3)	–0.1 (0.3)
GSS	4.8 (.6)	5.5 (0.3)	3.6 (0.5)	5.1 (0.5)	4.5 (0.6)
OSI	0.7 (0.8)	– 0.7 (0.5)	–2.1 (0.6)	–0.6 (0.5)	– 1.45 (0.8)

^*P < 0.05

• Note: The difference between boys and girls were not statistical significant when controlled for years.

Majority of the tutors (94.2 %) agreed that the course improved the student’s expression skills. Similarly 76.2% agreed that students were more specific and goal-oriented with the passage of time during their training. Moreover, 61.8 % tutors opined that PBL system is definitely better that the traditional system of learning in medical schools (Table 4).

Table (4).

Tutor response regarding PBL system of teaching and learning.

Items	Strongly agree 5	Agree 4	Neither agree/nor disagree 3	Disagree 2	Strongly disagree d 1
5. The course sharpened the analytic skills of students.	10 (29.4)	18 (52.9)	5 (14.7)	1 (2.9)	0
9. The students were more specific and goal-oriented with the passage of time.	7 (20.6)	19 (55.9)	6 (17.6)	2 (5.9)	0
10. The student interest in PBL activity increased with the passage of time.	9 (26.5)	11 (32.4)	9 (26.5)	5 (14.7)	0
11. The course improved student’s expression skills.	16 (47.1)	16 (47.1)	1 (2.9)	1 (2.9)	0
12. The students usually reach an acceptable level of understanding the problem at the end of the PBL session.	7 (20.6)	21 (21.8)	4 (11.8)	2 (5.9)	0
27. Students follow the seven jumps method regularly.	9 (26.5)	13 (38.2)	4 (11.8)	4 (11.8)	2 (5.9)
25. In my opinion, PBL system is better than traditional system.	9 (26.5)	12 (35.3)	13 (38.2)	0	0

Regarding the Weekly Problem Solving Exercise, 49.8% students said that they spent allocated time in understanding the problem and determining the objectives, 31.9% spend it in literature search and, 86.9% in preparing presentation. 85.2% students consulted books, 25.2% searched the Internet and, 74 % sought help from tutors. Majority of the students (68.4%) revealed that they are satisfied with the PBL system as it is superior to the traditional system and 16.8 % students did not agree with their opinion (Table 5).

Table (5).

Results of added items otherwise not included in any Scale.

Items	Strongly agree (5)	Agree (4)	Neither agree/Nor disagree (3)	Disagree (2)	Strongly disagree (1)
27. For the weekly problem solving I spend my time in :
a: Understanding the problem & finding objectives	57 (16.5)	115(33.3)	47 (13.7)	61 (17.7)	49 (14.2)
b: Searching the literature	127(36.8)	145 (42)	40 (11.6)	25 (7.2)	6 (1.7)
c: Preparing presentation.	88 (25.5)	143(41.4)	52 (15.1)	33 (9.6)	16 (4.6)
28. I depended mostly on the following source/s
a- Books	207 (60)	87 (25.2)	22 (6.4)	8 (2.3)	1 (0.3)
b- Internet	21 (6.1)	66 (19.1)	79 (22.9)	72 (20.9)	101 (29.3)
c- Tutors and lectures	78 (22.6)	143 (41.4)	57 (16.6)	34 (9.90)	22 (9.8)
d- PBL Groups	22 (6.4)	85 (24.6)	88 (25.5)	73 (21.2)	69 (20)
30. In my opinion, PBL system is better than traditional system	141 (40.9)	95 (27.5)	49 (14.2)	18 (5.2)	40 (11.6)

Discussion

The study reveals that the PBL system helps developing student skills particularly problem solving skills and help sharpening analytic skills However, majority of the students are not satisfied with the evaluation system in the college as they think that it is not student-centered as it does not reflect the improvement made by the student with the passage of time. Although a small number (19.5%) of respondents accepted that the tutors gave them helpful feedback on how they are going, majority (67.4%) shows disagreement.

The majority of respondents agreed that PBL is better than the traditional system and consider it superior to the traditional Lecture-Based System in Medical Education. However, the teaching/learning facilities including audio-visual, reference books and Internet need further improvement.

Results of the regular programme evaluation indicate that after 18 months students become bored with the repetitious routine of working through problems. Students indicate that the problems and the approach used in the tutorial groups cease to be challenging and motivating. The PBL process tends to become ritualized, with students skipping the stage of elaboration of prior knowledge. The discussion in the second session is reduced to presentation of the main results, with no attempts being made to appraise opinions and synthesize findings. Students shirk their responsibility to take an active part in the discussion out of lack of interest, laziness and uncertainty as has been found in some similar stdies ^(6,7)

Students are satisfied with many objectives of the Problem Based Learning depicted comprehensively in the respective Blocks designed by the tutors and finalized by experts. By clearly specifying the educational outcomes in behaviorally measurable ways, we can change the way faculty teach and students learn. Where evaluation drives the curriculum, graduation should contingent upon demonstrating mastery of a defined set of competencies. ⁽⁸⁾

In several of the Clerkship performance measures, the PBL students performed significantly better and in no circumstances did they perform worse than the standard curriculum.⁽⁹⁾

The appropriate training for both faculty and students is an essential factor to ensure the successful implementation of PBL programme in medical schools.

This study reveals that the quality of implementation of curriculum and assessment methods needs to be improvised. Concept maps are suitable for quality management in education, thus enabling the paradigm shift in medicine. A series of communication, education, practice, system reforms and the PBL curriculum are required for the success of programme.

Curriculum maps need to be developed for the smooth functioning of PBL system. Cultural issues including language problems (Saudi students medium of instruction upto ‘A’ Level is Arabic. They learn English after passing ‘A’ Level (during the preparatory year after obtaining admission to professional colleges), in setting ground rules for PBL tutorials must be given due consideration. There is a strong need for well-trained teachers who can conduct small group PBL sessions skillfully. Assessment methods of the students need to be consistent with how students learn.

Conclusion

The growing popularity and increased application of PBL presents significant challenges. Our strategy for evaluating the success of PBL is ongoing and the results represent only an initial stage in analysis as we are still in the process of standardizing the process of collecting the outcome data particularly from those who are pass outs from the college and doing their internship. However, there is evidence that PBL is having an impact on the performance of students, their perspective on learning and teaching methods. Focus is required on improving the quality of teaching/learning as well as of the assessment methods. Students who used the problem-based learning method showed better interpersonal skills and psychosocial knowledge, as well as a better attitude towards patients. Further evaluation will allow us to determine the strengths and weaknesses of PBL system at College of Medicine, Qassim University.

Appendices

Appendix 1. Methods

Description of Scales

The Good Teaching Scale measures perceptions of the teachers’ ability to contribute to student learning. It is characterized by practices such as providing students with feedback on their progress, explaining things, making the course interesting, motivating students, and understanding students’ problems. High scores on the Good Teaching Scale are associated with the perception that these practices are present. Lower scores reflect a perception that these practices occur less frequently.

The Clear Goals and Standards Scale measures the extent to which students have a clear idea of what, at a broad level, is required of them in their degree.

Appropriate Assessment Scale [AAS] measure concentrates on one particular aspect of assessment and is not exhaustive in its measurement of assessment approaches. It focuses on the extent to which assessment emphasized recall of factual information rather than higher order thinking.

The Appropriate Workload Scale focuses on the extent to which the workload given is perceived to be manageable. High scores on this scale indicate perceptions of reasonable workloads.

Generic Skills Scale [GSS] takes into account the extent to which the course contributes to the key skills that graduates might be expected to possess. Skills typically identified in this context include communication skills, the capacity to learn new skills and procedures, the capacity to make decisions and solve problems, the ability to apply knowledge to the workplace, and the capacity to work with minimum supervision.

SCEQs have been designed to monitor students’ experience of the whole programme, and their satisfaction with that programme. The way students’ perceive key aspects of their learning/research context is related to the quality of their learning approach and to their outcomes of learning.

The SCEQ also contains four sets of related items that are not constituted as scales. The SCEQ is based on a way of thinking about teaching and learning at a more holistic level. The development of more student-centred ways of thinking is more likely to lead to changes in results and to the quality of student learning than a focus on individual items of the questionnaire. Initiatives that focus on encouraging perceptions of an environment where students’ value and enjoy learning are likely to have an effect on many of the items.

Research on student learning has shown that students adopt qualitatively different approaches to their studies, depending upon their prior experiences of studying and the particular context in which they find themselves. These different approaches lead to qualitatively different learning outcomes. Student approaches to study are not stable aspects, but are conceived of as relations between the student and the context.

The student learning perspective suggests that students’ experiences of teaching and learning contexts are a function of both their prior experiences and of the present context. It is in relation to these experiences that they approach their studies. In order to improve their learning outcomes, we need to be concerned about both the context and their experiences of that context. Institutional policies and practices of student evaluation of teaching would be expected to have substantial effects on the way staff approach their teaching and structure the teaching and learning context and institutions need to establish their policies and practices with an explicit view of student learning in mind.

The experiences of students are primary in determining the quality of the outcomes of their learning. Despite the pitfalls, to date, these have been the most common methods of evaluation used in PBL programme. Mixed method reviews involving both quantitative and qualitative evaluation programme provides depth and breadth to our analysis may if you put this up better.

Appendix 2.

Modified Qassim University Student Course Experience Questionnaire Items: Percentage Responses, Means and Standard Deviation

Items	I do not know	5	4	3	2	1
1. It was always easy to know the standard of work expected.	3.2	7.6	32.8	22.9	27	6.5
2. The PBL course developed my problem-solving skills, so far.	0.9	28.3	53.1	10.2	5.5	2
3. The tutors of this course motivated me to SDL.	2.1	6.2	22.9	28.2	25.3	15.3
4. The workload was too heavy.	0.3	52.3	33	7	6.4	0.9
5. The course sharpened my analytic skills.	1.2	17.0	54.3	18	7.7	1.8
6. I usually had a clear idea of where I was going and what was expected of me in this course.	3.9	5.6	30.6	23.7	27.0	9.2
7. I am satisfied with the facilities (books, internet, etc) in the PBL room.	2.6	7.9	11.1	6.4	16.0	56.0
8. To do well in this course all I really need is the ability to find target information.	4.2	23.1	48.7	11.9	8.6	3.6
9. The course helped me develop my ability to work as a team member.	0.6	23.5	46.0	13.8	12.6	3.5
10. As a result of my course, I feel confident about tackling unfamiliar problems.	4.4	14.8	46.2	21.6	9.8	3.3
11. The course improved my expression skills.	0.3	27.2	53.2	9.9	5.8	3.5
12. The staff seemed more interested in testing what I had memorized than what I had understood.	1.7	32.8	29.9	19.2	11.3	4.9
13. I develop more interest with the passage of time.	1.2	12.8	32.1	13.7	22.4	17.8
14. I was generally given enough time to understand the things I had to learn.	1.5	7.0	18.1	10.8	33.0	29.5
15. The staff made a real effort to understand difficulties I might be having with my work.	1.8	6.2	26.5	22.6	22.1	20.9
17. The teaching staff normally gave me helpful feedback on how I was going.	0.9	3.8	15.7	12.2	18.4	49.0
18. My lecturers were extremely good at explaining things.	1.2	7.0	36.8	27.2	21.6	6.1
19. Tutors asked me questions just about facts not concepts.	7.6	12.3	30.8	24.0	17.9	7.3
20. The teaching staff worked hard to make their subjects interesting.	2.1	5.3	23.9	29.1	24.7	15.0
21. There was a lot of pressure on me to do well in this course.	0.6	49.1	34.6	7.8	5.5	2.3
22. My course helped me to develop the ability to plan my own work.	1.5	20.7	43.7	14.6	13.7	5.8
23. The evaluation system in the college is not student-centered as it dose not reflect the improvement made by the student with the passage of time.	2.6	42.7	24.9	10.5	9.1	10.2
24. The staff made it clear right from the start what they expected from students.	3.0	4.7	23.4	24.6	26.7	17.5
25. Overall, I was satisfied with the quality of this course.	0.6	5.6	38.3	23.0	23.6	8.8
24. The staff made block clear right from the start and also what they expected from students.	0.3	3.5	25.7	24.6	30.4	15.5
25. Overall, I was satisfied with the quality of the course.	1.2	6.4	53.1	23.9	11.4	4.1
26. I was satisfied about   - Tutors role in PBL session.	1.2	11.7	41.4	19.5	20.4	5.8
- Problem scenario.	17.8	2.4	16.0	18.3	21.0	24.6
- My Group dynamics.	1.5	20.7	43.7	14.6	13.7	5.8
- Tutors evaluation	2.6	42.7	24.9	10.5	9.1	10.2
29- I am satisfied about the method of evaluation of our course and I think it is fair	2.9	2.4	13.6	17.7	21.8	41.6