Academic Accreditation Process: Experience of a Medical College in Saudi Arabia

Abstract

Objective

This study aims to describe effects of the accreditation process of the National Commission for Academic Assessment and Accreditation (NCAAA) and its impact on the quality of medical education in one medical college in Saudi Arabia.

Methods

We conducted a mixed (qualitative-quantitative) study in Qassim University College of Medicine, where an accreditation exercise led by NCAAA was recently completed. Data pertaining to impact of the accreditation process were collected through self-administered questionnaires and focus-group discussion. Data were aggregated and analyzed and compared with the information from prior to accreditation, where available.

Results

The accreditation process lasted about two years, culminating in the preparation of a self-evaluation report and a visit of external reviewers. The process itself brought significant changes in the educational processes and administration and implementation of the curriculum. Our analysis also indicated significant improvements in the quality of medical education in the College.

Conclusions

The accreditation process was successful in improving quality of medical education, without imposing radical changes in curriculum philosophy or orientation. Better monitoring of students learning outcomes and continuous quality improvement will improve the quality of medical education in the college.

Introduction

Until 2004, higher education institutes in Saudi Arabia, including medical colleges, required no accreditation. Rapid growth in the number of post-secondary institutions necessitated the creation of a government agency for quality assurance of higher education. In 2005, the Council of Higher Education approved the founding of National Commission for Academic Assessment and Accreditation (NCAAA). It is now mandatory for all institutes of higher education to be accredited by the NCAAA. The NCAAA has developed a comprehensive quality assurance and accreditation system that has benefited from international experience while keeping its local characteristics. The NCAAA is an independent body, financially and administratively, reporting to the Council of Higher Education. Its responsibilities include establishing standards, criteria and procedures for accreditation, reviewing and evaluating performance of existing and new institutions, accrediting institutions and programs, and supporting improvements in quality. ^{(1, 2, 3)}

Accreditation is defined as a process whereby officially appointed external regulatory bodies, accountable at government level, evaluate educational institutions sing established criteria, standards and procedures. It entails gathering data on various aspects of the educational institution and making decisions regarding compliance with the standards. This is done primarily to ensure the quality of education required to produce competent graduates, to guarantee equal standards for medical doctors graduating from all medical schools, and to define minimum essential requirements that every medical school should provide. ^{(4, 5)} While many accreditation processes are largely similar to those applied to any higher education institution some are more specific to the medical colleges. ^(6–11)

We present the results of a mixed (qualitative-quantitative) study of the NCAAA accreditation process of a medical college in Saudi Arabia. Our study examined the ‘mock’ accreditation process of the Qassim University College of Medicine conducted during 2008–2009. As the College was not yet eligible for a formal accreditation, the mock exercise helped identify the strengths and weaknesses of the medical education program. The mock accreditation was similar in every way to actual accreditation, short of the issuance of a formal letter of accreditation by the NCAAA. The specific objective was to examine the impact of the NCAAA’s accreditation process on the quality of medical education.

Data and Methods

This study was conducted in Qassim College of Medicine where an experimental accreditation process was completed by the NCAAA during 2008–2009. Data were collected through self-administered questionnaires of the NCAAA, which are used for self-evaluation in the institutions before accreditation (12–14) (N=51). We also conducted one focus group discussion (FGD) with the members of the QA Group of the College (six participants). Guidelines for the FGD were also developed from the NCAAA questionnaires by inserting probes to elicit in-depth information on the quality and key performance indicators that are covered in these questionnaires. The FGD was conducted in English and was moderated by one of the authors (FM). The faculty members were familiar with the questionnaires as they were previously exposed to the NCAAA questionnaires. In addition we used an abridged version of the NCAAA questionnaire for medical students (N=91); this questionnaire included only the teaching and learning sections of the NCAAA questionnaire. All of the NCAAA questionnaires are considered to be validated. We did not make any changes in the questionnaires.

Ethical approval was obtained from the Academic Council and the Medical Research Ethics Committee. All respondents were free to refuse from participating in the study; the responses were kept strictly confidential and personal information of the respondent was not revealed to anyone at any stage of the study.

The qualitative data were manually aggregated and summarized, then categorized into domains. The authors read the FGD notes independently and formed their own summaries and conclusions, which were later integrated. Quantitative data from the NCAAA questionnaire were entered into Excel spreadsheets and analyzed using SPSS for Windows (version 17.0). A brief summary of all the results was prepared and discussed with several other colleagues before finalizing our discussion and conclusions for this study.

Results

Impact of the Accreditation Process on the Quality of Medical Education

Student Learning Outcomes

Participants of this study – faculty and students alike – were aware that the College had formulated the goals and objectives of the curriculum to meet the student learning outcomes. They were also very familiar with the activities that the College is carrying out to achieve these goals. The participants believed that the students learning outcomes were consistent with requirements for professional practice and regulations in Saudi Arabia, as well as with the community’s health needs. Participants expressed the view that the high quality performance of our graduates in the National Saudi Selection Exam (SLE) (organized by Saudi Commission for Health Specialties) was a fair evidence for this conclusion.

A majority of faculty members believed that there were visible improvements in quality standards of education after accreditation: For example, student learning outcomes were significantly improved; course specifications became mandatory before the start of all courses; course specifications included learning outcome; and faculty members were encouraged to participate and provide their input in the formulation of learning outcomes.

The participants believed that learning outcomes were achievable through the educational formats that were currently in use. These positive changes were attributed to the exercise that preceded the evaluation and the external reviewers’ visit for the NCAAA accreditation.

Program Development

Both faculty and students felt that the College became more receptive to curriculum modifications to accommodate changes in national and international requirements in medical education. Faculty members felt that curriculum structure, composition and duration were meticulously planned and controlled. They believed that the behavioral/social sciences and medical ethics were better emphasized than before. Course specifications and course reports became integral parts for implementation of all basic and clinical courses. It was believed that accreditation process had a major role in stimulating regular curriculum review and improvement.

Program Evaluation and Review Processes

After the accreditation, courses are evaluated and reported regularly with information providing the extent to which learning outcomes were being achieved. Quality indicators are identified and used for all courses and programs. The records of student performance and completion rates in all courses, and the program as a whole, are obtained and included in course reports. The faculty members demonstrated that their awareness and concerns with program evaluation has increased substantially and they have become more critical to this issue.

Student Assessment

Students indicated that assessment mechanisms have become more appropriate, and reflect to a greater extent the teaching methods, the learning outcomes and the curriculum goals. Students also reported that assessment processes were clearly communicated to them at the beginning of courses. The faculty members believed that the mechanism used for verifying standards of student achievement in relation to relevant internal and external benchmarks is acceptable. Comparisons of student performance with other comparable institutions within Saudi Arabia are currently done using SLE results of students from different Saudi universities. Feedback on performance by students and results of assessments was given promptly to students, although the mechanisms for providing assistance if needed were not adequate. Criteria and processes for academic appeals are made known to students and administered equitably.

Educational Assistance for Students

Students confirmed that development of a system for office hours improved the availability of teaching staff for consultation and advice. Student academic counseling was introduced and included electronic communications through email, but the effectiveness of this process was not evaluated.

Recently, the college introduced the progress test, in which scoring of every student in subsequent tests is communicated to academic supervisor for continuous monitoring and for providing assistance and/or counseling to those facing difficulties. The college now provides adequate facilities for private study with access to computer terminals and other necessary equipment such as internet connection.

Students’ social, extra-curricular and sports activities and facilities are encouraged more. The College has also established students’ mentoring system at informal and formal level. In general, the College has substantially improved the educational assistance for students; however, faculty members are still not familiar with the range of support services available in the institution for students to refer them to the appropriate sources for assistance, as and when required.

Quality of Teaching and its Support

Faculty and students acknowledge that the College was using a wider range of teaching and learning methods. The use of the didactic teaching in certain courses and in certain circumstances has become more limited to settings where this method is essential. Other methods which encourage self-directed learning, team working, critical thinking and lifelong learning skills are more encouraged and supported. Interdisciplinary teaching and assessment is implemented in more settings such as symposium and Objective Structured Practical Exams (OSPE).

Educational resources such as textbooks and e-learning materials are better developed and are more accessible. The orientation workshops for newcomers, rapid growth of the college e-learning unit, and the use of course specification have contributed substantially in improving the quality of teaching. A faculty development unit was created and was given the responsibility to develop training programs to enhance the teaching and research skills.

Field Experience Activities

Training in the field has become more structured with clear objectives, tasks and assessment methods. Students are more satisfied with the training and more conscious about what is expected from them.

Partnership Arrangements with Other Institutions

Faculty and students highlighted positive achievements in the partnership arrangements. Medical professionals in Ministry of Health are now more familiar with the content and details of courses and standards of student assessments. Also, clinical training, assignments and examinations are currently adapted to the local settings.

Partnership arrangements with internationally recognized medical institutions is more recognized and needed by teaching staff and college administration. The College started recently a series of international partnership agreements in prevention and health promotion research and traditional medicine.

As shown in the following table, several changes have occurred as a result of the accreditation process, most of which are related to teaching and learning.

Changes in the Medical Education as an Impact of NCAAA Accreditation Process

	Before NCAAA accreditation		Through and after NCAAA accreditation
Item	2006	2007	2008	2009	2010
Mission & vision	✓	✓	✓	✓	✓
Program specification	-	-	✓	✓	✓
Courses specification	-	-	✓	✓	✓
Course evaluation survey	-	-	✓	✓	✓
Student experience survey	-	-	✓	✓	✓
Program evaluation by graduates	-	-	✓	✓	✓
Graduate evaluation by stakeholders	-	-	-	✓	✓
Course report	-	-	✓	✓	✓
Annual program report	-	-	✓	✓	✓
Program evaluation	-	-	✓	✓	✓
Student representation in committees	-	-	-	✓	✓
Academic supervision	-	-	✓	✓	✓
Declaration of Office hours	-	-	✓	✓	✓
Introduction of learning resources center	-	-	-	✓	✓
Structured Faculty development activities	-	-	✓	✓	✓
Independent evaluation	-	-	✓	✓	✓
Comparison of students achievement in different courses	-	-	✓	✓	✓
Follow up of student progression	-	-	-	-	✓
Comparison of boys & girls sections	-	-	-	-	✓

Discussion and Conclusion

The NCAAA standards are strong on strategic planning; management and resource issues, but these dominate the process, with relatively little emphasis on teaching, learning and assessment. Too much time is therefore spent in the review process on management and strategy, and not enough in looking at the curriculum, learning approaches and assessment. Some of the generic standards may be inappropriate or insufficient for medical education.

The findings from this study are based on a comparison between freshly collected data and the self-evaluation report submitted to the NCAAA in 2008. It is clear that these processes made the faculty and students more conscious and aware of quality issues in medical education. The awareness increased the critical judgment regarding the education processes in the college. The results of this study also provide the basis for evaluation of the medical education in the college for other areas in addition to the focus area of the study which is teaching and learning.

It is difficult to provide information on all areas of impacts of NCAAA accreditation. However, we have presented an account of some of the initial impacts of the first accreditation exercise in the College. Our study suggests that, while applicability of the NCAA standards to medical education is debatable, the impact of the accreditation processes (including intense data collection and analysis, documentation, and participation of and feedback to faculty members and students) was clearly felt at all levels.

The role of accreditation in quality assurance and quality improvement is well-recognized. However, there are differences in opinions regarding the mechanisms by which an accreditation process affects the quality assurance and quality improvement processes. Many educationists have stressed upon the fact that accreditation is useful not only to evaluate the educational quality of new and established programs, but also to allow individual institutions to monitor, reflect upon, and provide a means of continuous improvement of the curriculum. ^{(15, 16)}

The impact of the accreditation process on learning and teaching is obvious and common to all medical colleges that implemented an accreditation process. This impact has been shown to be effective and constructive in improving many aspects of curricular development and educational strategies used. Our study also shows that numerous changes happened as an impact of the accreditation process, which will lead to quality improvements. It has been shown that nearly all schools in Australia and New Zealand have made major changes to their courses in the last 10 years due to accreditation requirements. There was a greater focus on the teaching and assessment of communication skills, curricula were better integrated, there was a greater focus on student-centered learning, there was a better use of information technology in courses, assessment strategies were more diverse, and course evaluation was more consistent. ⁽¹⁶⁾ Similar results were found in a study in Nigeria. ⁽¹⁷⁾

Medical colleges across the globe are adopting an accreditation process to ensure that their educational programs meet the quality standards. Although accreditation processes use different strategies, techniques and standards, some themes and objectives are common to all. The primary purpose of accreditation and quality improvement is to adjust medical education to changing conditions in the health care delivery system, to prepare doctors for the needs and expectations of society, to help doctors cope with the explosion in medical and scientific knowledge and technology, and inculcate in them the ability for lifelong learning. ^(18–21)

Because accreditation and recognition is a variable process, the criteria, standards and procedures of this accreditation should be revised regularly. Review can be done and maintained through feedback from research studies, expert teams, institutions undergoing accreditation or recognition, government policy which effect clinical training, extensive stakeholder consultation, and networking with other regional, national, and international medical education agencies. Regular review against national and international standards for good practice should be a routine practice in every medical institution.

The NCAAA accreditation process conducted by our college has been successful in improving medical education in the college without imposing any radical changes in the curriculum philosophy or orientation. Better measurement of the students learning outcomes, regular review and updating of the quality standards and continuous quality improvement are anticipated to deliver better medical education in the college. Data from additional institutions would greatly enhance the relevance and validity of the study. The conclusions drawn and lessons learned from this study are nonetheless important.