The first formal report on the design, implementation, and evaluation of the objective structured clinical examination (OSCE) to assess the clinical skills of clinical psychology students in Iran

Abstract

Background

The objective of the present study was to provide the first formal report on the design, implementation, and evaluation of the Objective Structured Clinical Examination (OSCE) for assessing the clinical skills of clinical psychology students in Iran.

Methods

This analytical study encompassed all stages of designing, implementing, and evaluating a clinical psychology OSCE. The examination was conducted in 2024 at the Faculty of Welfare and Health Social Sciences, Islamic Azad University of Medical Sciences, Tehran, for all 90 final-year master’s students in clinical psychology. The clinical psychology OSCE consisted of nine separate stations. Each station was assessed via evaluation checklists previously developed by the university professors. After assessing the students, the test’s reliability and validity were examined. Students’ opinions on the examination were also analyzed quantitatively and qualitatively.

Results

The results indicate that the test has appropriate validity and reliability. The mean scores for the evaluation, interview, hospital procedures, treatment, and overall score were 17.44 (SD = 1.68), 16.61 (SD = 2.98), 17.18 (SD = 2.27), 17.84 (SD = 1.94), and 17.26 (SD = 1.41) (out of 20 points), respectively. Additionally, various sections of the test showed a good correlation with the overall score obtained. The overall score was also significantly correlated with the total grade point average (GPA) of the master’s students (P < 0.001; r = 0.54), demonstrating the criterion validity of the test. The obtained Cronbach’s alpha among the scores of different sections of the test was 0.68, indicating an acceptable level of reliability. Quantitative and qualitative analyses of student feedback regarding the OSCE exam revealed the appropriateness of the method and execution, despite insufficient time at some stations, and lack of prior practice.

Conclusions

Although the OSCE has rarely been utilized in clinical psychology within Middle Eastern countries, its structured assessment of practical skills makes it a suitable method for evaluating the clinical competencies of students in this field.

Background

Assessment constitutes a fundamental element in education, taking place in two primary forms: formative and summative. Evaluation of training is an integrated education process to improve learning. In contrast, summative assessment is conducted at the end of a learning period to gauge the extent of knowledge and is typically more formal in nature [1]. The issue of assessment holds particular significance in clinical domains, where it directly relates to practical interventions concerning individuals’ health. In addition to the intrinsic value of accurate assessment in health-related fields, many of these fields have both theoretical and practical components, which necessitate thorough evaluations that methodically and systematically assess the practical aspects of the material taught, particularly in scenarios similar to real-world ones.

In this sense, the Objective Structured Clinical Examination (OSCE) is a performance evaluation instrument that is frequently used in clinical education and assessment in a variety of clinical specialties. It was first used in the 1970s by Harden and associates to evaluate students’ clinical and practical skills in a simulated setting [2].

The OSCE comprises a series of stations, each presenting the student with a simulated patient (SP). During the examination, the practical assessment of crucial clinical skills such as history taking, information gathering, empathy, communication, treatment planning, and responding to unexpected situations becomes feasible [3]. At each station, examiners utilize standardized checklists to evaluate student performance [4]. This examination facilitates a standardized and structured evaluation of clinical skills in a simulated setting, allowing for the rapid and direct observation of multiple competencies [5].

This tool has been repeatedly utilized across various clinical domains, including medicine, dentistry, pharmacy, and nursing, at different academic levels, in both formative and summative assessments, and in diverse geographical locations worldwide [6–9]. Medical education in diverse specialties such as general practice, pediatrics, obstetrics and gynecology, surgery, and psychiatry has benefited from assessment using the OSCE [8]. Numerous studies have examined the validity and reliability of this examination, which has been applied with great success in the field of psychiatry [5, 10–16].

The OSCE has been used in the field of clinical psychology since 2012 by Yap and colleagues [17], and it is still utilized in many nations for the systematic evaluation of students in this subject. Reports from these evaluations [18–22] indicate its effectiveness in assessing clinical skills in psychology.

This test is especially useful because it can assess certain abilities in the field of clinical psychology. For instance, it can assess the necessary competencies for diagnosing psychological disorders, evaluating patients’ mental status, and delivering evidence-based treatments [9]. Given the critical role of patient interaction and interview-based assessments in clinical psychology, the OSCE provides an opportunity to evaluate these skills in controlled environments with immediate feedback. The OSCE’s capacity to objectively and consistently evaluate several clinical abilities at once is a major benefit in the field of psychology. Furthermore, it can evaluate both soft skills, such as effective communication and empathy, as well as more specialized skills like diagnosis and treatment planning [23]. The OSCE possesses specific characteristics such as objectivity and reproducibility, as students are assessed fairly under similar conditions, which helps mitigate the impact of individual biases [24].

In conclusion, the OSCE shows great value and effectiveness in assessing clinical decision-making and communication abilities, and it can help raise the standard of instruction and evaluation in these domains. However, despite the first administration of the psychiatric OSCE in Iran in 2004 and its subsequent importance, no reports have been received regarding the implementation of the OSCE for evaluating students in clinical psychology. This gap may stem from the fundamentally subjective nature of diagnostic processes and therapeutic interventions in psychology. In contrast to psychiatry and other medical disciplines, where both diagnosis and treatment tend to be more objective and amenable to standardized measurement, the systematic and objective evaluation of psychological assessment and intervention is considerably more challenging and demands substantially greater rigor and precision.

The present report appears to be the first formal account of an OSCE designed, implemented, and evaluated for clinical psychology students by the Islamic Azad University of Medical Sciences of Tehran in 2024. The current study aimed to design, implement, and evaluate an OSCE to examine the clinical skills of clinical psychology students.

Methods

The present study employed an analytical design. The research was conducted on 90 Master’s students in Clinical Psychology at the Faculty of Welfare and Social Health Sciences, Islamic Azad University of Medical Sciences of Tehran, using a census sampling method. All participants in this study were students who had completed a 240-hour clinical internship program, during which they had acquired extensive familiarity with psychological assessment techniques, diagnostic interviewing, hospital-based inpatient procedures, and evidence-based psychotherapeutic interventions. The criteria for participation in the OSCE were as follows:

• Inclusion Criteria: Being a Master’s (M.Sc.) student, having completed all approved academic coursework, and having finished the internship/practicum courses administered by the university.

• Exclusion Criteria: Failure to participate in any section/station of the exam, or withdrawal of consent to participate.

The university’s ethics committee provided the following code of ethics: IR.IAU.TMU.REC.1404.285. These individuals had completed their internship in accordance with the educational curriculum and internship guidelines of the Islamic Azad University of Medical Sciences of Tehran, across three sections: assessment, diagnostic interviewing based on the Structured Clinical Interview for DSM-5 Disorders: Clinician Version (SCID-5-CV) [25], and treatment. Following several sessions of discussion, deliberation, and decision-making regarding the first clinical psychology OSCE, which was undertaken by the General Education Office of the Islamic Azad University of Medical Sciences of Tehran and the Educational Development Office of the Faculty of Welfare and Social Health Sciences, the design, implementation, and evaluation of the examination were carried out in three stages:

1- Examination Design

Initially, a workshop was held to familiarize the faculty members of the Psychology Department and their teaching assistants, who were doctoral students, with the OSCE process. This workshop was led by a faculty member specializing in medical education. Subsequently, the number of stations for each section was determined. Following this, the faculty members responsible for each section designed the scenarios, checklists, and station guidelines. These materials were then discussed and reviewed in a meeting. It is worth noting that the design of each station was entrusted to faculty members based on their specialization within the field. The station scenarios were designed as requests to perform a specific activity. Content validity of the checklist was quantitatively assessed by six experts in clinical psychology and psychiatry. Item-level (I-CVI) and scale-level (S-CVI/Ave and S-CVI/UA) content validity indices were calculated according to Polit and Beck [26].

Example for the scenarios

Scenario for the psychotherapy station using Rational Emotive Behavior Therapy (REBT):

“As a psychologist, you are meeting with an individual named ‘Saeed’ who suffers from a fear of flying. Saeed has recently realized that he cannot even go to the airport. He tells you that the thought of a plane crashing always frightens him. Your task is to converse with Saeed, identify his irrational beliefs, and help him challenge these beliefs.”

The design of the evaluation checklists was also carried out by the station supervisors, considering the objective of each station, and was discussed with other members of the OSCE expert committee. Ultimately, for each expected item on the checklist, two statements were considered: “able to perform” and “unable to perform.” Additionally, the evaluators assessed the students on a 5-point Likert scale, ranging from “very good,” “good,” “average,” “weak,” and “very weak.”

After designing the stations, standardized patients (SPs) underwent 8 to 10 h of training under the supervision of the station supervisors and evaluators, all of whom were faculty members. It is important to note that the standardized patients were psychology students or teaching assistants. Nine faculty members from the Psychology Department and one faculty member from the University of Medical Sciences with a degree in medical education participated in the station design process. Furthermore, 8 faculty members and 15 teaching assistants served as evaluators, and 2 teaching assistants and 15 psychology students participated as standardized patients in the examination process.

The examination was conducted in the format of 9 stations, comprising two assessment stations, two diagnostic interview stations, one hospital procedures station, two treatment stations, and two rest stations (Fig. 1). Seven minutes were allocated to each station, with a 45-second interval for exiting one station and entering the next. The content of the stations was as follows in Table 1.

Fig. 1.

The 9 stations of the clinical psychology OSCE held

Table 1.

Table of Stations, skills being Assessed, student Assignment, and time

Station	Skill being assessed	Student assignment	Time
1-Psychotherapeutic intervention	Implementing therapeutic interventions for the disorder that the simulated patient was simulating, utilizing one of the following approaches: Acceptance and Commitment Therapy (ACT), Rational Emotive Behavior Therapy (REBT), Integrative Psychotherapy, or Intensive Short-Term Dynamic Psychotherapy (ISTDP)	Implementation of core therapeutic techniques from the primary treatment modality selected by the trainee—namely Acceptance and Commitment Therapy (ACT), Rational Emotive Behavior Therapy (REBT), Integrative Psychotherapy, or Intensive Short-Term Dynamic Psychotherapy (ISTDP)—in response to the clinical presentation enacted by the SP	7 min
2-Diagnostic interviewing	Skills in conducting a diagnostic interview using the SCID-5-CV	Conducting a structured diagnostic interview strictly adhering to the Structured Clinical Interview for DSM-5 Disorders–Clinician Version (SCID-5-CV) protocol, guided by the presenting symptoms and clinical phenomenology exhibited by the SP	7 min
3-Psychological assessment	Psychological assessment using one objective psychological test and one projective psychological test	Administration of an objective and/or projective psychological assessment instrument to the standardized patient (SP), selected in direct accordance with the clinical case and diagnostic hypothesis being evaluated	7 min
4-Rest	Offering refreshments to the students and allowing a rest period	Provision of a brief intermission or rest interval between stations/tasks	7 min
5-Hospital-based inpatient procedures	Assessing skills in dealing with acute patients	Implementation of immediate therapeutic and crisis-intervention measures for a hospitalized psychotic patient.	7 min
6-Psychotherapeutic intervention	Implementing therapeutic interventions for the disorder that the simulated patient was simulating, utilizing one of the following approaches: Acceptance and Commitment Therapy (ACT), Rational Emotive Behavior Therapy (REBT), Integrative Psychotherapy, or Intensive Short-Term Dynamic Psychotherapy (ISTDP)	Implementation of core therapeutic techniques from the primary treatment modality selected by the trainee—namely Acceptance and Commitment Therapy (ACT), Rational Emotive Behavior Therapy (REBT), Integrative Psychotherapy, or Intensive Short-Term Dynamic Psychotherapy (ISTDP)—in response to the clinical presentation enacted by the SP	7 min
7-Diagnostic interviewing	Skills in conducting a diagnostic interview using the SCID-5-CV	Conducting a structured diagnostic interview strictly adhering to the Structured Clinical Interview for DSM-5 Disorders–Clinician Version (SCID-5-CV) protocol, guided by the presenting symptoms and clinical phenomenology exhibited by the SP	7 min
8-Psychological assessment	Psychological assessment using one objective psychological test and one projective psychological test	Administration of an objective and/or projective psychological assessment instrument to the standardized patient (SP), selected in direct accordance with the clinical case and diagnostic hypothesis being evaluated	7 min
9-Rest	Offering refreshments to the students and allowing a rest period	Provision of a brief intermission or rest interval between stations/tasks	7 min

Before entering each station, students would read the relevant station guidelines posted at the entrance, becoming aware of the required task at each station before entering. Each station was staffed by one evaluator and one standardized patient, except for the assessment stations, where only an evaluator was present, and students were presented with a projective test card or an objective test psychological profile and performed the required task according to the station scenario. In each section of the examination, a faculty member served as an observer and executive manager of the examination, and two individuals, who were assistants or Master’s students, were present to guide students along the station route. Additionally, one person was responsible for managing the bell, which was timed using specialized software. It is worth mentioning that a mock run was held at the examination site the day before the actual examination to familiarize the evaluators and standardized patients with the examination process, address any potential issues, answer questions, and ensure the accurate timing of the bells.

2- Examination Implementation

The first clinical psychology OSCE was held on Sunday, September 9, 2024 (corresponding to Shahrivar 18, 1403 in the Iranian calendar), at the Faculty of Welfare and Social Health Sciences of the Islamic Azad University of Medical Sciences of Tehran, in two sessions, morning and afternoon. Given that the students had received training in four different therapeutic approaches during their internship – Acceptance and Commitment Therapy, Rational Emotive Behavior Therapy, Intensive Short-Term Dynamic Psychotherapy, and Integrative Psychotherapy – the examination was conducted separately for the respective therapy groups in 4 sections across the two morning and afternoon sessions. Students arrived at the examination site at the scheduled time. After depositing their personal belongings, they entered a waiting area and were called to the stations in groups of 9. The order of students’ participation in the examination process was determined randomly by software.

3- Evaluation

In the evaluation phase, the evaluators present at each station marked the pre-determined checklists based on each student’s performance. Subsequently, based on the score assigned to each item on the checklist and by summing the scores of all items on the checklist, they recorded the score for that section. Each student received a total of 4 scores from the four sections: assessment, diagnostic interviewing, hospital procedures, and treatment. Scoring in each section was out of 20 points, and ultimately, a score representing the overall average of the clinical psychology OSCE was announced to the students.

In addition to the examination scores, the students also answered 16 survey questions regarding their satisfaction with the examination. The questions used in the examination survey were adapted from the 32-item De Lisle questionnaire [27] (as cited in Pierre et al., 2004 [28]), which, although concise, examined three areas: implementation quality and examinees’ opinions regarding the reliability and validity of the test. This 16-item questionnaire had previously been used in several other domestic studies [29, 30] to assess satisfaction with the OSCE in various disciplines. The questionnaire used is adapted from standard OSCE instruments [27, 28], and is grounded in the Kirkpatrick Model for evaluating training effectiveness. It is a formative, three-option self-assessment tool designed to evaluate the content, structure, and organization of the OSCE through student feedback. The primary objective of this questionnaire is to measure students’ satisfaction and initial perception of the OSCE as a teaching/assessment tool, allowing for the identification of strengths and weaknesses to improve educational programs. Essentially, this is a formative assessment tool, not a diagnostic one, and it focuses on Level 1 of the Kirkpatrick Model (Reaction).

Since this questionnaire serves more as a qualitative-descriptive tool for feedback rather than a standard psychometric instrument measuring a latent trait, deep statistical analyses such as Cronbach’s alpha are not typically found in the primary source or secondary references. However, it has been utilized in numerous reputable studies to measure participant satisfaction in OSCEs, including Pierre et al. and De Lisle et al. [27, 28], and in Iran by Imani & Hosseini and Rafati et al. [29, 30].

Regarding structure, the questionnaire consists of 16 closed-ended questions based on a three-point Likert scale. The options are: “Completely Agree” (Score 3), “Somewhat Agree” (Score 2), and “Disagree” (Score 1). This simple scale is appropriate for descriptive analysis (percentage of agreement). The open-ended question, “What are your suggestions for improving the OSCE?“, was designed to collect qualitative data, which was then examined using thematic analysis.

Furthermore, students’ comments and suggestions regarding the examination, which were provided in written form as qualitative data, were analyzed using Graneheim and Lundman’s [31] content analysis method. The qualitative research question (Question 17) asked: “What is your opinion or suggestion for improving the test?” This question was designed to uncover latent themes, complement the quantitative data, and practically improve the examination.

To analyze the qualitative data, the approach outlined by Graneheim and Lundman [31] was followed through these steps:

1. Preparation: Transcribing responses into raw text, considering each response as a unit of analysis, and numbering the data.

2. Immersion: Reading the text repeatedly to achieve a deep understanding of the overall meaning.

3. Identifying Meaning Units: Extracting meaning units and separating phrases related to the objective, treating each unit as an independent idea.

4. Condensation: Shortening the meaning units without losing their core content.

5. Coding: Assigning a descriptive code to each condensed phrase that remained close to the text.

6. Categorization: Grouping similar codes into sub-categories.

7. Thematic Synthesis: merging sub-categories to form the main themes.

8. Reporting: Compiling and declaring the final report.

In this process, the students’ written texts were initially read, and then relevant meaning units were identified and coded. Finally, the codes were categorized based on their similarities and analyzed.

Six academic members analyzed and approved the scenarios and checklists’ material in order to evaluate the exam’s content validity. The overall Master’s GPA of the students was used to evaluate the criterion validity. Cronbach’s alpha coefficient and the correlation between the final exam score and the scores from each station were used to evaluate the exam’s dependability.

Regarding research ethics, all matters related to the confidentiality of individuals’ information, maintaining fairness in scoring, and the voluntary nature of participation in the post-examination survey were observed. All stages of the examination, including planning, implementation, and evaluation, were carried out under the supervision and approval of the General Education Office of the Islamic Azad University of Medical Sciences, and all examination processes were accompanied by the approval of esteemed experts in this field.

A summary of the key activities undertaken in the three stages of examination design, implementation, and evaluation is shown in Fig. 2.

Fig. 2.

Summary of activities undertaken during the three stages of design, implementation, and evaluation of the clinical psychology OSCE

Results

This study was conducted on 90 final-semester Master’s students in Clinical Psychology at the Faculty of Welfare and Social Health Sciences, Islamic Azad University of Medical Sciences of Tehran. Among these students, 20 (22.2%) were male and 70 (77.8%) were female. The descriptive statistics of the scores obtained from the clinical psychology OSCE are presented in Table 2. The descriptive findings include the minimum, maximum, mean, standard deviation, skewness, and kurtosis values for each variable. Considering the mean scores for each variable (except the rest station), (assessment = 17.44 (SD = 1.68), interview = 16.61 (SD = 2.98), hospital procedures = 17.18 (SD = 2.27), treatment = 17.84 (SD = 1.94), and final score = 17.26 (SD = 1.41)), the students’ performance appears to be at a satisfactory level. Regarding the standard deviations, the scores in the interview section (SD = 2.98) seem to have a wider distribution and greater dispersion from the mean. Based on the skewness and kurtosis values obtained for the variables, there is a possibility of non-normal data distribution, necessitating further investigation to determine normality more precisely. Therefore, we examined the normality of the data using both the Kolmogorov-Smirnov and Shapiro-Wilk tests. The results of these two tests indicated that the score distribution was not normal for any of the variables (Kolmogorov-Smirnov P < 0.05; Shapiro-Wilk P < 0 0.05). Thus, non-parametric statistics were used for subsequent analyses.

Table 2.

Descriptive statistics of scores obtained from each section of the clinical psychology OSCE

Variable	Minimum	Maximum	Mean	Standard Deviation	Skewness	Standard Error	Kurtosis	Standard Error
Assessment	13.00	20.00	17.44	1.68	-0.69	0.25	0.45	0.50
Interview	4.00	20.00	16.61	2.98	-2.17	0.25	5.73	0.50
Hospital Procedures	10.00	20.00	17.18	2.27	-0.60	0.25	0.07	0.50
Treatment	8.50	20.00	17.84	1.94	-1.60	0.25	4.81	0.50
Final Score	13.37	19.62	17.26	1.41	-0.71	0.25	0.18	0.50

Spearman’s correlations between the different sections of the examination (as shown in Table 3) indicate a significant positive correlation between assessment scores and the final examination score (r = 0.49, P < 0.001). Diagnostic interview scores show significant positive correlations with hospital procedures scores (r = 0.27, P = 0.008), treatment scores (r = 0.31, P = 0.003), and the final examination score (r = 0.70, P < 0.001). Hospital procedures scores have significant positive correlations with treatment scores (r = 0.22, P = 0.033) and the final examination score (r = 0.62, P < 0.001). Treatment scores also show a significant positive correlation with the final examination score (r = 0.64, P < 0.001).

Table 3.

Correlations between research variables

Variable	1	2	3	4	5
1. Assessment	-
2. Interview	0.18 P = 0.082	-
3. Hospital Procedures	0.18 P = 0.090	**0.27 P = 0.008	-
4. Treatment	0.20 P = 0.054	**0.31 P = 0.003	*0.22 P = 0.033	-
5. Final Score	**0.49 P < 0.001	**0.70 P < 0.001	**0.62 P < 0.001	**0.64 P < 0.001	-

**Significant at the 0.01 level

* Significant at the 0.05 level

To assess the content validity of the examination, the content of the checklists and scenarios was reviewed and approved by 6 faculty members. To assess criterion validity, the Spearman correlation coefficient between the final examination score and the students’ overall Master’s GPA was calculated, indicating a significant and appropriate correlation between them (r = 0.54, P < 0.001).

The correlations observed between the overall examination score and each of the stations (as reported above) also suggest internal consistency of the examination scores. Furthermore, to assess the reliability of the examination, Cronbach’s alpha coefficient was used, and the obtained alpha value was 0.68 (α = 0.68).

In supplementary analyses of the examination, we compared the final examination scores between male and female students. This comparison did not reveal a significant difference between the two groups (t = -0.16, P = 0.49). The mean final scores were 17.22 for male students and 17.28 for female students, indicating a close similarity between the scores of these two groups.

In the survey section assessing students’ satisfaction with the assessment aspects, implementation quality, and examinees’ perceptions of the examination’s validity and reliability, the results were as shown in Table 4. As observed, 88.5% of the examinees believed that the examination assessed a reasonable range of their professional knowledge. 80.7% rated the examination administration as good and well-organized. 80.7% considered this examination a suitable learning opportunity, and 65.4% believed that the examination identified students’ learning weaknesses. Notably, 92.3% of the examinees found the OSCE to be stressful.

Table 4.

Survey questions on the clinical psychology OSCE

Question Number	Questions	Agree (%)	Disagree (%)	No opinion (%)
1	The examination was fair.	42.3%	57.7%	0%
2	The examination covered a reasonable range of professional knowledge.	88.5%	11.5%	0%
3	The examination administration was good and well-organized.	80.7%	19.3%	0%
4	The examination was stressful.	92.3%	7.7%	0%
5	The OSCE was a suitable method for assessing students’ clinical competence.	53.8%	46.2%	0%
6	The examination identified students’ learning weaknesses.	65.4%	34.6%	0%
7	The examination covered a wide range of clinical skills needed for psychotherapy.	53.8%	46.2%	0%
8	Necessary information was provided to students before the examination.	38.5%	61.5%	0%
9	The tasks required at the stations were based on prior learning.	46.1%	53.9%	0%
10	The time allocated to each station was sufficient.	30.7%	69.3%	0%
11	The physical space for conducting the examination was appropriate.	65.3%	34.7%	0%
12	The instructions were unambiguous.	53.9%	46.1%	0%
13	The requested tasks were fair and appropriate.	46.2%	53.8%	0%
14	The sequence of stations was logical and appropriate.	73.1%	26.9%	0%
15	The examination was a learning opportunity.	80.7%	19.3%	0%
16	The performance of the simulated patients was appropriate.	57.7%	42.3%	0%

*Adapted from Pierre et al. (2004) [28]

Analysis of the qualitative data obtained from students’ comments regarding the psychology OSCE revealed both positive and negative aspects. Two main categories emerged from the positive data: 1- Comprehensive coverage of professional knowledge, and 2- Accurate and appropriate examination implementation. A significant number of students believed that despite their first encounter with this examination, its structure was generally suitable for assessing students’ professional knowledge regarding the diagnosis, evaluation, and treatment of patients. They also positively evaluated the examination administration and its clear organization and structure.

Four categories emerged from the negative data: (1) Lack of acquaintance with the conditions of the test and the requirement for additional practice tests before the test. Because this kind of evaluation was new to them, several students felt that even though they had been given instructions about the OSCE before the test, they still needed additional simulation exercises. (2) Some simulated patients’ performances were of poor quality (For example, some students stated that the standardized patients’ portrayal of the role was somewhat weak and artificial).

In contrast to actual clients, some students thought that some simulated patients’ performances were overly dramatic and contrived. 3. Not enough time at certain stations. Many students believed that additional time was required to apply therapeutic procedures at the stations linked to therapy. 4. The stressful nature of the test. Many students stated that they experienced a great deal of stress as a result of the OSCE.

Discussion

As highlighted earlier, assessment in the field of clinical psychology is of paramount importance due to its combined theoretical and practical nature, particularly considering the subjective and abstract aspects of the practical interventions involved. Traditional theoretical assessments in universities often fall short of comprehensively evaluating all the necessary practical skills in this discipline. Furthermore, the utilization of role-playing models in assessing clinical skills faces significant limitations, including a lack of structure and an inability to fully evaluate the various facets of students’ clinical competencies. This is despite the critical importance of skill assessment in clinical psychology due to its direct engagement with individuals’ mental health. Consequently, implementing an examination that can objectively, systematically, and structurally assess the diverse dimensions of clinical skills in situations resembling real-world scenarios is of substantial value. However, despite this significance, the implementation of the OSCE in psychology has been largely overlooked in Iran. This is in contrast to numerous countries, such as Australia [22, 32], Sweden [19], the United States [33, 34], and Spain [35], where the OSCE has been employed in clinical psychology and has reportedly been highly successful in evaluating clinical psychology students.

In the present study, we provided a descriptive report of the implementation of the first OSCE in clinical psychology by the Faculty of Welfare and Social Health Sciences of the Islamic Azad University of Medical Sciences of Tehran. We also presented the validity and reliability of the examination, the percentage of student satisfaction with it, and an analysis of their attitudes towards the examination. Consistent with the findings, the implemented examination demonstrated appropriate and reliable validity and reliability. The mean scores obtained across the various stations indicated satisfactory performance levels and a reasonable level of examination difficulty and student preparedness. Regarding the correlations between the scores of different stations, despite the presence of multiple correlations between various sections, the assessment section scores, which resulted from the administration and interpretation of objective and projective tests, showed relative independence and a weak correlation with other sections except for the final examination score. This could be attributed to the fact that in clinical practice, the assessment of clients through objective and projective tests is a skill somewhat distinct from the skills required for the diagnosis and psychological treatment of disorders, relying more on students’ knowledge of the items embedded in psychological tests. In contrast, the remaining sections of the implemented OSCE (interview, hospital procedures, and treatment) were primarily related to students’ practical skills in the field of therapy. On the other hand, the high correlation between interview scores and hospital procedures and treatment sections could reflect the reality that stronger therapeutic interventions are associated with more accurate initial diagnostic interviews. Similarly, the correlation between hospital procedures and treatment sections may indicate a relative overlap between the activities and skills involved in them. Given the obtained and desirable validity and reliability of the examination, it can be confidently stated that the OSCE effectively assessed students’ practical skills in a structured manner.

These findings are consistent with the results of studies related to the implementation of the clinical psychology OSCE in other countries [19, 36], as well as with the results obtained from the implementation of psychiatric OSCEs in Iran [37, 38], highlighting the significance of the obtained results.

The findings from the quantitative and qualitative analysis of students’ opinions regarding the implemented OSCE suggest that, firstly, the overall structure of the examination and its implementation were satisfactory to the students and, in some cases, even enhanced their prior knowledge. However, it appears that, given this was the first clinical psychology OSCE in Iran and students had no prior direct experience with this type of examination, participation was stressful for them. Additionally, students in some instances felt the need for more practice in simulated situations before the examination or found the allocated time, especially for implementing specific therapeutic skills and techniques, to be insufficient. This underscores the necessity of reviewing and addressing these issues for the optimal implementation of the examination in subsequent cycles.

Regarding the similarities and differences between the findings of the present study and previous research on the use of OSCEs in clinical psychology training, the following points can be highlighted: The psychometric properties and student perceptions of the study presented here are closely aligned with international evidence concerning OSCEs and simulation-based assessments in clinical psychology training, while also highlighting the specific challenges of a first-time implementation in a non-Western context. Moderate internal consistency (α = 0.68) and significant inter-station and station-to-total correlations (rₛ = 0.22–0.70), performance was appropriate (83–89% of max score) and criterion validity was strong (r = 0.54 with GPA), although reliable estimates in the present study were lower than established programs in Australia (α ≈ 0.70) [22, 36] and in the United States (α = 0.72, r = 0.52) [33, 34], and when examining the summary of studies provided globally (reliabilities from 0.60 to 0.75 and 0.30 to 0.70 correlations) [32]. The students agreed about the breadth of knowledge covered and the value of learning (80.7–88.5%) and were consistent in qualitatively describing the need for additional preparation time and the limitations of time in completing assessment stations, including simulated patients presenting as unnatural, which was consistent with all of the studies presented. However, the observed student reported much higher perceived examination-related stress (92.3%) than ranges indicated in literature (70–85%) and reported less adequate preparation time and time allocated to assessment and authenticity of the SP’s portrayal when modeled across samples of Western programs [22, 33, 34, 36]. These results provide supporting evidence regarding the cross-cultural consistency and adaptability of the OSCE methodology, while underlining the important influence of improved orientation and pilot simulations in ‘novel educational contexts’ [32].

Limitations and future research

Limitations of the conducted psychology OSCE include the limited experience of the simulated patients, the stressfulness of the examination, and the relatively limited time allocated to the treatment stations, which require greater attention in future OSCE implementations. However, it also has strengths, such as the ability to assess psychology students’ clinical practical skills in a structured and objective manner.

In light of the aforementioned limitations, it is recommended that in future administrations of the psychology OSCE, students be provided with more mental preparation before the examination, and more simulation exercises be incorporated whenever possible. Furthermore, other stressors for students should be carefully examined and analyzed, and strategies to reduce student stress should be provided. Finally, adjusting the location and duration of the examination also appears necessary.

Conclusion

The results of this research indicate that the implementation of the OSCE in clinical psychology, despite its difficulties and limitations in psychological domains, can, while possessing appropriate validity and reliability, also provide a very suitable opportunity for a comprehensive assessment of the various clinical skills of clinical psychology students. It can also enhance their specific learning while making them aware of their weaknesses in operational areas. Furthermore, the appropriate correlation between the examination scores and students’ final GPAs in their Master’s program suggests that this examination, like traditional written examinations that assess students’ knowledge, can effectively evaluate their practical skills in situations similar to real therapeutic settings. The present examination and report can serve as a suitable model for implementing the psychology OSCE in all universities that train clinical psychology students.

Abbreviations

OSCE

Objective Structured Clinical Examination

REBT

Rational Emotive Behavior Therapy

ACT

Acceptance and Commitment Therapy

ISTDP

Intensive Short-Term Dynamic Psychotherapy

GPA

Grade Point Average

SCID-5-CV

Structured Clinical Interview for DSM-5 Disorders: Clinician Version

Authors’ contributions

M.V. contributed to designing the study and management of the project. Z.A.H.S.T.T. contributed to the experimental procedures. M.P. contributed to the statistical analyses and manuscript writing. All authors critically reviewed the final version of the manuscript and approved it for submission.

Funding

Not applicable.

Data availability

The datasets used and analyzed during the current study are available from the corresponding author on reasonable request.

Declarations

Ethics approval and consent to participate

The Ethics Committee of Islamic Azad University of Medical Sciences of Tehran provided the following code of ethics: IR.IAU.TMU.REC.1404.285. All participants provided informed consent before participating in the study. The study was conducted in accordance with the ethical principles outlined in the Declaration of Helsinki.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.