Standardization of Clinical Skill Evaluation in Physical/Occupational Therapist Education –Effects of Introduction of an Education System Using OSCE–

Abstract

[Purpose] A major issue in physical/occupational therapist education is the improvement of students' clinical techniques. In this study, we introduced an education system using an Objective Structured Clinical Examination (OSCE), and made an attempt at standardization of its evaluation. [Subjects] The subjects were 227 students in the classes of 2008 to 2010 who enrolled at our university between 2004 and 2006, before the introduction of the education system using OSCE, and 221 students in the classes of 2011 to 2013 who enrolled between 2007 and 2009, after the introduction. [Methods] Performances in attitude and skills (performance in clinical training and OSCE) were compared between before and after the introduction of OSCE. OSCE results were compared between before and after clinical trainings at each OSCE Level; and the correlation of between performances in clinical training and OSCE was examined. [Results] Performances in OSCE and clinical training (attitude, skills) were improved by the introduction of the education system using OSCE, but no significant correlation was observed in the relationship between performances in OSCE and clinical training. [Conclusion] Further studies should be conducted aiming at the standardization of clinical skill evaluation in postgraduate education to establish an education system using OSCE.

INTRODUCTION

Currently Now, a variety of postgraduate educations programs are provided in clinical settings; however, it is hard to say that the education methods are well integrated, and the methodology for the evaluation of the clinical skills of novice therapists and for postgraduate education are still unclear. Therefore, we believe that early and objective evaluation of clinical skills and integration of postgraduate education and clinical teaching methods suitable for acquisition of their skills of great importance for the development are the most important for growth of novice therapists.

We consider physical and occupational therapies as “studies of treatment”, or clinical sciences. At the same time, we consider student education as professional education, or “education to cultivate specialists with clinical skills”. Based on these philosophies, we have implemented education focusing on clinical practice since the establishment of our department in 2004. In other words, we believe that a central issue in education is in clinical skill improvement. In current therapist education, however, teaching staff of training institutions does not evaluate the clinical skills of students, and the evaluation is entrusted to instructors in a practical training facilities. This is a crucial problem in therapist education that cultivates “clinical therapists”. Our teaching staff treats patients in clinical practices on a routine basis, and at same time, we always conduct student guidance while discussing with therapists in the field. At the university, we are also trying to standardize the evaluation conducted by instructors of clinical training by introducing an Objective Structured Clinical Examination (OSCE), which is an independent examination for therapists, to clarify the criteria for clinical education in the training institutions¹⁾. The OSCE is a clinical competency evaluation proposed by Harden in 1975²⁾. It has been reported that this approach is suitable for evaluating learning effects in the psychomotor and emotional domains, which are difficult to evaluate with written examinations.

To establish an education system using OSCE, therefore, we have started OSCE before the clinical trainings in 2005, and introduced OSCE exercises from 2008. In the OSCE exercises which students learn OSCE systematically to experience the attitude and skills required in clinical settings as a simulation to smoothly expedite proceed clinical trainings. Smoothly; and then OSCE were introduced after the clinical trainings in addition to OSCE before the clinical trainings sessions in 2007 (hereinafter referred to as the “education system using OSCE”).

We have demonstrated the correlation between performances in OSCE, at the university and clinical trainings in previous studies to date^{3, 4)}, and the aim of present study was standardize clinical skill evaluation using OSCE in the physical/occupational therapist education. We believe that the quality of rehabilitation in medical practices would be improved if standardization were established through evaluation in therapist education and treatment. The education system using OSCE was conducted with a phased approach to the attitude and skill aspects that are required in clinical settings: OSCE Level 1 (evaluation before and after early experiential training), acquisition of communication and assistance skills; OSCE Level 2 (evaluation before and after clinical training), acquisition of testing and measuring skills; and OSCE Level 3 (evaluation before and after advanced clinical training), acquisition of analytical and therapeutic skills (assistance and guidance). For validation, comparative analysis of the attitude and skill aspects of the performances in clinical training was conducted using students in the classes of 2008 to 2010 who enrolled between 2004 and 2006, that was before the introduction of the education system using OSCE, and those in the classes of 2011 to 2013 who enrolled between 2007 and 2009, that was after the introduction. Furthermore, the correlation of the attitude and skill aspects of students' performances in clinical training sessions between the 3rd and 4th years was investigated. We also report the correlation of between performances in clinical trainings and OSCE of the in classes of 2011 to 2013.

SUBJECTS AND METHODS

The subjects were 227 students in the classes of 2008 to 2010 who enrolled at our university between 2004 and 2006 and graduated within four years, and 221 students in the classes of 2011 to 2013 (class of 2011, 76 students; class of 2012, 82; and class of 2012, 63) who enrolled at our university between 2007 and 2009 and graduated within four years. Meanwhile, Performances in clinical psychiatry training of occupational therapists were excluded from this study.

The OSCE was conducted implemented by as follows: Several rooms (stations) were arranged, and one problem was set at each station. Examinees (students) took the examination with rotating among the stations following signals as instructed. At each station there were two evaluators and one mock patient, who were both teachers of physical or occupational therapy, were placed in each station. The test time at each station was 5 minutes, and the students received a feedback of 2 minutes at the end of each problem. The examination was conducted without giving any advance notice about the skills being problems tested. Performances at the university were the annual average score of each class. Evaluation was decided through discussion by the two evaluators and was used for OSCE performances because the concordance rate of the scores indicated by the two evaluators was low in some cases. For performances in clinical trainings, average scores of clinical trainings in the 3rd year and average scores of advanced clinical trainings in the 4th year were used.

Statistical processing was conducted using Predictive Analytics Software (PASW) Statistics 18.0. One-way analysis of variance and Tukey-Kramer's multiple comparison were used for comparison of averages among 3 or more groups in different clinical trainings sessions (the 1st, 2nd and 3rd terms of the 3rd year; the 1st and 2nd terms of the 4th year). Unpaired comparisons between two groups were performed for comparisons among the classes of 2008 to 2010, students before the introduction of the education system using OSCE (hereinafter referred to as the “pre-introduction group”), and the classes of 2011 to 2012 students after its introduction (the “post-introduction group”). One-way analysis of variance and Tukey-Kramer's multiple comparison were used for comparison of averages among three or more groups in different clinical trainings sessions for the post-introduction group. In addition, for the post-introduction group, paired comparison between two groups was performed for a comparison of grades before and after the clinical training at each OSCE Level, and unpaired comparisons among between three groups were used for a comparisons among three groups at a total of 6 time points, before and after a clinical training at OSCE Levels 1 to 3. Spearman's rank correlation used to evaluate the correlations among performances in attitude and skills in different clinical trainings sessions, and between performances in OSCE and clinical training. Scores rates for students' the performances in OSCE and clinical training were calculated; and the correlation with each performance as well as the relationship of performances in attitude and skills in the clinical trainings among clinical trainings to use the conducted collected different times were also examined using Spearman's rank correlation coefficient. The Consent regarding data used in this study was obtained from the study students, and this study was approved by the Ethical Review Board of Clinical Research of the Fujita Health University (10-121).

RESULTS

In comparisons between the groups before and after the introduction of the education system using OSCE, the performances in attitude of the pre-introduction group (n=227) were 78.2 ± 13.1% in the 1st term of the 3rd year, 81.7 ± 11.9% in the 2nd term of the 3rd year, 83.8 ± 10.3% in the 3rd term of the 3rd year, 83.5 ± 13.3% in the 1st term of the 4th year, and 83.3 ± 8.3% in the 2nd term of the 4th year. There was a significant improvement between the 1st and 2nd terms of the 3rd year (p<0.05). In the post-introduction group (n=221), the performances in attitude were 79.9 ± 10.2% in the 1st term of the 3rd year, 83.0 ± 9.8% in the 2nd term of the 3rd year, 84.8 ± 9.1% in the 3rd term of the 3rd year, 88.7 ± 8.7% in the 1st term of the 4th year, and 89.7 ± 8.3% in the 2nd term of the 4th year. There were significant improvements between the 1st and 2nd terms of the 3rd year, and between the 3rd term of the 3rd year and the 1st term of the 4th year (p<0.05). The performances in the 1st and 2nd terms of the 4th year in the post-introduction group were significantly higher than those of the pre-introduction group (p<0.05). The performances in skills of the pre-introduction group were 40.2 ± 23.7% in the 1st term of the 3rd year, 54.2 ± 18.4% in the 2nd term of the 3rd year, 59.7 ± 15.3% in the 3rd term of the 3rd year, 71.0 ± 15.2% in the 1st term of the 4th year, and 70.6 ± 17.0% in the 2nd term of the 4th year. There were a significant improvements between the 1st and 2nd terms of the 3rd year, between the 2nd and 3rd terms of the 3rd year, and between the 3rd term of the 3rd year and the 1st term of the 4th year (p<0.05). The performances in skills of the post-introduction group were 48.2 ± 23.7% in the 1st term of the 3rd year, 54.9 ± 9.8% in the 2nd term of the 3rd year, 59.1 ± 11.1% in the 3rd term of the 3rd year, 72.6 ± 12.1% in the 1st term of the 4th year, and 74.6 ± 11.6% in the 2nd term of the 4th year. There were significant improvements between the 1st and 2nd terms of the 3rd year, between the 2nd and 3rd terms of the 3rd year, and between the 3rd term of the 3rd year and the 1st term of the 4th year (p<0.05). The performances in the 1st term of the 3rd year and the 2nd term of the 4th year in the post-introduction group were significantly higher than those of the pre-introduction group (p<0.05) (Table 1).

Table 1. Comparison in performances of attitude and skills in clinical trainings between the pre- and post-introduction groups.

		1st term of the 3rd year		2nd term of the 3rd year		3rd term of the 3rd year		1st term of the 4th year		2nd term of the 4th year
		the pre-intro- duction groups	the post-intro- duction groups	the pre-intro- duction groups	the post-intro- duction groups	the pre-intro- duction groups	the post-intro- duction groups	the pre-intro- duction groups	the post-intro- duction groups	the pre-intro- duction groups	the post-intro- duction groups
attitude	average	78.2±13.1	79.9±10.2	81.7±11.9	83.0±9.8	83.8±10.3	84.8±9.1	83.5±13.3	88.7±8.7	83.3±15.0	89.7±8.3
	maximum	100.0	100.0	100.0	100.0	100.0	100.0	100.0	100.0	100.0	100.0
	minimum	37.5	46.0	45.0	54.0	55.0	50.0	10.0	61.8	10.0	56.0
	median	80.0	80.0	85.0	84.0	85.0	86.0	85.0	90.5	85.0	90.9
skills	average	40.2±23.7	48.2±23.7	54.2±18.4	54.9±9.8	59.7±15.3	59.1±11.1	71.0±15.2	72.6±12.1	70.6±17.0	74.6±11.6
	maximum	80.0	97.3	88.0	76.6	92.0	84.7	100.0	98.4	100.0	100.0
	minimum	0.0	17.9	0.0	33.3	0.0	28.0	18.0	39.3	0.0	30.7
	median	44.0	48.0	60.0	55.3	60.0	60.0	72.0	72.0	72.0	75.3

For the performance in attitude, there was a significant improvement between the 1st and 2nd terms of the 3rd year in the pre-introduction group (p<0.05). In the post-introduction group (n=221), there was significant improvement in the performance in attitude between the 1st and 2nd terms of the 3rd year, and between the 3rd term of the 3rd year and the 1st term of the 4th year (p<0.05). The performance of the 1st and 2nd terms of the 4th year in the post-introduction group was significantly higher than that in the pre-introduction group. (p<0.05). For the performance in skills, there were a significant improvements between the 1st and 2nd terms of the 3rd year, between the 2nd and 3rd terms of the 3rd year, and between the 3rd term of the 3rd year and the 1st term of the 4th year in the pre-introduction group (p<0.05). In the post-introduction group, there were significant improvements in the performance of skills between the 1st and 2nd terms of the 3rd year, between the 2nd and 3rd terms of the 3rd year, and between the 3rd term of the 3rd year and the 1st term of the 4th year (p<0.05). The performance of the 1st term of the 3rd year and the 2nd term of the 4th year in the post-introduction group was significantly higher than that in the pre-introduction group (p<0.05).

In comparisons of performances in attitude of students in different classes, in the group after the introduction of the education system using OSCE, the performances in attitude of the class of 2011 were 81.7 ± 10.0% in the 1st term of the 3rd year, 84.2 ± 10.3% in the 2nd term of the 3rd year, 86.1 ± 9.9% in the 3rd term of the 3rd year, 87.9 ± 9.3% in the 1st term of the 4th year, and 90.0 ± 8.9% in the 2nd term of the 4th year. There were a significant improvements between the 1st and 3rd terms of the 3rd year, and between the 1st term of the 3rd year and 2nd term of the 4th year (p<0.05). In the class of 2012, the performances in attitude were 79.1 ± 9.1% in the 1st term of the 3rd year, 82.2 ± 9.8% in the 2nd term of the 3rd year, 84.8 ± 8.3% in the 3rd term of the 3rd year, 88.7 ± 8.8% in the 1st term of the 4th year, and 90.1 ± 8.2% in the 2nd term of the 4th year. There were a significant improvements between the 1st and 3rd terms of the 3rd year, and between the 3rd term of the 3rd year and 1st term of the 4th year (p<0.05). In the class of 2013, the performances in attitude of the class of 2013 were 78.4 ± 11.5% in the 1st term of the 3rd year, 82.6 ± 9.1% in the 2nd term of the 3rd year, 83.5 ± 9.3% in the 3rd term of the 3rd year, 89.7 ± 8.0% in the 1st term of the 4th year, and 88.7 ± 7.9% in the 2nd term of the 4th year. There were a significant improvements between the 1st and 3rd terms of the 3rd year, and between the 3rd term of the 3rd year and 1st term of the 4th year (p<0.05). This class showed no significant improvements after all of the clinical training sessions. In the class of 2011, the performances in skills of the class of 2011 were 51.3 ± 9.3% in the 1st term of the 3rd year, 58.1 ± 9.6% in the 2nd term of the 3rd year, 61.7 ± 10.8% in the 3rd term of the 3rd year, 72.1 ± 11.0% in the 1st term of the 4th year, and 76.9 ± 11.8% in the 2nd term of the 4th year. The performance of this class was significantly improved between the 1st and 2nd terms of the 3rd year, between the 3rd term of the 3rd year and the 1st term of the 4th year, and between the 1st and 2nd terms of the 4th year (p<0.05). In the class of 2012, the performances in skills of the class of 2012 were 47.5 ± 10.5% in the 1st term of the 3rd year, 52.9 ± 8.7% in the 2nd term of the 3rd year, 57.1 ± 10.9% in the 3rd term of the 3rd year, 73.2 ± 12.2% in the 1st term of the 4th year, and 73.0 ± 11.2% in the 2nd term of the 4th year. The performance of this class was significantly improved between the 1st and 2nd terms of the 3rd year, and between the 3rd term of the 3rd year and the 1st term of the 4th year (p<0.05). In the class of 2013, the performances in attitude of the class of 2013 were 45.7 ± 11.9% in the 1st term of the 3rd year, 53.6 ± 10.2% in the 2nd term of the 3rd year, 58.6 ± 11.8% in the 3rd term of the 3rd year, 72.1 ± 13.4% in the 1st term of the 4th year, and 73.8 ± 11.9% in the 2nd term of the 4th year. The performance of this class was significantly improved between the 1st and 2nd terms of the 3rd year, and between the 3rd term of the 3rd year and the 1st term of the 4th year (p<0.05). This class showed no significant improvements after all of the clinical training sessions (Table 2).

Table 2. Comparison of performances in clinical trainings among students in different classes in the post-introduction group.

		class of 2011					class of 2012					class of 2013
		1st term of the 3rd year	2nd term of the 3rd year	3rd term of the 3rd year	1st term of the 4th year	2nd term of the 4th year	1st term of the 3rd year	2nd term of the 3rd year	3rd term of the 3rd year	1st term of the 4th year	2nd term of the 4th year	1st term of the 3rd year	2nd term of the 3rd year	3rd term of the 3rd year	1st term of the 4th year	2nd term of the 4th year
attitude	average	81.7±10.0	84.2±10.3	86.1±9.9	87.9±9.3	90.0±8.9	79.1±9.1	82.2±9.8	84.8±8.3	88.7±8.8	90.1±8.2	78.4±11.5	82.6±9.1	83.5±9.3	89.7±8.0	88.7±7.9
	maximum	100.0	100.0	100.0	100.0	100.0	96.0	100.0	100.0	100.0	100.0	100.0	96.0	98.0	100.0	100.0
	minimum	46.0	56.0	54.0	62.0	58.0	54.0	54.0	60.0	64.0	56.0	52.0	54.0	50.0	61.8	65.5
	median	84.0	87.0	88.0	90.0	91.0	78.0	84.0	86.0	90.0	92.0	80.0	84.0	86.0	90.9	90.9
skills	average	51.3±9.3	58.1±9.6	61.7±10.8	72.1±11.0	76.9±11.8	47.5±10.5	52.9±8.7	57.1±10.9	73.2±12.2	73.0±11.2	45.7±11.9	53.6±10.2	58.6±11.8	72.1±13.4	73.8±11.9
	maximum	82.0	76.0	84.7	94.0	100.0	97.3	73.3	79.3	94.7	100.0	73.8	76.6	82.1	98.4	97.3
	minimum	25.3	33.3	30.0	45.3	30.7	32.7	36.7	28.0	39.3	46.0	17.9	33.8	35.2	41.7	49.1
	median	50.7	58.0	62.7	72.0	78.7	45.3	51.7	59.7	72.3	73.0	43.4	55.2	58.6	71.3	74.4

In the class of 2011, performance in attitude was significantly improved between the 1st and 3rd terms of the 3rd year, and between the 1st term of the 3rd year and 2nd term of the 4th year (p<0.05). In the class of 2012, the performance was significantly improved between the 1st and 3rd terms of the 3rd year, and between the 3rd term of the 3rd year and the 1st term of the 4th year (p<0.05). In the class of 2013, the performance was significantly improved between the 1st and 3rd terms of the 3rd year, and between the 3rd term of the 3rd year and the 1st term of the 4th year (p<0.05). There was no significant difference among these classes at respective clinical trainings. In the class of 2011, performance in skills was significantly improved between the 1st and 2nd terms of the 3rd year, between the 3rd term of the 3rd year and the 1st term of the 4th year, and between the 1st and 2nd terms of the 4th year (p<0.05). In the class of 2012, the performance was significantly improved between the 1st and 2nd terms of the 3rd year, and between the 3rd term of the 3rd year and the 1st term of the 4th year (p<0.05). In the class of 2013, the performance was significantly improved between the 1st and 2nd terms of the 3rd year, and between the 3rd term of the 3rd year and the 1st term of the 4th year (p<0.05). There was no significant difference among these classes at respective clinical trainings.

Regarding the correlation of performances in attitude and skills between the groups before and after the introduction of the education system using OSCE, the contribution ratios of the pre- and post-introduction groups were 10.2% and 28.4% in the 1st term of the 3rd year, 21.5% and 40.1% in the 2nd term of the 3rd year, 21.9% and 46.5% in the 3rd term of the 3rd year, 30.1% and 39.5% in the 1st term of the 4th year, and 28.3% and 50.7% in the 2nd term of the 4th year, respectively (Table 3).

Table 3. Relevancy of performances in attitude and skills in clinical trainings between the pre- and post-introduction groups.

		the 1st term of the 3rd year		the 2nd term of the 3rd year		the 3rd term of the 3rd year		the 1st term of the 4th year		the 2nd term of the 4th year
the pre-intro- duction groups		0.102		0.215 *		0.219 *		0.301 *		0.283 *
the post-intro- duction groups		0.284 *		0.401 *		0.465 *		0.395 *		0.507 *
		the 1st term of the 3rd year		the 2nd term of the 3rd year		the 3rd term of the 3rd year		the 1st term of the 4th year		the 2nd term of the 4th year
		the pre-intro- duction groups	the post-intro- duction groups	the pre-intro- duction groups	the post-intro- duction groups	the pre-intro- duction groups	the post-intro- duction groups	the pre-intro- duction groups	the post-intro- duction groups	the pre-intro- duction groups	the post-intro- duction groups
attitude	80~	53.3	57.9	63.4	70.6	70.5	74.2	71.8	85.5	69.6	88.7
	60~79	37.9	37.6	31.3	27.6	27.8	24.4	23.8	14.5	24.7	10.4
	40~59	8.4	4.5	5.3	1.8	1.8	1.4	4.0	0.0	4.4	0.9
	20~39	0.4	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.4	0.0
	0~19	0.0	0.0	0.0	0.0	0.0	0.0	0.4	0.0	0.9	0.0
skills	80~	1.8	0.5	2.2	0.0	7.5	2.7	28.6	31.2	31.7	33.5
	60~79	27.8	20.8	48.5	30.3	48.5	48.4	53.3	55.2	49.8	54.8
	40~59	28.2	66.5	36.6	63.3	37.0	44.3	15.9	12.7	15.0	11.3
	20~39	21.1	11.3	5.3	6.3	4.8	4.5	1.8	0.9	2.2	0.5
	0~19	21.1	0.9	7.5	0.0	2.2	0.0	0.4	0.0	1.3	0.0

Contribution ratios of the pre- and post-introduction groups were 10.2% and 28.4% in the 1st term of the 3rd year, 21.5% and 40.1% in the 2nd term of the 3rd year, 21.9% and 46.5% in the 3rd term of the 3rd year, 30.1% and 39.5% in the 1st term of the 4th year, and 28.3% and 50.7% in the 2nd term of the 4th year, respectively.

In performances of OSCE conducted before and after clinical trainings sessions for the group after the introduction of the education system using OSCE, the performances at OSCE Level 1 conducted before the early experiential training were 78.1 ± 6.9% by the class of 2011, 75.3 ± 9.3% by the class of 2012, and 71.0 ± 12.8% by the class of 2013. The performance of the class of 2011 was significantly higher those of the classes of 2012 and 2013 (p<0.05). At OSCE Level 1, conducted after the early experiential training, the performances were 80.9 ± 9.2% by the class of 2012, and 78.9 ± 12.4% by the class of 2013. There was no significant difference was observed between the classes of class 2012 and 2013 (p<0.05). At OSCE Level 2, conducted before the clinical training, the performances were 76.6 ± 8.5% by the class of 2011, 66.2 ± 11.6% by the class of 2012, and 69.9 ± 8.5% by the class of 2013. The performance of the classes of 2011 was significantly higher those of the classes of 2012 and 2013 (p<0.05). At OSCE Level 2, conducted after the clinical training, the performances were 81.6 ± 7.4% by the class of 2011, 72.7 ± 10.0% by the class of 2012, and 84.1 ± 5.6% by the class of 2013. The performance of the class of 2013 was significantly higher those of the classes of 2011 and 2012 (p<0.05). At OSCE Level 3, conducted before the advanced clinical training, the performances were 67.0 ± 8.7% by the class of 2011, 62.2 ± 12.1% by the class of 2012, and 75.3 ± 7.0% by the class of 2013. The performance of class of 2013 was significantly higher than that of class 2011 and 2012, and the performance of the class of 2011 was significantly higher those of the classes of 2012 (p<0.05). At OSCE Level 3, conducted after the advanced clinical training, the performances were 75.0 ± 7.2% by the class of 2011, 76.7 ± 8.6% by the class of 2012, and 80.0 ± 7.6% by the class of 2013. The performance of the class of 2013 was significantly higher those of the classes of 2011 and 2012 (p<0.05). After the introduction of the OSCE education system, the OSCE performances conducted after clinical trainings sessions significantly improved as compared to the OSCE performances conducted before clinical trainings sessions all OSCE levels from Levels 1, 2, and 3 (p<0.05) (Table 4).

Table 4. Comparison of OSCE performances before and after clinical trainings among students in different classes of the post-introduction group.

	OSCE level 1		OSCE level 2		OSCE level 3
	before clinical trainings	after clinical trainings	before clinical trainings	after clinical trainings	before clinical trainings	after clinical trainings
class of 2011	78.1±6.9		76.6±8.5	81.6±7.4	67.0±8.7	75.0±7.2
class of 2012	75.3±9.3	80.9±9.2	66.2±11.6	72.7±10.0	62.2±12.1	76.7±8.6
class of 2013	71.0±12.8	78.9±12.4	69.9±8.5	84.1±5.6	75.3±7.0	80.0±7.6

At OSCE Level 1 conducted before the early experiential training, the performance of the class of 2011 was significantly higher those of the classes of 2012 and 2013 (p<0.05). At OSCE Level 1 conducted after the early experiential training, no significant difference was observed the classes of 2012 and 2013 (p<0.05). At OSCE Level 2 conducted before the clinical training, the performance of the class of 2011 was significantly higher those of the classes of 2012 and 2013 (p<0.05). At OSCE Level 2 conducted after the clinical training, the performance of class of 2013 was significantly higher those of the classes of 2011 and 2012 (p<0.05). At OSCE Level 3 conducted before the advanced clinical training, the performance of class of the 2013 was significantly higher those of the classes of 2011 and 2012, and the performance of the class of 2011 was significantly higher those of the classes of 2012 (p<0.05). At OSCE Level 3 conducted after the advanced clinical training, the performance of the class of 2013 was significantly higher that of class 2011 and 2012 (p<0.05). After the introduction of the OSCE education system, the OSCE performances conducted after clinical trainings were significantly better than compared with the OSCE performances conducted before clinical trainings at all OSCE levels from Levels 1, 2, and 3 (p<0.05).

A significant correlation was observed between the performances of OSCE Level 2 before a clinical training and the performances of the clinical training in the 1st term of the 3rd year in the post-introduction group (contribution ratio, 24.95%; p<0.05). No significant correlations were observed between the performances in the clinical training in the 3rd term of the 3rd year and the performances of OSCE Level 2 after a clinical training, between the performances of the clinical training in the 1st term of the 4th year and the performances of OSCE Level 3 before a clinical training, and between the performances of the clinical training in the 2nd term of the 4th year and the performances of OSCE Level 3 after the clinical training (Table 5).

Table 5. Relevancy between performances of clinical trainings and OSCEs in the post-introduction group.

		the performances of OSCE
		pre level 2	post level 2	pre level 3	post level 3
the performances of clinical trainings	the 1st terms of the 3rd year	0.2495 *
	the 3rd terms of the 3rd year		0.0665
	the 1st terms of the 4th year			0.0986
	the 2nd terms of the 4th year				0.1026

* p<0.05. A significant correlation was observed between the results of OSCE Level 2 before a clinical training and the performances in the clinical training in the 1st term of the 3rd year in the post-introduction group (contribution ratio, 24.95%; p<0.05).

DISCUSSION

Recently, the OSCE has been introduced into predominantly medical education and others fields as an examination that enables the objectively evaluation of clinical skills^5,6,7). The OSCE can evaluate the psychomotor, emotional and cognitive domains better than with existing written exams, and can clarify clinical competencies and inadequate aspects⁶⁾. In addition, Saitoh et al., reported that OSCE in the medical education enhanced student' acquisition of acquire basic abilities including the skills and attitude required of physicians, and also pointed out that it means that students acquire knowledge, skills and attitude, forming a trinity, following the introduction of the OSCE⁸⁾.

Therefore, we examined student' performances in results of attitude and skills in clinical training performances to establish an evaluation standard for the clinical education of therapists. As a result, the performances in clinical trainings (attitude, skills) were generally improved by the introduction of the therapist education system using OSCE. In addition, an increase of in the correlation of between skills and attitude in the clinical trainings was also observed.

In the comparison of performances in attitude and skills in clinical trainings between the pre- and post-introduction groups, both the attitude and skill aspects indicated significantly high values in the 2nd term of the 4th year in the post-introduction group. Therefore, we considered that this improvement was observed because the students experienced the actual clinical practices. Nakahara et al. reported that many exhibited by students in the rehabilitation field, including physical therapy and occupational therapy were related to the emotional domain and included immature social skills and fundamental attitude according to instructors in clinical trainings⁹⁾. Saitoh opined that teaching staff in the professional education field of rehabilitation should not be “a specialists of education” but “a specialists of rehabilitation medicine as well as a teachers of expertise” as the way is should be; and that students should learn from the teaching staff as an excellent specialists by imitating them¹⁰⁾. The results of our present study suggest, it is considered that OSCE skill exercises conducted to improve attitude and skills, which has been sought in clinical practices, before and after clinical trainings from the 1st term of the 3rd year successfully produced effective improved in the final attitude and skills in the 4th year following the establishment of the education system using OSCE. In the skill aspect, therefore, teaching staff should diligently educate reception and treatment skills, which are essential for clinical practice, more than ever through OSCE in the clinical practices.

Further analysis of the attitude and skill performances indicated improvement of attitude between before and after clinical trainings sessions; however, no differences were observed among students of different classes in the post-introduction group. This result for the attitude aspect may be due to a ceiling effect. Thus, we suggested that there is a need for further education for students with low performances in the future. In the aspect of skills, although no significant differences were observed among students of different classes, about 20% of improvement was observed after the clinical training and advanced clinical training conducted in the 3rd term of the 3rd year and the 1st term of the 4th year, respectively. We considered that the performances reflected the consistency of the education through the clinical trainings and guidance given at the university.

In the comparison of performances in attitude and skills in clinical trainings between the pre- and post-introduction groups, the correlation between the performances of attitude and skills of the post-introduction group became higher than that of the pre-introduction group in all 5 terms from the 1st term of the 3rd year to the 2nd term of the 4th year. In addition, there was a decrease in the number of decrease of students with low skill performances in the post-introduction group in the 5 terms from the 1st term of the 3rd year to the 2nd term of the 4th year. This improved level may have been the result of caused by retests conducted for students who failed the OSCE exam before clinical trainings in the post-introduction group. The correlation between attitude and skills was low in our previous studies^{3, 4)}; however, we considered that the improvement in attitude resulted in the high correlation between attitude and skills. It truly is “no skills without attitude”.

In the comparison of OSCE performances conducted before and after clinical trainings (the early experiential training, clinical training, and advanced clinical training) in the post-introduction group, the performance at the every OSCE level (level 1, 2, and 3) was better improved in the all of the respective classes. Especially, the OSCE performances showed the greatest improvement most increased in the class of 2013, and it was considered that this was because of repeated learning with innovation in the contents of the OSCE exercises as well as the use of model videos.

Performances in clinical trainings and OSCE in the post-introduction group showed low correlation in any terms (r=0.0665–0.2495), and problems remain left in this respect. The clinical training conducted in the 3rd term of the 3rd year contains therapeutic elements for an integrated training, and its relevancy to OSCE Level 2 might be low. OSCE performance in the 4th year increased, but it was not reflected into the performances students in the clinical training. This might be because practical clinical abilities other than OSCE, such as the therapeutic elements and cooperation with other professionals, are required at this stage, and this might be the cause of the low correlations.

As mentioned above, it seems that the integration of the education systems for therapists between our department (training institution) and practical training facilities is taking a shape form little by little. However, problems remain with the correlation of the left in relevancy in performances of clinical trainings and OSCE. Rehabilitation medicine is a practical system that brought learning of patients into the medical services. Therefore, one goal of therapist education aiming to create turns out specialists of learning is not to transplant a structure built for medical education into the therapist education, but to investigate the meaning of the OSCE education thoroughly again from the standpoint of the rehabilitation medicine and to establish its structure with several new ideas.

We are aiming to integrate our training institution with practical training facilities by incorporating evaluation items of OSCE into evaluation lists in clinical trainings. To further this aim, we introduced the contents of OSCE conducted in our department to instructors of clinical training from 2011 to integrate the evaluation of students. As mentioned above, we are aiming to integrate of the education systems for therapists between training institution and practical training facilities, something which no other therapist training institutions has not achieved in this country, by bridging the clinical skill evaluations between educations conducted at the university and the clinical training with OSCE.

We would also like to promote the use of OSCE for student education and standardization the clinical skill evaluation in the postgraduate education used to educate novice therapists by mid-career physical therapists and occupational therapists in practical training facilities in the future, to aim at further the establishment of the education system using OSCE.