Abstract
[Purpose] This study examines the relationship between the results of computer-based testing (CBT) and level of satisfaction with learning, school life, graduation research, and national examination results among freshman and sophomore undergraduate physical therapy students. [Subjects and Methods] The subjects of this survey were 56 male and 42 female physical therapy students who graduated from the International University of Health and Welfare, Ohtawara, in March 2017. The students were ranked according to four 25th-percentile groups based on the results of CBT, which was conducted at the end of freshman and sophomore years. A visual analog scale was used to assess satisfaction levels at the end of sophomore, junior, and senior years. The results of the national examination were scored independently. [Results] Compared with the freshman-year CBT results, we found a significant difference in learning satisfaction during the senior year and in the national examination. In addition, compared with the sophomore-year CBT results, there was a significant difference in learning satisfaction for sophomore, junior, and senior years, as well as in the national examination. [Conclusion] We found a link between the CBT results from freshman and sophomore years and those from the national examination. The results suggest that CBT has an educational effect.
Key words: Computer-based testing, Satisfaction with learning, National examination
INTRODUCTION
In 2005, common achievement tests, including Computer-Based Testing (CBT) and Objective Structured Clinical Examination (OSCE), were implemented on a full-scale in medical and dental schools across Japan. Their purpose was to evaluate whether clinical practice is able or unable by Common Achievement Tests Organization. The CBT evaluates knowledge, while the OSCE evaluates skills and attitudes1, 2).
Pharmaceutical Common Achievement Tests Organization were also implemented on a full-scale in medical schools in 20093).
On-campus education is important before partaking in clinical practice, because physical therapy, especially during senior year, combines physical examination with treatment. To our knowledge, objective criteria have not been established to evaluate whether students have the necessary knowledge, skills, and attitudes to participate. In addition, criteria have been depend on each school.
In 2010, the Common Achievement Tests Organization for Physical Therapy was established for rehabilitation.
We previously found that CBT results during junior and senior year were correlated with those from the national examination and practice examination created by vendors, both showed correlation coefficients of 0.42–0.60 (p<0.01)4).
In this study, we sought to further assess the relationship between CBT results in freshman and sophomore undergraduate physical therapy students, and the level of satisfaction with learning, school life, graduation research, and those from the national examination. Educational usefulness was also evaluated.
SUBJECTS AND METHODS
We included 56 males (age=22.0 ± 0.7 [years mean ± SD]) and 42 females (age=22.9 ± 4.1 years) physical therapy students who graduated from the International University of Health and Welfare in March 2017, and who agreed to participate in this survey. The study was approved by the institutional review board of the corresponding author’s University (Approval No. 2017-Io-122).
The CBT was used to operate by Common Achievement Tests Organization for Physical Therapy. The test lasted 60 minutes, and consisted of 50 questions, with each question worth one point. The scope of the CBT administered to freshmen comprised anatomy, physiology and kinesiology. For those in sophomore year, it included motor dysfunction disease, neurological disease and internal medicine. Students were ranked into four 25th percentile groups based on CBT results from the end of freshman and sophomore year.
The results of the national examination were scored by themselves based on answers from several update reports published after the completion of the national examination. The results were also ranked. The national examination consisted of 200 questions, including 100 on basic medical science and others on physical therapy. Practical problems testing clinical skills and knowledge included 40 questions in specialized field. Each question in them was worth three points. The total score for the national examination was 280 points.
The satisfaction level for learning and school life was assessed at the end of sophomore, junior, and senior year using a visual analog scale. The level of satisfaction with graduation research was assessed two weeks before the national examination. They were performed using a standard form based on previous studies5,6,7,8). The subjects indicated “not at all satisfied” on the left end or “extremely satisfied” on the right end of 100-mm straight line. These sessions were conducted by the lead author. No specific explanation was provided to students regarding the level of satisfaction, because the degree of satisfaction was subjective.
The level of satisfaction and results of the national examination were analyzed using the Kruskal-Wallis test in the four percentile groups. Statistical analysis of the results was performed using the SPSS for Windows 22.0 (IBM Corporation). P values <0.05 indicated statistical significance. The rank in the top and bottom group of the CBT was compared with that of the national examination.
RESULTS
Level of satisfaction and national examination results from the four CBT percentile groups are shown in Table 1. We found a significant difference in the level of learning satisfaction from the CBT results during freshman year, compared with those during senior year and national examination results. No significant difference was found for the other items. Compared with CBT results from sophomore year, the level of learning satisfaction was significantly different in sophomore, junior, and senior year, as well as from the national examination results. No significant difference was found in other items. Among 31 students ranked between 1st and 25th place on the CBT freshman year, 67.7% ranked in the top 25 and 3.2% ranked below 76th place on the national examination. Of the students who ranked below 76th place, 34.6% again ranked below 75th place for the national examination and 7.7% ranked within the top 25 for the national examination. Of the students who ranked between 1st and 25th place for the CBT in sophomore year, 52.0% ranked within the top 25 for the national examination and 5.6% ranked below 76th place for the national examination. However, of the 18 students who ranked below 76th place, 33.3% ranked below 75th place for the national examination and 16.7% ranked within the top 25 for the national examination.
Table 1. The level of satisfaction and the national examination results in four CBT percentile groups.
| Group | 1st–25th | 26th–50th | 51th–75th | below 76th | Total |
| CBT in freshman year | n=31 | n=18 | n=23 | n=26 | n=98 |
| Level of satisfaction with | |||||
| learning in sophomore year | 56 ± 22 | 54 ± 18 | 51 ± 19 | 42 ± 23 | 51 ± 22 |
| school life in sophomore year | 64 ± 20 | 64 ± 22 | 74 ± 13 | 64 ± 18 | 66 ± 19 |
| learning in junior year | 58 ± 21 | 59 ± 18 | 48 ± 23 | 52 ± 19 | 54 ± 21 |
| school life in junior year | 66 ± 20 | 72 ± 14 | 71 ± 20 | 65 ± 20 | 68 ± 19 |
| learning in senior year* | 76 ± 10 | 70 ± 15 | 68 ± 18 | 59 ± 18 | 68 ± 16 |
| school life in senior year | 73 ± 18 | 76 ± 14 | 80 ± 12 | 67 ± 23 | 74 ± 18 |
| graduation research | 72 ± 20 | 76 ± 16 | 73 ± 12 | 63 ± 30 | 71 ± 21 |
| National examination* | 229 ± 12 | 212 ± 13 | 219 ± 14 | 212 ± 17 | 219 ± 16 |
| CBT in sophomore year | n=25 | n=31 | n=24 | n=18 | n=98 |
| Level of satisfaction with | |||||
| learning in sophomore year* | 58 ± 15 | 45 ± 16 | 48 ± 20 | 39 ± 26 | 51 ± 22 |
| school life in sophomore year | 70 ± 21 | 64 ± 19 | 69 ± 18 | 63 ± 16 | 66 ± 19 |
| learning in junior year* | 66 ± 18 | 55 ± 18 | 49 ± 20 | 45 ± 24 | 54 ± 21 |
| school life in junior year | 72 ± 20 | 67 ± 18 | 68 ± 20 | 65 ± 17 | 68 ± 19 |
| learning in senior year* | 79 ± 8 | 65 ± 18 | 65 ± 14 | 65 ± 19 | 68 ± 16 |
| school life in senior year | 79 ± 14 | 70 ± 20 | 74 ± 18 | 72 ± 18 | 74 ± 18 |
| graduation research | 75 ± 16 | 72 ± 19 | 69 ± 22 | 65 ± 28 | 71 ± 21 |
| National examination* | 228 ± 11 | 218 ± 13 | 215 ± 17 | 214 ± 18 | 219 ± 16 |
*p<0.05, mean ± standard deviation.
DISCUSSION
We found that CBT results from freshman year were associated with the level of learning satisfaction in the senior year before the national examination. CBT results from sophomore year were also associated with the level of learning satisfaction in three years after the sophomore.
A significant difference was seen in national examination results between CBT results from freshman and sophomore years. The top group had more than 10 points than the other groups. Questions on the CBT administered freshman year included basic medical knowledge such as anatomy, physiology, and kinesiology, while the one administered to sophomore included topic such as motor dysfunction disease, neurological disease and internal medicine. The knowledge evaluated on the CBT reflected national examination results. It further suggests that administering the CBT during freshman and sophomore year has an educational effect.
Tomiki et al.9) reported that the results of the CBT conducted during the second half of 4th year were linked to graduation examination results. In particular, 50% of the students who ranked below 76th place on the CBT, also ranked below 75th place on the graduation examination. It should be noted that students cannot prepare for the CBT in a short period of time (e.g. overnight). CBT results predicted students’ attitude for learning and graduation examination results. They suggested a greater need for educational intervention among students who ranked lower. CBT results may confirm a fixing of daily learning and the impact of the level of satisfaction with learning.
Among students who ranked between 1st and 25th place on the CBT, 50% or more ranked in the top 25 on the national examination, and 6% or less ranked below 76th place on the national examination. However, of the 18 students who ranked below 76th place, 33.3% also ranked below 75th place on the national examination and 16.7% ranked in the top 25 for the national examination. We believe that CBT results during freshman and sophomore year trended to be fixed. However, no significant difference was found in the level of satisfaction with school life and graduation research, compared CBT results during freshman and sophomore year. We acknowledge that the main limitation of this study is that the subjects were only from one campus, and were not a large cohort. Future studies should include a wider range of subjects, as well as other schools.
Conflict of interest
The authors report no conflicts of interest in this work.
REFERENCES
- 1.Common Achievement Tests Organization (CATO): “Common Achievement Tests” before beginning clinical training (2012), 9th ed. Ishigakukyouikushisutemukenkyusenta, 2011 (in Japanese). [Google Scholar]
- 2.Nakaya H: Medical education in japan and national medical practitioner examination. Igakukyouiku, 2015, 46: 14–17 (in Japanese). [Google Scholar]
- 3.Irie T: Pharmaceutical common achievement tests. Pharm Soc Jpn, 2016, 52: 653–655 (in Japanese). [Google Scholar]
- 4.Kubo A, Maruyama H, Kuramoto A T, et al. : Relationships between scores of computer-based testing for PT students and the self-marking of the results after the national examination for physical therapists. Rehabilitation Kyouiku Kenkyu, 2014, 19: 210–211. (In Japanese). [Google Scholar]
- 5.Kubo A, Tani H, Kobayashi K, et al. : Level of satisfaction with study and life in the first four months after entering a university physical therapy department. Rigakuryoho Kagaku, 2016, 31: 625–628 (In Japanese). [Google Scholar]
- 6.Kubo A, Han H, Sadakiyo K, et al. : Second-year physical therapy undergraduates’ satisfaction with learning and life at the end of first and second term. J Int Univ Health Welf, 2016, 21: 28–30 (In Japanese). [Google Scholar]
- 7.Kubo A, Kurosawa K, Maruyama H: Changes in QOL of undergraduate physical therapy students after the great east japan earthquake: satisfaction with life and learning at the end of the junior year of college. Rigakuryoho Kagaku, 2014, 29: 1007–1009 (In Japanese). [Google Scholar]
- 8.Kubo A, Kuramoto AT, Kobayashi K, et al. : Influence of satisfaction with learning and living at the end of the junior and senior college years of PT students on the results of the national examination for physical therapists. Rigakuryoho Kagaku, 2015, 30: 115–117 (In Japanese). [Google Scholar]
- 9.Tomiki Y, Dambara T, Okada T, et al. : Do academic rankings based on common achievement tests predict the results of graduation examination results? Juntendou Med J, 2012, 58: 431–435 (In Japanese). [Google Scholar]