Evaluation of midwifery students’ competency in providing intrauterine device services using objective structured clinical examination

Abstract

BACKGROUND:

Delivering IUD services is one of the important competencies that midwifery students must obtain during academic period. As Objective Structured Clinical Examination (OSCE) can be reasonably reliable, valid and objective method for clinical skills assessment, this study was conducted to assess midwifery students’ skill in delivering intrauterine device (IUD) services using a clinical examination and their satisfaction from the OSCE.

METHODS:

All of the 62 eligible Bachelor of Science midwifery students of Mashhad University of Medical Sciences participated in a ten-station OSCE about delivering IUD services for 50 minutes in 2006. Students performed technical skills or interacted with standard patients in 6 stations and in 4 stations they answered to the related questions. Students’ performance in 6 stations was rated by observer or standard patients using validated checklists. Students’ level of satisfaction and also their experience of participating in OSCE examination were gathered.

RESULTS:

Performance of 98.2% of students was poor. On average, the students gained 49% of total score in counseling and screening, 35.7% in inserting the IUD, 40% in IUD removal and 24.4% in management of IUD side effect. Eighty percent of students rated their satisfaction from the OSCE high and very high. Students reported the OSCE as an enjoying examination experience.

CONCLUSIONS:

Students’ skill in delivering IUD services was lower than expected level that shows the need to change the current teaching methods. OSCE is a valid evaluation method which provides valuable information which cannot be obtained by more traditional assessment modalities. Based on the finding of this study a workshop program on providing IUD services for midwifery students and family planning providers should be prepared.

Assessment of clinical skills has a central role in medical education and the selection of suitable methods has been a matter of permanent concern for clinical teachers, course directors and medical educators.¹ Proper selection of a method of assessment can improve students’ performance. Faulty methods of assessment can lead to wrong decisions that might be deleterious to the future activities of students. Thus assessment techniques appear to have an impact on study strategies and to influence the performance of students.² Clinical competencies can be measured using a variety of methods: written tests, computerized tests, records of performance, different type of simulations, job samples, patient management program, essays, objective structured clinical examination (OSCE) and so on.³,⁴ The conventional clinical and practical examinations are beset with several problems. The correlation coefficient between marks awarded by different examiners for the same candidate performance is low. The marks also reflect only the global performance of the candidate and are not based on demonstration of individual competencies. Attitudes are usually not tested at all. Even in clinical skills, often the student is questioned only regarding his final conclusion. The ability to examine a patient and arriving at that conclusion is not observed by the examiners. In addition, there is no significant feedback to the candidate.^5–7 These defects of clinical and practical examinations have been realized for long and have given rise to attempts at improving the current scenario.⁵ Based on the pyramid of competence which introduced by Miller, different facets of competency can be assessed by different type of assessment methods.⁴,⁸ But production of a valid assessment methods which measure all facets of clinical competencies has remained an unsolved challenge. For this reason assessment research has focused on developing valid ways of assessing the actual performance over the last four decades.⁴ As a potential solution, OSCE has gained increasing importance.⁴,⁹ The OSCE is now a commonly used method of assessing clinical competence of medical students and practical physicians in such areas as history taking, physical examination, diagnostic reasoning and management and is now established as one of the most valid, reliable and effective tests for the assessment of clinical skills.¹,²,⁶,¹⁰,¹¹ Gilson et al. found OSCE a reliable and valid test for evaluating medical students’ competence in primary health care of women.¹² As a result, in the late 20^th century, OSCE was said to represent the ‘gold standard’ for medical student assessment.¹³ It can be used to measure some clinical skills that other tests do not permit.¹⁴ It fulfills all the criteria of an ideal method of assessment of clinical competence.² OSCE is more accurate than traditional methods previously utilized¹ and provides information that is not obtained by more traditional modalities.¹² Delivering intrauterine device (IUD) services is one of the important competencies that midwifery students should have at graduation.¹⁵ Sure, safe and effective use of IUD requires competent and well trained health care providers.¹⁶ Studies show that many of students do not practice necessary competencies, so their skill is insufficient at the graduation.¹⁷,¹⁸ Abraham and McBride found that 69% of final year medical students had not handled an IUD.¹⁹ Academic teachers try to assess clinical competency with valid and reliable test methods to issue licensure to those who will be entering the workforce. Because the OSCE has great scope to adequately test students’ competency¹¹ and has not been conducted in the university so far, this study was conducted to assess midwifery students’ skill in delivering IUD services using OSCE and their satisfaction from it.

Methods

In this descriptive research, 62 midwifery students of 2^nd to 4^th semesters of discontinuous bachelor program in Mashhad University of Medical Sciences participated in the study in 2006. All participants were healthy without any history of severe stresses and no IUD insertion during the past 6 months. Data on personal, demographic and educational characteristics gathered using self-report questionnaires. Students’ cognitive, interactive and technical skills in providing the IUD services were assessed through an objective structured clinical examination (OSCE) constitute of 10 stations that were hold for the purpose of this study in the School of Nursing and Midwifery. Two multiple choice questionnaires and 8 checklists were prepared to be used in the OSCE examination. Checklists mostly were developed from the checklists introduced by JHPIEGO²⁰ and rated as satisfactory, unsatisfactory and not observed. The content validity of checklists and the questionnaires were confirmed by 10 faculties from midwifery, nursing, medical education, education and psychology disciplines. Using alpha Cronbach coefficient test, the reliability of OSCE was 0.81 and reliability of 8 checklists applied in stations 3 to 10 were between 0.68 and 0.83. Using split-half methods, the reliability of 2 multiple choice questionnaires were 0.82 and 0.84. The time of each station was 5 minutes. All necessary instruments and instructions for students in each station were already provided. Stations 1 and 2 were grouped stations. In each of them one related questionnaire (constitute of 11 items in each station) was filled by the students about applied knowledge on the IUD and infection prevention in IUD services. In the stations 3 to 10, students performed requested skills and their performances were assessed based on the related rating scales filled by observers or standard patients. Standard patients were selected among the expert midwives and post graduate students employing in the family planning units of health centers. They were trained for the participation in the study. The observers and standard patients did not change in each station during the examination. In station 3, students’ skills in screening and counseling with a client who chose the IUD was evaluated and rated by standard patient using related checklists (constitute of 9 items). In the stations 4, 6, 7 and 9 the students’ technical skills including pelvic examination, loading the IUD in the sterile package with no touch technique, IUD insertion and IUD removal were assessed respectively. Four observers rated students’ performance in these 4 stations using related rating scales (constitute of 16,10,19,15 items in each station respectively). In the station 5 students recorded the results of their pelvic examination in a client's dossier. In the station 8 their performance in post-insertion counseling was assessed and rated by standard patient using related checklists (constitute of 7 items). In the station 10, there was the dossier of a new user of IUD with complain of bleeding, in which her medical history, findings of her pelvic examination and laboratory tests were recorded. The students were asked to review dossier records, response to the related questions, give a diagnosis and decide about necessary interventions. Students’ records and responses in stations 5 and 10 were rated by two faculty staff using related checklists (constitute of 11 and 6 items in each station respectively). Students then filled a questionnaire about their satisfaction that contained 6 items rated from 0 to 4. Students’ point of view and experience about the OSCE was gathered in a focus group cession after the end of the OSCE. Descriptive statistics for overall variables were reported. Statistical analysis was performed using SPSS for windows (version 13). Average values were reported as mean (Standard deviation).

Results

In this study the mean age of students was 25.25(2.46) years. History of work in health centers was reported by 49.8% of students. The students had passed 1.98(2) units of theoretical and 1.79(0.7) units of clinical courses in mother and child health before the entrance to the study. In average, they had performed 2.47(1.7) IUD insertion under supervision and 7.98(14.99) IUD insertion alone. In OSCE 98.2% of students had poor skills in delivering IUD services. The mean of students’ skill score in delivering IUD services was 74.06(10.11) out of 202 (36% of total score).

As figure 1 show, students gained 39% of total score in station 1 related to knowledge about the IUD, 49% of total score in station 2 on the infection prevention, 49% of total scores in station 3 that was related to counseling and screening, 37% in station 4 on pelvic examination, 60% in station 5 regarding the record of their pelvic exam results, 10% of total score in station 6 on loading the IUD, 35.7% in inserting the IUD in station 7, 49% in station 8 on post-insertion counseling, 40% in station 9 related to IUD removal and 24.4% in station 10 in management of IUD's side effect. Students’ performance in some of the key actions in delivering IUD was as follows. In station three, 55.4% of students did not make attention to the counseling about other types of contraceptive methods and 26.3% of them had unsatisfactory performance in this regard. More than 70% and 64.3% of students had poor performance in client screening and assessing client’ knowledge about IUD’s major side effects respectively.

Figure 1.

The percent of total scores gained by students in each of stations 1-10 of OSCE in providing IUD services

In station four, 67.9% of students did not check if client has recently emptied her bladder and washed her genital area if necessary. In addition, 94.6% of students did not check for lower abdominal especially supra-pubic tenderness and mass or other abnormality. Moreover, 96.4% of students had unsatisfactory performance in assessing external genitalia and 60.7% of students could not remove speculum well. In station six, 98.2% of students did not check the expire date of the IUD and 91.1% of them did not make attention if the package of IUD has remained undamaged. In station 7 almost 60% and 68% of students had unsatisfactory performance in applying antiseptics for disinfecting cervix and vagina and grasping cervix with tenaculum respectively; 98.2% of them could not detect the position of uterus and depth of uterine cavity correctly; 64.3% of students had unsatisfactory performance in releasing arms of IUD using withdrawal technique; 53.6% of students did not push in on the inserter tube until slight resistance is felt; 92.9% of students did not examine cervix if there is bleeding at the tenaculum puncture sites and the number of students which did not check to be sure that client is not having excessive cramping and did not answer to her question was 91.1%.

In station eight, 94.6% of students had unsatisfactory performance in providing instructions about follow up visits. In station 9 after IUD removal, 96.4% of students did not counsel about a new contraceptive method. Furthermore, 80% of students rated their satisfaction from the OSCE high and very high. The mean of satisfaction scores was 18.25(3.64) out of 24. In addition, 82.2% of students had strongly recommended OSCE should be used for assessment of their other skill too and 80% of students reported OCSE as a valid test for assessment of their skill.

In a focus group hold after the end of the examination, students reported that OSCE was enjoying examination experience that provide rapid feedback for them, with a lower anxiety compared to examinations hold in real situations and without any fear of harm to the clients. They mentioned that they need such a structured examination for evaluating their other competencies. They also believed that measurements based on traditional observational are less reliable as the examinees are often evaluated by different examiners encountering with different clients. They also pointed to the enough time of each station and believed also that OSCE can cover all aspects of their skills.

Discussion

In our study, 98.2% of students had poor skills in delivering IUD services. In 9 of 10 stations, students gained less than 50% of total skill's scores. In part, this finding may be related to the low quality of students’ education in the associated degree of midwifery. Previous studies in other countries, also, have shown a lack of adequate training in contraception. For example, studies in US showed 38% of graduating in OB chief residents and 66% of Family practice residents had no experience in IUD.²¹

Steinauer et al. also mentioned that most doctors who graduate from family practice programs in the US receive little or no clinical training in reproductive health care.²² In the study of Cheng et al. 50% of final year residents in FP and 20% of OB residents had never inserted an IUD. All of FP residents and 80% of OB residents had not inserted more than 10 IUD.²³ This finding is important because in some parts of the country, many of the family planning services in health centers are delivered by graduates of associate degree of midwifery. Previous studies showed that providers who have minimal experience in placing IUD did not recommend it to family planning clients and did not insert the IUD.¹⁷ This may be a cause of the low utilization of IUD in our country.

In our study, the score of students in loading the IUD in the package was low; may be due to the fact that it was not included in their formal education previously. In addition, students’ skill in management of IUD's side effects was low. It shows the need for more attention to this important aspect of delivering IUD services. As imperfect management of IUD's side effects can lead to negative attitude of IUD users that finally can lead to reject of this method by family planning clients. In other stations, the scores was between 35.7% and 60% that was not sufficient. These results is consistent with findings of previous studies and show the need for improving the teaching and learning methods in educating the IUD providing services to midwifery students.

Most of the students reported OCSE as a valid test for skill evaluation and recommended OSCE should be used for assessment of their skill. In the study of Kurz et al. the use of OSCE was contributed with high students’ satisfaction.²⁴ Students also stated that OSCE is an enjoying examination experience with a higher reliability compared to traditional methods of skill evaluation. Levine and Swartz also mentioned that traditional bedside oral examination that uses direct observation by expert physician examiners as a natural context for assessing clinical performance are often unreliable and point to some causes such those mentioned by our participants.²⁵ Nalesnik et al. also showed that an OSCE in the obstetrics and gynecology clerkship can be administered with limited resources and applied a valuable assessment method.²⁶

Although the use of an OSCE requires a precise evaluation program designing and a great deal of efforts for coordinating personnel and also time, financial and facility resources, it can provide a comprehensive evaluation of students’ skills needed for providing IUD services and measuring if the students has acquired the necessary related skills. Thus, even with limited resources, an effective OSCE can be incorporated into the students’ clinical courses and workshops curriculum. We designed and used the OSCE as a suitable method for evaluation of students’ competency in IUD providing services and applied it with purpose of educational need assessment. Based on the finding of this study, a workshop program on providing IUD services for midwifery bachelor students and family planning providers was prepared. The use of our designed examination program and its validated checklists is suggested to the future researchers and staff who examine the students’ family planning skills.

The authors declare no conflict of interest in this study.