Table 2.
Articles (n = 47) evaluating the prescribing competencies of final‐year medical students
Author | Setting | Study design | Participants | CPT topic | Methods | Main results | Kirkpatrick level c | MERSQI‐score a |
---|---|---|---|---|---|---|---|---|
Abbo et al. (2013) 23 | USA; 3 medical schools | Single‐group, cross‐sectional descriptive | 317 final‐year (4th) medical students | Antimicrobial prescribing and resistance | 24‐point self‐administered electronic survey instrument about perceptions, knowledge and attitudes to antimicrobial use and resistance, including 11 clinical vignettes | 92% agreed that knowledge of antimicrobials is important. Mean correct knowledge score was low (51%; max. 100%). Only one‐third perceived their preparedness to be adequate in some fundamental principles of antimicrobial use. 90% wanted more education on appropriate use of antimicrobials | 1, 2b | 12 |
Abubakar et al. (2015) 30 | Malaysia: one medical school | Multiple groups, cross‐sectional descriptive | 37 penultimate‐(4th) year and 50 final‐year (5th) medical students | Pharmacovigilance | 25‐point self‐administered written questionnaire about knowledge, attitudes and practices regarding ADRs and pharmacovigilance | 78% agreed that ADR reporting is a professional obligation. 59% recognized the right functions of pharmacovigilance. Only 21% knew the method used to report ADRs; 47% had never received any training in how to report ADRs | 1, 2b | 11 |
Agrawal et al. (2013) 31 | India; one university | Multiple groups, cross‐sectional descriptive | 102 2nd‐year and 98 final‐year (5th) medical and dental students | Contraception | 55‐point self‐administered written questionnaire about knowledge, attitudes, and barriers to practice regarding oral contraceptive pills | Average knowledge score was low (16.7; max. 30); final‐year students had higher knowledge scores compared with 2nd year students. Most students had negative perceptions and attitudes to oral conceptive pills | 2b | 8.5 |
Al‐Wadaani (2013) 87 | Saudi Arabia; one medical school | Single‐group, cross‐sectional descriptive | 73 final‐year (6th) medical students | Cardiovascular | 22‐point self‐administered written questionnaire about knowledge, attitudes and practices regarding diabetes and diabetic retinopathy | Overall knowledge, attitude and practice scores were moderate (64.8; max. 100). Students lacked knowledge about the follow‐up of diabetic patients for the screening of diabetic retinopathy. 90% of students did not know the proper angle of insulin injection | 2b | 10.5 |
Awad et al. (2012) 32 | Kuwait; one university | Multiple groups, cross‐sectional descriptive | 54 junior and 68 senior (4th–7th‐year) medical and pharmacy students | Complementary and alternative medicines | Self‐administered written questionnaire about knowledge, perceived effectiveness and barriers to complementary and alternative medicine | Medical students had poor knowledge of complementary and alternative medicine. Over two‐thirds had positive attitudes. Lack of trained professionals and lack of scientific evidence were the most common perceived barriers | 2b | 9 |
Azeem et al. (2010) 80 | Pakistan; one university hospital | Single‐group, cross‐sectional descriptive | 105 medical graduates | General prescribing | Self‐administered written questionnaire about perceptions regarding preparation and competency to prescribe as junior doctors | 19% considered themselves competent to prescribe at the time of graduation. <35% felt comfortable in providing information about treatment options to patients. The majority (59–95%) indicated a lack of formal teaching and practice at skills relating to drug therapy | 1, 2a | 6.5 |
Beyhun et al. (2016) 85 | Turkey; one medical school | Single‐group, cross‐sectional descriptive | 148 final‐year (6th) medical students | Narcotics | Self‐administered written questionnaire about awareness of synthetic cannabinoids | Students’ mean awareness score was moderate (4.7; max. 9). Students lacked awareness of usage, illegal status and clinical effects of synthetic cannabinoids | 2b | 8.5 |
Brinkman et al. (2017) 10 | Europe; 17 medical schools from 15 countries | Single‐group, cross‐sectional descriptive | 895 final‐year (5th–6th) medical students | General prescribing | Online assessment of prescribing knowledge (24 MCQs) and skills (5 patient cases). Self‐administered questionnaire about self‐reported confidence, perceived preparedness for prescribing, and received CPT education | Overall knowledge score was moderate (70%, max. 100%). 46% of the therapies were inappropriate and 55% of the prescriptions contained an error. Students lacked confidence in essential prescribing skills and felt not adequately prepared for prescribing responsibilities. >60% were not satisfied with the quantity and quality of undergraduate CPT education | 1, 2b | 13.5 |
Brinkman et al. (2014) 27 | The Netherlands; one medical school | Single‐group; cross‐sectional descriptive | 483 final‐year (6th) medical students | General prescribing | Prescribing performance during 4 therapeutic consultations with real patients during clinical clerkship, self‐administered online questionnaire on self‐reported confidence in prescribing | Adequate performance score (7.93; max. 10). Majority (66–88%) lacked self‐reported confidence in essential prescribing skills | 2a, 3 | 9 |
Cheng & Yu (2012) 33 | China; one medical school | Multiple groups; cross‐sectional descriptive | 103 penultimate‐year (6th) and 80 final‐year (7th) medical students, 59 clinical master degree candidates, 50 clinical doctors | Narcotics | 45‐point self‐administered written questionnaire about knowledge of cancer pain management | Senior medical students had a poor knowledge of cancer pain management, especially among non‐oncological medical students | 2b | 11.5 |
Chuenchom et al. (2016) 76 | Thailand; three medical schools | Single‐group; cross‐sectional descriptive | 455 final‐year (6th) medical students | Antimicrobial prescribing and resistance | Self‐administered written questionnaire about perception, attitude and knowledge of antimicrobial resistance, appropriate antimicrobial use and infection control | Mean knowledge scores were low (1.61; max. 5). Nearly all participants had a positive attitude about antimicrobial resistance. >10% reported that they had never learned about antimicrobial resistance and appropriate antibiotic use. Nearly all students wanted more education about rational antimicrobial use | 1, 2b | 10 |
Coombes et al. (2008) 81 | Australia; two teaching hospitals | Single‐group; cross‐sectional descriptive | 101 final‐year (4th) medical students | General prescribing | 21‐point self‐administered written questionnaire about students' perceptions of their readiness to prescribe, associated risks and outcome if involved in an error, as well as their perceptions of available support | 84% felt able to prescribe for simple complaints, whereas only 54% felt comfortable to prescribe warfarin and 66%to prescribe IV fluids. 99% were under the misapprehension that potentially harmful prescriptions would be safely administered. 79% perceived that they would be blamed if they made a prescribing error | 2a | 8.5 |
Dyar et al. (2013) 35 | France; one medical school | Multiple groups; cross‐sectional descriptive | 34 penultimate‐year (5th) and 26 final‐year (6th) medical students | Antimicrobial prescribing and resistance | 41‐point self‐administered online questionnaire about knowledge, perceptions and beliefs about antibiotic resistance and prescribing | Final‐year students were highly aware of and concerned about the overuse of antibiotics. Areas of non‐confidence and gaps in knowledge were found. Students consistently overestimated the current burden of resistant bacteria and were unaware of success in reducing MRSA infections. 79% wanted more training in antibiotic selection | 1, 2b | 9.5 |
Dyar et al. (2014) 77 | Europe; seven medical schools from seven countries | Single‐group; cross‐sectional descriptive | 338 final‐year (5th–6th) medical students | Antimicrobial prescribing and resistance | 41‐point self‐administered online questionnaire about knowledge, perceptions and beliefs about antibiotic resistance and prescribing | Students lacked confidence in combination therapy and dose selection of antibiotics. Students consistently overestimated the current burden of resistant bacteria and were unaware of success in reducing MRSA infections. 74% wanted more training in antibiotic selection | 1, 2b | 10.5 |
Dyasanoor & Urooge (2016) 28 | India; one university hospital | Multiple groups; cross‐sectional descriptive | 52 undergraduates, 221 internsa, 227 postgraduates | General prescribing | Quality of 500 prescriptions dispensed by participants analysed according to 19 parameters | 39% of the prescriptions by interns were not adequate because important parameters were missing. Prescriptions written by undergraduates were better than those written by interns and postgraduates | 3 | 10 |
Grossman & Sheidler (1985) 25 | USA; one university hospital | Multiple groups; pretest intervention | 4 2nd‐year, 12 penultimate‐year (3rd), 1 final‐year (4th) medical student, 68 house officers | Narcotics | 3 clinical vignettes about converting narcotic analgesic regimen in patients with pain | 2% of the answers of medical students were within the correct range. There were no significant differences between the responses among individuals at different education levels | 2b | 10 |
Gyawali et al. (2016) 46 | Nepal; one university hospital | Multiple groups; cross‐sectional descriptive | 120 final‐year (5th) medical students, 117 internsa | Generic prescribing and substitution | Self‐administered written questionnaire about knowledge and perceptions of generic medicine and generic prescribing | Only 5% knew the correct regulatory limits for bioequivalence. 46% had the impression that brand‐name medicines were required to meet higher safety standards than were generic medicines. 17% had the impression that generic medicines caused more side effects and were less effective than brand‐name medicines | 2b | 12.5 |
Haque et al. (2016) 36 | Malaysia; one medical school | Multiple groups; cross‐sectional descriptive | 41 3rd‐year, 55 penultimate‐year (4th) and 45 final‐year (5th) medical students | Antimicrobial prescribing and resistance | 21‐point self‐administered written questionnaire about knowledge, beliefs and practices regarding antibiotic resistance and prescribing | >50% of final‐year students lacked confidence in combination therapy and dose and duration selection of antibiotics. Students consistently overestimated the current burden of resistant bacteria. 88% wanted more training in rational antibiotic selection | 1, 2b | 12.5 |
Harakeh et al. (2015) 37 | Saudi Arabia; two medical schools | Multiple groups; cross‐sectional descriptive | 1042 1st‐year to final‐year (6th) medical students | Antimicrobial prescribing and resistance | 20‐point self‐administered written questionnaire about knowledge, attitudes and practices regarding the use of antibiotics for upper respiratory tract infections | 93% of the final‐year medical students had adequate knowledge of antibiotic effectiveness. Similarly, most were aware of antibiotic abuse and development of drug resistance in bacteria | 2b | 9.5 |
Hassali et al. (2007) 51 | Australia; 10 universities | Multiple groups; cross‐sectional descriptive | 400 final‐year (5th) medical students and 289 final‐year (5th) pharmacy pre‐registrants | Generic prescribing and substitution | 20‐point self‐administered online questionnaire about knowledge and perceptions of generic medicine, generic prescribing and generic substitution | Both groups scored poorly on the allowable bioequivalence limits, with pharmacy pre‐registrants scoring better. Both groups incorrectly believed that generic medicines met lower safety standards, were inferior in quality, less effective and produced more side effects than brand‐name medicines | 2b | 11.5 |
Heaton et al. (2008) 82 | UK; 25 medical schools | Multiple groups; cross‐sectional descriptive | 2413 medical students graduating in 2006–2008 | General prescribing | 17‐point self‐administered online questionnaire about undergraduate training and assessment in pharmacology and therapeutics, the acquisition of prescribing skills and self‐reported confidence in prescribing | Only 38% felt confident about prescription writing and 35% had filled in a hospital prescription chart more than three times during training. 74% felt that there was (far) too little teaching and 56% disagreed that their assessment thoroughly tested knowledge and skills | 1, 2a | 10 |
Hema et al. (2012) 47 | India; one university hospital | Multiple groups; cross‐sectional descriptive | 80 final‐year (5th) medical students, 88 internsb and 132 postgraduates | Pharmacovigilance | 25‐point self‐administered written questionnaire about awareness, knowledge and methods to apply pharmacovigilance | Compared with interns and postgraduates, final‐year students had a lower awareness score (2.5; max. 5), lower knowledge score (2.3; max. 8) and lower method of application score (5.8; max. 12) | 2b | 10 |
Holstein et al. (2000) 38 | Germany; 21 academic teaching and university hospitals | Single‐group; cross‐sectional descriptive | 310 final‐year (6th) medical students | Cardiovascular | 20‐point self‐administered written questionnaire about clinically relevant diabetes treatment knowledge and experience | Marked gaps in manual experience and clinically relevant knowledge of diabetes, particularly in practical aspects of diabetes therapy. 45% had never injected insulin and 68% had no experience with the insulin pen. 81% were not satisfied with the diabetes teaching | 1, 2b | 10.5 |
Hoque et al. (2016) 39 | Bangladesh; one medical school | Multiple groups; cross‐sectional descriptive | 40 3rd‐year, 33 penultimate‐year (4th) and 34 final‐year (5th) medical students | Antimicrobial prescribing and resistance | 21‐point self‐administered written questionnaire about knowledge, attitudes and perceptions about antimicrobial resistance and rational use of antimicrobials | Participants had an average level of confidence in the rational use of antimicrobials, with final‐year students being more confident than earlier‐year students. Students underestimated the prevalence rate of MRSA infection. 94% wanted more training in antibiotic selection | 1, 2b | 11.5 |
Huang et al. (2013) 40 | China; three teaching hospitals | Multiple groups; cross‐sectional descriptive | 1236 medical students, including 241 final‐year (5th year) students, 852 nonmedical students | Antimicrobial prescribing and resistance | 30‐point self‐administered online questionnaire about knowledge, attitudes and perceptions about antibiotic use | Senior students had more knowledge of and positive attitudes to antibiotic use compared with junior students. No difference between penultimate‐year (4th) and final‐year (5th) students was observed. There was more excessive use of antibiotics among senior medical students | 2b | 12.5 |
Humphreys et al. (2006) 78 | Ireland; three medical schools | Single‐group; cross‐sectional descriptive | 108 medical graduates | Antimicrobial prescribing and resistance | 19‐point self‐administered written questionnaire about knowledge and understanding of treatment and prevention of common infections | ±60% of the questions were answered correctly; questions regarding the rational use of antibiotics were poorly answered compared with other questions | 2b | 9 |
Ibia et al. (2005) 24 | USA; 21 medical schools | Multiple groups; cross‐sectional descriptive | 30 penultimate‐year (3th) and 959 final‐year (4th) medical students | Antimicrobial prescribing and resistance | Six paediatric vignettes regarding clinical management of different upper respiratory tract infections | 99% were informed about the problems of antibiotic resistance. Majority of students 30–56%) prescribed antibiotics inappropriately for viral upper respiratory diseases. No subanalysis for done for study year | 2b | 11.5 |
Iffat et al. (2014) 41 | Pakistan; multiple universities | Multiple groups; cross‐sectional descriptive | 78 3rd‐year, 212 penultimate‐year (4th) and 241 final‐year (5th) dental and medical students | Pharmacovigilance | 31‐point self‐administered written questionnaire about knowledge, attitudes and perceptions about ADR reporting | 88% had knowledge of ADRs. Medical students showed generally positive attitudes to ADR reporting. 55% did not know the definition of pharmacovigilance. Only 10% knew where to report ADRs. 72% indicated that they had not received any training in reporting ADRs | 2b | 12.5 |
Ismail et al. (2015) 83 | Malaysia; one medical school | Single‐group; cross‐sectional descriptive | 23 final‐year (5th) medical students | Pharmacovigilance | Self‐administered written questionnaire about knowledge, attitudes and practice regarding ADRs and pharmacovigilance | 96% knew the term ADRs. 57% did not know the method for reporting ADRs, although 78% had observed ADRs during clinical clerkships. 87% had not received any training in ADR reporting during their medical training | 1, 2b | 10 |
Jamshed et al. (2014) 52 | Malaysia; one medical school | Multiple groups; cross‐sectional descriptive | 72 final‐year (5th) medical students and 51 final‐year (5th) pharmacy students | Antimicrobial prescribing and resistance | Self‐administered written questionnaire about understanding of antibiotic use and antibiotic resistance | 94% showed adequate knowledge regarding the course content related to antibiotics. Only 30% were able to recognize penicillin and vancomycin‐resistant bacteria. Pharmacy students had better knowledge than medical students | 2b | 11 |
Kaki (2011) 86 | Saudi Arabia; one medical school | Single‐group; cross‐sectional descriptive | 325 final‐year (6th) medical students | Narcotics | 18‐point self‐administered written questionnaire about knowledge, beliefs and attitudes to cancer pain management | 46% considered cancer pain as untreatable. 59% considered that the risk of addiction is high with opioid prescriptions. 68% limited opioid prescriptions to patients with poor prognosis. 63% indicated that they had received no education in pain medicine during their medical training | 1, 2b | 12 |
Khan et al. (2013) 42 | India; one medical school | Multiple groups; cross‐sectional descriptive | 148 3rd‐year medical students and 130 medical graduates | General prescribing | Self‐administered written questionnaire about knowledge, attitudes and behaviour regarding safe prescribing, including one patient case | The knowledge and attitudes of medical graduates were satisfactory. 12% prescribed unnecessary antibiotics. Prescribing skills were generally inadequate as errors in written prescriptions were commonly found | 2b | 10.5 |
Meher et al. (2015) 43 | India; one medical school | Single‐group; cross‐sectional descriptive | 60 2nd‐year, 60 penultimate‐year (4th), 60 final‐year (5th) medical students | Pharmacovigilance | 21‐point self‐administered written questionnaire about knowledge, attitudes and practice of pharmacovigilance | Mean knowledge and attitude scores for final‐year students were low (4.3; max. 7 and 1.7; max. 4, respectively). Penultimate‐year students had better knowledge and attitude scores than final and 2nd‐year students | 2b | 9.5 |
Olakunle et al. (2014) 79 | Nigeria; three medical schools | Single‐group; cross‐sectional descriptive | 241 final‐year (6th) medical students | Antimicrobial prescribing and resistance | Self‐administered written questionnaire about knowledge of diagnosis and management of tuberculosis under DOTS | Only 46% correctly mentioned the various categories of the DOTS treatment regimen. 7% were able to identify the correct treatment duration for new tuberculosis | 2b | 10.5 |
Oshikoya et al. (2008) 21 | Nigeria; one medical school | Single‐group; cross‐sectional descriptive | 34 final‐year (6th) medical students | General prescribing | Self‐administered written questionnaire about knowledge of principles of good prescribing and prescribing skills, including two clinical cases | 92% knew that rational prescribing involved prescribing correct dosage of an appropriate medicine formulation. Less than 50% of the prescriptions included name, case file number, age and gender of the patient | 2b | 10.5 |
Ozoh et al. (2014) 89 | Nigeria; one medical school | Single‐group; cross‐sectional descriptive | 139 final‐year (6th) medical students | COPD | Self‐administered written questionnaire about knowledge of COPD diagnosis and treatment | 38% had good, 37% fair and 20% poor knowledge of COPD management. 55% were familiar with the GOLD guidelines. 28% correctly identified inhaled steroids and bronchodilators as therapy for stable COPD | 2b | 9.5 |
Puchner & Zacharisen (2002) 48 | USA; two academic and two teaching hospitals | Multiple groups; cross‐sectional descriptive | 36 medical students (junior/senior), 342 medical specialists | Antimicrobial prescribing and resistance | Self‐administered written questionnaire about cephalosporin prescribing and knowledge of penicillin allergy | There was marked variation between study groups in the prescribing of cephalosporin and requesting penicillin skin testing. In general, medical specialists had a better knowledge of penicillin allergy than medical students | 2b | 8.5 |
Rehman et al. (2011) 88 | USA; one medical school and four teaching hospitals | Multiple groups; cross‐sectional descriptive | 121 final‐year (4th) year medical students, 354 junior doctors | Cardiovascular | 11‐point self‐administered written questionnaire about knowledge of hypertension diagnosis and management | Only 8% answered the questions regarding hypertension treatment correctly. Mean number of correct answers given by final‐year students was low (5.2; max. 11), although they did better than junior doctors | 2b | 12.5 |
Rothwell et al. (2012) 29 | UK; three medical schools | Single‐group; cross‐sectional descriptive | 65 medical graduates | General prescribing | Multiple methods, including face‐to‐face and telephone interviews, questionnaires and secondary data from prescribing safety assessment | 73–80% of the graduates felt generally underprepared for prescribing. Only 16–19% of the graduates passed the prescribing safety assessment at the first attempt | 2b | 11 |
Scaioli et al. (2015) 44 | Italy; one medical school | Multiple groups; cross‐sectional descriptive | 465 1st year, 187 2nd‐year, 190 3rd‐year, 70 4th‐year, 79 penultimate‐year (5th), 59 final‐year (6th) healthcare students | Antimicrobial prescribing and resistance | 31‐point self‐administered written questionnaire about knowledge and attitude about antibiotic usage and antibiotic resistance | More than 90% answered all items correctly. However, 20% were not aware that antimicrobial drugs were not effective against viruses and that these drugs can cause secondary infections. However, no subanalysis by study year was performed | 2b | 11.5 |
Schutte et al. (2017) 45 | The Netherlands; eight medical schools | Multiple groups; cross‐sectional descriptive | 354 3rd‐year, 92 4th‐year, 106 penultimate‐year (5th), 298 final‐year (6th) medical students | Pharmacovigilance | 10‐point self‐administered online questionnaire about pharmacovigilance awareness, skills and knowledge | Final‐year students had marginally better knowledge scores than 3rd‐year students (68% vs. 64%). Only 48% of the final‐year students felt sufficiently prepared to report ADRs in future practice, and a substantial proportion did not know the authority to which they should report ADRs | 1, 2b | 12.5 |
Sharrad & Hassali (2011) 84 | Iraq; six medical schools | Single‐group; cross‐sectional descriptive | 546 final‐year (6th) medical students | Generic prescribing and substitution | 23‐point self‐administered written questionnaire about perception and knowledge of generic medicines and generic prescribing | >70% of the respondents did not possess sufficient knowledge about bioequivalence limits for generic medicines. >60% of the students thought that generic medicines were inferior, less effective and produced more side effects than brand‐name medicines | 2b | 13 |
Thriemer et al. (2013) 49 | Democratic Republic of Congo; one medical school and multiple hospitals | Multiple groups; cross‐sectional descriptive | 106 final‐year (6th) medical students and 78 medical doctors | Antimicrobial prescribing and resistance | 23‐point self‐administered written questionnaire about knowledge, attitudes and practices regarding antibiotic prescribing | Mean knowledge scores were generally low (4.9; max. 9) and did not differ significantly between the groups. Students had high self‐confidence in antibiotic prescribing (86%), although this was lower than that of qualified doctors (95%) | 2b | 12.5 |
Upadhyaya et al. (2012) 22 | India; one teaching hospital | Single‐group; cross‐sectional descriptive | 50 medical graduates | General prescribing | 22‐point self‐administered written questionnaire about self‐rated knowledge of the principles of good prescribing | 48% rated their prescribing knowledge as average, and 18% as (very) poor. 58% felt that their medical training did not prepare them for rational prescribing. 42% felt that they had some problems with writing a prescription during their internship | 1, 2a | 7 |
Weber et al. (1986) 50 | Canada; one paediatric hospital | Multiple groups; cross‐sectional descriptive | 28 final‐year (4th) medical students, 21 paediatric residents, 22 paediatricians | General prescribing | Six paediatric patient cases about knowledge of the price of frequently prescribed medications | Students' estimations were adequate in 40% of the cases, which was lower than that of paediatric residents (52%) and paediatricians (62%) | 2b | 8 |
Weinstein et al. (2000) 34 | USA; one medical school | Multiple groups; cross‐sectional descriptive | 88 1st‐year medical students, 180 final‐year (4th) medical students | Narcotics | 49‐point self‐administered written questionnaire about knowledge and attitudes on pain and the use of opioid analgesics | Senior medical students scored as well as, or better than, 1st‐year students on questions related to knowledge about pain. However, they had a more negative attitude to relieving chronic pain and seemed to have less understanding about the causes of pain in cancer patients | 2b | 10.5 |
Wheeler et al. (2004) 26 | UK; one medical school | Multiple groups; cross‐sectional descriptive | 87 2nd‐year, 98 penultimate‐year (3rd), 78 final‐year (4th) medical students | General prescribing | Online multiple‐choice examination about dose‐calculating skills | The mean knowledge score for all students was low (1.2; max. 3), although final‐year students had significantly better knowledge scores than 1st‐year students | 2b | 9.5 |
ADR, adverse drug reactions; COPD, chronic obstructive pulmonary disease; CPT, clinical pharmacology and therapeutics; DOTS, directly observed treatment, short‐course; GOLD, Global Initiative for Chronic Obstructive Lung Disease; IV, intravenous; MCQ, multiple choice question; MERSQI, Medical Education Research Quality Instrument; MRSA, methicillin‐resistant Staphylococcus aureus
The MERSQI has a minimum score of 5 and a maximum score of 18.
Interns in India and Malaysia have graduated from medical school but receive their full registration as a medical doctor after completion of the internship
Kirkpatrick's four levels of training evaluation are:
Level 1: Participation – covers learners' views on the learning experience, its organization, presentation, content, teaching methods and aspects of the instructional organization, materials and quality of instruction.
Level 2a: Modification of attitudes or perceptions – outcomes here relate to changes in the reciprocal attitudes or perceptions between participant groups towards intervention or simulation.
Level 2b: Modification of knowledge and skills – for knowledge, this relates to the acquisition of concepts, procedures and principles; for skills, this relates to the acquisition of thinking and problem solving, and psychomotor and social skills.
Level 3: Behavioural change – this documents the transfer of learning to the workplace or the willingness of learners to apply new knowledge and skills.
Level 4a: Change in organizational practice – this relates to wider changes in the organization or delivery of care, attributable to an educational programme.
Level 4b: Benefits to patients or clients – this relates to any improvement in the health or well‐being of patients and clients as a direct result of an educational programme.