Notes for Primary Care Teachers: Assessments – the Basics

As a primary care teacher, you will be involved in student assessment. This article will touch briefly on the purpose of assessment, definitions of some terms commonly used, what should be assessed and the assessment instruments.

Assessment is defined as a measure of student learning. This is different from evaluation which is the appraising of teaching. The purposes of assessments are:

• To pass or fail a student

• To grade or rank students

• To provide feedback to both students and teachers

• To motivate students

Formative assessment is a means of monitoring a student’s progress through a course, and providing feedback to help the student improve. It does not count towards the final grades obtained by the student for the course. Continuous assessment means a student is assessed throughout the course by various means including assignments, tests and case presentations. Continuous assessment can be formative or the marks/grades obtained may be added to the end-of-course assessments to become a part of the summative assessment. Summative assessment is designed to find out if a student has achieved the course’s objectives. It is usually at the end of the course, and a mark/grade is given to the student.

Examinations can be norm-referenced or criteria-referenced. Norm-referencing describes a student’s performance in relation to the rest of the group. By ranking, students can be compared with each other. The passing mark may be set to cut off a pre-determined percentage of students at the bottom. The disadvantage is that it cannot provide a clear picture as to what students can or cannot do. In criterion-referencing, students must meet a minimum standard of competence set, in order to pass. It gives a clearer picture of what students can or cannot do. However the percentage of passes for each particular year of the course can vary widely depending on that particular cohort of students, whether they are largely good or weak students.

Validity: A test is considered valid if it measures what it is supposed to measure. For example, multiple choice questions tests factual recall but, by itself, it cannot test how well a student can utilise the factual information in managing a patient.

Reliability: A test is considered reliable if it consistently measures whatever it measures.

What should be assessed?

This usually follows the educational objectives of the course and includes:

• knowledge (what the student knows)

• skills (what the student can do)

• attitude (what the student feels)

Core knowledge should be tested, not trivial knowledge (nice to know).

What assessment instruments to use?

This depends on what you are trying to examine. Table 1 gives a summary of commonly used written assessments.1,2,3 In addition, open book examination tests reference techniques and projects tests the ability to seek relevant information.

Table 1. Summary of commonly used written assessment instruments, their objectives, advantages and disadvantages.

Assessment instruments	Objectives	Advantages	Disadvantages
Essay Questions extended long essay short essay	Tests students thought process & understanding of subject	Easy to set, allows exploration of students’ attitude	Time consuming to mark, Difficult in scoring Problem of reliability, less commonly used now in medical schools.
Objective tests A. Supply - item types short answer completion item	Tests student’s ability to recall terms, facts, principles & procedures	Easy to develop. Prevents guessing	Not ideal for measuring complex learning outcome, e.g. problem solving.
B. Multiple Choice Questions (MCQs) True/False items Single best items Extended matching items	Tests knowledge, factual recall Tests applied knowledge if constructed well	Easy to mark High reliability, Easy to mark, opportunity to guessing less than True/False items	Susceptibility to guessing, (reduced through negative marking), tends to assess trivial knowledge, difficult to construct flawlessly, needs training to avoid pitfalls in construction. Measures what students know and understand but do not measure performance, needs training to write good MCQs.
Modified Essay Question (MEQ) & Patient Management Problems (PMP)	Problem solving ability	Uses actual problems and events in clinical practice	Requires some skill in developing
Key Feature problems	Problem solving ability & clinical decision making skills	Good reliability	New, construction is time consuming, best used in high stakes examinations

Clinical assessments

The observed structured clinical examination (OSCE) is now widely used because it is a reliable approach in assessing basic clinical skills. It is based on a circuit of patient based “stations”. It can vary from 8 to 20 stations, with 5 to 15-30 minutes per station. Both real and simulated patients are used. The scoring uses rating scales or checklists, by observers and “patients”. The disadvantage of an OSCE is that only isolated aspects of a clinical encounter can be assessed. Some OSCEs include data or visual interpretation stations on clinical problems.

Alternative assessments include the traditional long consultation which allows the observers to be present throughout the doctor-patient interaction. The main disadvantage is poor reliability and it is time consuming, especially when the student numbers are large. One way of overcoming this is to incorporate a number of long consultations into the course that the students must pass before they can proceed to the end-of-course examinations.

For short cases with real patients (usually a series of 5-6 patients), one or two examiners observe a students performance, ask questions and made a judgment based on student’s performance. Such short cases can be incorporated into OSCE stations.

The oral examination allows exploration of students’ understanding of topics considered relevant to clinical practice.

Individual assessment instruments test different aspects such as knowledge or skills. An examination usually uses a combination of different types of assessments including written and clinical assessments, to test the competencies acquired by students. An example of this will be shown in the article “Assessments – the Conjoint MAFP/FRACGP examinations” (page 84).

Timing of assessments

For short courses some have a pretest assessment (before the teaching begin) followed by a post test assessment to compare how much was learned from the course. A pretest can also help in determine how much prior knowledge the students have, so that the teacher can plan the lessons to build on it. Usually a combination of formative, continuous and summative assessments is used.

Attitude

This is difficult to assess. To some extent it can be assessed in essay type questions, oral examination and in the communication skills stations of the OSCE. The supervisor’s or GP tutor’s report as part of the continuous assessment may also reveal some attitude problems. Portfolio assessment is another means of assessing attitudes that will be discussed under a separate article.

Involvement in assessments

You may be asked to do one or more of the following:

• Write a report on the student attached to your clinic (usually a standardised checklist is provided)

• Assess his/her performance in your teaching session including case presentation.

• Mark student assignments or projects such as audit or research projects. Usually checklists or guidelines are provided.

• Vet and/or mark written examination questions

• Participate as an invigilator or examiner in clinical assessments such as OSCE.

• Contribute questions to the existing bank of examination questions. If you are asked to help construct examination questions, ensure the questions prepared by you meet your course objectives. It should cover core knowledge and skills taught by you. You can ask a more senior colleague to guide you.

Conclusion

If you have recently become a primary care teacher, do familiarise yourselves with the different assessment instruments used in the course you are teaching. Reading materials on assessment instruments are also available.1-4 Be ready to participate in any assessment workshop and teaching the teachers courses made available.

The Family Medicine Digital Resources Library [FMDRL]

http://www.fmdrl.org/

“FMDRL contains user-posted conference presentations and handouts, and shared curricular materials such as PowerPoint lectures, learning modules, syllabi, digital images, video and audio recordings, recommended Web sites and more.”

Jacob Reider, MD, Project Coordinator

Richard P. Usatine, MD, Editor-in-chief

Sandra K. Burge, PhD, Associate Editor

THE GLOBAL FAMILY DOCTOR

http://www.globalfamilydoctor.com/

This Online Educational Resource Centre is Wonca’s repository of educational material from a variety of sources

http://www.globalfamilydoctor.com/education/edresourcecentre/edresourcecentre.asp

Some examples of what is in there:

Chronic Respiratory Diseases - a clinical series for family doctors

Guidelines for International Breast Health and Cancer Control

COPD exacerbations

Educational Resources on Tobacco

Time to Live - a new report on COPD

IPAG Diagnosis & Management Handbook - Chronic Airways Diseases, A Guide for Primary Care Physicians

CE Medicus - An educational experience for health care professionals

Clinical Nutrition Updates