Viva Voce in Postgraduate Surgical Examinations in Anglophone West Africa

Abstract

The Viva Voce is a form of assessment dating back to antiquity. It is widely used by the English-speaking West African Postgraduate Medical Colleges for intermediate and exit level examinations. Although it is still popular till the present day, there is a growing awareness of its limitation as an examination method. This paper explores the origin, format, advantages and limitations of oral examinations in postgraduate surgical assessment and proffers practical guidelines on its usage.

“Viva Voce” in Postgraduate Surgical Examinations

“Viva Voce” (Medieval Latin) or examination “by word of mouth,” “orally,” “by the living voice,” (Webster’s Dictionary) was the earliest form of formal assessment of medical and other apprentices, dating back to pre-medieval times1. Historically and for a long time, great prominence was given to the spoken word. Plato explicitly denigrated writing, stating that the spoken word was superior to the written2.Aristotle3 in 350 BC similarly defended robust oral arguments from which many a philosophical principle and logic emanated. Apprentices were deemed truly worthy of certification in their chosen fields of endeavor after they had satisfied a panel of more than one erudite assessor who tested the candidate’s scope and depth of knowledge on sundry topics through an oral discourse within a specified time frame. The candidate would obviously have been well prepared for this session. Although the invention of the printing press and the consequent increased popularity of mass-produced books have since reduced the dependence on the spoken word, Oral Examination (OE) as an examination method remains an important method of assessment in many fields of medical training till date4. In university settings, the Viva is used often to defend theses. It may also be used discretionally to clarify final marks for candidates with borderline marks, unexpected results or sometimes in extenuating circumstances5.

While it is well known that no single examination technique adequately assesses surgical skills, the oral examination is still very popular with assessors and is widely used in medical curricula worldwide6. In viva voce summative assessment at postgraduate surgical fellowship examinations, the candidate has ample chance to assume the teacher mode and verbally display his depth of knowledge on a number of proffered topics. It is the closest an assessor can get in probing for a candidate’s competence. This method allows examiners to get a firsthand eyeball-to-eyeball “feel” of the candidate’s grasp of the subject, mettle and verbal skills. The examiner can judgmentally “size-up” the candidate’s ability. At times, the examiners make judgment, going by their “gut feeling” about the person in front of them.

Setting

The Viva Voce is a timed process during which a candidate’s performance is orally assessed. Although a typical surgical textbook may contain more than 60 chapters, only 3-5 randomly asked questions are used at the viva to judge his total knowledge of the syllabus. Different questions, variations and combinations of sundry topics are given to candidates who may be required to compose an essay, solve a puzzle, show deductive reasoning, display his knowledge, describe a procedure or defend a given position or dissertation. It is an interactive format, affording much spontaneity in both the range of questions asked and the candidate’s latitude to answer. The aim of the oral examination is to assess the candidate’s cognitive knowledge, psychomotor skills and best attitude for good outcome in the chosen specialty.

The West African College of Surgeons (WACS), the Faculty of Surgery of the National Postgraduate Medical College of Nigeria(NPMCN) and the Ghana College of Physicians and Surgeons(GCPS) no longer conduct the OE at the Primary Examinations but continue to use it for the intermediate and exit examinations (Parts I and II for WACS or NPMCN and parts II and III for the GCPS examinations). In addition, this method is employed to assess the defense of Part II/III theses, a time honored format of dissertation assessment. In each of the three English-speaking postgraduate Colleges the oral examination is used at the Part I examination as follows:

ORAL I: Principles of Surgery

ORAL II: Operative Surgery & Pathology

The initial question may be as broad as “Describe Whipple’s procedure.” “What fluids would you use for resuscitation of a patient in an emergency?” What are the principles of cancer chemotherapy?” A trunk question and the level of performance may be raised at the discretion of the examiners in order to test the depth and field of knowledge of the candidate. It is not usual to pin a candidate down to a point where he/she demonstrates weakness. Rather, the examiners are expected to move onto other areas or a different question altogether so as to give the candidate a good spread of questions in the specialty before a decision is made on his/her competence/suitability. Questions already taken in the theory papers are usually avoided at the oral examination.

Scoring

The candidate’s final score is usually computed after all the questions have been asked within the allotted time and after the “bell has gone,” when the candidate has departed. This inevitably creates a time-lag and examiners usually compare notes to reach a consensus. In many Faculties, the “close-marking” system is employed whereby the highest score is 12 over 20 while the lowest is 8. The examiners at the oral examination are usually about three to five in number. The examiners score the candidates under several headings and sum up at the end of the oral examination; block scoring is not used in order to avoid bias and stick to objectivity. In order to maintain the credibility of oral examination, anyone with proven records of corruption or fraud is not used as an examiner.

Defense of dissertation is not much different from the traditional Viva Voce examination except that the candidate who wrote the thesis knows his/her book inside out. Nonetheless, the assessors would ask probing questions to test the candidate’s understanding of his/her subject topic, methodology, results and their interpretations. They also question and criticize the style of writing and point out typographical errors that need to be corrected before the book is finally accepted.

What are the advantages of Oral Examinations?

The Viva Voce requires only a few resource staff and very simple logistics. It is an examination method best suited to small candidate numbers if examiner fatigue is to be avoided. It tests depth of knowledge and gives a measure of the candidate’s understanding of practical concepts on the topic. It is also a highly flexible examination format which thus easily detects rote learning. In fact, it tests “thinking on one’s feet” as the candidate has to formulate an answer and verbalize it with little room for re-ordering the given answer once it is spoken. For disciplines such as teaching, broadcasting, legal practice, marketing and for men of the collar - vocations where verbal skills are most essential - the Viva Voce tests oratory communication skills, being most rewarding for candidates with the gift of garb. Preparation for the examination requires the least effort for examiners as they may simply jot down some topics from the syllabus on the morning of the event. Lastly, the oral examination is greatly gratifying to the examiners’ ego, who are “seen to be sitting in dreadful judgment” with great fanfare.

Drawbacks

1. Low Objectivity Score

Although no single examination method is perfect, the Viva Voce rates least among other popular examination methods on any scale of objectivity. This low objectivity score derives from many factors broadly inherent in the methodology itself or traceable to various biases at the examiners / candidate interface7.

a. Inherent to the Methodology

Narrow Scope

Inherent to the examination method is its narrow scope, a necessary price for depth of probing. Because so much time is taken in the questioning process on each topic, only 4-6 questions can be adequately taken within an allotted time of 15-20 minutes. The candidate’s performance on these 4-6 areas is then extrapolated to mean an average performance throughout all the topics in the syllabus and he/she is scored as having passed or failed the oral examination. This obviously may not in fact be so.

Non-Uniform Questions

Candidates are not given the same questions. This non-uniform, random and unwieldy format of administered questions, while preventing repetition, may sometimes seem like comparing apples with oranges. The performances of three examination candidates who were asked the following questions will be difficult to compare:

• ν“Describe Ladd’s Procedure”
• ν“Describe apendicectomy”
• ν“How would you drain a nail-bed abscess?”

Some obviously seem easier to answer than others and if proffered by an examiner, may be a reflection of his like/dislike of a particular candidate. This problem is sometimes mitigated through the pooling of questions by examiners in a group who may then allow candidates to ballot for questions they will be asked. Other examination methods seek to better address this problem.

Delayed, Varied Scoring

Scoring is delayed till the candidate has left and is done one candidate at a time, therefore often shifting the reference point, depending on a good or bad performance of the previous candidate. Even when there is a structured scheme of marking, it does not provide for the individual variations in response among candidates for different segments of the answers. The examiner, meanwhile, forms an opinion on each aspect and stores it in the mind till the end when notes are compared with the other examiner(s) to arrive at a consensus mark. Scoring the candidate under various headings agreed by the examiners may partly solve this problem.

b. Examiners’ Factors

Biases

A number of biases and other factors ascribed to the examiners further diminish the objectivity rating of oral examinations. These, largely, are human factors operating at the examiners/candidate interface, usually not immediately apparent but contributory to the final pronouncement on each person assessed. These, it must be pointed out, may contribute positively or negatively to the outcome of the assessment but each of these influences is a bias. An examiner may exhibit a personal prejudice towards the candidate because of the candidate’s (or another candidate’s) performance at a lower and higher percentile. Personal bias may also be influenced by previous encounter(s) between the examiner and examinee. A visually pleasing candidate evokes a pleasant disposition in the assessors mind while a sloppy appearance may trigger a dislike. The fairer gender may provoke sympathy while a candidate who has shown up at the same examination on a number of occasions may be perceived as a habitual low performer and be so rated. Ethno-regional and racial partialities are a part of human nature which examiners constantly strive to overcome.

Fatigue

It is a well-known fact that an examiner’s mood may influence the candidate’s result. A feisty examiner (from any reason) may exhibit unfair intolerance of minor errors and thus unfairly judge a candidate’s performance. Fatigue, low blood sugar and displeasure with host institution’s hospitality may contribute to this and inform the reason why host communities must keep examiners in stress-free comfort! Similarly, a candidate fatigued by the rigorous Fellowship examination process may not present his/her best.

Harassment

Harassment is an untoward effect of the confrontational nature of Viva Voce when the two parties assume adversarial stances. The examination may easily degenerate into an unpleasant event, with the candidate usually becoming a victim and the examiner feeling the victor. The converse may also occur, with the examiner feeling put down by a candidate who “knows too much,” and ending in an equally disastrous consequence for the candidate as well.

c. Candidate related factors

The difficult first question

The oral examination method has never been very popular with candidates for the above many reasons as well as others relating to candidates. A difficult first question may throw off even the best candidate and set the tone of the whole encounter with each party wrongly assuming “he cannot be good.” Luckily, this is not always the case, as such examinee will be given other questions to tackle. Given an abysmal performance on the first question however, a candidate can only hope to escape with an average score at best.

Stage fright

While the oral examination presents an undue advantage to skillful speakers, stage-fright (brain block, tongue-tie) is more common and remains the most dreaded problem for examination candidates. There are anecdotal accounts of students who have had to prime themselves with ß-blockers (e.g. Propanolol) to stem the panic that usually sets in on sighting a panel of dreaded examiners. This problem can be minimized by constant practice at public speaking and ample opportunities abound for teaching of junior residents, house officers, medical students and nurses at ward rounds, clinic setting, seminars and tutorials.

Question pooling

Since candidates are assessed at different times, those who are examined later tend to pool questions from candidates exiting the venue and soon, examiners run out of topics and recycle earlier ones. In some instances candidates get to know the favourite questions of examiners and successfully predict the questions they are likely to ask thus diminishing the objectivity of the oral examination.

2. Slow Pace

The Viva Voce is a very slow process for examining large numbers as it cannot examine en-mass like a number of other formats such as essays, MCQ and the Objective scored Clinical Examination (OSCE). This contributes to the high cost (vide infra)

3. High Cost

The Viva Voce is a very costly method when examining large candidate numbers as it requires many examiners whose travel expenses, upkeep will have to be borne by the institution for the duration of the examination process. At the West African College of Surgeons, available data suggests that the use of Viva Voce increased the unit cost of administering an examination six-fold per candidate when compared with assessments where only the MCQ system was used. In fact, overall, over 80% of the WACS examination cost was spent on examiners’ overheads because of the widespread use of oral examination by all Faculties of the WACS^. The Viva Voce has been proven to be fraught with subjectivity, thus adding very little to the overall assessment from other examination methods8.

Why then is the Oral Examination Method Still Used?

Popularity

It is a very popular method that dates back to antiquity. Since tradition is very difficult to change, assessors often stick to the dictum “If I went through it, others should” . Thus, those who are adjudged to have passed would seem to have performed to the standard by which the examiners had been certified of yore. The method is also perceived as a flexible tool which adequately tests depth of clinical and academic skills, both of which are closely associated with medical practice9.

Enormous logistics of alternative options

An institution embarking on changing its examination format from the oral to MCQ or Objective Structured Clinical Examination (OSCE) or the steeple-chase will need to invest some initial material and human cost to retraining its examiners and design proper questions and materials to adequately test areas intended. Such changes are usually reluctantly made, as they often pitch older academics with younger ones and highlight the non-awareness and unfamiliarity of many with alternative examination modalities.

Mitigating the Disadvantages of Viva Voce

Some of the undesirable features of oral examinations may best be minimized through regular training workshops for examiners to highlight the several limitations of this method and teach about newer concepts on candidate assessment. Under no circumstance should an oral examination ever be performed by a solo examiner. Examining bodies should also realize the inherent limitations of Viva Voce and place less emphasis on its scores in the final computation of marks, but rather use such scores only as an adjunct to those obtained from other methods. Objectivity of scoring is increased when a different examiner scores the answer to questions asked by another examiner as practiced in some Royal Colleges in the United Kingdom. Examiners should be well selected, trained, guided and monitored on well entrenched examination protocols10. In addition, questions to be administered should be peer-reviewed and assigned to candidates by balloting. Examiners should be kind, polite and interactive with candidates, rather than intimidating. An examiner should start with simple questions and gradually probe deeper7. It is unacceptable to make fun of a poorly performing candidate. The examiner may give a clue when a candidate encounters a brain-block. The marks awarded for any oral examination should not carry an equal or weightier value of the total computation of marks with other examination methods such as the OSCE and MCQ which have all been proven to be superior to Viva in reliability11. The gradual adoption of more objective assessment methods by our Colleges will eventually relegate orals to the shelves of antiquity. The monitoring role of both internal and external assessors at the examinations of each of the three Anglophone Colleges cannot be overemphasized. The training of trainers’ courses organized at both Faculty and College levels have been equally useful.

How should candidates prepare for the oral examination?

The Viva Voce will remain an integral part of candidate assessment in postgraduate surgical training for the foreseeable future, hence, candidates should appraise themselves of strategies to cope with this much dreaded examination format. A robust postgraduate training programme should include ample opportunities for residents to teach as this improves recall, thought organization, style of delivery and confidence in verbal skills. In West Africa, postgraduate surgical training is hospital-based and residents have lots of teaching programmes to test their teaching wings. Reading and understanding does not guarantee true comprehension, the ability to recall and give back a set of facts in a form that is clearly understood by its recipients. Teachers should also strive to engage their wards in mock adversarial roles, challenging and correcting them regularly in simulated preparation for oral examinations. Study groups where roles are played and reversed among participants are also a very helpful preparation strategy. Here, a list of possible /past questions and topics are randomly assigned and candidates speak to them ex tempore and are corrected on the spot for perceived errors of style, language or facts.

The traditional orals to test basic knowledge for large groups of candidates is considered expensive and at times, wasteful. It also runs the risks of low objectivity, bias, non-uniform application and narrow scope. Worldwide, the Oral Examination is however an important adjunct to other forms of assessment for the higher levels of medical specialists’ cadres and its use remains as popular as ever. Examiners and candidates should therefore be prepared for, and find ways to mitigate its shortcomings in order to maximize its derivable benefits.