Notes for the Primary Care Teachers: PORTFOLIO ASSESSMENT

Abstract

The portfolio consists of a variety of documentation of a learner’s proof of learning. It has been promoted as one way to verify a learner’s personal and professional development, especially for the more mature trainees and doctors at work. It has not been widely accepted as a summative tool because the amount of time imposed on the learners may be considerable. Ways to improve the reliability of assessments on unstandardised portfolios are needed for its wider application.

WHAT IS A PORTFOLIO?

The Concise Oxford Dictionary defines ‘portfolio’ as “a set of pieces of creative work intended to demonstrate a person’s ability to a potential employer.” For an artist, his portfolios are in large folders that contain his best artworks, and, when presented to prospective clients, show that he can paint. In medical education, a portfolio is a collection of documents and materials that show what a trainee has learnt and reflects his learning. The contents of a portfolio can vary a great deal from one trainee to another (see example in Box 1).

Box 1: Types of portfolios1

1. Proof of completion of a specific course of study: Certificates of attendance at conferences or workshops on specific clinical skills,, professional degrees

2. Assessment results and reports, including tutors’ testimonials

3. Record of learning activities: Log book of cases seen or procedures performed;, videotapes of communication skills;, clinical audit activities, case commentaries, elective reports, presentation slides, critical incident analyses, reflective diaries

4. Performance data: Patient satisfaction questionnaires, prescription data, appraisal forms from peers or patients

5. Publications in professional journals

As can be seen from Box 1, portfolios include written documentation of the many learning activities of a trainee. However, these portfolios should not be merely a listing of teaching-learning activities that have been planned by a medical school or vocational training programme. Much flexibility is given to the trainees to collect various portfolios that fit into his learning plan. Also, the trainee is encouraged to write down his reflections (e.g. admitting to having difficulty in obtaining a sexual history and identifying steps to overcome this problem). These reflections can be appended to a log diary as free text entries. Alternatively it can take the form of a critical incident analysis – a case write-up focusing on negative events (e.g. a missed diagnosis or an unexpected admission) with an emphasis on the lessons learnt.

WHY INTRODUCE PORTFOLIO-BASED ASSESSMENT?

A portfolio is consistent with experiential learning and reflection as described by Kolb and Schon.2 Maintenance of portfolios by trainees can lead to deep learning and achievement of learning outcomes if the trainee reflects on his deficiency and proceeds to ‘close the loop’ (demonstrating steps taken to overcome a deficiency). For medical students, a learning plan accompanied by ‘evidence’ of learning and achievement in the form of various portfolios is felt to be more holistic compared to written or clinical examinations. Furthermore, practising doctors are increasingly expected to demonstrate continuing professional development. Revalidation of doctors for a continuing licence may require the maintenance of a portfolio for the future. At the moment, most undergraduate and postgraduate medical programmes have some elements of portfolios as part of formative assessment but few, if any, rely entirely on the portfolio for the summative examination; University of Dundee being a notable exception.2

PROBLEMS IN THE PREPARATION AND ASSESSMENT OF A PORTFOLIO

Many trainees do not like portfolios, saying that the extra time required in building portfolios could be better used for learning ‘real medicine’ in the clinics and wards. Most trainees also have considerable difficulty reflecting on their learning. If portfolios were to become mandatory, it is expected that most trainees would do the minimum to satisfy the requirements. There is also the possibility of sharing of materials between trainees and even extensive copying unless these are carefully checked by the assessors. Guidance on how to prepare a portfolio with the mentor demonstrating on how to reflect and think critically is essential for the junior trainees.

For the assessors, the voluminous portfolios collected by a trainee will take quite a bit of time to go through. In view of the variable experiences undergone by trainees, the portfolios collected by trainees from the same programme may differ considerably between one to another. The unstandardised portfolio, coupled with inter-rater variability between assessors, means that a sufficiently high level of reliability and validity is not achievable.3-5

CONCLUSION

The portfolio is consistent with the principles of adult learning and self-directed learning. Whether it should be widely introduced in undergraduate medical courses and postgraduate vocational training is still being debated. Maintenance of a portfolio promotes elements of ‘personal and professional development’, and may be introduced in future revalidation or recertification of doctors.

Ovarian screening by annual transvaginal ultrasonography and CA125 is inefficient at detecting early-stage ovarian cancers

Woodward ER, Sleightholme HV, Considine AM, et al. Annual surveillance by CA125 and transvaginal ultrasound for ovarian cancer in both high-risk and population risk women is ineffective. BJOG. 2007;114(12):1500-9

This is a retrospective audit of 341 asymptomatic women of variable risk of developing ovarian cancer. They underwent annual transvaginal ultrasound and CA125 screening. Of the 4 ovarian cancers that occurred, only one was detected by screening (the other three presented symptomatically between screenings). Thirty women underwent exploratory surgery because of abnormal findings at surveillance (two cancers were detected, one ovarian cancer and the other endometrial cancer). Sensitivity, specificity, PPV and NPV for transvaginal ultrasound in the whole cohort were 33.3%, 85.8%, 0.6% and 99.8%, respectively. Combining both modalities for the whole cohort, the sensitivity, specificity, PPV and NPV were 66.7, 82.9, 1.5 and 99.8%.