Medical Student Portfolios: A Systematic Scoping Review

Abstract

Phenomenon

Medical Student Portfolios (MSP)s allow medical students to reflect and better appreciate their clinical, research and academic experiences which promotes their individual personal and professional development. However, differences in adoption rate, content design and practice setting create significant variability in their employ. With MSPs increasingly used to evaluate professional competencies and the student's professional identity formation (PIF), this has become an area of concern.

Approach

We adopt Krishna’s Systematic Evidence-Based Approach to carry out a Systematic Scoping Review (SSR in SEBA) on MSPs. The structured search process of six databases, concurrent use of thematic and content analysis in the Split Approach and comparisons of the themes and categories with the tabulated summaries of included articles in the Jigsaw Perspective and Funnelling Process offers enhanced transparency and reproducibility to this review.

Findings

The research team retrieved 14501 abstracts, reviewed 779 full-text articles and included 96 articles. Similarities between the themes, categories and tabulated summaries allowed the identification of the following funnelled domains: Purpose of MSPs, Content and structure of MSPs, Strengths and limitations of MSPs, Methods to improve MSPs, and Use of E-portfolios.

Insights

Variability in the employ of MSPs arise as a result of a failure to recognise its different roles and uses. Here we propose additional roles of MSPs, in particular, building on a consistent set of content materials and assessments of milestones called micro-competencies. Whislt generalised micro-competencies assess achievement of general milestones expected of all medical students, personalised micro-competencies record attainment of particular skills, knowledge and attitudes balanced against the medical student’s abilities, context and needs. This combination of micro-competencies in a consistent framework promises a holistic, authentic and longitudinal perspective of the medical student’s development and maturing PIF.

Introduction

At a time when medical education is embracing a more personalised approach to knowledge attainment, skills training and development of professional behaviours, portfolios promise a means for medical students to better understand, reflect upon and actively shape their learning and development ¹ . Complementing traditional assessment methods with wider longitudinal appraisals of an individual’s growth, portfolios add a personalised dimension to logbooks^4,5, by serving as a repository for written examinations, tutor-rating reports and bedside assessments ⁶ as well as individual reflections and analyses.

Indeed, portfolios offer medical students “a self-regulated, cyclical process in which [they may] mentally revisit their actions, analyse them, cogitate alternatives, [and] try out alternatives in practice” ⁷ . It is this platform to showcase individual educational, research, ethical, personal and professional development^1,8, and guide specific, holistic and timely feedback and remediation throughout the individual’s medical education that underscores growing interest in portfolio use among medical students (henceforth medical student portfolios or MSPs)^4,12. However, despite their growing traction ¹³ , MSPs show significant variability in their structure and content. With local, practical, sociocultural, educational and healthcare considerations prioritising different types of data, the role of MSPs remains limited.

Need for the Review

With MSPs representing a sustainable and effective educational undertaking that provides insight into the medical student’s development, needs, values and beliefs that may guide their professional identity formation (PIF), better understanding of the principles behind their use, the key elements within them and a framework for consistent utilisation is required.

Methods

To determine what is known about MSPs, a systematic scoping review (SSR) is proposed to study current literature to enhance understanding of their roles and structure. These insights will also help guide the design of a consistent framework for MSPs to be used across different settings, purposes and specialities given their ability to evaluate data ¹⁴ from “various methodological and epistemological traditions” ¹⁹ .

To overcome SSR’s variable methodological steps, guidance and standards, this review adopts the Systematic Evidence Based Approach (SEBA) ²⁰ . A SEBA guided SSR (henceforth SSR in SEBA) facilitates the synthesis of an evidence-based, accountable, transparent, and reproducible analysis and discussion.

Steering this process and boosting accountability, oversight, and transparency, this SSR in SEBA sees an expert team involved in all stages of this review. The expert team comprised of medical librarians, local educational experts, and clinicians.

SSRs in SEBA are built on a constructivist perspective acknowledging the personalised, reflective, and experiential aspect of medical education and recognising the influence of particular clinical, academic, personal, research, professional, ethical, psychosocial, emotional, legal and educational factors upon the medical student’s learning journey, professional development and personal growth ²⁷ .

To operationalise the SSR in SEBA, the research team adopted the principles of interpretivist analysis to enhance reflexivity and discussions^18,32 in the six stages outlined in Figure 1 .

Figure 1.

The SEBA process.

(Insert Figure 1. The SEBA Process)

Stage 1 of SEBA: Systematic Approach

1. Determining the title and background of the review

The expert and research teams determined the overall goals of the SSR and the population, context and concept to be evaluated.

2. Identifying the research question

Guided by the PCC (population, concept and context), the expert and research teams agreed upon the research questions. The primary research question was “what is known about medical student portfolios?”. The secondary questions were “what are the components of MSPs?”, “how are MSPs implemented?” and “what are the strengths and weaknesses of MSPs?”.

3. Inclusion criteria

All peer reviewed articles, reviews and grey literature published from first January 2000 to 31^st June 2021 were included in the PCC and a PICOS format was adopted to guide the research processes^35,36. The PICOS format is found in Table 1 .

Table 1.

PICOS, inclusion and exclusion criteria.

PICOS	INCLUSION CRITERIA	EXCLUSION CRITERIA
Population	Undergraduate and postgraduate medical students	Allied health specialties such as Pharmacy, Dietetics, Chiropractic, Midwifery, Podiatry, Speech Therapy, Occupational and Physiotherapy Non-medical specialties such as Clinical and Translational Science, Alternative and Traditional Medicine, Veterinary, Dentistry
Intervention	The use of portfolios for medical students
Comparison	Comparison of the various use of portfolios (approaches, modalities, processes, objectives, motivations, challenges, facilitating characteristics/resources)
Outcome	Approaches, modalities, processes, objectives, motivations, challenges, facilitating characteristics/resources in the current and potential uses of portfolios Impact of the use of portfolios on medical students
Study design	Articles in English or translated to English Grey literature, case reports and series, ideas, editorials and commentaries Electronic and print information not controlled by commercial publishing All study designs including: Mixed methods research, meta-analyses, systematic reviews, randomised controlled trials, cohort studies, case-control studies, cross-sectional studies, descriptive papers Date of Publication: Jan 2000 – June 2021

4. Searching

A search on six bibliographic databases (PubMed, Embase, PsycINFO, ERIC, Google Scholar and Scopus) was carried out between first to 10^th September 2021. Limiting the inclusion criteria was in keeping with Pham et al’s (2014) approach to ensuring a sustainable research process ³⁷ . The search process adopted was structured along the processes set out by systematic reviews.

5. Extracting and charting

Using an abstract screening tool, members of the research team independently reviewed the titles and abstracts identified by each database to identify the final list of articles to be reviewed. Sambunjak et al’s (2010) approach to ‘negotiated consensual validation’ was used to achieve consensus on the final list of articles to be included ³⁸ . The six members of the research team independently reviewed all the articles on the final list, used the Medical Education Research Study Quality Instrument (MERSQI) ³⁹ and the Consolidated Criteria for Reporting Qualitative Studies (COREQ) ⁴⁰ , discussed them online and were in consensus that none should be excluded (Supplementary File 1).

Stage 2 of SEBA: Split Approach

Three teams of researchers simultaneously and independently reviewed the included full-text articles. Here, the combination of independent reviews by the various members of the research teams using two different methods of analysis provided triangulation ⁴¹ , while detailing the analytical process improved audits and enhanced the authenticity of the research ⁴² .

The first team summarised and tabulated the included full-text articles in keeping with recommendations drawn from Wong et al’s (2013) “RAMESES publication standards: meta-narrative reviews” ⁴³ and Popay et al’s (2006) “Guidance on the conduct of narrative synthesis in systematic reviews” ⁴⁴ . The tabulated summaries served to ensure that key aspects of the included articles were not lost (Supplementary File 1).

Concurrently, the second team of three trained reviewers analysed the included articles using Braun & Clarke’s (2006) approach to thematic analysis ⁴⁵ . In phase one, the research team carried out independent reviews, actively reading the included articles to find meaning and patterns in the data. In phase two, ‘codes’ were constructed from the ‘surface’ meaning and collated into a code book to code and analyse the rest of the articles using an iterative step-by-step process. As new codes emerged, these were associated with previous codes and concepts. In phase three, the categories were organised into themes that best depict the data. An inductive approach allowed themes to be “defined from the raw data without any predetermined classification”. In phase four, the themes were refined to best represent the whole data set. In phase five, the research team discussed the results of their independent analysis online and at reviewer meetings. ‘Negotiated consensual validation’ was used to determine a final list of themes.

A third team of three trained researchers employed Hsieh & Shannon’s approach to directed content analysis and independently analysed the included articles ⁴⁶ . This analysis using involved “identifying and operationalising a priori coding categories”. The first stage saw the research team draw categories from Davis et al.’s (2001) “AMEE Medical Education Guide No. 24: Portfolios as a method of student assessment” ⁴⁷ to guide the coding of the articles. Data not captured by these codes were assigned a new code in keeping with deductive category application. Categories were reviewed and revised as required. In the third stage, they discussed their findings online to achieve consensus on the final codes. These final codes were compared and discussed with the final author.

Stage 3 of SEBA: Jigsaw Perspective

As part of the reiterative process, the themes and categories identified were discussed with the expert team. Here, the themes and categories were viewed as pieces of a jigsaw puzzle and areas of overlap allowed these pieces to be combined to create a wider/holistic view of the overlying data. The combined themes and categories are referred to as themes/categories.

Creating themes/categories relied on use of Phases 4 to 6 of France et al.’s (2016) adaptation ⁴⁸ of Noblit and Hare's (1998) seven phases of meta-ethnography ⁵² . To begin, the themes and categories were contextualised by reviewing them against the primary codes and subcategories and/or subthemes they were drawn from. Reciprocal translation was used to determine if the themes and categories could be used interchangeably.

Stage 4 of SEBA: Funnelling Process

To provide structure to the Funnelling Process, we employed Phases 3 to 5 of the adaptation. We described the nature, main findings, and conclusions of the articles. These descriptions were compared with the tabulated summaries. Adapting Phase 5, reciprocal translation was used to juxtapose the themes/categories identified in the Jigsaw Perspective with the key messages identified in the summaries. These verified themes/categories then form the line of argument in the discussion synthesis.

Results

A total of 14501 abstracts were reviewed, 779 full text articles were evaluated, and 96 articles were included (see Figure 2 .). The funnelled domains identified were: Purpose of MSPs, Content and structure of MSPs, Strengths and limitations of MSPs, Methods to improve MSPs, and Use of E-portfolios.

Figure 2.

PRISMA flow chart.

Funnelled Domain 1: Purpose of MSPs

The purpose behind the employ of MSPs are often poorly explained and have been summarised in Table 2 for ease of review.

Table 2.

Purpose of MSPs.

CONTENT	ELABORATION AND/OR EXAMPLES
Learning	Reflective learning:7,11,47,53,54,56,57,61,62,64,66,68,69,72,74,75,79,80,86,87,91,101,108 Links practical experience with pre-existing medical knowledge 108 Collection and selection of work samples to allow for reflection and analysis of learning 125 Provides a platform to express inner dialogue 7 Self-directed learning7,11,53,54,62,64,65,67,71,73,91,101,102,105,109,115,117,126 Identify personal learning needs11,115 Individualise learning goals and plans 110 Workplace-based learning66,115,131 Encouraged by clinical components of portfolio 66 Group learning130,132
Assessment	Formative Assessment1,3,5,7,47,60,61,68,75,76,78,84,85,101,106,116,117,124,133 Platform to receive constructive feedback60,68,116,134 Summative Assessment1,5,13,47,54,59,68,70,71,78,80,82,102,105,106,116,121,123,124,130,135,136,139 Ensure students have met curriculum learning objectives by quantifying their performance through grades or numerical marks5,13,47,68 Results are utilised to inform decisions on promotion, graduation and licensing47,54,62,68,71,106,139 A combination of formative and summative assessment1,47,53,106 establishes portfolios as a “very powerful assessment tool” 47 Self-Assessment1,5,7,47,54,62,66,67,71,72,91,110,111,115,126,131,135 Students assess their own learning5,7,126, strengths and weaknesses 54 and performance47,72,111 Encourages positive changes in behaviour7,62,127 Modalities include interviews to discuss portfolio content between students and assessors1,4,11,57,68,108,123,131,136 or portfolio review by assessors1,4,8,11,54,59,70,75,91,105,111,112,116,117,123,131,134,140

Funnelled Domain 2: Content and structure of MSPs

1. Content in MSPs

Similarly, discussions on the contents of MSPs are limited and have been summarised in Table 3. The content can be broadly categorised into content provided by the institution, medical students, and feedback/assessments by other stakeholders.

Table 3.

Content in MSPs.

CONTENT	ELABORATION AND/OR EXAMPLES
Contributed by institution
Learning objectives	Institutions list out clear learning objectives that students can refer to as a guide for their learning5,7,47,54,57,67,76,79,81,102,108,111,121,124,140 Some institutions refer to professional accreditation guidelines5,7,67,76,79,82,102,108,111 For example, several institutions have made use of the Canadian CanMEDS framework79,82. Other institutions utilised descriptions of professional roles to substitute learning objectives54,57 as these are easier to comprehend 54 . For example, the university of Maastricht requested for its students to include evidence within their portfolio regarding their role as a ‘researcher’, ‘healthcare worker’, ‘medical expert’ and ‘person’. 54
Educational Resources	Web links 72 Graphics and streamed videos 72 Checklists to highlight OSCE steps 72 Training package on specific topics 72 Reflection writing framework68,129 E-Learning cases 82
Reflective prompts	Questions to stimulate student reflection3,7,54,64,65,68,69,86,103,108,115,127,144.
Contributed by medical student
Evidence of Activities	Curriculum Vitae54,138 Research projects47,55,67,80,145 Elective reports1,91 Presentations67,80,112 Personal achievements63,68,129 Membership in professional societies 145 Extracurricular activities 80 Evidence of learning activities Learning diaries53,65,72,144 Case summaries, reports, discussions11,47,67,128,131,136,146 Logbooks55,66,70,80,83,91,115 Essays to document progress in meeting competency standards47,70,105 Group Learning Assignments130,132 Graded assignments Workplace Based Assessments Mini CEX59,67,75,131 Direct observations59,75,85 Multi-source feedback (MSF) assessments59,75,99,131 Case based discussions59,75,85 Patient write-ups 67 Summative assignment and assessment grades59,67,80,85,91 Critical appraisals of a topic 131 Standardised patient assessments 67 Evidence based medicine project 67 Posting learning outcome grades 1 Progress test results 59 Anatomy lab 134 Small group assessments showcasing student’s teamwork skills 134 Longitudinal clinical preceptorships 134
Evidence of reflection	Written reflections from students1,3,6,7,11,53,54,57,59,62,63,65,67,69,72,75,86,87,91,103,106,108,111,115,116,131,135,147 Topics: ▪ Professional development/skills acquisition6,54,57 ▪ Plans for future self-development/improvement54,57,62,131 ▪ Personal learning goals1,53,57,62,72,116 Content: ▪ Patient encounters1,86,131 ▪ Short summaries of patients seen by the student and reflections on what they had learned in the process 1 ▪ Learning activities53,108 ▪ Activities may be those conducted internally or extra-curricular activities 108
Evidence of self-assessment	Performance in competencies59,70,72,80,111,115,123 and roles 7 Personal strengths and weaknesses 111 Personal learning61,115 and growth 47 Professionalism 47
Contributed by other stakeholders (eg assessors, peers)
Assessments	Assessors Tutors53,55,59,60,85,111,116,135,139,146 Faculty70,122,138 Peer assessors72,111,131,134 Patients 72 Examiners from courses taken in other faculties 63 Domains Clinical skills/competencies1,3,6,47,59,63,108,112,115,128,129,131,133,139 Communication skills111,115,117,139 Behavioural competencies 142 Authentic learning, referring to the learning of practical knowledge54,128,148 Personal and professional development1,47,55,67,70,91,131

2. Structure of MSPs

Standardisation within and across portfolios may be achieved through the use of a clear template ⁴ or set of guidelines ⁵³ . MSPs with clear delineation of contents required ⁵⁴ were found to boost student receptivity^55,56 and enhanced reliability and validity during portfolio assessment^47,55,57.

However, a flexible approach allowing medical students to personalise their MSPs ⁵⁸ and express themselves more freely ⁵⁹ facilitates portfolio student-centricity^60,61 and ownership ⁵³ . By encouraging students to incorporate their own content, such as reflective diary entries ⁵⁵ , reflective essays ⁵⁷ , video recordings ⁵⁸ , audio recordings ⁵⁹ , poetry or art ⁶² , improvements may be seen in the quantity and quality of their reflections ⁵⁶ .

Funnelled Domain 3: Strengths and Limitations of MSPs

Given the lack of elaboration, much of the data for this domain is summarised in tables to aid easy review.

1. Strengths

Strengths of MSPs are highlighted in Table 4 .

Table 4.

Strengths of MSPs.

STRENGTHS	ELABORATION AND/OR EXAMPLES
Learning	Highlights important skills and competencies Allows medical educators to reshape and redefine core concepts of medical practice through the development of portfolio criteria Streamlines learning and teaching focused on important competencies4,11,53,72,80,115,123,124,133 Stimulates learning5,11,74,77,102,109,118,130,132,135 Feedback provided highlights potential areas for improvement5,6,9,66 “Act of logging ‘learning moments’ helped facilitate memorisation” 109 May improve performance in other knowledge-based assessments 132 Promotes development of important skills Problem solving 132 Communication56,63,105,111,115,121,128,131,142 Ethical and legal responsibility7,53,87,149 Professional development5,8,11,47,53,56,63,74,78,86,103,105,111,116,123,131,135,141,142 Teamwork63,87,111,130,132,135,142 Critical thinking 121
Assessment	Examiners and faculty generally accept portfolios6,60,65,74,114,116,150 and their assessment methods1,6,116,135,140 as they are: Individualised47,55,63,117 Portfolio assessment can cater to a range of learning styles 117 because it can be easily personalised based on the student 55 . Unique evidence may be selected to express their capabilities to examiners 63 . Comprehensive1,54,61,70,83,117,123,126,135,137,140 “Combines information from both subjective and objective assessment procedures ‘to see the whole picture’” 140 Able to evaluate competencies that are otherwise not easily assessed1,54,83 such as professionalism123,137 Longitudinal1,47,67,74,80,99,117,133,141 Portfolios are assembled over a period of time and hence can be used to monitor student’s progress over the period of compilation Educational Use in assessment has helped stimulate learning1,66,73,74 Guides tailored teaching by faculty members54,91,126,133,134 Guides remediation plans for underperforming students1,62,91,105,111,116,135,140,142 Specific to summative portfolio assessment: Ensures that students take the portfolio exercise seriously57,114 Students will be spurred on to improve themselves should they receive negative feedback 75 Better demonstrates achievement in competencies such as professionalism, teamwork, and communication skills 111 Specific to formative portfolio assessment: Enables constant improvement through feedback and reflection6,7,60,71,75,105,116,127,133,140 Fosters self-motivation5,69 and intrinsic motivation to reflect91,115.
Others	Encourages students to discuss their private thoughts 103 Prepares students for postgraduate work Easily transferable when needed in the future 80 to facilitate job applications103,104 or acquisition of letters of recommendation for future training 80 Helps to ease transition to postgraduate educational practice 74 as portfolios and portfolio assessment are often utilised at postgraduate level 55 Improves teaching within undergraduate programs Improves faculty’s understanding of students ▪ Better understand students’ thinking and attitudes 65 ▪ Directs discussion during meetings with advisees65,74 Identifies gaps in the curriculum56,101 such as through providing opportunities for students to evaluate teaching activities 56 Helping students to develop better rapport with others including patients62,118,122, clinical teams 62 and other students 132

2. Limitations

The limitations of MSPs are highlighted in Table 5 .

Table 5.

Limitations of MSPs.

LIMITATIONS	ELABORATION AND/OR EXAMPLES
Learning	Limited use for theoretical knowledge 121 Limited use for reflective learning Does not guarantee that reflection will take place7,54,56,64,78,87,103 Students are sceptical about the reflective process53,67,68,87,110 Challenging for individuals who are not intuitively reflective64,72 Overly prescriptive structure of reflective prompts may hinder reflective process 64
Assessment	Limited reliability and validity4,54,55,59,62,63,71,72,91,108,111,112,117,135,137 Inauthentic ▪ Provide only vignettes of a student’s journey 59 , and students may hide evidence of their weaknesses54,59,63,70,104,126, fail to express their authentic views 63 or even fabricate reflections 78 ▪ They may also perform poorly under stress during assessments included in their portfolios such as directly observed work-based assessments59,137 ▪ Students tend to have a poor self-assessment capacity72,111,151 ▪ Perceived quality of portfolio relies heavily on the individual’s reflective ability55,105,121 which is unfavourable for students with poor reflective skills Subjective ▪ Students may create their portfolios differently based on their own interpretation of the purpose of the portfolio 59 ▪ Student’s portfolios may unknowingly be judged on irrelevant aspects such as layout and format 4 ▪ This may be amplified if student identity is not anonymised to examiners evaluating the portfolios 119 Overly structured47,53,57,59,62,64,119 ▪ Highly structured portfolios with a rigid format can lead to students including less of their personal observations and reflections, which diminishes the portfolio’s capacity for authentic assessment of the student and their development Problematic assessment process Poor student understanding11,53,62,63,73,104,116 Time consuming ▪ There may be insufficient time for comprehensive assessments in the clinical setting as taking time to assess students must be balanced with providing quality patient care 59 ▪ Time consuming for assessors1,5,11,13,53,55,60,63,65,68,74,104,112,116,140 ▪ Human resource intensive6,112,137,140 ▪ Excessive paperwork1,55,74,106 Lack of standardisation among examiners ▪ Poorly standardised assessment procedure leads to poor consensus among assessors 117 Lack of training for assessors limits the use of work-based assessments within portfolios for assessing student competence 137
Portfolio Implementation	Negative student sentiments Resistance5,11,53,59,61,63,66,67,74,102,104,106,126 ▪ Perceived to be redundant61,102 and incompatible with studying format61,77,78 Non-priority ▪ Students prioritise coursework that contributes towards their final examination marks 146 ▪ Interference with other studies 123 , including clinical learning 91 and time that should be spent with patients 1 or studying for exams 78 Poor understanding and engagement1,4,54,61,66,74,78,108,150 ▪ Unaware of how portfolios can be integrated into their education 110 ▪ Stressful 78 and difficult to fill out61,78 Burdensome ▪ Time consuming11,66,79,108,115,116 ▪ Excessive paperwork1,55,77,102,106,108 Worried about the negative comments they could receive from their mentors 61 Felt the time given to complete their portfolios was too short, leading to reduced value 123 Lack of support from mentors64,66,110 Not all mentors provided feedback and engaged the students64,78,103,118 Factors leading to faculty’s lack of support ▪ Poor time management 64 ▪ Failure to understand role as portfolio mentors64,110 ▪ Did not engage in reflection personally 64 ▪ Difficulty finding methods to help students 78 ▪ Poor impression of portfolios and their role in education66,78 ▪ Poor relationship with student 103

Funnelled Domain 4: Methods to Improve MSPs

The potential methods to improve MSPs are highlighted in Table 6 .

Table 6.

Methods to improve MSPs.

METHODS	ELABORATION AND/OR EXAMPLES
Increase Mentorship Mentorship refers to a system where students are assigned to faculty throughout their training and portfolio creation to coach them54,57,101, engage them in supportive dialogue63,64,108,118,148, provide feedback1,61,63,64,133 and encourage them to fully engage with their portfolios74,78,103,131,146.
Benefits of Mentorship	Crucial to portfolio success4,7,63,64,78,79,87,104,131 because it helps guide the students’ reflective process57,65,131,146, enhances learning1,57,74,135 and increases student receptivity towards their use 7,64,103
Improving quality of mentorship	Train mentors66,78,87,123 and utilise verified teaching methods that foster reflection 152 and ensure mentors are able to stretch their students in their reflective practice 78 Recruit good mentors Willing to engage students 108 Understands reflection 129 and their responsibility to teach students how to utilise reflections purposefully 79 Able to build trust and rapport with students 64
Having a structured mentoring programme to guide portfolio use	Some institutions encourage frequent weekly meetings with mentees 108 , while others believe that mentorship can occur as infrequently as two to three times a year4,57,64 Keep the student to mentor ratio small such as having one-to-one interactions6,70,79
Encourage portfolio uptake
Improve understanding	Students with a better understanding of portfolio usage had more positive attitudes towards portfolios 108 Introduce and orientate students to the portfolio6,54,57,61,63,73,104,108 Educate students on purpose and objectives of portfolio62,64,67,70,101,104,123 Provide clear instructions and portfolio guidelines7,61,63,70,73,102,104,108,114,116,118,123,143 Structure portfolios clearly4,7,53,54,56,57,60,64,65,70,91,102,114,121,123
Increase Exposure	Students who had been exposed to them for some time6,91 had more positive attitudes towards portfolios. Embed portfolio into the curriculum54,64,72,104 and encourage faculty and department staff to reference it in daily practice 77 Early portfolio introduction54,129
Structure portfolio appropriately
Organise portfolio based on its purpose	Organise the portfolio based on its purpose 125 . For a portfolio focused on enhancing learning, the portfolio should include more self-reflection54,56 and reasoned tasks that demonstrate student learning 56 . For a portfolio meant for assessment, content should mainly compose of evidence that competencies have been achieved 5 and prompts should be minimal as the student's choice of reflection is also important in assessment 143 If the portfolio is meant to promote reflection, design the portfolio to ensure it is conducive for reflection Provide reflective prompts3,7,54,64,65,68,86,108,119,127,143,144 Increase emphasis on writing reflections rather than describing activities 108 Refrain from limiting word count 62 Utilise innovative tools such as the visual analogue scale 151 or audio recordings 59 Portfolios should also be organised to facilitate effective teaching by faculty 56
Improving portfolio assessment process
Enhance learning through assessment process	Focus assessment on promoting student development 88 through providing useful feedback121,124 Enhance reflective learning Ensure assessment does not compromise reflection 54 Assess students based on the authenticity of their reflections 53 Institute a central committee to review assessments and ensure ample learning experiences and assessment evidence exist to guide student learning 70
Standardisation	Standardisation improves the reliability of the assessment process8,72,116,131 The following may be standardised Portfolio content8,72,116,131 Standardising assessment criteria1,8,47,55,72,112,116,124,131,135 including standardising portfolio interview questions1,13
Improve assessment procedure	Prepare students adequately for the assessment91,105,116,131 by providing guidelines on the purpose and format of the assessment 116 , clarifying expectations 91 , providing guidance from trained portfolio advisors105,131. Ensure assessment occurs immediately after a clinical experience 129 Increase number of assessment points such as by adopting more work-based assessments within the portfolio 137 Reduce subjectivity of assessment Create and validate clear rubrics to assist assessors in their grading of students 121 Increase number of assessors to achieve better inter-rater reliability62,72,112,121 Provide training to assessors4,53,62,64,67,68,74,85,87,104,111,121,124,135 Providing opportunities for discussion or feedback between assessors4,8,63,72,105,111,116,117,124 Introduce portfolio interviews where students can discuss and elaborate upon their portfolios personally4,8,53,72,105,116,140 or even assess their own portfolios5,55
Improve self-assessment process	Encourage students to include evidence to support their self-assessments to reduce inaccurate self-assessments 111
Evaluate Feedback
Importance	Student empowerment and feedback have all been valuable tools in successful portfolios47,53: Allows for evaluation and alignment of portfolio with teaching, learning and assessment data 113 Help to ensure the portfolio is being used appropriately11,68,74 Helps to introduce positive changes11,47,62,78

Funnelled Domain 5: E-Portfolio

The electronic portfolio (e-portfolio) is a form of MSP that is hosted on electronic platforms^{5,6,9,47,53,56,58,61,63}, and may be created using unique software^{47,63,65,76,86}. Compared to hardcopy portfolios, they are more durable ⁶⁶ , user friendly^63,75,77, accessible^{6,53,58,61,80} collaborative^{5,67,73,76,81} and superior for assessment in certain areas ⁶¹ . Furthermore, they are able to include a wider variety of evidence including videos or website links^{5,63,75,78,79}, provide increased privacy and confidentiality for users including students and coaches^67,73,86 and allow for instant comparison between students ⁷⁶ . These factors enhance their receptivity among medical students^53,61,63.

However, accessibility may be limited by poor interface design^{64,67,73,74,77,87,88}, limited administrative support^67,73,88, poor technology^66,67,73,79, and a lack of time or finances to upgrade and support e-portfolio technology ⁶⁷ . Similarly, the lack of immediate access to computers in a clinical setting^58,66,73, poor data security^58,65,66, issues with communicating with mentors online ⁶⁴ or mentors not being tech-savvy ⁶⁷ also limit their applicability.

Stage 5 of SEBA: Analysis of Evidence-Based and Non-Data Driven Literature

Evidence-based data from bibliographic databases were separated from grey literature such as opinion pieces, perspectives, editorial, letters and non-data based articles drawn from bibliographic databases and both groups were thematically analysed separately. The themes from both groups were compared to determine if there were additional themes in the non-data driven sources that could influence the narrative. In this review, the themes from the two data sources overlap, suggesting no undue influence upon the findings of this review.

Stage 6 of SEBA: Synthesis of SSR in SEBA

The narrative produced from consolidation of the funnelled domains was guided by the Best Evidence Medical Education (BEME) Collaboration guide ⁸⁹ and the STORIES (Structured approach to the Reporting In healthcare education of Evidence Synthesis) statement ⁹⁰ .

Discussion

In answering its primary and secondary research questions, this SSR in SEBA reveals that MSPs have expanded beyond merely repositories of assessments and are now seen as a means of triangulating and contextualising assessments and their impact upon individual medical students. MSPs also allow students, faculty, and institutions to better understand the medical student’s needs, abilities, expectations, and aspirations, aiding the provision of personalised mentoring and remediation. However, to meet these wider roles, manageable ⁸⁷ and “authentic” portfolios that improve levels of engagement ⁹¹ are key. Here, authenticity refers to the “extent to which the outcomes measured represent appropriate, meaningful, significant and worthwhile forms of human accomplishments” ⁴⁷ and serves to enhance the trustworthiness of what is largely qualitative data, and the validity of longitudinal assessments that help to map the development of their clinical competency ⁴ and professional identity formation^4,12,92.

However, current MSPs lack a consistent structure. While broad commonalities including learning objectives and professional expectations and roles to be met, and reflections, learning activities, self-assessments, achievements, and other evidence of competencies, MSPs vary significantly in their focus and content. Yet, these variations and particularities are unsurprising given the different practice settings, structure and program goals established by the host institution. These differences underpin the presence of different types, “depth” and nature of content prioritised. Inherent variability brought about by personalisation of longitudinal data, “choice of materials by the student” ⁵⁴ and “individualised selection of evidence” ⁴⁷ , ultimately limits the use of portfolios beyond the confines of a specific institution. This lack of consistency raises concerns about the efficacy of MSPs in providing a holistic perspective of the medical student’s personal, academic, clinical, and professional development.

We believe that these concerns may be bridged in part by harnessing the ability of current MSPs to capture education and assessment in specific areas of practice. Our findings suggest that current MSPs encapsulate several entrustable professional activities (EPA)s ⁹⁴ . Each EPA however shares common aspects of other EPAs that may not be directly contained within a particular MSP. We believe that it is possible to harness these overlapping aspects to make MSPs more widely applicable. Here, we build upon the notion that micro-credentialling that incorporates “circumscribed assessments” of a specific EPA, such as “interpreting and communicating results of common diagnostic and screening tests”, may be extrapolated to other EPAs such as “[communicating] in difficult situations” in a different practice setting ⁹⁷ .

Hong et al’s (2021) and Zhou et al’s (2021) adaptations^98,99 of Norcini’s (2020) concept of micro-credentialling and micro-certification in medical education ¹⁰⁰ which forward the concepts of generalised and personalised micro-competencies provide a viable bridge between prevailing MSP content without compromising the rich mix of structure and customisation within MSPs. Based on the certification of micro-competencies within an EPA, Zhou et al. (2021) suggest that generalised micro-competencies are the standards and expectations applicable to all medical students. They are small, professional learning milestones that all students need to attain before proceeding to the next competency-based stage. These are requisite knowledge, skills and attitudes all soon-to-be clinicians must have. Personalised micro-competencies, in turn, are determined by the individual’s particular goals, training, abilities, skills and experiences. They are determined by the medical student and tutors and must be consistent with institutional codes of conduct and expectations. They underscore the importance of assessing the student's individual needs and circumstances which influence which in turn shape the kind of training and support proffered. With expectations differing across practice settings and levels of training, both generalised andpersonalised micro-competencies must be clearly conveyed to the medical student and tutors in a timelyand structured manner. To encapture their learning and attainment, MSPs must forward clear learning plans to align expectations with evidence of diverse learning activities, reflective prompts and diaries, multisource formative and summative evaluations via standardised assessment tools and constructive feedback. These standardised baseline guidelines will lend clarity to portfolio developers and users. This may boost the latter’s trust and receptivity towards regular portfolio use^55,56.

We believe that structured and consistent micro-certification of micro-competencies could be extrapolated beyond the initial goals of the MSPs and could provide a longitudinal perspective of the medical student’s development. This is especially useful when considering competencies such as interpersonal, communication skills and systems-based practices. Perhaps here, too, the silver lining to changes in medical education practices due to the COVID-19 pandemic can be harnessed.

With many institutions incorporating online learning, e-portfolios should be institutionally sanctioned ⁸⁵ with a dedicated team of portfolio developers and invested faculty members onboarding and overseeing their implementation. These considerations foreground the need for orientation sessions^{10,62,64,67,104} to educate students and faculty on the identified EPAs as well as the use of generalised and personalised micro-competencies to ensure learning and assessment congruity and objectivity^91,105,106. Embedding the portfolios into the formal curricula, assigning students mentors trained in reflective engagement, and establishing protected time for regular portfolio reviews would help to facilitate their consistent usage. Concurrently, portfolio use must be part of a continuous quality improvement process, building on feedback ¹⁰⁷ and lessons learnt to promote further improvement to MSPs and portfolio assessment^{10,11,47,62,78}. Indeed, both forms of micro-competencies underline the need for effective recording and oversight. This is especially important when micro-competencies provide a holistic appraisal of the medical student’s progress and achievements, needs and abilities and provides insights into their professional identity formation. Capturing this data in a comprehensive, longitudinal manner replete with the medical student’s reflections reveals a new dimension to portfolio use.

Limitations

Firstly, the review is limited by the omission of articles not published in English. This creates the risk of missing key papers. Furthermore, the focus on papers published in English led to focus on studies in North America and Europe.

Secondly, while the articles comment on the sentiment of users including medical students on the effectiveness of portfolios for learning and assessment, there are a limited number of articles highlighting the perspectives of doctors who previously undertook the task of undergraduate portfolios. Hence, the review is limited by its inability to assess the long-term effectiveness and acceptability of portfolio usage after medical students enter the workforce as practicing medical professionals.

Conclusion

This SSR in SEBA reveals that if portfolios are to remain relevant and maintain their user-friendliness and accessibility, the future of MSPs must lie in improving assessments and in enhancing the manner in which they are designed.

While it is clear that assessments tools need to be enhanced to meet new perspectives of education and training, it is perhaps timely that this SSR in SEBA suggests key changes to portfolio use. In adopting e-portfolios for its accessible and expansive potential, it is clear that a robust and well-supported platform is critical. This platform ought to accommodate all manner of data and assessment results and remain a comprehensive repository of data. Categorised into different, sometimes overlapping, domains, data from this repository may be drawn to populate different designs of MSPs. Changing from one goal to another should therefore be simple. Such flexibility will still allow medical students to personalise their e-portfolios in a manner that they feel best represents their development without compromising faculty evaluation. A flexible yet robust e-portfolio such as this will also enable collaborations and facilitate input of corroborative data from third parties where required.

Moving forward, further research may be undertaken to identify the long-term effects of portfolio usage, the manner that portfolios are evaluated, and the impact it has on professional identity formation throughout and beyond medical school.

Glossary Terms

Professional Identity Formation

An adaptive developmental process that involves the psychological development of an individual, and the socialisation of the individual into appropriate roles and participation at work.

Krishna’s Systematic Evidence-Based Approach (SEBA )

A structured and accountable approach used to guide analyses to ensure reproducible and robust data.

Split Approach

Combines content and thematic analysis of data to enhance the trustworthiness and depth of an analysis.

Jigsaw Perspective

Comparing overlaps between the themes and categories delineated by content and thematic analysis are considered in tandem, like complementary ‘pieces of the jigsaw’. This allows for holistic perspective of data.

List of abbreviations

EPA

Entrustable Professional Activities

MSP

Medical Student Portfolios

PCC

Population, concept and context

SEBA

Systematic Evidence-Based Approach

SSR

Systematic Scoping Review

Biography

Ms Rei Tan is a medical student at the Yong Loo Lin School of Medicine, National University of Singapore, Singapore. Email: e0232945@u.nus.edu

Ms Jacquelin Jia Qi Ting is a medical student at the Yong Loo Lin School of Medicine, National University of Singapore, Singapore. Email: jacting@gmail.com

Mr Daniel Zhihao Hong is a medical student at the Yong Loo Lin School of Medicine, National University of Singapore, Singapore. Email: hongzhihao@live.com

Ms Annabelle Jia Sing Lim is a medical student at the Yong Loo Lin School of Medicine, National University of Singapore, Singapore. Email: annabellelimjs@gmail.com

Ms Yun Ting Ong is a medical student at the Yong Loo Lin School of Medicine, National University of Singapore, Singapore. Email: yunting.ong08@gmail.com

Ms Anushka Pisupati is a medical student at the Yong Loo Lin School of Medicine, National University of Singapore, Singapore. Email: anushka.pisupati@u.nus.edu

Ms Eleanor Jia Xin Chong is a medical student at the Yong Loo Lin School of Medicine, National University of Singapore, Singapore. Email: eleanor.chong@u.nus.edu

Ms Min Chiam, MSc (Medical Humanities) is a researcher at the Division of Cancer Education, National Cancer Centre Singapore, Singapore. Email: chiam.min@nccs.com.sg

Ms Alexia Sze Inn Lee, BSc (Psychological Science) works at the Division of Cancer Education, National Cancer Centre Singapore, Singapore. Email: "lee.sze.inn@nccs.com.sg

Dr Laura Hui Shuen Tan, MBBS graduated from the Yong Loo Lin School of Medicine, National University of Singapore, Singapore. Email: laura.tan.hs@gmail.com

Ms Annelissa Mien Chew Chin, MSc (Info & Lit) is a Senior Librarian at the Medical library, National University of Singapore libraries, Singapore. Email: annelissa_chin@nus.edu.sg

Dr Limin Wijaya MBBS (Melbourne), MRCP (UK), DTM&H (Liverpool), is a senior consultant at the Division of Infectious Disease, Singapore General Hospital, Singapore. Email: limin.wijaya@singhealth.com.sg

Dr Warren Fong MBBS, MRCP, FAMS, is a doctor at the Department of Rheumatology and Immunology, Singapore General Hospital, Singapore. Email: warren.fong.w.s@singhealth.com

Prof Lalit Kumar Radha Krishna MBChB, FRCP, FAMS, MA (Medical Education), MA (Medical Ethics), PhD (Medical Ethics) is a Senior Consultant at the Division of Supportive and Palliative Care at the National Cancer Centre Singapore, Singapore. Prof LKRK holds faculty appointments with the Centre for BioMedical Ethics, Duke-NUS Medical School and the Yong Loo Lin School of Medicine, National University of Singapore, Singapore. Email: lalit.radha-krishna@liverpool.ac.uk