Assessing Professionalism in Medicine – A Scoping Review of Assessment Tools from 1990 to 2018

Abstract

Background:

Medical professionalism enhances doctor-patient relationships and advances patient-centric care. However, despite its pivotal role, the concept of medical professionalism remains diversely understood, taught and thus poorly assessed with Singapore lacking a linguistically sensitive, context specific and culturally appropriate assessment tool. A scoping review of assessments of professionalism in medicine was thus carried out to better guide its understanding.

Methods:

Arksey and O’Malley’s (2005) approach to scoping reviews was used to identify appropriate publications featured in four databases published between 1 January 1990 and 31 December 2018. Seven members of the research team employed thematic analysis to evaluate the selected articles.

Results:

3799 abstracts were identified, 138 full-text articles reviewed and 74 studies included. The two themes identified were the context-specific nature of assessments and competency-based stages in medical professionalism.

Conclusions:

Prevailing assessments of professionalism in medicine must contend with differences in setting, context and levels of professional development as these explicate variances found in existing assessment criteria and approaches. However, acknowledging the significance of context-specific competency-based stages in medical professionalism will allow the forwarding of guiding principles to aid the design of a culturally-sensitive and practical approach to assessing professionalism.

Background

Medical professionalism underpins the development of trusting doctor-patient relationships that help inform and guide the delivery of socioculturally sensitive, patient-centric care and enhance healthcare outcomes and overall patient satisfaction.^1-4 It forms the cornerstone of effective self-regulation and the promulgation of a transparent, accountable and evidence-based clinical practice.^1-3 Ensuring that medical professionalism is effectively practised is thus pivotal to the standing of the medical profession, the preservation of public trust and the provision of quality healthcare.^5-8

However, the perception of healthcare services as commodities subjected “to the forces of commercialisation and profit-making in the free-market economy”⁹ has seen the erosion of professional, moral and ethical values in medical practice. In addition to this scrupulous emphasis on productivity, the profession has struggled to “hold up against [the] prevailing philosophy of moral scepticism and relativism in the present post-modernist society”.⁹ Pervasive technological changes to the therapeutic relationship has also led to the blurring of professional boundaries between doctors and patients^10-12 and these issues have complicated oversight of appropriate professional behaviour in medicine.^11,13 These considerations underscore the need to better assess medical professionalism.^14-17

Defining medical professionalism

A consistent understanding of medical professionalism remains elusive in part due to the influence of practical and local sociocultural, legal, financial, educational and healthcare considerations.^3,18-30 This diversity is captured in Birden, et al’s³¹ comprehensive overview of key concepts in professionalism and in the definitions/domains of medical professionalism proffered by the Accreditation Council for Graduate Medical Education (ACGME)³² in the United States, the General Medical Council (GMC)³³ in the United Kingdom and the Canadian Medical Education Directives for Specialists (CanMEDS)³⁴ (Table 1).

Table 1.

Definition/domains of medical professionalism.

Healthcare regulatory organisations	Definitions/domains of medical professionalism
Accreditation Council for Graduate Medical Education (ACGME)	(a) Demonstrating professional conduct and accountability. (b) Demonstrating humanism and cultural proficiency. (c) Maintaining emotional, physical, and mental health. (d) Pursuing continual personal and professional growth.
General Medical Council (GMC)	The domains for professionalism include: (a) Knowledge, skills, and performance • Make the care of your patient your first concern. • Provide a good standard of practice and care. (b) Safety and quality • Take prompt action if you think that patient safety, dignity or comfort is being compromised. • Protect and promote the health of patients and the public. (c) Communication partnership and teamwork • Treat patients as individuals and respect their dignity. • Work in partnership with patients. • Work with colleagues in the ways that best serve patients’ interests. (d) Maintaining trust • Be honest and open and act with integrity. • Never discriminate unfairly against patients or colleagues. Never abuse your patients’ trust in you or the public’s trust in the profession.
Canadian Medical Education Directives for Specialists (CanMEDS)	As Professionals, physicians are committed to the health and well-being of individual patients and society through ethical practice, high personal standards of behaviour, accountability to the profession and society, physician-led regulation, and maintenance of personal health. • Commitment to patients • Commitment to society • Commitment to the profession • Commitment to self

Recognising this diversity in concepts, the Ottawa Consensus Group meeting in 2018 acknowledged the need to explore “the perspectives of patients and the tensions of individual and institutional values in regard to professionalism” as opposed to “a standardised, single, and reductionist definition”.² This position underlines the need for context-specific, culturally appropriate and linguistically sensitive assessments.^1,2

The need for this review

The absence of such a context-specific, culturally appropriate and linguistically sensitive assessment tool within the Singapore setting inspired this scoping review. It is hoped that the insights proffered will guide design of a tool for the local context.

Methods

Building upon Li et al’s,³⁵ Wilkinson et al’s⁴ and Veloski et al’s³⁶ reviews, this scoping review explores the size and scope³⁷ of published data in peer-reviewed literature on tools used to assess medical professionalism.^38-43 Use of a scoping review also allows for the systematic extraction, synthesis and summarising of actionable and applicable information⁴⁴ in available literature^45,46 across a wide range of pedagogies, assessment contents and practice settings.^47-51 Levac et al’s⁵² adaptation of Arksey and O’Malley’s³⁸ framework for scoping reviews was used to map key concepts, sources and types of evidences available to guide future research.^39,53,54 Levac et al’s framework was also guided concurrently by the PRISMA-P 2015 checklist⁴⁵ and a 6-stage scoping review protocol was developed for this study.^38-42,52

Stage 1: Identifying the research question

The 11-member research team was guided by a team of local clinicians, educators and researchers from Yong Loo Lin School of Medicine at National University of Singapore (YLLSoM), Duke-NUS Medical School, the University of Liverpool and the National Cancer Centre Singapore (NCCS), and two librarians from the medical libraries at YLLSoM and NCCS (henceforth the expert team). The expert and research teams discussed prevailing issues, needs, and practices pertaining to assessment tools used in medical professionalism and determined the primary research question to be: ‘what tools are available to assess medical professionalism among medical students or physicians?’ The teams determined the secondary research questions to be: ‘what domains are assessed and what approaches are used to carry out these assessments?’ and ‘what existing frameworks are used to guide them?’ These questions were designed based on prevailing population, conceptual and contextual considerations⁵⁵ using a PICOS format (Table 2).

Table 2.

PICOS, inclusion and exclusion criteria.

PICOS	Inclusion criteria	Exclusion criteria
Population	• Undergraduate and postgraduate medical students, residents or doctors within the clinical, medical, research or academic settings	• Allied health specialties such as Pharmacy, Dietetics, Chiropractic, Midwifery, Podiatry, Speech Therapy, Occupational and Physiotherapy • Non-medical specialties such as Clinical and Translational Science, Alternative and Traditional Medicine, Veterinary, Dentistry
Intervention	• Curricular assessment tools on professionalism of medical students or doctors
Comparison	• Comparisons of the various assessment tools, including the assessment principles, modalities and criteria
Outcome	• Assessment principles, modalities, and criteria • Impact of assessment on those being assessed
Study design	• Articles in English or translated to English • All study designs including:  ○ Mixed methods research, meta-analyses, systematic reviews, randomised controlled trials, cohort studies, case-control studies, cross-sectional studies, and descriptive papers • Year of Publication: 1 January 1990 to 31 December 2018 • Databases: PubMed, ERIC, Google Scholar, Web of Science databases	• Grey Literature/electronic and print information not controlled by commercial publishing • Case reports and series, ideas, editorials, and perspectives

These collaborative consultations between the research and expert teams also took place at Stage 1, 2, 3 and 6 of the research process.

Stage 2: Identifying relevant studies

Guided by the expert team and prevailing definitions and descriptions of medical professionalism by ACGME, GMC and CanMEDS,^32,33,56 the research team developed their search strategy which may be found in Online Appendix A.

In keeping with Pham et al’s⁵⁴ approach to ensuring a viable and sustainable research process, the research team confined the searches to articles published between 1 January 1990 and 31 December 2018 on PubMed, ERIC, Web of Science and Google Scholar. All qualitative and quantitative research methodologies in peer-reviewed articles published in English or had English translations were included. The independent searches of the four databases were carried out between 13 February 2019 and 24 April 2019.

Stage 3: Selecting studies to be included in the review

The nine members of the research team independently screened the titles and abstracts by exporting and organising the articles using the Endnote software. This allowed for the removal of duplicate titles and the generation of a tentative list. Sandelowski and Barroso’s⁵⁷ ‘negotiated consensual validation’ approach was used to agree on the final list of articles to be reviewed and, later, the final list of full text articles to be scrutinised.

A summary of the PRISMA process may be found in Figure 1.

Figure 1.

PRISMA flow chart.

Stage 4: Data characterisation and analysis

Braun and Clarke’s⁵⁸ approach to thematic analysis was adopted to scrutinise articles with different goals and populations of medical students and doctors. It circumnavigates the context-specific and socioculturally influenced nature⁴² of medical professionalism^{1-3,31,35,58-60} and the wide range of research methodologies present among the included articles that prevent the use of statistical pooling and analysis.^58,61-66

The members of the research team independently reviewed and ‘actively’ read the included articles to find meanings and patterns in the data to construct ‘codes’ from the ‘surface’ meaning of the text.^58,67-69 The initial codes from ‘open coding’ were grouped into categories then into themes. The reviewers reconvened at online and face-to-face meetings to discuss their individual findings.

Stage 5: Collating, summarising, and reporting the results

A total of 3799 abstracts were identified from four databases, 138 full-text articles reviewed, and 74 full-text articles were analysed.

The narrative produced was guided by the Best Evidence Medical Education (BEME) Collaboration guide⁶⁴ and the STORIES (Structured approach to the Reporting In healthcare education of Evidence Synthesis) statement.⁷⁰

In addition, two reviewers carried out individual appraisals of quantitative studies using the Medical Education Research Study Quality Instrument (MERSQI)⁷¹ and the Consolidated Criteria for Reporting Qualitative Studies (COREQ)⁷² to evaluate the quality of qualitative and quantitative studies included in this review. They met face-to-face to reconcile any differences in their assessments and forwarded a consensus-based appraisal of the included studies. A summary of these quality assessments may be found Online Appendix B.

Stage 6: Consultations with expert team and key stakeholders

Whilst the findings were well-received by the expert team, several members stressed the need to consider the impact of the diverse assessment methods, approaches, goals, and populations in relation to the local educational, healthcare and linguistic setting. This is so as to facilitate development of a cohesive, longitudinal and holistic system of assessments with appropriate integration of individual assessment tools calibrated for various training stages within the Singaporean setting.

Results

For transparency and ease of review the themes are delineated in tables. The initial themes identified were attributes and assessment criteria for professionalism (Table 3a and b); tools, approaches and modalities used to assess professionalism (Table 4); and prevailing assessment frameworks established by healthcare regulatory organisations (Table 5).

Table 3a.

Attributes of professionalism.

General attitudes and behaviour • Confidence73 • Innovation74 • Collegiality73 • Altruism21,24,29,75-77 • Self-motivation78,79 • Commitment to lifelong learning4,73,75,77,78,80 • Commitment to profession23,73,75,81,82 • Upholding standards of excellence21,29,76,83

Ethical attitudes and behaviour • Integrity21,24,29,30,73-77,83-87 • Equity24 • Respect21,24,25,73,76-78,85,88-91 • Confidentiality75,77,83,84,87,88,91,92 • Trustworthiness21,23,29,30,93,73,77,78,80,81,84-87,94-97 • Responsibility21,28,75,82,83,88 • Accountability77,78,81,83,85,86,88,98-100 • Upholding justice and social responsibility21,75,78,80,89,92,95,100 • Upholding moral fiduciary relationship and duty24,82

Patient care • Patient-centred approach78,89,90,95-97 • Empathy85,87,90-92,97,101 • Humanism73,80,82,102-104 • Compassion75,77,83,84,87,88,91,92 • Cultural sensitivity and proficiency29,73,77,80,82,91-93,97,102

Table 3b.

Assessment criteria for professionalism.

Attitudes	• (Perceived) facilitators and barriers to professional and unprofessional behaviour25,85,100,105-107 • Awareness of exposure to unprofessional behaviour in the informal and hidden curriculum28-30,86,92,100,105,107-110 • Adopting a humanistic attitude towards medicine10 • Self-efficacy in acquiring professionalism skillsets and in interpersonal interactions and teamwork86,89,93,105,111,112 • Focusing on patient’s needs above self and empathising with and respecting patient’s dignity90,91,96,101,103,106
Behaviours	Individual level • Balancing competing demands of research, clinical, personal and fellowship life90 • Asking for help when needed87 such as for mental and physical abuse87 • Maintaining composure in difficult situations87,104 • Establishing personal-professional lifestyle balance • Ensuring “protected time” for scholarly pursuits and spontaneously pursuing own area of interest106,113 • Adhering to and role modelling ethical conduct and professionalism25,29,77-79,100 • Respecting and cooperating with authority4,73,80 • Displaying lifelong learning and adaptability as well as taking initiatives, being punctual and attentive4,23,26,30,73,77-80,85,87,97,104,105 • Responding constructively to assessments and criticism23,30,73,75,77-79,87,91,92,93,97,99,104,114 Interpersonal level • Ensuring the following unprofessional behaviours are not performed • Acting compulsively30 • Venting emotions against patients, family, and colleagues90,96 • Using derogatory language92,104 • Pushing the blame onto others112 • Denigrating the medical profession30 • Respecting and collaborating with fellow healthcare professionals30,78,90,97,104,111 • Participating proactively in team-based learning115 • Teaching and assisting peers, juniors and students78,101,113 • Providing constructive feedback,25,74,89,92,97,100,101,111,115 accepting and soliciting feedback73,74,81,86,92,104,115 • Reporting dishonesty78 • Avoiding abuse of power78,92 • Respecting and collaborating with patients30,78,89,90,95,96,103,104,111 • Providing holistic care and showing interest in patient as a person73,80,82,102-104 • Sharing decision-making with patients and acknowledging and advocating for patient autonomy91,92,97,103 • Communicating honestly92,103 and empathetically92,101,103,104 • Respecting patient’s family30,78,89,90,95,104 • Managing conflicts of interest78,95 Organisational and Societal Level • Involved in citizenship and professional engagement85 • Treating underprivileged patients and improving access to medical care such as through joint and appropriate distribution and use of healthcare resources78,82,89,95,104,113 • Bringing positive influence to the work environment87
Clinical skills and competencies	Interpersonal skills with fellow healthcare professionals • Communication,30,83,92,104 leadership,87,89,99 teamwork,77,89,99 educational knowledge and competencies94 Interpersonal skills with patients • Communicating well with patients25,26,73,74,80,84,85,88,89,91,92,94,96-98,104,105,112,116 such as appropriately breaking bad news,93,112 and discerning appropriate language to use to ensure patient understands85,96 Self-development skills • Coping against depression and burnout23,81,87,90 • Self-directed learning,23,78,88,97-99,104,111,113 self-awareness and self-regulation30,73,75,83,92,93,104 • Reflectivity4,104 • Time management104 • Problem-solving and critical thinking91 Clinical skills • Risk assessment89,99,104,112,114 • Clinical decision-making82,89,94,96,97,99,104,114 and situational awareness in times of medical or ethical dilemma89,91,96,99,104,114 • Quality and error management23,30,73,75,77-79,87,91-93,97,99,104,114 Competencies in procedural medical skills,89 providing continuity of care,87,89,104,114 medication prescription and counselling,93,96,97,114 caring for the dying,90 obtaining informed consent93 and medical documentation112
Other measures	• Improvement in clinical117 and evidence-based practice73 • Improvement in patient satisfaction and reduction in complaints118

Table 4.

Tools, approaches and modalities used in the assessment of professionalism.

Assessment approaches and modalities
Tool	• Dreyfus and Dreyfus Level of Mastery105 • Miller’s Performance Level3,26,74,85 • Pangaro’s Performance Level85 • REFLECT rubric119 • Kirkpatrick model118 • Entrustable Professional Activities21,23,29,74-76,79,84,87,89,93,94,96,98,105,119-121
Setting	• Undergraduate medical school education18,20,24-26,28,74,80,83,85,96,98,104,106,107,109,111,115,117,119,120,122,123-127 • Graduate medical school education88 • Postgraduate clinical training4,21-23,29,30,76-79,81,86,90,93-95,97,99,100,102,103,105,108,110,112-114,116,118,121,128 • Continuing professional development from medical school education to medical doctor training3,75,84,101,129,130
Target population	• Medical Students18,20,24,25,28,83,85,96,98,104,106,107,109,111,115,117,119,120,122-127 • Residents4,23,29,30,77,79,81,86,87,89,93,100,102,103,108,110,112,114,116,121 • Medical Doctors21,22,76,78,82,94,95,97,99,100,113,118,128,131 • Clinical fellows90 • Faculty staff or tutors75,92,100,101,105,123
Methods	Mixed method • Questionnaire22,75,83,85,88,92,96,100,109,112,114,119,120,122,124,131 • Portfolio4,122 Quantitative • Likert Scale Questionnaire22-24,28-30,76-78,83,85-88,90,92-94,96,97,99,101-107,109,110,111,112,116-122,124-127,129 • Frequency of behaviour scale21,84 • Yes/No questionnaire76,84 • Checklist84,115,128 • Multiple-choice questions/responses118,131 Open-ended • Interview25,81,85 • Reflective Writing/Diary85,100,119 • Focus group discussion84,124 • Open-ended Questionnaire-Free Text and Essay22,83,85,88,92,96,97,109,114,120,124,131 • Comment card122
Goals	• Formative21,98,117 • Summative93,95,104
Source	• Multi-modal75,118,122 • Multi-source3,4,23,26,30,74,80-84,105,108,115,116,122,130 • Self-assessment20,21,24,25,28,76-79,83,85,88-90,97-100,106,109,111-117,119-121,126,127,131 • Peer-assessment83,97 • Faculty-assessment29,86,87,93,95,100,104,107 • Patient-assessment85,96,97,103 • Nurse-assessment23,80,116

Table 5.

Assessment frameworks established by healthcare regulatory organisations.

Assessment frameworks

• Accreditation Council for Graduate Medical Education (United States) • American Board of Internal Medicine Medical Professionalism Framework • American Board of Medical Specialties • American College of Physician Professionalism Domains • Canadian Medical Education Directives for Specialists Roles for Family Medicine • European Federation for Internal Medicine’s Physician Charter for Medical Professionalism • General Medical Council Professionalism Capabilities Framework (United Kingdom) • Indian Medical Council Professional Conduct, Etiquette and Ethics International Classification of Primary Care Competency

Upon recommendation from the expert team, these three initial themes were reconfigured into two in this review^132-134 – the context-specific and competency-based nature of assessments. The competency-based nature of assessments correspond to various stages of a medical student’s or doctor’s training (henceforth competency-based stages).

Context-specific nature of professionalism assessments

Although many of the included articles adopted definitions of medical professionalism outlined by frameworks established by key healthcare regulatory organisations, their assessments were revealed to be significantly context specific. For example, whilst the ACGME definition was adopted by Hochberg et al⁹³ and Fontes et al²³ in surgical education, the former’s assessments focused on the prevailing surgical culture, the informal and hidden curriculum, and specific competencies practised in their specialty. The latter, however, focused on “exposure to professionalism and interpersonal and communication skills concepts”. In further contrast, Malakoff et al’s¹⁰⁸ assessments of internal medicine and transitional year residents revealed a heavier focus on participation in events such as grand rounds and sponsored conferences despite following the same ACGME definition guidelines.

Differences in settings, specialties, target populations and rationales also influenced assessments of professionalism. Further examples of the context-specific nature of assessments of professionalism are featured in Table 6.

Table 6.

Shared professionalism frameworks with variances in assessment context and domains.

Framework	Author, year	Setting	Rationale	Key domains assessed
				PCAC	HACP	MEPMH	PCPPG	Commitment to:
								Patients	Society	Profession	Self
Accreditation Council for Graduate Medical Education (ACGME)	Chandler et al116	Postgraduate (PG), Paediatrics, Residents	To determine if non-faculty ratings of resident professionalism and interpersonal skills differ from faculty ratings through use of 360-degree evaluations suggested by ACGME	X
ACGME	Fontes et al23	PG, Neuro surgery, Residents	To foster teaching and continuous evaluation of ACGME core competencies through findings from two interventions implemented on residents	X		X	X			X	X
ACGME	Gauger et al30	PG, Surgery, Residents	To develop an instrument to measure specific aspects of professionalism in surgical residents	X
ACGME, American Board of Medical Specialties (ABMS), American Board of Internal Medicine (ABIM), Association of American Colleges (AAC)	Gillespie et al29	PG, Various, Senior residents in emergency medicine, internal medicine, paediatrics, psychiatry and surgery	To assess perceptions of professional competence and professionalism in residents’ learning environment	X
ACGME	Hochberg et al93	PG, Surgery, Residents	To assess whether professionalism has taken root in their surgical resident culture three years after implementing professionalism curriculum	X	X	X	X
ACGME	Malakoff et al108	PG, Internal medicine, Medical students and transitional year residents	To objectively assess professionalism	X
ACGME	Picho et al79	Undergraduate (UG)	To assess alumni perceptions of clinical practice preparedness using ACGME competencies	X
ACGME	Rawlings et al87	PG, General surgery, Residents	To develop and evaluate narrative cases representing the five levels of the ACGME professionalism milestones	X
ACGME	Santosa et al121	PG, Psychiatry, Residents	To develop an ACGME-adapted instrument to assess professionalism in psychiatric residents	X
ACGME	Tanaka et al81	PG, Anaesthesiology, Residents	To define optimal professionalism feedback; develop, test, and implement a web-based feedback tool; and map the results to ACGME anaesthesiology milestones	X
ACGME ABMS	Williams et al105	PG, Physicians at a large Midwestern regional healthcare provider and hospital system	To determine convergence of Miller’s framework with the ACGME/ABMS Core Competency framework	X
General Medical Council (GMC)	Campbell et al97	PG, Non-training grade doctors, colleagues and patients	To investigate potential sources of systematic bias arising in the assessment of doctors’ professionalism	X			X	X		X	X
GMC	Johnston et al109	UG, Medical students at Queen’s University, Belfast, United Kingdom	To assess professional attitudes at different curriculum stages and investigate the influence of the hidden curriculum using a novel tool based on GMC standards	X
GMC	Olsson et al94	PG, Family Medicine, Residents	To assess the internal consistency of components in the Swedish adaptation of the GMC questionnaires and to determine aspects of good medical practice reflected in their latent variable structure	X	X
Canadian Medical Education Directives for Specialists (CanMEDS)	Al-Abdulrazzaq et al19	UG, Final year medical students at Kuwait University	To explore experiences and views of Kuwait students on professionalism					X	X	X
CanMEDS	Kalen et al120	UG, Medical students at Karolinska Institutet, Sweden	To explore perceptions of specific learning activities and their relation to professional development as defined by CanMEDS			X	X
CanMEDS	Ortwein et al99	PG, Anaesthesiology, Specialist and registrar	To evaluate the validity of a core competency catalogue based on CanMEDS and to ascertain differences in perceptions by specialists and registrars	X	X		X	X		X
CanMEDS	Peterkin et al119	UG, Third year medical students	To determine if using reflective writing in teaching roles delineated by CanMEDS increases students’ understanding of clinical roles	X
CanMEDS	Rademakers et al127	UG, Sixth year medical students	To determine value of specific CanMEDS competencies as perceived by students	X
CanMEDS	Warren et al75	PG, Canadian residency program directors in anaesthesiology, diagnostic radiology, general surgery, internal medicine, obstetrics and gynaecology, paediatrics and psychiatry	To understand the program directors’ perceptions of the CanMEDS Professional Role, and identify teaching and assessment methods employed				X
American Board of Internal Medicine (ABIM)	Askarian et al76	UG, Medical students	To investigate perceptions of professional behaviour exhibited by peers	X
American Board of Internal Medicine (ABIM)	Blackall et al24	UG, Medical students	To evaluate the development and factorial validity of an instrument used to measure attitudes toward professionalism in medical education among students, residents and faculty	X
ABIM American College of Physicians (ACP) European Federation of Internal Medicine (EFIM)	Humphrey et al84	UG, Preclinical and clinical students, and residents	To increase awareness of medical professionalism across institution and better understand changes in medical trainees’ professional behaviours as a result of learning environment	X	X	X	X
ABIM	Roberts et al123	UG, Faculty	To determine unprofessional behaviours endemic in their institution and particular departments	X
ABIM	Tsai et al20	UG, Seventh year medical students	To identify and understand structure of latent traits underlying concept of medical professionalism for Taiwanese students	X
Indian Medical Council (IMC)	Bahus Førde135	UG, Medical students at a private medical school in Pondicherry, India	To evaluate awareness of the ethical code of conduct for medical practitioners	X

Abbreviations: PCAC, Professional Conduct and Accountability; HACP, Humanism and Cultural Proficiency; MEPMH, Maintaining Emotional, Physical and Mental Health; PCPAPG, Pursuing Continual Personal and Professional Growth.

Competency-based stages in professionalism assessment

The competency-based nature of assessments is evidenced by ten studies which put forth a stepwise approach to the assessment of professionalism knowledge and skills informed by the learner’s development.^{23,93,100-102,114,118,119,122,131}

In the initial stages, assessments focus on evaluating the understanding of concepts of professionalism as well as their key attributes and role in medical education. Next, perceptions and attitudes towards the roles and responsibilities of a professional are assessed. The final stage culminates in the sustained development and display of key professional attitudes and behaviours over time.^{23,93,100-102,114,118,119,122,131}

The presence of such progressive and longitudinal competency-based stages highlights the need for a safe and supportive environment as well as structured training processes.¹²⁵

Discussion

In highlighting available tools used to assess medical professionalism as well as pertinent domains to consider, this systematic scoping review successfully meets its primary and secondary research objectives. The context-specific nature of assessments is evident in the fact that prevailing tools or systems of assessments are based on regnant characterisations^{23,28,29,74,75,79,84,87,89,93,94,98,104,105,119-121} and prioritisation of specific professional attributes of medical professionalism.^{21,29,75,76,78,80,83,89,92,95,100}

However, despite these contextual differences, it is equally important to acknowledge that there are consistent features to most of them. These include use of a variety of modalities, sources of assessments, and focus on knowledge, attitudes, behaviours, skills, competencies and outcomes.^{3,4,23,26,30,74,75,80-84,105,108,115,116,118,122,130}

The findings further reinforce the notion that professionalism assessments should be competency-based, highlighting the need to carefully design their various stages. It is posited here that the stages of professionalism in the included articles correspond with the stages of Miller’s Pyramid (Table 7). In turn, assessments of professionalism are consistent with Kirkpatrick’s Model of Assessment (Table 8).^{23,93,118,122} Here, it is inferred that well-established milestones and competency levels are able to guide the learner’s progress as well as inform assessment processes.

Table 7.

Miller’s pyramid (reproduced from Moore et al¹³⁶).

Miller pyramid	Definition	Outcomes reported
Does	Demonstration of Professionalism in Everyday Clinical Practice	Likelihood of students to report physicians, nurses and fellow students125
		Impact of faculty development process on faculty’s humanistic teaching92,101
		Medical students’ professionalism,91,111 manifest through behaviour and communication skills80,83,104 • Self-assessment83 • Peer-assessment83 • Multi-source feedback115 • Assessed by Patients85
		Physicians’ professionalism30,22,90,96,118,122 such as through behaviour73,86 and/or demonstration of Core Competencies set out by ACGME79,89,99 • Peer-assessment97 This may also be carried out via evaluating the interpersonal and communication skills,23,116 self-control77 and empathy102
		The professionalism curriculum as a cultural change agent93
Shows how	Demonstration of Professionalism in a controlled setting such as in response to clinical vignettes and case scenarios	Likelihood of students to report physicians, nurses and fellow students for unprofessional behaviour125
		Response to case scenarios of ethical issues18,128 and unprofessional behaviour131
		Demonstration of professional behaviour using clinical vignettes129 and standardised patients103
Knows how/Understand	Being able to articulate understanding of professionalism	Faculty understanding of feedback delivery for professionalism competency pre- and post- faculty development program100
Knows/Knowledge	State the definition of professionalism (but not necessarily demonstrate understanding and internalisation)	Perception and attitudes of professionalism24,25,84,88,109,117,130 and professional behaviour21,76,109
		Change in attitudes towards professionalism after professionalism course25,109,114,117,119
		Residents’ perceptions of their own professionalism and the professionalism of their learning environment29

Table 8.

Kirkpatrick’s model of assessment.

Articles	Learning
	Level 1 (participation)	Level 2a (attitudes and perception)	Level 2b (knowledge and skills)	Level 3 (behavioural change)	Level 4a (organisation practice)	Level 4b (patient benefits)
Aggarwal and Kheriaty125	X	X
Akhund et al126	X
Arnold et al106
Arun Babu et al18			X
Asghari et al21		X
Askarian et al76		X
Blackall et al24		X
Branch et al101	X	X	X	X
Brauch et al100	X	X	X
Bryan et al83				X
Campbell et al97				X
Chandler et al116				X
Cruess et al104				X
Cuesta-Briand et al25		X
Davis et al85		X
Delport et al91				X
Domen et al131	X	X	X
Dyrbye et al28
Elcin et al117		X
Emanuel82
Emke et al115				X
Fontes et al23	X	X	X	X
Gauger et al30				X
Gillespie et al29				X
Gisondi et al128			X
Goldie3
Guraya et al26
Haque et al88		X	X
Hershberger et al77			X
Hochberg et al93	X	X	X	X	X
Hultman et al118	X	X	X	X	X
Humphrey et al84		X		X
Iramaneerat95
Johnston et al109		X
Kalen et al120	X
Kalet et al122	X	X	X	X
Katic et al96				X
Kesselheim et al90				X
Mak-van der Vossen et al107
Malakoff et al108				X
Menna et al80				X
Nagler et al22		X		X
O’Sullivan, Toohey129		X	X
Olsson et al94				X
Ortwein et al99				X
Pavon et al114	X	X	X
Peterkin et al119	X	X	X
Peterson et al89				X
Picho et al79				X
Rademakers et al127		X
Raee et al111				X
Rawlings et al87
Roberts et al102	X	X	X	X
Roberts et al123
Sang et al113				X
Santen et al98
Santosa et al121				X
Stockley, Forbes124
Strowd et al78		X
Sullivan et al110
Tanaka et al81
Taylor et al112			X
Todhunter et al92			X	X
Tsai et al20		X
van de Camp et al73				X
Vora et al103	X	X	X	X
Wilkinson et al4
Williams et al74
Williams et al105
Yazdankhah et al86				X

The stages of assessments also reveal the longitudinal nature of professional development and the need for longitudinal assessments. There are a number of considerations.

One, reliance upon pre-existing knowledge and skills foregrounds the need for effective assessments prior to commencement of professionalism training. Whilst it is crucial that standards and codes of conduct are consistently applied, personalised training and assessments are also paramount as learners may have different baseline abilities and often require different forms of support at different junctures to achieve their desired goals.

Two, longitudinal assessments must be sensitive not only to the personalised aspect of professional development but also the appropriate stage of their learning. Indeed, emphasis on regular assessments require acknowledgement of the learner’s prevailing contextual, linguistic and cultural sensitivity as well as local understanding of professionalism. These also underscore the importance of discerning appropriate tools to be employed at each stage, with the learner’s movement from didactic learning to independent display of practical knowledge and skills. This stage-wise consideration is important given that the data accrued at each developmental stage will guide subsequent teaching, assessments and remediation for the learner.

Three, most tools do not incorporate multisource data required to effectively appreciate the evolving, adaptive nature of a developing professional identity in an individual and their practice. As far as possible, tools should be specific to different stages of professionalism development or be sufficiently flexible to account for differences in the learner’s abilities and setting.

Four, clear and realistic milestones must be established at each competency-based stage. Remediation plans should be also made available and actively integrated to ensure that learners in need do not slip between the cracks and are offered targeted and timely support.

Five, given that assessments are largely dependent upon the supervisor, mentor, coach and/or tutor, assessors must be trained on which tool to use and how each tool is to be effectively used. To facilitate communication between assessors in different settings and acknowledging the longitudinal nature of professional development, a portfolio-based assessment method should be employed to streamline the assessment process.

Consequently, these aspects of assessing professionalism affirms the fact that medical professionalism cannot be restricted to a “standardised and reductionist” definition and assessment method.^2,31,137 Rather, these findings underline the need for a portfolio-based assessment program^4,122 where a mixture of generalised tools and context-specific ‘specialised’ assessment methods can be employed to assess competencies and milestones achieved. The portfolio must also consider the prevailing practice culture, availability of resources and receptivity of its implementation to ensure the sustainability of professionalism assessment.

A portfolio approach would also allow reflections on individual experiences and allow assessors to evaluate and provide feedback on these reviews of experiences and refinements in practice and thinking. An additional consideration is that learners can also provide their perspective of the assessments and may even provide their views of a particular assessment and challenge the appraisal. This introduces the need for assessments to overseen and reviewed by independent third parties who may offer a review of the overall progress of the learner or a review of a particular assessment.

Mapping these findings have generated much insight and a general guidance for the design of professionalism assessments is featured in Figure 2 where key principles corresponding to each competency-based stage is foregrounded.

Figure 2.

Competency-based stages of professionalism assessment.

Limitations

Whilst this review focused solely on medical professionalism assessments, the articles included explored broader concepts of professionalism. The limited scope of this review also saw the exclusion of closely associated concepts such as professional identity formation and ethics³¹ which may have hampered a holistic picture of assessment practices in medical professionalism.^2,31,137 Due to time, manpower and resource constraint, the exclusion of other health professional literature may have also led to the omission of key ideas potentially transferable to the field of medical education.

Moreover, as the included articles were restricted to those in English, majority of the accounts originated from North America and the European countries. Although the utilisation of similar internationally acclaimed and accredited professionalism frameworks may suggest universal similarities and agreement over the domains of medical professionalism, it is evident that clinical and educational committees evaluate and adapt their guidelines in tandem with local healthcare and educational contexts and cultures. As the findings may not be representative of professionalism assessment practices beyond these countries, this review is limited in guiding educators on assessment attributes and criteria that will overcome nuanced geographical and cultural boundaries.^{1-3,31,35,58-60}

Future directions to consider

Mapping of prevailing medical professionalism assessment practices highlight the following potential areas for future research and action:

• Delineate a working definition of medical professionalism that acknowledges their longitudinal and competency-based nature and integrates local sociocultural and contextual factors

• Establish consensus on a medical professionalism framework that incorporates universally agreeable codes of conduct whilst granting flexibility for geographical and cultural considerations and specialties

• Develop principles to guide the design of professionalism assessments with more robust understanding of how various assessment methods, criteria, content, goals, and needs may be integrated with due consideration for prevailing resources, settings and target populations

• Evaluate the significance of sociocultural and linguistic idiosyncrasies in the design of assessment tools

• Carry out more detailed and context-specific systematic reviews to better determine unique factors influencing how professionalism could be best assessed in psychiatry and other medical and surgical specialities

• Adequately train and support assessors to ensure efficacious use of assessment tools and transparently convey to learners on the specifics of what they will be assessed on and how they will be assessed

• Consider the use of portfolios which will allow for longitudinal, multidimensional reviews and learner initiated commentaries and reflections – this would provide greater opportunities for personalised remediation and training, as well as more streamlined communications between assessors