Table 2.

List and Description of Themes Identified from Qualitative Content Analysis

Primary Themes		Secondary Themes	Description	Examples	Frequency
					All Articles (N = 63)		CBME Articles (N = 11)
					#	%	#	%
1.	Identification of residents in difficulty [3–21]	A. Assessment	Includes articles discussing the association between assessment tool type, accuracy or frequency, and the identification of residents in difficulty / in need of remediation	The utility of Standardized Direct Observation Tools (SDOT) and OSCEs to identify deficiencies in clinical performance (Medical Expert)	26	41%	7	64%
		B. Faculty Development	Includes articles discussing how faculty development / training could help improve faculty’s ability to diagnose resident difficulties / deficiencies	Through improving the knowledge and skills of faculty around proper assessment and feedback methods; teaching faculty how to define and classify deficiencies so they can be more readily identified in practice	8	13%	1	9%
		C. Other	Other topics discussed in relation to the identification of resident deficiencies	Factors predictive of resident deficiencies / their need for remediation in the future (e.g. age of application to residency, having transferred from another institution) [18]; identification of deficiencies by creating and having a better understanding of the taxonomy of the “problem resident” [19, 20]	3	5%	0	0%
2.	Defining and classifying resident deficiencies		Discusses the importance of creating a framework for defining and classifying resident problems in an effort to design appropriate programs tailored to the issue(s) at hand. Addresses the questions of “What is the problem or deficiency?”; “What are its causes?”; “What signals a deficiency in this competency domain?”; etc.	Defining the problem through subjective and objective measures	19	30%	3	27%
				Classifying problems, e.g. by rating their severity, determining whether the problem is inherent or contextual, what the contributing factors are
3.	Improving assessment tools and/or methods for tracking the progress of residents undergoing remediation		Improving the accuracy and/or frequency of assessment to better track residents’ progress throughout their remediation	Advocating for more frequent formal and informal evaluations and feedback	10	16%	5	45%
				Often found to improve resident learning outcomes
4.	Individualizing or tailoring of the remediation plan / program	A. To the resident	Many articles discuss the importance of tailoring remediation plans to the resident	Customizing the approach and structure and the resources needed for the remediation plan for each resident, with consideration given to the problem type and severity, and the resident’s characteristics (such as their learning style, personality, and level of insight).	16	25%	3	27%
		B. To the specialty	Different specialties may require special consideration when developing a remediation plan, due to the unique nature of their training program and/or special clinical/training environment(s)	Emergency Medicine interns are found to have a significantly higher chance of under-performing than those in other disciplines, thought to be possibly related to differences in assessment practices and/or training environments	3	5%	2	18%
5.	Defining terms relating to remediation		Demonstrates the importance of having institutional-level consistency in the definitions of terms relating to remediation. This can help Program Directors (PDs), their faculty, and residents to better understand the expectations for training and the repercussions of not meeting them	Need to better differentiate and understand the difference between “need for improvement,” “need for remediation” (formal vs. informal), and “need for probation” (notice of potential for dismissal), e.g. CanMEDS, ACGME Roles	8	13%	2	18%
6.	Demands placed on faculty by remediation		Any strain on faculty as a result of their participation in the remediation of residents, including their time and effort spent and the complexity of their role(s)	Most refer to how time-consuming it is for faculty to participate in remediation programs and the great effort usually required	9	14%	2	18%
7.	Hidden curriculum		Discusses the “hidden curriculum” in terms of attending role modelling, which can either positively or negatively impact residents	Faculty and residents need to be held to the same standards of professionalism, e.g. studies find that residents are given passes on their behaviour relative to the learner’s level of training	2	3%	0	0%
8.	Associations with past performance		Correlations observed between past performance (e.g. in medical school) and performance during residency	Performance in medical school may be predictive of performance in residency	7	11%	1	9%
9.	Pilot testing of plan / program		Discusses the pilot testing of a novel remediation plan or program, including program description and program effectiveness and/or outcomes	Looks at the effects of a 4-month training program implemented in an Emergency Medicine residency program to improve residents’ American Board of Emergency Medicine exam scores	3	5%	0	0%
10.	Roles and responsibilities of players involved in remediation		Understanding and identifying the many “players” involved in remediating residents, in identifying those in difficulty, and defining their roles	Identifying the key individuals to be involved in the remediation process and specifying their roles for varying severities of resident problems, e.G. minor problems can be managed by the ward or department, whereas more serious problems might merit a formal investigation	7	11%	3	27%
11.	Other		Topic areas that were less prominent in the literature reviewed	Critique of literature on remediation (review); PD survey of incidence / prevalence of resident problems and possible predictors; plan for improved self-reflection integrated into remediation programs; general description of remedial process; and benchmark scale for residency training	13	21%	3	27%