ABSTRACT
Background
The Accreditation Council for Graduate Medical Education (ACGME) requires residency programs to support residents in developing individualized learning plans (ILPs) to promote self-reflection and continuous improvement. However, there is no standardized ILP template available for programs and residents to use, making implementation and evaluation across programs variable and time-consuming.
Objective
To develop a consensus-based ILP template for use in graduate medical education.
Methods
A modified Delphi process was performed between February 2022 and May 2023 to create consensus on essential components of an ILP. A literature review informed panelist recruitment and item generation. A panel of 14 experts in ILPs from several specialties and disciplines in the United States and Canada participated in 3 rounds of surveys. Consensus was defined as more than 80% agreement.
Results
Fourteen of 28 invited experts (50% response rate) agreed to participate in our study. All panelists completed Round 1, 93% (13 of 14) completed Round 2, and 100% (14 of 14) completed Round 3. Panelists identified key outcomes for ILPs, including self-reflection, needs identification, goal setting, and progress tracking. Key ILP content included long-term planning, feedback and assessment, strengths and areas for growth, personal and career goals, and structured goal planning. Expert recommendations were used to refine the template for clarity and usability. All panelists endorsed the final ILP template as ready for dissemination.
Conclusions
This modified Delphi study of ILP researchers and educators identified consensus-based recommendations for content for an ILP designed to be implemented in any graduate medical education specialty.
Introduction
The Accreditation Council for Graduate Medical Education (ACGME) requires residency programs to assist residents in developing individualized learning plans (ILPs) to identify strengths and areas for growth.1 The ACGME reflective practice and commitment to personal growth subcompetency requires implementing and refining an ILP in the context of performance data.2 Typical use of an ILP in residency programs varies, but may include residents and fellows filling out an ILP alone or with an advisor or coach one or more times a year. While the ACGME gives specific recommendations for the components of an ILP, there is no standardized ILP template, and there is limited evidence to guide the development and implementation of ILPs in residency programs.2-4
The requirement to implement ILPs without a standardized framework has resulted in residency programs implementing their own ILP processes. This is time consuming for busy residency directors and may lead to variable individualized learning across programs and specialties. Although recent specialty-specific forms have been developed, they may not be applicable to other specialties.5,6 Further, the lack of standardization of ILPs makes evaluating their outcomes challenging. For medical educators to make progress toward a shared goal of implementing ILPs that will facilitate self-reflection, goal setting, and performance improvement, an important first step is to determine the key components that should be included on an ILP form, and best practices for implementation.
The aim of this study was to identify key content for an ILP for use in graduate medical education (GME). We used a modified Delphi approach to find consensus on key components and best practices for ILPs, and to revise and finalize an ILP template. The proposed template is intended to be comprehensive and applicable to any specialty training program.
KEY POINTS
What Is Known
The ACGME requires individualized learning plans (ILPs) in residency training. However, no standardized template exists, which leads to variability and inefficiency across programs.
What Is New
Using a modified Delphi process, experts reached consensus on essential ILP components, including self-reflection, needs identification, goal setting, progress tracking, and structured planning aligned with feedback, strengths, and career goals.
Bottom Line
A consensus-based, refined ILP template has been developed and endorsed for broad use across graduate medical education, offering a standardized framework to support resident development and program implementation.
Methods
We used a modified Delphi approach7-9 between February 2022 and May 2023 to identify key components of an ILP and then to iteratively develop a standardized ILP based on expert consensus. To inform initial content validity for the ILP, as well as identify potential experts for the panel, a literature search was performed through MEDLINE regarding established ILP practices using the search terms “graduate medical education,” “individual plan,” “personal plan,” “curriculum,” “instruction,” “intern,” “resident,” “design,” and “goal” which identified common themes. A total of 635 possible article abstracts were screened by the primary author (C.A.B.) to determine relevance. Thirty-six articles were identified as contributory to the study aims and were manually reviewed to (1) identify potential content for a standardized ILP, and (2) identify ILP experts from primary authors to invite to participate in our panel. Four members of the study team (C.A.B., L.M.Y., P.R., B.A.P.) reviewed all 36 included articles to develop a shared mental model about the current literature on ILP best practices. The literature review did not identify standardized recommendations for ILP content.
Selection of Expert Panel
To select a panel of experts who would be able to identify key content areas for a standardized ILP, we invited a sample of first and senior authors from our literature search to participate in a Delphi study via email. Aiming for a final panel of 8 to 20 experts, the study team made a list of authors and selected individuals to participate, based on number of articles authored, authorship position (first and last authors prioritized), and availability of an email address.7 Ten of 15 authors selected from our literature search agreed to participate in Round 1. Additionally, we asked authors to identify other experts in the field. Thirteen additional experts were identified, with 4 agreeing to participate in our study. The demographics of the final 14 panelists are demonstrated in Table 1.
Table 1.
Panel Demographics
| Round 1, n (%) (N=14) | Round 2, n (%) (N=13) | Round 3, n (%) (N=14) | |
|---|---|---|---|
| Country of residence | |||
| United States | 13 (93) | 12 (92) | 13 (93) |
| Canada | 1 (7) | 1 (8) | 1 (7) |
| Specialty | |||
| Emergency medicine | 6 (43) | 6 (46) | 6 (43) |
| Pediatrics | 3 (21) | 2 (15) | 3 (21) |
| Otolaryngology | 1 (7) | 1 (8) | 1 (7) |
| Internal medicine | 1 (7) | 1 (8) | 1 (7) |
| PhD | 2 (14) | 2 (15) | 2 (14) |
| EdD | 1 (7) | 1 (8) | 1 (7) |
Delphi Process
Three rounds of consensus-building were conducted via electronic surveys (Google Sheets and Qualtrics) between February 2022 and May 2023 (Figure). To optimize content validity, items were selected based on our literature search and discussions among our author group, who all have expertise in ILPs. The author group read survey drafts aloud and revised them for clarity, brevity, and comprehension to optimize response process validity. We defined an agreement rate of 80% for consensus for each round to identify the minimum necessary content that should be included in an ILP. Surveys were distributed to experts via email. Three reminder emails were sent in addition to the initial survey invitation for each round.
Figure.
Flow Diagram for Delphi Process
Abbreviation: ILP, individualized learning plan.
Round 1
The first round of the Delphi process aimed to identify experts’ perceptions of best practices in ILPs, as well as begin to refine key content for an ILP. The survey asked: “If you were to develop a gold standard Individualized Learning Plan (ILP), what would be on it?” The panel was also asked about the intended outcomes of an ILP; to suggest available literature, resources, or templates that should be considered in creating a standardized ILP; to suggest conceptual frameworks important to ILPs; and for any other additional input. Panelists were also prompted to suggest experts in ILPs to include in snowball sampling. The responses to the open-ended questions were analyzed by 2 internal medicine ILP experts (B.A.P., P.R.) and 3 emergency medicine ILP experts (C.A.B., K.R.S, L.M.Y.) using thematic analysis. The Round 1 questionnaire is included as online supplementary data Appendix A.
Round 2
The aim of Round 2 was to refine the content outline for a standardized ILP and explore subsections and preferred formatting for each content domain. The study team reviewed the Round 1 results to create a list of candidate components of a standardized ILP, based on both expert consensus and the study team’s review of the literature. The list of candidate items was presented to the panel in Round 2, and experts were asked to rate each item as “definitely include,” “potentially include,” or “don’t include” on a standardized ILP and invited to suggest additional items for consideration in subsequent rounds. If experts chose “definitely include” or “possibly include,” they were asked to describe specific subcomponents as “very important,” “somewhat important,” and “not important.” Experts were asked to provide commentary on their selection. Consensus was defined as at least 80% of experts choosing “definitely include” or “possibly include” and “very important” or “possibly important” for content items and subcomponents, respectively. Based on the panel’s responses to Round 2 questions, as well as ILP best practices synthesized from the literature search, the authors developed an initial draft of the standardized ILP. The author team revised this draft for response process validity by reading aloud the items as a group and revising for clarity and brevity. The Round 2 questionnaire is included as online supplementary data Appendix B.
Round 3
The aim of Round 3 was to present the standardized ILP to the expert panel to determine if there was 80% agreement that this instrument was ready for implementation (defined as experts responding “yes” or “possibly with modifications”) and to collect further content and response process validity information. The draft template ILP, along with anonymized results from the Round 2, was presented to the panel. The experts were asked to rank the readiness of the template ILP for dissemination by responding “yes,” “possibly with modifications,” or “no.” Experts were asked to provide modifications to the overall template design and subsections of the template. The Round 3 questionnaire is included as online supplementary data Appendix C.
This study was deemed not human subjects research by the primary institution’s review board and therefore did not require full review.
Results
Fourteen of 28 invited experts (50% response rate) agreed to participate in our study. One hundred percent (14 of 14) of panelists completed Round 1; 93% (13 of 14) completed Round 2; and 100% (14 of 14) completed Round 3.
Round 1
In Round 1, the panel suggested content for a standardized ILP. These content areas are demonstrated in the Box.
Box Candidate Key Content Items and Subsections for a Standardized Individualized Learning Plan
Feedback section
Strengths section
Areas for improvement section
Assessment section
Self-assessment/reflection section
Goals section
Plan/strategy to achieve goals section
Intended ILP Outcomes:
Experts identified the following as intended outcomes of a standardized ILP: self-reflection, identification of learners’ needs, development of goals, and demonstration of progress toward goal completion. Multiple experts also highlighted accountability and professional identity formation as important outcomes of an ILP.
Conceptual Frameworks Relevant to ILPs:
Expert panelists recommended several theories and conceptual frameworks that should inform the design of an ILP. These include master adaptive learner,10 self-regulated learning theory,11 goal-setting frameworks such as WOOP12 and I-SMART,13 R2C2,14 reflective practice,15 and informed/guided self-assessment.16
Round 2
The second-round questionnaire asked panelists to identify sections that should be included within an ILP and the importance of corresponding subsections. Table 2 displays content items that met consensus for inclusion (80% answering “definitely” or “possibly” include) and the subsections that met consensus for inclusion (80% answering “somewhat” or “very” important). Given our aim to develop a comprehensive tool that could be adapted by individual programs, we included the “possibly include” and “somewhat important” categories with the belief that these items would likely be useful to some programs, and that others could delete items that were not needed.
Table 2.
Panel Responses Regarding ILP Content Areas and Subsections
| Do Not Include, n/N (%) | Possibly Include, n/N (%) | Definitely Include, n/N (%) | Not Important, n/N (%) | Somewhat Important, n/N (%) | Very Important, n/N (%) | |
|---|---|---|---|---|---|---|
| Feedback section | 0/13 (0) | 5/13 (38) | 8/13 (62) | |||
| Individual shift feedback | 1/13 (8) | 7/13 (54) | 5/13 (38) | |||
| Rotation feedback | 1/13 (8) | 5/13 (38) | 7/13 (54) | |||
| CCC feedback | 2/13 (15) | 4/13 (31) | 7/13 (54) | |||
| Individual learner self-reflection | 1/13 (8) | 2/13 (15) | 10/13 (77) | |||
| Strengths section | 0/13 (0) | 2/13 (15) | 11/13 (85) | |||
| Learner identified strengths | 0/13 (0) | 2/13 (15) | 11/13 (85) | |||
| Faculty identified strengths | 0/13 (0) | 3/13 (23) | 10/13 (77) | |||
| CCC identified strengths | 0/13 (0) | 4/13 (31) | 9/13 (69) | |||
| Areas for improvement section | 0/13 (0) | 1/13 (8) | 12/13 (92) | |||
| Learner identified areas for improvement | 0/13 (0) | 1/13 (8) | 12/13 (92) | |||
| Faculty identified areas for improvement | 0/13 (0) | 4/13 (31) | 9/13 (69) | |||
| CCC identified areas for improvement | 0/13 (0) | 4/13 (31) | 9/13 (69) | |||
| Assessment section | 0/13 (0) | 7/13 (54) | 6/13 (46) | |||
| Milestone ratings | 3/13 (23) | 5/13 (38) | 5/13 (38) | |||
| EPA ratings | 2/13 (15) | 5/13 (38) | 6/13 (46) | |||
| “Does not meet”/“Meets”/“Exceeds expectations” ratings | 3/13 (23) | 7/13 (54) | 3/13 (23) | |||
| “Meets criteria for promotion” ratings | 3/13 (23) | 7/13 (54) | 3/13 (23) | |||
| Self-assessment section | 0/13 (0) | 1/13 (8) | 12/13 (92) | |||
| Goals section | 0/13 (0) | 1/13 (8) | 12/13 (92) | |||
| Rotation specific goals | 1/12 (8) | 8/12 (67) | 3/12 (25) | |||
| 3-month goals | 2/11 (18) | 4/11 (36) | 5/11 (45) | |||
| 6-month goals | 0/12 (0) | 5/12 (42) | 7/12 (58) | |||
| Yearly goals | 0/12 (0) | 5/12 (42) | 7/12 (58) | |||
| Long-term goals | 2/12 (17) | 9/12 (75) | 1/12 (8) | |||
| Personal goals | 2/12 (17) | 6/12 (50) | 4/12 (33) | |||
| Scholarship goals | 0/12 (0) | 10/12 (83) | 2/12 (17) | |||
| Plan/strategy to achieve goals section | 0/13 (0) | 0/13 (0) | 13/13 (100) | |||
| Learner-driven strategy | 0/13 (0) | 1/13 (8) | 12/13 (92) | |||
| Faculty-driven strategy | 5/13 (38) | 6/13 (46) | 2/13 (15) | |||
| Program director-driven strategy | 5/13 (38) | 7/13 (54) | 1/13 (8) | |||
| CCC-driven strategy | 6/13 (46) | 6/13 (46) | 1/13 (8) | |||
Abbreviations: ILP, individualized learning plan; CCC, Clinical Competency Committee; EPA, entrustable professional activity.
An initial standardized ILP template was created based on the responses (online supplementary data Appendix D). The sections included were long-term planning, self-reflection and assessment, personal and career goals, and additional topics for conversation. The following subsections were also included: core values, career aspirations, summary of feedback and assessment data, opportunities for development and growth, reflection on patient encounters, reflection on the academic year, and personal and professional goals subsection that was organized using an I-SMART13 framework.
Round 3
During the third round, panelists were asked whether the proposed draft document was complete and ready for distribution or in need of further modifications. We received responses from 100% (14 of 14) of the experts. Experts responded that the ILP template was ready for dissemination (4 of 14) or could be with modifications (10 of 14).
The panelist recommendations for ILP template modification fell under 3 categories: long-term planning, self-reflection and assessment, and personal and career goals. The author team reviewed all recommendations and implemented those that aligned with the consensus recommendations on the most important content and best practices. Modifications were made for clarity, to improve response process validity, and to provide further guidance on how to use the ILP for both trainees and coaches (which emerged in Round 3 as an important element of the ILP). Suggested modifications and team responses are included in online supplementary data Appendix D. The final ILP template is in online supplementary data.
Discussion
In this modified Delphi approach to develop a standardized ILP, experts agreed that an ILP should include resident self-reflection, an assessment of strengths and areas for improvement, and a framework that allows residents to create actionable educational goals. The majority favored using the I-SMART goal framework to guide residents in goal creation.13 Additionally, experts reached consensus on the following key ILP sections: feedback, strengths, areas for improvement, assessment, self-assessment, goals, and plan/strategy to achieve goals.
While there is extensive literature on the use of ILPs in GME, there are few published ILP forms. Those that do exist have been made at an institutional level or by experts within a specific medical discipline.7,8,17-22 Our study is the first to create a standardized ILP template that can be adapted to any specialty training program. The next steps include evaluating residents’ and faculty’s satisfaction with the ILP template and their experience using it across medical specialties, as well as exploring best practices for implementation and in conjunction with a coaching approach. On a broader scale, it will be important to see if this template impacts trainee development, faculty coaching, and other outcomes of interest.
There are several limitations of our study. First, our method of identifying experts may have missed potential content experts. We aimed to develop a consensus-based standardized ILP, and therefore we invited authors who had published articles on ILPs to create an expert group well-matched to our study aim. Although our panelists represent 4 medical specialties and 2 doctorate disciplines, there are more emergency physicians (6 of 14 participants) than those from other specialties/disciplines, which may limit the external validity of our results to other fields. Another potential limitation is that it is possible that some specialties may find our template restrictive or lacking sufficient detail. To address this, we plan to further evaluate this tool in our program of research. Additionally, we asked our panel to come to consensus about ILP content and best practices, not to develop the form. Although the panel did review and endorse a near-final version, the final modifications were made by the author team rather than the panel. Future research may collect additional validity evidence to support this ILP’s use. Lastly, although an ILP template may support residents in creating plans to achieve learning goals, it does not teach residents how to approach ILPs, the importance of iteration in ILPs, or how to use I-SMART goals to create action plans. Our study aim was to create the ILP template, but we did not specifically seek consensus regarding coaching practices using the ILP. We are currently piloting the ILP in order to determine feasibility and acceptability, and this is an area for future work. Our template should be paired with learner and faculty development to optimize the ILP implementation process, and further work related to implementation and coaching with ILPs in GME may support its use.
Conclusions
In this modified Delphi study of ILP researchers and educators, we identified consensus-based recommendations for content for a standardized ILP designed to be implemented in any GME specialty.
Supplementary Material
Author Notes
Funding: The authors report no external funding source for this study.
Conflict of interest: The authors declare they have no competing interests.
Disclosure: AI was not used in writing this article.
Editor’s Note
The online supplementary data contains the surveys used in the study, panel recommendations for individualized learning plan (ILP) modification based on Round 3 with study group response, and an ILP template.
References
- 1.Accreditation Council for Graduate Medical Education. ACGME Common Program Requirements (Residency) Published 2026 Accessed May 21, 2026. https://www.acgme.org/globalassets/pfassets/programrequirements/2026-prs/cprresidency_2026_feb_revision.pdf.
- 2.Accreditation Council for Graduate Medical Education. Harmonizing the Practice-Based Learning and Improvement Milestones. Accessed June 19, 2026. https://www.acgme.org/globalassets/pdfs/milestones/harmonizingpbli.pdf.
- 3.Challis M. AMEE medical education guide no. 19: personal learning plans. Med Teach. 2000;22(3):225–236. doi: 10.1080/01421590050006160. doi: [DOI] [Google Scholar]
- 4.Wheat S, Cole S. Individualized learning plans—who, what, when, where, why, and how? Ann Fam Med. 2023;21(6):560–562. doi: 10.1370/afm.3062. doi: [DOI] [Google Scholar]
- 5.Alliance for Academic Internal Medicine. Individualized learning plan (ILP) Accessed July 14, 2026. https://www.im.org/resources/resource/aaim-individualized-learning-plan-(ilp)
- 6.Society of Teachers of Family Medicine. Individualized learning plan template for family medicine residents. Accessed October 17, 2025. https://stfm.org/teaching resources/resources/cbme-toolkit/cbme-individualized-learning-plans/
- 7.Humphrey-Murto S, Varpio L, Gonsalves C, Wood TJ. Using consensus group methods such as Delphi and nominal group in medical education research. Med Teach. 2017;39(1):14–19. doi: 10.1080/0142159X.2017.1245856. doi: [DOI] [PubMed] [Google Scholar]
- 8.Humphrey-Murto S, Wood TJ, Gonsalves C, Mascioli K, Varpio L. The Delphi method. Acad Med. 2020;95(1):168. doi: 10.1097/ACM.0000000000002887. doi: [DOI] [PubMed] [Google Scholar]
- 9.Chan TM, Yarris LM, Humphrey-Murto S. Delving into Delphis. CJEM. 2019;21(2):167–169. doi: 10.1017/cem.2019.3. doi: [DOI] [PubMed] [Google Scholar]
- 10.Cutrer WB, Pusic MV, Gruppen LD, Hammoud MM, Santen SA. The Master Adaptive Learner. Elsevier; 2019. [Google Scholar]
- 11.Boekaerts M. Self-regulated learning: where we are today. Int J Educ Res. 1999;31(6):445–457. doi: 10.1016/S0883-0355(99)00014-2. doi: [DOI] [Google Scholar]
- 12.Saddawi-Konefka D, Baker K, Guarino A, et al. Changing resident physician studying behaviors: a randomized, comparative effectiveness trial of goal setting versus use of WOOP. J Grad Med Educ. 2017;9(4):451–457. doi: 10.4300/JGME-D-16-00703.1. doi: [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Reed S, Lockspeiser TM, Burke AE, et al. Practical suggestions for the creation and use of meaningful learning goals in graduate medical education. Acad Pediatr. 2016;16(1):20–24. doi: 10.1016/j.acap.2015.10.005. doi: [DOI] [PubMed] [Google Scholar]
- 14.Sargeant J, Lockyer J, Mann K, et al. Facilitated reflective performance feedback: developing an evidence- and theory-based model that builds relationship, explores reactions and content, and coaches for performance change (R2C2) Acad Med. 2015;90(12):1698–1706. doi: 10.1097/ACM.0000000000000809. doi: [DOI] [PubMed] [Google Scholar]
- 15.Mamede S, Schmidt HG. The structure of reflective practice in medicine. Med Educ. 2004;38(12):1302–1308. doi: 10.1111/j.1365-2929.2004.01917.x. doi: [DOI] [PubMed] [Google Scholar]
- 16.Sargeant J, Armson H, Chesluk B, et al. The processes and dimensions of informed self-assessment: a conceptual model. Acad Med. 2010;85(7):1212–1220. doi: 10.1097/ACM.0b013e3181d85a4e. doi: [DOI] [PubMed] [Google Scholar]
- 17.Li STT, Burke AE. Individualized learning plans: basics and beyond. Acad Pediatr. 2010;10(5):289–292. doi: 10.1016/j.acap.2010.08.002. doi: [DOI] [PubMed] [Google Scholar]
- 18.Getto LP, Drake J, Kern A, Fredette J. Implementation of a monthly individualized learning plan with emergency medicine residents. AEM Educ Train. 2021;5(4):e10710. doi: 10.1002/aet2.10710. doi: [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Pennock M, Svrakic M, Bent JP., 3rd Completion of an individualized learning plan for otology-related milestone subcompetencies leads to improved otology section otolaryngology training exam scores. Otol Neurotol. 2019;40(10):1392–1398. doi: 10.1097/MAO.0000000000002392. doi: [DOI] [PubMed] [Google Scholar]
- 20.Stuart E, Sectish TC, Huffman LC. Are residents ready for self-directed learning? A pilot program of individualized learning plans in continuity clinic. Ambul Pediatr. 2005;5(5):298–301. doi: 10.1367/A04-091R.1. doi: [DOI] [PubMed] [Google Scholar]
- 21.Burke A. Individualized learning plans: faculty as facilitators. MedEdPORTAL. 2009;5:1684. doi: 10.15766/mep_2374-8265.1684. doi: [DOI] [Google Scholar]
- 22.Pincavage AT, Gandhi A, Falk E, et al. Evaluation of an individualized learning plan template for the transition to residency. J Grad Med Educ. 2023;15(5):597–601. doi: 10.4300/JGME-D-23-00040.1. doi: [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.