Abstract
Background
The transition to residency marks a significant shift in the financial circumstances of medical trainees. Despite existing resources, residents still cite uncertainty in this domain. A personal finance curriculum is needed to close this educational gap and improve the financial well‐being of trainees.
Methods
The curriculum was developed using Kern's framework. Two needs assessments informed the consensus development of goals and objectives, educational strategies, and assessments. Course material was hosted online for asynchronous review and complemented by two 1‐hour webinars. The curriculum was piloted at one institution. Participants completed (1) knowledge assessments before and after the intervention, (2) a survey of reactions to the curriculum, and (3) an assessment of financial behavioral changes after the intervention.
Results
Thirty‐seven residents (37/49, 76%) enrolled in the curriculum. Among participants, 20 (20/37, 54%) completed the curriculum. Most participants agreed or strongly agreed that the content was relevant (20/20, 100%) and clearly presented (19/20, 95%) and that they would recommend the curriculum to other residents (20/20, 100%). Performance on the knowledge assessment improved 21% after the intervention (mean ± SD = pretest 57% ± 17%, posttest = 78% ± 12%; p < 0.001). Most residents (17/20, 85%) also reported behavioral changes including setting new financial goals (12/20, 60%), taking new action toward financial planning (11/20, 55%), and changing financial habits (6/20, 30%). There were no direct financial costs incurred in the implementation of this pilot.
Conclusions
This is a successful pilot of a virtual personal finance curriculum with positive outcomes data. Addressing this problem at scale will require buy‐in from educators around the country to deliver this information to residents that may not otherwise seek it out. Future study should assess curricular outcomes in other settings and the durability of acquired knowledge and behavioral changes over time.
NEED FOR INNOVATION
The transition from medical school to residency marks a significant shift in the financial circumstances of medical trainees: the shift from an organization that charges tuition to one that pays a salary. This rapid transition forces residents to make several time‐sensitive decisions regarding their new income. Despite existing resources, residents still cite uncertainty in this domain. 1 A personal finance curriculum is needed to close this educational gap and improve the financial well‐being of trainees.
BACKGROUND
The projected economic viability of various medical career paths was recently reported 2 ; however, the average debt burden remains high and the road to financial success is complicated by important questions that serve as a significant source of stress for trainees. 1 , 3 , 4 , 5 , 6 , 7 , 8 This stress is reflected in the listing of personal finance as one of six key aspects of resident physician wellness 9 and underscores the urgency of equipping trainees with the resources needed to make informed financial decisions.
Efforts to improve the financial knowledge and well‐being of residents have been described in the academic literature; however, there are no reports of comprehensive financial curricula. To date, financial planning interventions have significant limitations in scope, 10 scalability, 11 , 12 and evidence of impact. 13 , 14 , 15 , 16 , 17 Resources targeting medical professionals also exist online and in the lay press, including some that are robust, freely available, and popular with some in the medical community. However, residents report a lack of time to find, filter, and utilize these resources. 1 These limitations leave a significant education gap on personal finance resources for medical professionals that needs to be filled by a high‐yield, pragmatic, and scalable personal finance curriculum for medical trainees.
OBJECTIVE
The objective was to develop, implement, and demonstrate outcomes of a virtual personal finance curriculum that is targeted in scope for residents and easily scalable.
DEVELOPMENT PROCESS
We developed this curriculum using Kern's curriculum design framework. 18 An overview of the curriculum development process is provided in Figure 1.
FIGURE 1.
Curriculum development process of the personal finance curriculum for residents, University of Chicago Emergency Medicine Pilot, 2018
Needs assessments
We completed two needs assessments in the process of developing this curriculum: first, a multidisciplinary qualitative study of residents’ financial circumstances and interests 1 and, second, using the instrument development model of mixed methods, a quantitative survey of a broader and more diverse resident group. 3 These needs assessments provided a conceptual framework and background data that guided the content development and delivery processes.
Goals and objectives
Informed by the above needs assessments, a modified nominal group consensus process was used to generate curricular goals. The consensus group consisted of local physician educators with experience teaching personal finance to residents. Goals with unanimous agreement were adopted for the curriculum. Objectives were drafted by two authors (ES, JA) to align with consensus‐derived goals and desired curricular outcomes.
Education strategies
Given residents’ interest in financial education resources that are accessible at any time, 1 we chose written materials as the primary educational approach and created a course website to host the content for learners to access asynchronously. This virtual learning approach also fit our goal of creating a curriculum that will be easily scaled to additional programs. Four authors (ES, JA, ME, MP) composed the written content in alignment with each objective, organized into five topic areas (education debt, long‐term disability insurance, life insurance, investing, and financial advisors).
Because residents also reported interest in being able to ask questions specific to their own financial circumstances, 19 we planned two webinar sessions hosted by course contributors (ES, MP). The webinar structure was: (1) introduce participants, (2) field questions, (3) review covered topics, and (4) have facilitators share financial “pearls and pitfalls” from their own experience.
Assessment of learners
We sought to assess residents’ attitudes about the curriculum, knowledge improvement, and financial behavioral changes occurring during or after the curriculum. Attitudinal assessments included residents’ perceived relevance of the curricular content, effectiveness and clarity of presentation of curricular content, and whether participants would recommend the curriculum to their peers.
Upon review of the literature, we were unable to find an existing tool to assess financial knowledge with the depth and breadth we targeted for residents. Therefore, we developed a new financial knowledge assessment for graduate medical trainees. Using a test blueprint from curricular objectives and guided by established standards, 19 two authors with formal training in test development and experience teaching personal finance to graduate medical trainees (ES, JA) authored three test items for each of five topics to yield a 15‐item test. This limited test was designed as a coarse assessment of knowledge acquisition in the setting of this pilot program rather than a comprehensive assessment of content mastery. Residents provided open‐ended written feedback on the test items in the curricular evaluation; this feedback was incorporated into the curricular evaluation process to improve response process validity. Additionally, psychometric measures of internal structure validity evidence including item difficulty, item discrimination, and reliability were calculated.
Measures of behavioral changes included residents’ setting of financial goals, taking new financial planning action, and changing of financial habits, each assessed at the end of the curriculum.
IMPLEMENTATION PHASE
We piloted this online curriculum with a convenience sample of emergency medicine residents at the University of Chicago. This curriculum was supported by residency program leadership. Participants were offered a $10 gift card. This study was granted an exemption by the institutional review boards at the University of Chicago and the University of Illinois at Chicago.
An email message introducing the curriculum and study was sent to all 49 residents in the 3‐year program. During a 2‐week enrollment period, residents opted in by completing an online entry assessment via Qualtrics (Provo, UT).
We arranged the course content to be delivered over 5 weeks, each focusing on one of the five topics areas. Each week we sent an email message including a summary of the main themes for the previous topic, a brief introduction to the new topic of the week, and a link to the relevant section on the course website. Residents were encouraged to read the provided material and consider applications to their own financial circumstances.
Synchronous webinar sessions were hosted on Google Hangouts (Mountain View, CA). One‐hour interactive sessions were scheduled at the beginning of the third week and the end of the fifth week of curriculum implementation.
After the final week, we sent exit assessments and evaluation forms via Qualtrics (Provo, UT). Only data from residents that completed the entry, exit, and evaluation forms were included in the knowledge and behavioral analytics used for evaluation of the program.
OUTCOMES
Outcomes are outlined in Table 1 using Yardley and Dornan's adaptation of Kirkpatrick's levels of evaluation. 20 At the first Kirkpatrick level (participation), 37 residents (37/49, 76%) began the curriculum by completing the entry assessment. Among participating residents, 20 (20/37, 54%) completed the curriculum within the time constraints of the pilot. Feedback from residents who completed evaluation forms but did not participate in all aspects of the curriculum suggested that most encountered scheduling conflicts or clinical workloads that prevented full participation.
TABLE 1.
Kirkpatrick outcome measures of the personal finance curriculum for residents, University of Chicago Emergency Medicine Pilot
| Kirkpatrick level21 | Outcome measure | Result | Detail |
|---|---|---|---|
| 1–Participation | Residents: curriculum initiated | n = 37/49, 76% |
PGY‐1: 13 PGY‐2: 12 PGY‐3: 12 |
| Residents: curriculum completed | n = 20/37, 54% |
PGY‐1: 7 PGY‐2: 7 PGY‐3: 6 |
|
| 2a–Attitudes |
Resident agreement with statement: “The content in this curriculum was relevant to me.” |
Strongly agree: 13 Agree: 7 Neutral: 0 Disagree: 0 Strongly disagree: 0 |
|
|
Resident agreement with statement: “The information was clearly presented.” |
Strongly agree: 11 Agree: 8 Neutral: 1 Disagree: 0 Strongly disagree: 0 |
||
|
Resident agreement with statement: “I would recommend this curriculum to other residents.” |
Strongly agree: 12 Agree: 8 Neutral: 0 Disagree: 0 Strongly disagree: 0 |
||
| 2b–Knowledge | Multiple‐choice assessment |
Pre‐test (mean): 57% Post‐test (mean): 78% |
Cohen's d = 1.23, paired t‐test: t(19) = 5.50, p < 0.001 |
| 3–Behavioral | Resident report: new financial goal |
n = 12/20, 60% |
Short‐term a : 8/20, 40% |
|
Long‐term a : 8/20, 40% | |||
|
Short‐term and long‐term: 4/20, 20% | |||
| Resident report: new financial planning action | n = 11/20, 55% |
Topic
a
|
|
| Resident report: changed financial habit |
n = 6/20, 30% |
Topic
a
|
Abbreviations: LTD, long‐term disability; PGY, postgraduate year; PSLF, Public Service Loan Forgiveness.
Nonexclusive category.
Regarding attitudes (Kirkpatrick level 2a), nearly all residents agreed or strongly agreed that the content was relevant (20/20, 100%) and clearly presented (19/20, 95%) and that they would recommend the curriculum to other residents (20/20, 100%).
Performance on the knowledge assessment (Kirkpatrick level 2b) improved 21% after the intervention (mean ± SD = pretest 57% ± 17%, posttest 78% ± 12%; paired t‐test t(19) = 5.50, p < 0.001). This difference indicates a substantial effect size (Cohen's d = 1.23). Internal consistency reliability (Cronbach's alpha) was 0.62 on the pretest and 0.63 on the posttest. Using the post hoc Spearman‐Brown formula, reliability projections for the number of test items required to achieve a Cronbach's alpha statistic of 0.75 is 28.
Most residents (17/20, 85%) also reported behavioral changes (Kirkpatrick level 3). Nonexclusive categories of behavioral changes included setting new financial goals (12/20, 60%; example: “Set up disability and life insurance before residency ends”), taking new action toward financial planning (11/20, 55%; example: “Opened Roth IRA”), and changing financial habits (6/20, 30%; example: “Keeping closer eye on spending habits to estimate how I can best optimize savings”).
Faculty effort required for implementation of the pilot was 2 hours of small‐group facilitation (two 1‐hour webinar sessions) and sending the templated emails for course introduction, content, assessment, and evaluation. There were no costs incurred in the implementation of this pilot.
REFLECTIVE DISCUSSION
This is a successful pilot of a virtual personal finance curriculum with positive initial outcomes data. In addition to significant improvement in knowledge assessments as evidenced by the large educational effect size (Cohen's d = 1.23), the majority of residents completing this curriculum (85%) reported changes in their financial behavior. These findings underscore the notion that—in spite of previous efforts and existing resources related to resident financial education—residents still have unmet needs in this area. This curriculum represents an important step forward by addressing the limitations of existing resources identified in the needs assessments, including the desire for resident‐specific educational materials and the format that allows both independent self‐study and question‐and‐answer sessions. 19 This notion is further supported by 100% of respondents indicating that they would recommend this curriculum to other residents. Limitations of these results include the use of identical tests for the pre‐ and posttest, which may have contributed to strong learner performance on the second test. Future studies may also analyze factors contributing to learner improvement, including their financial circumstances or year of training.
Addressing this problem at scale will require buy‐in from local program leaders around the country not only to manage the logistics of curricular deployment but also to present this information to residents that may not otherwise seek it out. In an effort to simplify dissemination, we limited implementation requirements to facilitation of two 1‐hour webinars and sending templated emails. Because the course content and assessments are hosted online and Web‐based videoconferencing platforms are freely available, there were no other direct financial costs associated with implementation.
In summary, personal finance education remains an unmet need for graduate medical trainees. This virtual curriculum was strongly endorsed by residents and achieved significant cognitive and behavioral outcomes with minimal implementation requirements. Future study should assess curricular outcomes in other settings, as well as the durability of acquired knowledge and behavioral changes over time.
CONFLICT OF INTEREST
ES owns and operates MDintheBlack.com, which was created to host the asynchronous material for this curriculum. Since the time that this study was completed, ES received sponsorship funding from the Council of Residency Directors in Emergency Medicine (CORD) to publish the MD in the Black book, for which he is the Editor‐in‐Chief. ES has no ongoing financial relationship with CORD related to this manuscript. The other authors have no potential conflicts of interest.
AUTHOR CONTRIBUTIONS
Study concept and design: Eric Shappell, James Ahn, Yoon Soo Park, Ryan McKillip, Michael Ernst, Matthew Pirotte, Ara Tekian. Acquisition of the data: Eric Shappell, James Ahn. Analysis and interpretation of the data: Eric Shappell, James Ahn, Yoon Soo Park, Ryan McKillip, Michael Ernst, Matthew Pirotte, Ara Tekian. Drafting of the manuscript: Eric Shappell, James Ahn, Yoon Soo Park, Ryan McKillip, Michael Ernst, Matthew Pirotte, Ara Tekian. Critical revision of the manuscript: Eric Shappell, James Ahn, Yoon Soo Park, Ryan McKillip, Michael Ernst, Matthew Pirotte, Ara Tekian. Statistical expertise: Yoon Soo Park. Acquisition of funding: N/A.
ACKNOWLEDGMENTS
The authors thank Mohammed Minhaj, James Woodruff, and Aalok Kacha for their assistance with this project.
Shappell E, Ahn J, Park YS, et al. Affective, cognitive, and behavioral outcomes from a resident personal finance curriculum pilot project. AEM Educ Train. 2021;5:e10619. 10.1002/aet2.10619
Supervising Editor: Sorabh Khandelwal
Portions of this study previously reported in the form of an MHPE thesis defense.
REFERENCES
- 1. Shappell E, Ahn J, Ahmed N, Harris I, Park YS, Tekian A. Personal finance education for residents: a qualitative study of resident perspectives. AEM Educ Train. 2018;2:195‐203. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Marcu MI, Kellermann AL, Hunter C, Curtis J, Rice C, Wilensky GR. Borrow or serve? An economic analysis of options for financing a medical school education. Acad Med. 2017;92:966‐975. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. McKillip R, Ernst M, Ahn J, Tekian A, Shappell E. Toward a resident personal finance curriculum: quantifying resident financial circumstances, needs, and interests. Cureus. 2018;10:e2540. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Ahmad FA, White AJ, Hiller KM, Amini R, Jeffe DB. An assessment of residents’ and fellows’ personal finance literacy: an unmet medical education need. Int J Med Educ. 2017;8:192‐204. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. Glaspy JN, Ma OJ, Steele MT, Hall J. Survey of emergency medicine resident debt status and financial planning preparedness. Acad Emerg Med. 2005;12:52‐56. [DOI] [PubMed] [Google Scholar]
- 6. Witek M, Siglin J, Malatesta T, et al. Is financial literacy necessary for radiation oncology residents? Int J Radiat Oncol Biol Phys. 2014;90:986‐987. [DOI] [PubMed] [Google Scholar]
- 7. Yoo PS, Tackett JJ, Maxfield MW, Fisher R, Huot SJ, Longo WE. Personal and professional well‐being of surgical residents in New England. J Am Coll Surg. 2017;224:1015‐1019. [DOI] [PubMed] [Google Scholar]
- 8. Young TP, Brown MM, Reibling ET, et al. Effect of educational debt on emergency medicine residents: a qualitative study using individual interviews. Ann Emerg Med. 2016;68:409‐418. [DOI] [PubMed] [Google Scholar]
- 9. Physician Wellness: Preventing Resident and Fellow Burnout . American Medical Association website. 2017. Accessed May 5, 2018. https://www.stepsforward.org/Static/images/modules/23/downloadable/resident_wellness.pdf
- 10. Dhaliwal G, Chou CL. A brief educational intervention in personal finance for medical residents. J Gen Intern Med. 2007;22:374‐377. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Chui MA. An elective course in personal finance for health care professionals. Am J Pharm Educ. 2009;73:6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. Liebzeit J, Behler M, Heron S, Santen S. Financial literacy for the graduating medical student. Med Educ. 2011;45:1145‐1146. [DOI] [PubMed] [Google Scholar]
- 13. Mizell JS, Berry KS, Kimbrough MK, Bentley FR, Clardy JA, Turnage RH. Money matters: a resident curriculum for financial management. J Surg Res. 2014;192:348‐355. [DOI] [PubMed] [Google Scholar]
- 14. Hill AD, Ortega ME, Williams AC. Personal finances for the physician: a primer on maintaining and protecting your earnings. J Orthop Trauma. 2014;28:S50‐S58. [DOI] [PubMed] [Google Scholar]
- 15. Greene AK, Puder M. A resident's guide to personal finance. Curr Surg. 2002;59:423‐425. [DOI] [PubMed] [Google Scholar]
- 16. Thacker PG. Personal finance for the radiology resident: a primer. J Am Coll Radiol. 2014;11:205‐208. [DOI] [PubMed] [Google Scholar]
- 17. Wherry JE, Thomalla K. The transition from resident to private practice–important financial decisions. Oral Maxillofac Surg Clin North Am. 2008;20:109‐118. [DOI] [PubMed] [Google Scholar]
- 18. Kern DE, Thomas PA, Hughes MT. Curriculum Development for Medical Education: A Six‐Step Approach. Baltimore, MD: Johns Hopkins University Press; 2009. [Google Scholar]
- 19. Constructing Written Test Questions for the Basic and Clinical Sciences. 4th ed. Philadelphia, PA: National Board of Medical Examiners; 2016. [Google Scholar]
- 20. Yardley S, Dornan T. Kirkpatrick's levels and education ‘evidence'. Med Educ. 2012;46:97‐106. [DOI] [PubMed] [Google Scholar]