Cost-Efficient Medical Education: An Innovative Approach to Creating Educational Products

Abstract

Background

Cost is a barrier to creating educational resources, and new educational initiatives are often limited in distribution. Medical training programs must develop strategies to create and implement cost-effective educational programming.

Objective

We developed high-quality medical programming in procedural instruction with efficient economics, reaching the most trainees at the lowest cost.

Methods

The Just-In-Time online procedural program was developed at the University of Toronto in Canada, aiming to teach thoracentesis, paracentesis, and lumbar puncture skills to internal medicine trainees. Commercial vendors quoted between CAD $50,000 and $100,000 to create 3 comprehensive e-learning procedural modules—a cost that was prohibitive. Modules were therefore developed internally, utilizing 4 principles aimed at decreasing costs while creating efficiencies: targeting talent, finding value abroad, open source expansion, and extrapolating efficiency.

Results

Procedural modules for thoracentesis, paracentesis, and lumbar puncture were created for a total cost of CAD $1,200, less than 3% of the anticipated cost in utilizing traditional commercial vendors. From November 2016 until October 2018, 1800 online instructional sessions have occurred, with over 3600 pageviews of content utilized. While half of the instructional sessions occurred within the city of Toronto, utilization was documented in 10 other cities across Canada.

Conclusions

The Just-in-Time online instructional program successfully created 3 procedural modules at a fraction of the anticipated cost and appeared acceptable to residents based on website utilization.

Introduction

High costs, which are often multifaceted and hidden,¹ can impede the development and implementation of locally relevant educational resources, including those that support procedural training (table). Performing bedside procedures is an essential competency in internal medicine postgraduate training, but the majority of residents report a lack of comfort with at least one aspect of common bedside procedures,² and describe a lack of exposure to procedural skills training during residency.³ Residents desire more opportunities for procedural training to improve their skills.⁴

Table.

Levin's Framework for Cost Ingredients in Standard Educational Interventions

Cost Ingredient	Example Expenditures
1. Facility costs	Facility rental fees
	Facility maintenance costs
2. Personnel costs	Development costs
	Hired staff wages
	Administrative staff wages
	Volunteer time
3. Equipment and materials costs	Equipment purchases
	Equipment maintenance
	Furnishing
	Instructional materials
4. Client inputs	Learner travel costs
	Learner opportunity costs
5. Other program inputs	Information technology costs
	Communication and promotional costs

We sought to increase opportunities for procedural training by developing a Just-in-Time procedural program, including the creation of comprehensive, e-learning procedural modules for thoracentesis, paracentesis, and lumbar puncture. Commercial vendors with expertise in digital development quoted costs ranging from CAD $50,000 to $100,000 for the creation of 3 modules that we would be unable to modify independently in the future and would only be available within the hospital intranet. We needed to minimize the cost and wanted our educational modules to be freely available on the internet, allowing trainees at community sites, and those outside our institution, to use them. We sought to utilize economic principles efficiently to build a cost-effective and widely useful program.

Methods

Our intervention was conducted in an internal medicine residency program at the University of Toronto in Toronto, Canada, a large, urban, 3-year core program with approximately 200 residents. Recognizing residents' time limitations, we designed the modules to be succinct and self-directed. They were designed to highlight key procedural tasks, with review of the relevant material occurring immediately before performing a live procedure (Figure). We aimed to be mindful of best practices in instructional design and principles when using multimedia in educational interventions.

Figure.

Core Steps in Performing Internal Medicine Bedside Procedures

Challenging the way e-learning modules have traditionally been created, we constructed our modules utilizing the following 4 core principles.

Target Talent: Match Complexity of Task With Skill Level of Provider

Completing a project requires execution of a broad range of tasks with varying difficulty. Inefficiency arises when highly skilled, and concordantly expensive, workers are tasked with projects below their skill level. This waste of resources is commonplace when hiring a single high-level agency or contractor to deliver a product. Both in micro- and macroeconomics, productivity is reliably maximized by matching the skills of workers to the skills required for the task.⁵

Consider the economic argument for the use of dental hygienists. Even though a dentist may be marginally more skilled than hygienists at routine dental hygiene, the dentist holds a substantial comparative advantage (see Figure provided as online supplemental material) over the hygienist in complex dental procedures and is therefore best served by dedicating time to those tasks. It is estimated the optimal use of dental hygienists can increase a dentist's productivity by more than 50%.⁵

We sought to similarly maximize our efficiency by splitting project tasks into smaller components and delegating each portion to the most appropriately matched professional. Rather than hiring a single developer, we hired multiple individuals of varying skill levels and proficiencies. Individual hires for photo editing, graphic creation, video-splicing, and photography were able to complete the majority of work at an hourly rate less than one-tenth of that of our website developer. Project management was performed by a resident within the internal medicine program who undertook this role as a part-time project in addition to clinical duties, spending approximately 150 hours over 9 months. Two faculty members served in supervisory roles, dedicating approximately 15 hours each to the project.

Find Value Abroad: Leverage the Internet to Assemble a Cost-Effective Team

Assembling a diverse team of professionals can be a complex endeavor. It is challenging to assemble the necessary talent when recruiting from a local pool of expertise. Fortunately, the internet has created a global marketplace, eliminating many of these barriers.

This new reality is exemplified by 5Crowd, a marketing production agency that has succeeded by looking abroad. Faced with ballooning marketing costs quoted by agencies in North America, 5Crowd developed a proprietary web platform to procure, evaluate, and employ digital media professionals around the world, executing traditional marketing projects with tremendous efficiencies. To date, they have completed thousands of projects for Fortune 500 companies, including Johnson & Johnson and Pepsi, with savings relative to traditional firms of 20% to 50%.⁶

We took a similar approach in outsourcing key technological jobs to optimal and cost-effective bidders around the world, utilizing global freelancing platforms Upwork (Upwork Global Inc, Santa Clara, CA) and Fiverr ( Fiverr International Ltd, Tel Aviv, Israel). Collaborating virtually, we engaged an artist and video editor in Argentina, a copy editor in the United Kingdom, a graphic designer in Afghanistan, a Photoshop artist in Russia, a computer programmer in the United States, and a website developer and illustrator in Canada.

Open Source Expansion: Remove Barriers to Widespread Dissemination of Knowledge

We wanted our modules to be available to as wide a population as possible. Traditional educational initiatives are locally implemented and only serve to benefit learners proximate to their implementation. The proliferation of online content can serve to ameliorate this inefficiency, but cumbersome logins or fees required for use can be hindrances.

The case for open and accessible educational resources has been championed by the success of Khan Academy, which provides free online content for math and science, subjects traditionally reserved for structured educational institutions. Khan Academy has delivered over 1 billion lessons via easily accessible, web-based platforms, without the need for payment or mandated registration.⁷

Although on a much smaller scale, we took a similar approach to making our educational modules freely available on the internet, allowing our residents to access the modules when training at community sites, or after completing their residency, and enabling those outside our institution to utilize the modules. Although some of the content is focused on specific local context, other portions are more broadly relevant.

Extrapolate Efficiently: Apply a Winning Formula to New Challenges

Once a successful model is piloted, there is a continual challenge to expand and achieve broader impact. It is difficult to efficiently translate concepts, values, and materials to overcome the obstacles of a new application. Core concepts must be distilled, formalized, and templated to be effectively transposed.

We developed our educational modules to conform to a standardized template. The color scheme, layout, interactive programming, and content organization map are programmed and available as a ready-made platform. Other procedural instruction modules or medical education initiatives can be easily built on this platform in an expedient and low-cost manner, and current modules can be easily adapted to the local context at other institutions.

Guided by these 4 core principles, we created 10-minute instructional e-learning modules that include materials, indications and contraindications, suggestions for appropriate investigations, and video-guided, step-by-step instructions based on existing best practices. Prefilled consent sheets, standardized procedure checklists, and documentation notes are automatically printed to ensure quality of patient care. Module utilization was measured via embedded analytics, programmed within the website code. Number of sessions, pages of content utilized, and geographic area of utilization were abstracted in raw form.

The Mount Sinai Hospital Research Ethics Board at the University of Toronto approved the Just-In-Time procedural project, including evaluation and feedback from end users locally. Non-identifying analytic data was received from a third-party program based on an intrinsic website design, which was declared exempt from review at our institution.

Results

We were able to complete all 3 modules for approximately CAD $1,200, less than 3% of the cost quoted using a traditional approach to development. The Just-in-Time program was initially developed over a 9-month period and launched in November 2016. From November 2016 to October 2018, our modules facilitated 1800 online instructional sessions, utilizing over 3600 pageviews of content. The largest period of utilization was between July and October 2018 (30% of overall sessions). Modules have been regularly used within University of Toronto hospitals. While 48% of sessions occurred within the city of Toronto, the modules have also been viewed in multiple practice locations across Canada, with 10 cities participating in at least 5 sessions.

Discussion

We successfully created high-quality, broadly applied medical programming by utilizing economic principles. Medical trainees have actively engaged in our Just-in-Time initiative and have provided positive feedback, pointing to success at Level 1 of Kirkpatrick's model of education.⁸ Anecdotally, we have found residents who use the modules better prepared before performing a procedure at the bedside, and their documentation of the procedure has improved. The principles used here can be extrapolated to other procedures and educational initiatives. Just-in-Time interventions have been effective for diverse fields,^9,10 with web-based interventions successful as well.⁹ Emulating our principles of cost-efficiency can therefore be beneficial to many different specialties striving for procedural competence.

Limitations to this approach include the time and associated opportunity cost required from the project manager to recruit and manage the multiple stakeholders involved in module creation. Costs associated with updating and evaluating the modules should be accounted for as initiatives expand. The major limitation of our study is that success was measured in participation, rather than true knowledge retention or quantified change in behavior. We were also unable to ascertain the distribution of new users versus recurrent users.

Next steps involve examining resident improvement on objective structured clinical examinations attributable to the procedural modules. Ultimately, behavioral change and subsequent higher procedural success rates would be the ideal downstream effects to quantify.

Conclusions

Utilizing the economic principles of targeting talent, finding value abroad, open source expansion, and extrapolating efficiency, we were able to create and implement online instructional modules at very low costs that were widely accepted and utilized locally, with spontaneous access documented outside our institution.