Developing an Online and In-Person HIT Workforce Training Program Using a Team-Based Learning Approach

Abstract

Workforce training in health information technology (HIT) is in demand as electronic health record adoption becomes a nationwide priority. Columbia University and Weill Cornell Medical College worked together to develop a 6-month ONC-supported certificate course. To identify relevant skills and knowledge, we conducted a needs assessment that included: interviews and focus groups with potential employers and current HIT employees; an analysis of both published literature on competencies and actual job listings; and the development of a diverse operations-oriented curriculum advisory committee, which help to synthesize the findings into 6 core curriculum modules. We selected a team-based learning approach, allowing us to train a diverse student body and providing opportunities to build collaboration skills. Our novel hybrid adaptation of team-based learning combines online and in-person activities. Lessons learned from the development of this program are likely to have widespread applicability as training programs in the field become more prevalent.

Introduction

With the federal electronic health record (EHR) incentive program and related initiatives in the HITECH act of 2009, adoption of health information technology (HIT) is expected to grow rapidly. Estimates of workforce needs have varied, but may be more than 50,000 workers over the next several years.^1–3 As healthcare organizations move to more advanced electronic systems, the need for HIT employees grows.⁴ With the support of the Office of the National Coordinator for Health Information Technology (ONC), training programs are rising to the challenge of training competent workers in a short period of time. At Columbia University and Weill Cornell Medical College, we have embraced this challenge, with funding by the ONC, and have undertaken to train approximately 200 new members of the HIT workforce over the next two years, using a jointly developed certificate curriculum across our two campuses.

Both institutions have histories of informatics-based training programs, Columbia through its Department of Biomedical Informatics,⁵ and Weill Cornell through its Division of Quality and Medical Informatics.⁶ Nevertheless, a training program specifically focused on the HIT workforce was novel to both institutions. We faced three challenges. The first was to develop a new curriculum that prioritized workforce-oriented knowledge and skills over research. The second was one common to most informatics training programs, which was the need to provide concurrent education for students with extremely different training backgrounds: some students enter the program with strong technology or computer science background, while others enter with clinical or public health training or healthcare management or support experience. Finally, in keeping with ONC’s priority to train certificate students quickly, we sought to determine how to deliver this curriculum to a large number of students without diluting their educational experience or over-burdening faculty.

In this paper, we discuss our approaches to solving all three challenges. First, we describe a research project to elicit the perspectives of potential employers and students about the content for the training program, along with the creation of an operationally oriented curriculum committee to translate these perspectives into curriculum outlines and learning objectives. Second, we discuss our selection of an educational approach, namely team-based learning, that not only works well with a diverse student body but that takes advantage of their differing expertise to enrich the educational experience. Finally, we discuss our work to translate the team-based learning approach to an online environment, creating a novel hybrid program that incorporates the scalability of distance learning with the advantages of in-person classes. As the need for health IT workforce training continues to expand, we anticipate that these lessons may be broadly applicable to other institutions.

Methods

Our training grant focused on 3 ONC-defined roles: health information management and exchange specialist, health information privacy and security specialist, and programmer/software engineer. Our needs assessment included multiple approaches to determine critical knowledge and skills for these roles: semi-structured interviews with professionals in HIT, a scan of current HIT job postings, a review of existing informatics and HIT competencies and literature, and the incorporation of an advisory committee.

A. Interviews and focus groups

We conducted interviews and focus groups with a total of 38 participants (Table 1). We targeted potential employers, professionals who were currently performing the target roles, and potential certificate students. The first focus group included clinical directors of outpatient organizations (n = 9). The second (n = 9) and third (n = 14) included employees of an academic medical center: most of these were employed in health IT, but a few were working in related fields (such as general IT services) and seeking to move into HIT. We also conducted 6 individual interviews with potential employers in health care organizations and public health. The research was approved by the IRBs of Columbia and Weill Cornell, and participants provided written informed consent.

Table 1:

Semi-structured interview participants (n = 38)

Workforce role	Setting
Clinical Directors (n=9)	Outpatient clinic
Manager (n=1)	EHR vendor
HIT services (n=1)	Hospital
Chief Information Officer (n=1)	Outpatient clinic
Chief Medical Information Officer (n=1)	Hospital
Chief Executive Officer (n=1)	Outpatient clinic
Manager (n=1)	Health department
Current employees (n=23)	Academic medical center

The semi-structured interview guide included questions about health IT organizational structure and job openings, the roles that certificate graduates might fill, and skills and knowledge students would need to be successful in these roles. For potential students, we also asked what skills they thought they needed to enter this field. Participants also reviewed and commented on curriculum drafts, which were iteratively revised and updated. Key questions were:

1. What are the biggest personnel challenges faced when dealing with EHRs at clinical sites?

2. What skills are needed from individuals responsible for dealing with EHRs at these sites?

3. Do the proposed curriculum topics address the needed skills?

4. What types of individuals are needed to maintain EHRs at the sites?

Interviews were recorded or captured with field notes, which were later reviewed for common themes.

B. Environmental scan of existing training resources and job listings

We reviewed other training resources and curricula, including some of the ongoing work of the Curriculum Development Centers Program, the ONC initiative to develop materials useful for the ONC job roles being trained at community colleges. We also reviewed recently published sets of competencies for informatics specialists.

To ensure a close match with the current job market, we also conducted an environmental scan of job listings and advertisements from websites such as www.monster.com, the HIMSS JobMine (http://jobmine.himss.org), Health Care IT News JobSpot (http://jobspot.healthcareitnews.com/). We excluded jobs that involved mainly coding or transcription and focused on jobs that centered on EHR implementation, maintenance, and use.

C. Curriculum advisory committee

Concurrently with the efforts described above, we convened a diverse advisory committee for weekly teleconferences to discuss broad curriculum topics and analyze the products of the interviews, focus groups, and environmental scans. The group included representatives working in operational informatics in large and small healthcare settings as well as industry, and faculty in nursing, medicine, dentistry, public health, and informatics. In addition, we invited representation from people with expertise in teaching methods in adult education, one of whom was from a community college. (Members of the advisory committee are listed here: http://www.sce.cornell.edu/exec/hitcp/faculty.php).

Results

Our needs assessment included focus groups and interviews, environmental scans of published literature and job listings, and advisory board contributions.

Focus group and interview findings

Potential employers and current members of the HIT workforce mentioned a wide variety of types of knowledge and potential skills.

Understanding clinical operations and workflow was a very important topic mentioned in every focus group and by several of the individual interviewees. For example, one HIT professional said, “I think [they need to learn] workflows from beginning to end. Patient walks in door, patient walks out the door. Technical people in general, we tend to overlook that stuff.” Provider workflow and documentation practices were a related issue.

Teamwork to bridge multiple disciplines was another theme mentioned in nearly every interview and group. One manager said, “You have to bring together 3 elements: the IT side, the medical side, and the people side.” Along similar lines, some focus group members mentioned ways to address the potential variability between different types of people who would be interested in entering the training program, recommending education in clinical terminology for information technology specialists, and training in IT and network concepts for clinical professionals.

The importance of project management arose multiple times in multiple interviews and focus groups. Other themes included: interfacing between disparate information systems; privacy and security; decision support as well as evidence-based medicine and medical decision making techniques; databases, data warehouses, and analytics; customizing vendor EHR systems; understanding of healthcare system issues such as the roles of payers, regulators, and providers; ability to create reports and communicate well orally and in writing; and ability to perform staff training.

After an iterative review process, we classified the themes into 3 umbrella categories: (1) Clinical concepts, clinical workflow, and health care organizational structure, (2) HIT business and operations, and (2) Technical knowledge and skills.

In addition, we collected specific suggestions by the potential students. They mentioned the importance of flexibility in scheduling, including distance learning options, maintaining a relatively low cost, and direct applicability of all learning to the workforce. Potential students were also interested in hands-on learning experiences and mentorship or networking opportunities with where possible, to add value any lectures and readings.

Environmental scan results

We determined that the Health Information Management and Informatics Core Competencies for Individuals Working with Electronic Health Records was a very relevant competency set for our program (Table 2).⁷ This set of competencies, broadly covering the information needed by people working daily with EHRs, was developed by the American Medical Informatics Association (AMIA) and the American Health Information Management Association (AHIMA). We also drew content from the Core Content for the Subspecialty of Clinical Informatics⁸; however, this list was developed with the intention of creating a new medical subspecialty, and as a result is focused on a higher level of management. In addition, we identified considerable similarity between our student needs and those covered in the work of the Curriculum Development Centers Program, which was going on concurrently with our curriculum development, even though the targeted roles were different.

Table 2:

Informatics Competencies

Health Information Management and Informatics Core Competencies for Individuals Working with EHRs7	Core Content for the Subspecialty of Clinical Informatics8
Health information literacy and skills Health informatics skills using the EHR Privacy and confidentiality of health information Health information/data technical security Basic computer literacy skills	Fundamentals (background and culture of clinical informatics and the health system) Evidence-based patient care Health information systems Leading and managing change

We collected more than 50 advertisements and job listings and scanned them for descriptions of required skills and knowledge. In addition, we noted that many of the job listings described specific documents that employees would be required to create, review, or work with. Table 3 presents the results by each of the 3 ONC-defined roles in our training program. In our analysis, we found that it was very easy to match specific jobs to the health information privacy and security specialist role, but more difficult to do so for the other two roles. This was because the job listings used relatively general job titles (such as project manager, site manager, and analyst), or else were highly specific, requiring expertise or certification on specific EHR systems.

Table 3:

Mapping roles to real job titles with required qualifications and skills

ONC-defined role	Job title	Most common qualifications/skills	Documents used in job
Health Information Management and Exchange Specialist	Project Manager EMR Site Manager Quality Assurance Engineer Clinical Analyst Interoperability & analytics director EHR Clinical Project Director EHR Workflow Improvement Manager HIM Services Director	Develop standards, policies, procedures, processes EHR implementation Business applications Evaluation--systems and procedures User training System testing System design System maintenance Workflow analysis	Workflows, user requirements, procedures, defect report, test cases, project work plan
Health Information Privacy and Security Specialist	Senior Information Security & Compliance Analyst HIM Director/Privacy Officer Electronic Medical Records Security Administrator	IT risk management knowledge Compliance regulations knowledge Security framework knowledge Security standards knowledge Risk assessment/risk management Security administration	Processes/tools for IT compliance, security policy, reporting protocols, roles/permissions, coding system regulations
Programmers and Software Engineer	Application Analyst Software Development Engineer Software Engineer Product Development Software Engineer Database Engineer Network Engineer	Specific technologies (SQL, Epic, Crystal Reports, C#, UML, HL7) Technical writing, system documentation, design documents Software development lifecycle Product and system testing Release control System/database design, analysis, development, testing, implementation Data modeling	Logs, requirements, test programs, installation instructions, release notes, design documents, disaster recovery processes
All	N/A	Communication skills Organization skills Problem solving skills Project management	Project plan Budget System specifications Test plans

Advisory committee

We assembled a highly diverse advisory committee, with an emphasis on people representing operational jobs in health information technology as well as educational methods. We launched the committee’s work with an in-person meeting in order to facilitate in-person introductions, teambuilding, and brainstorming. Subsequently, we established a regular weekly or biweekly teleconference supported by screen sharing and web-based teleconferencing software, over a period of approximately 6 months. After the initial brainstorming, we guided the discussions by identifying 8 broad topics from the research described above, and presented one of these topics every week to the committee. The group came to consensus about whether the topic was important, what components it should include, and what the learning objectives should be. They also assisted in identifying appropriate faculty to teach each segment, suggested text books and online resources, and provided suggestions and discussions of teaching and learning methodologies.

Curriculum

After the efforts described above, we agreed upon a curriculum with 5 major components required of all students:

1. EHRs and related systems

   • This segment includes introductions to the types of systems, to federal policy including “meaningful use”, and to core informatics concepts such as terminologies, knowledge representation, standards, and relationships between technology and clinical workflow.
2. Business operations and project management

   • This segment includes an overview of project management methods and their application to health information technology, with discussion of process modeling and process consulting.
3. Health information services

   • This section describes how to select, implement, and support a HIT system in a health care organization.
4. Supporting health care decisions

   • Core concepts in this segment include clinical decision support, quality measurement and improvement, public health informatics, and patient technologies such as personal health records.
5. Health data exchange and security

   • This segment includes a discussion of standards and infrastructure for health information exchange and interfaces, plus core concepts in privacy and security.

The 6th component was the specialty training to serve the ONC roles of health information management and exchange specialist, health information privacy and security specialist, and programmer and software engineer. Each student had to select one to complete. This curriculum fit well within a 6-month format, with one month devoted to each of the 5 core components listed above, and the final month devoted to the specialty component.

Finally, to address the issue of the varied backgrounds of the students, we created prerequisites to help students fill in potential knowledge gaps. Each student was required to complete either the “US healthcare system” module (if they lacked a clinical background), or the “Computer science concepts for clinicians” module (if they lacked a technology background) within the first month of the course.

We also developed a capstone project, a mock implementation plan for a fictitious ambulatory practice and compliance with meaningful use criteria. To provide real-world context, we gave all students an account with a free web-based ambulatory EHR, Practice Fusion (San Francisco, CA; www.PracticeFusion.com); developed exercises to learn the basics of using it; and designed the project around the strengths and weaknesses of this specific product.

In developing the content of these curriculum components, we were able to draw upon the rich existing educational and operational resources of Columbia University, Weill Cornell Medical College, and New York-Presbyterian Hospital, which serves as the teaching hospital for both medical schools. We drew contributors from the Columbia Department of Biomedical Informatics master’s and doctoral programs, the Weill Cornell fellowship program in quality and medical informatics, the Weill Cornell Provider Organization faculty practice, the New York-Presbyterian Hospital information services division, and Cornell University in Ithaca. We also supplemented these in-house resources with guest lectures and mentorship provided by HIT professionals from regional health information organizations, community hospitals, regional extension centers (RECs), software vendors, consulting firms, and other organizations.

Educational approach

We knew that we would face the challenge of training different types of students in the same cohort (specifically, information technology specialists and more clinically oriented students). In addition, our qualitative findings and our curriculum committee both emphasized that a key skill in informatics is the ability to work with people from a variety of backgrounds to solve a problem collaboratively. This suggested that we could consider a diverse student body to be an educational opportunity rather than a problem.

Our faculty advisory committee recommended team-based learning (TBL), a methodology used in medical education that appeared well-suited to our purpose. TBL is an instructional strategy designed to “(a) support the development of high-performance learning teams, and (b) provide opportunities for these teams to engage in significant learning tasks.”⁹ In the TBL approach, instructors create learning teams of 5–8 students that are intentionally designed to contain people with diverse backgrounds. The methodology is highly structured (Figure 1). In the first phase, students prepare individually through readings and lectures. In class, students complete a quiz individually and then complete the same quiz as a team. This is followed by a discussion by the entire class. The next step is that the teams are presented with cases to discuss with their team, followed by group discussion and feedback. These cases typically involve application of more complex concepts.

Figure 1:

Original team-based learning model (top row) and adapted online version (bottom row)

Distance learning

We sought a scalable approach that would allow us to deliver the content to large numbers of students without either diluting their educational experience or overwhelming the faculty, and distance learning seemed to be an option to accomplish this. It had the additional benefit of allowing scheduling flexibility for students, which is important for individuals who are working while in school. Team-based learning seemed well suited to an online format because it would create a sense of student community through networks of peers that would be likely to persist even after the course.

Distance learning has been successful in informatics, and generally includes interactive features such as virtual groups and discussion boards.^10,11 However, formal application of the team-based learning paradigm has largely been in-person,¹² with only a few examples of online or hybrid TBL. These online versions have received positive qualitative feedback from students, with high student retention and performance.¹³ Other reports of online TBL have described motivation and perception of learning by students.¹⁴

We opted for a hybrid approach, in which the majority of material would be covered through distance learning, but in which students would assemble for intensive day-long classes once per month. This approach offers the scalability and flexibility of asynchronous distance learning while still allowing us to assemble the students to monitor their learning, provide in-person mentoring, and give opportunities for face-to-face discussion and guest presentations from industry professionals.

We followed this general format in developing the following distance-learning materials:

1. Voice over PowerPoint recorded lectures (2–4 per week), which could be streamed or downloaded.

2. Weekly readings from textbooks, internet resources, and news.

3. Weekly online quizzes. Each student had to complete a quiz to demonstrate mastery of the basic concepts in the weekly lectures and readings. Student teams assembled (through e-mail, chat room, teleconference, or any other communication method of their choice) to complete the same quiz.

4. Weekly cases. During the 1st month, teams were given open-ended discussion cases and had to work collaboratively to develop short written answers.

5. Capstone project. During the subsequent months, student teams worked on the capstone project, which had monthly deliverables. These deliverables were tailored to match the month’s learning materials and incorporate real-world tasks. For example, during the month in which students learned about project management techniques, their monthly deliverables included developing elements of a project plan.

For the online adaptation of TBL, we used online course management software to enforce deadlines for a weekly schedule of activities that allowed students to work asynchronously and at a distance while ensuring that they adhered to the general framework of the TBL sequence constraints (Figure 1). Quizzes were graded automatically by online software; cases were graded manually by faculty members. Additional online resources included a discussion board and messaging system.

We designed monthly in-person sessions to present a combination of team-based learning activities to reinforce the previous month’s course content, present new course content relevant to the subsequent month, and provide an opportunity for guest talks by members of the HIT workforce. We developed a strong network of guest speakers throughout the region to provide students with a chance to learn from and network with working professionals. In some sessions, we also arranged local site visits to institutional HIT departments.

Evaluation plan

Because the program is midcourse at the time of this writing, evaluation results are not yet available. The student evaluation plan includes performance assessment through team-based learning quizzes and cases, written midterm and final exams, within-team peer evaluations, and the capstone project. The evaluation plan follows Miller’s pyramid of competence, a framework for assessing students’ knowledge and skills that is prevalent in medical education.¹⁵

The course evaluation includes periodic assessments of the content, format, appropriateness, and clarity of the lectures, quizzes, cases, and project assignments. A full assessment at the end of the course will determine the effectiveness of the course in terms of the TBL method, the online format, the in-person sessions, and the instructor interactions. We also developed an anonymous online suggestion box to be available to students through the course management system for unstructured feedback; we also have received continuous feedback from students via e-mail and discussion boards.

Initial session

For the first cohort, we simultaneously trained 51 students at Columbia and 26 at Weill Cornell, all of whom shared the same distance learning resources through a single web-based course management system. The educational backgrounds of students were diverse. Sixteen percent held an advanced, professional or terminal degree, including 7 MDs, 2 PhDs, and 3 JDs. Forty-three percent (n=33) were master’s-prepared in areas such as biomedical sciences, public health, health policy, business management and administration, counseling and education, and nursing education. Forty-eight percent (n=37) held a bachelors degree in a clinical health profession such as nursing or a science, technology, and engineering field. Women comprised slightly more than half of the group (57%). Students reported residence within the tri-state area of New York, New Jersey and Connecticut as well as other states (Florida, Georgia, Maryland, California and Utah).

Discussion

In this initiative, we sought to create a highly relevant, workforce-specific health IT training program. To collect information about course content, we scanned the literature in published informatics competencies but found that our interviews and focus groups with current members of the HIT workforce and potential employers were extremely important in firmly grounding or training in current workforce needs. We also performed an environmental scan of current job listings to ensure that we were providing a variety of the types of highly desired skills and knowledge. Our diverse and active curriculum advisory committee was critical in broadening our prospective and helping us synthesize the information we were receiving from the qualitative research and the environmental scans into actionable learning objectives.

Although we were able to draw upon rich resources within our academic departments, we also had to reach out more broadly into the health IT workforce within our medical institutions and within our communities and regionally in order to develop an entirely new curriculum with a strong emphasis on currently needed workforce skills and competencies. We also faced a challenge common to many informatics training programs, which was a student body with very diverse training backgrounds, ranging from clinical expertise to information technology to health care management. In addition, we needed to create a scalable solution suitable for a large potential student body.

The solutions we present here were both to implement a team-based learning model in which students interacted closely with each other to share knowledge and build teamwork skills, and to develop a distance-learning approach supplemented with regular in-person classes. Although in-person team-based learning has been used in medical education, the team-based approach is novel to informatics. Furthermore, hybrid or online team-based learning has not been reported in HIT training.

Online team-based learning has allowed us to offer the course to a relatively large number of students with limited faculty resources. We found some challenges adapting the team-based learning methodology to the online platform, but were able to develop workable solutions with a standard course management system.

As demand increases for workforce-oriented training programs, we expect that more applied programs will be developed by multiple institutions nationwide. This provides an opportunity to share lessons learned across settings in order to develop the most effective solutions to meet the nation’s health information technology needs. More research is underway to determine what skills and knowledge are needed in the workforce to support the use of EHRs, and we intend to continue to incorporate the new findings into what is anticipated to be a continuously evolving curriculum.

Figure 2:

Developing training and assessing student competence using Miller’s pyramid of competence