Prioritizing Workforce Development Training in State Health Departments Using TRAIN: Challenges and Opportunities

Abstract

Objective:

The objectives of this study were (1) to obtain data on the current status of public health workforce training and the use of the Training Finder Real-Time Affiliate Network (TRAIN), a public health learning management platform, in state health departments, and (2) to use the data to identify organizational features that might be affecting training and to determine barriers to and opportunities for improving training.

Methods:

We conducted structured interviews in 2014 with TRAIN administrators and performance improvement managers (n = 14) from 7 state health departments that were using TRAIN to determine training practices and barriers to training. We determined key organizational features of the 7 agencies, including training structure, required training, TRAIN administrators’ employment status (full time or part time), barriers to the use and tracking of core competencies in TRAIN, training needs assessment methods, leadership support of training and staff development, and agency interest in applying for Public Health Accreditation Board accreditation.

Results:

We identified 4 common elements among TRAIN-affiliated state health departments: (1) underuse of TRAIN as a training tool, (2) inadequate ownership of training within the organization, (3) insufficient valuation of and budgeting for training, and (4) emerging collaboration and changing perceptions about training stimulated by agency preparation for accreditation.

Conclusions:

Public health leaders can increase buy-in to the importance of training by giving responsibility for training to a person, centralizing training, and setting expectations for the newly responsible training leader to update training policy and require the use of TRAIN to develop, implement, evaluate, monitor, and report on agency-wide training.

A prepared workforce is an important component of the infrastructure needed to effectively perform the core functions and essential services of public health and to improve health outcomes.^1,2 Yet, most US government public health workers have little or no formal training in public health. This situation may have contributed to the 1988 and 2003 Institute of Medicine characterizations of the public health infrastructure as being in disarray.^3,4 Although public health institutions have recognized the need to train the public health workforce, training has been a low priority and an enormous challenge.⁵

Although the public health workforce needs more training, the performance, tracking, and visibility of public health training have improved. Progress during the past 20 years is primarily attributable to 3 events. In 2001, the Faculty Agency Forum (now called the Council on Linkages Between Academia and Public Health Practice) created the core competencies for public health practice by public health professionals (hereinafter, core competencies). The core competencies were subsequently revised in 2010 and 2014.⁶ In 2003, the Public Health Foundation developed TRAIN (Training Finder Real-Time Affiliate Network), a specialized public health learning management system.⁷ In 2011, the Public Health Accreditation Board (PHAB) launched a national accreditation program for public health agencies. Of the 12 domains of PHAB agency requirements for accreditation,⁸ domain 8 requires that agencies identify organizational learning needs and that they create workforce development plans that use core competencies for tracking and reporting staff training activities.⁹

TRAIN was originally designed to “disseminate, track, and share trainings for the health workforce on a single national online learning management system platform,”⁷ and it contains more than 29 000 training courses, many of which are available to users (or learners) throughout the United States (Table 1). Public health agencies pay an annual fee to access TRAIN. As of 2014, 25 state health departments and several federal agencies were registered TRAIN affiliates. TRAIN was available to 1 million registered users by 2015 and more than 1.75 million registered users by 2018.¹⁰

Table 1.

Number of active learners, courses, and course providers in the Training Finder Real-Time Affiliated Integrated Network (TRAIN),^a by TRAIN-affiliated agencies and study participants, United States, 2014

Sample Group	Unique TRAIN Active Learners,b No. (%)	Active TRAIN Courses,c No. (%)	Trained TRAIN Course Providers,d No. (%)
TRAIN-affiliatede participant agenciesf (n = 7)	270 588 (36.1)	5889 (20.3)	595 (14.9)
All TRAIN-affiliatede agencies (n = 28)	750 000g (100.0)	29 000 (100.0)	4000 (100.0)

^a TRAIN is a public health learning management system sponsored by the Public Health Foundation.⁷

^b Active learners are TRAIN users who have an active open account in TRAIN.

^c Active courses are courses in TRAIN with content that is still current and that can be offered again in the future.

^d Course providers are expert TRAIN users who have more skills and access than regular users but fewer skills and access than TRAIN administrators. Course providers are able to create courses, run reports, and reset passwords.

^e TRAIN-affiliated agencies have a contract with the Public Health Foundation to use the TRAIN system. Of 28 US TRAIN-affiliated agencies in 2014, 25 were state health departments and 3 were either federal agencies or national organizations.

^f Of 25 state health departments that were TRAIN-affiliated agencies, 2 were excluded from the study to eliminate potential bias. Of the remaining 23 agencies that were invited to participate in the study, 7 agencies (Arkansas, Illinois, Kentucky, Oregon, Texas, West Virginia, and Wyoming) participated.

^g TRAIN had more than 1 million active learners by 2015 and 1.75 million active learners by 2018.¹⁰

TRAIN allows agencies to categorize training courses by the core competencies that are addressed by those courses.¹¹ For example, training in disaster and emergency preparedness was available through TRAIN in the early 2000s.¹² TRAIN also allows agencies to create training plans that use the core competencies to report on and track staff training activities. Consequently, health departments that use TRAIN are equipped to meet the PHAB domain 8 accreditation requirements for staff training, which include designing, implementing, and monitoring a workforce development plan (Figure).

Figure.

Relationships among state health departments, domain 8 of the Public Health Accreditation Board (PHAB) accreditation requirements, and the Training Finder Real-Time Affiliated Integrated Network (TRAIN). TRAIN is a public health learning management system, sponsored by the Public Health Foundation, that can be used for training needs assessment, public health training, tracking of the core competencies covered in training, and reporting.⁷ Within TRAIN-affiliated agencies, performance improvement managers usually oversee workforce development plans and TRAIN administrators oversee TRAIN (including managing it, cleaning data, running reports, training users and course providers, providing user support, and creating courses in the system). The PHAB accreditation requirements have 12 domains,⁸ 10 of which mirror the 10 Essential Public Health Services.¹³ The PHAB domain 8 accreditation requirement is to maintain a competent public health workforce.⁹ Of 28 TRAIN-affiliated agencies in the United States in 2014, 25 were state health departments.

Despite the availability of TRAIN, training public health workers is a challenge. A national survey conducted in 2014 and administered to the employees of state health departments found that at least 38% of the public health workforce planned to retire by 2020.¹⁴ The study also identified gaps in public health worker competency in the areas of policy analysis and development, business and financial management, systems thinking and social determinants of health, evidence-based public health practice, and collaboration and engagement with diverse communities. Although systems thinking and financial management have not traditionally been taught in public health training, the gap in competencies in other areas, particularly evidence-based public health practice and policy analysis, speak to the need for better public health workforce training.

The objectives of this qualitative study were to (1) obtain data on the current status of public health workforce training and the use of TRAIN in state health departments by using a sample of agencies with active access to TRAIN and (2) use these data to identify organizational features that might affect training and determine barriers to and opportunities for improving training in the sample agencies.

Methods

For this cross-sectional study, we looked at the entities that were TRAIN affiliates with the Public Health Foundation. As of 2014, 28 entities had signed an affiliation agreement with the Public Health Foundation, including 3 federal agencies. We excluded the 3 federal agencies and 2 state health departments because their inclusion may have created potential bias in our results (one was the department that had previously employed the first author and the other was a department that differed from all other departments because it received substantial additional support from a local university). We invited the remaining 23 agencies, which were TRAIN affiliates and state health departments, to participate in this study. Of these 23 agencies, 7 agencies participated: Arkansas, Illinois, Kentucky, Oregon, Texas, West Virginia, and Wyoming. The University of Illinois Chicago Institutional Review Board approved the data user agreement and our interview tool.

For each agency, we planned 1-hour structured interviews with the TRAIN administrator, who was responsible for administering the system (including managing it, cleaning data, running reports, training users and course providers, providing user support, and creating courses in the system), and the performance improvement manager, who was responsible for the workforce development plan, a PHAB accreditation requirement. We interviewed 8 TRAIN administrators (2 TRAIN administrators from 1 agency were interviewed together) and 6 performance improvement managers (1 performance improvement manager declined) in 2014 using a mix of telephone calls and written responses. We asked TRAIN administrators and performance improvement managers about the training capacity, policies, and challenges at their agencies. We queried TRAIN administrators about training needs assessments, resources, and reports at their agencies and about their use of TRAIN and the core competencies in the training process. We queried performance improvement managers about the status of their workforce development plans, any plans for public health accreditation, and the role TRAIN might play in addressing accreditation requirements.

We based the interviews on a standard questionnaire. Five participants provided written responses to the interview, and 8 participants responded by telephone. For the telephone calls, after obtaining interviewee approval, we audio-recorded, transcribed, and imported the content into NVivo 10.¹⁵ The lead author and a public health graduate student experienced in qualitative data analysis then each independently coded the data. The coders noted no substantial discrepancies in their coding results, and the κ coefficient for intercoder reliability was 0.769. However, the coders did adjust the coding names and structure by mutual agreement. The coders subsequently identified common themes that emerged from the interviews and prepared a list of participant quotes that fit within each theme.

Results

Based on the interviews, we found the following key organizational features pertaining to training in the 7 agencies: training structure, required training, TRAIN administrators’ employment status (full time or part time), barriers to the use and tracking of core competencies in TRAIN, training needs assessment methods, leadership support of training and staff development, and agency interest in applying for PHAB accreditation (Table 2). Based on further qualitative data analysis, we also identified 4 key themes that described the status of workforce training and TRAIN use and the effect of pursuing accreditation among the 7 participating state health departments: training is underused, ownership of training is unassigned, training is undervalued, and accreditation preparation is positively affecting training (Table 3).

Table 2.

Organizational features reported by 7 Training Finder Real-Time Affiliated Integrated Network (TRAIN)^a–affiliated agencies,^b United States, 2014

Feature	Agency 1	Agency 2	Agency 3	Agency 4	Agency 5	Agency 6	Agency 7
Training responsibility
Human resources department handles their training	X	X	X	X		X
Every program handles their own training	X	X	X	X		X	X
Training in transition—not yet assigned					X
Training required on topics other than emergency preparedness	Yes	Yes	Yes	Yes	No	Yes	Yes
No. of TRAIN administrators
Full time	0	1	0	0	0	0	1
Part time	1	0	1	1	1	2	0
Barriers to using and tracking core competenciesc in TRAINd
Core competencies are confusing				X	NAe		X
Lack of understanding	X			X	NAe	X
No verification method	X	X	X		NAe		X
Training needs assessment methods (2012-2014)	Survey	Dialogue among key staff members	Online survey	Survey	Survey	TRAIN self- assessment and survey	None
Leadership supports training and staff development	Moderate to strong	Insufficient data	In transition	Weak	In transition	Moderate to strong	Weak
Interested in applying for Public Health Accreditation Board (PHAB) accreditation	Yes	Yes	Yes	Yes	Yes	Yes	Yes

^a TRAIN is a public health learning management system sponsored by the Public Health Foundation.⁷

^b Of 25 state health departments that were TRAIN-affiliated agencies, 2 were excluded from the study to eliminate potential bias. Of the remaining 23 agencies that were invited to participate, 7 agencies (Arkansas, Illinois, Kentucky, Oregon, Texas, West Virginia, and Wyoming) participated. Information from agencies was obtained during 1-hour structured interviews with their TRAIN administrators and performance improvement managers.

^c The use of core competencies defined for training is required by the Public Health Accreditation Board.⁸

^d As reported by TRAIN administrators at agencies, barriers to the use and tracking of core competencies in TRAIN included the following: core competencies were confusing, lengthy, and/or hard to define corresponding skills; lack of understanding, inadequate knowledge to assign a core competency to a training course when not already done by instructor; and even when core competency was assigned and tracked, no test was available to verify trainees’ acquired knowledge and/or skills.

^e Agency 5 was in the early stages of implementing TRAIN, and the use of core competencies was not yet defined.

Table 3.

Key themes describing the status of workforce training and use of the Training Finder Real-Time Affiliated Integrated Network (TRAIN)^a and the effect of pursuing agency accreditation, as reported by 7 TRAIN-affiliated agencies,^b United States, 2014

TRAIN Is Underused	Ownership of Training Is Unassigned	Training Is Undervalued	Accreditation Preparation Is Positively Affecting Training
TRAIN administrators have multiple responsibilities and are not always full-time. The TRAIN system is complex and not intuitive to use. Course providers have difficulty interpreting and using the core competencies. Buy-in to promote the widespread use of TRAIN is lacking. TRAIN reports are generated only for simple uses (eg, class roster, list of attendees, users completing courses). More training on ad hoc TRAIN reporting tool is needed. TRAIN enhancements would facilitate more use of some features.	Centralized responsibility for training and training evaluation is lacking. A training division, if it exists, is likely to be insufficiently staffed and/or unable to meet agency-wide needs. Frequency of or consistency of tools used for training needs assessments is irregular. Training participation, training needs assessment responses, and evaluations are mostly voluntary. Training courses are provided in certain units or divisions primarily to meet grant or other requirements.	A culture that values training and staff development is needed. Budget does not allocate money for training, budgeting is conducted in silos, and resources are tight, including TRAIN administrator’s time. Inadequate leadership support of and staff member engagement in and perception of training as a benefit rather than a burden. Buy-in from all parts of the organization (staff and leaders) is needed. Leadership personnel and/or leadership priorities are constantly changing.	Agencies are preparing workforce development plans for the first time. Committees and workgroups are being formed to prepare workforce development plans. Training needs assessments are being conducted, training tools are being reviewed, and implementation of findings is being discussed. Training in new topics is being prepared (ie, new employee orientation). TRAIN language and reports are being used in workforce development plans.

^a TRAIN is a public health learning management system sponsored by the Public Health Foundation.⁷

^b Of 25 state health departments that were TRAIN-affiliated agencies, 2 were excluded from the study to eliminate potential bias. Of the remaining 23 agencies that were invited to participate, 7 agencies (Arkansas, Illinois, Kentucky, Oregon, Texas, West Virginia, and Wyoming) participated. Information from agencies was obtained during 1-hour structured interviews with TRAIN administrators and performance improvement managers.

TRAIN Is Underused

Only 2 of the 7 agencies had a full-time TRAIN administrator to manage TRAIN. The TRAIN administrators at the other 5 agencies either worked part time or worked full time but had both TRAIN and non-TRAIN (including other program and software) responsibilities. All 7 TRAIN administrators reported having insufficient time to conduct TRAIN system maintenance, including cleaning data and creating training plans. Comments included:

Administering TRAIN is only a small part of my job; big projects would [sic] pop up, I will knock them down and then we won’t do anything for a year.

TRAIN is really being underutilized. There is not anybody pushing it, reminding people that [TRAIN] is there, offering people to become course providers or things of that sort.

Although 6 agencies required their staff members to have some training (eg, on preparedness and confidentiality), all 7 TRAIN administrators noted that training reports, which allow monitoring of learner profile completion, training plans, evaluation completion, and course feedback, were never requested. Comments included the following:

Reports are just run when someone requests it and it is predominantly for the [TRAIN administrator] information.

I have not been asked in 10 years to run a report of any kind from TRAIN in terms of performance or anything.

We create reports to let people know we are still alive and doing a lot of work, but nobody asks for it.

I pull reports to evaluate the training and share with my supervisor and manager; they are not really asking.

All 7 TRAIN administrators identified challenges to using key TRAIN features, particularly tracking the core competencies being covered in each training course. Although TRAIN is set up to track core competencies, TRAIN administrators noted that ascertaining the core competencies being addressed during a given training course could be difficult. Furthermore, TRAIN administrators reported that because filling in this core competency tracking field within TRAIN was often optional, this feature frequently went unused. As a result, they noted that many TRAIN reports related to core competency coverage that could be generated would have limited accuracy and usefulness. Comments included the following:

It takes time to think who is the audience and what it is you are trying to accomplish, what skill you are addressing and identifying [core competencies] and going through an orderly process.

[W]hen asked what core competencies you are addressing, [some course providers] have no idea or are confused about making the selection; they are not clear on the competencies.

[There is no] systematic approach to collect this information.

[We] don’t spend a lot of time looking at them or taking the effort to validate if those core competencies were actually fulfilled.

Ownership of Training Is Unassigned

TRAIN administrators and performance improvement managers from all 7 agencies reported that although TRAIN was available to all state health department staff members, no single person had oversight or ownership of training at the agency, which resulted in a lack of accountability. It also meant that not all training was being delivered through TRAIN, which reduced the benefit of using TRAIN to track workforce training. In addition, they reported that because the number of staff-training opportunities depended on the schedules of other staff members at the agency who volunteered to lead courses, staff training was often inconsistent. Comments included the following:

Training is done in pockets, not organized or coordinated, and not reported.

No one person [is] in charge of developing a training strategy.

The team [doing training] is completely volunteering; they are people who see value in training and are taking on this extra responsibility.

TRAIN administrators noted that training tended to occur more frequently in some divisions of the agency (eg, human resources or emergency preparedness groups) where it was required (eg, on topics such as confidentiality and the incident command system) than in other parts of the agency. They reported that because the use of TRAIN for training was not required and ownership of training was lacking, agency-wide tracking of and reporting on training was inconsistent. Comments included the following:

Each group…creates their trainings with their own subject matter experts.

[Human resources] is responsible for personnel training.

Each program is responsible for training in their own area.

Everyone is required to take FEMA [Federal Emergency Management Administration] courses.

TRAIN administrators reported that this unsystematic, uncoordinated training resulted in infrequent assessments of training needs and the use of various tools and approaches when these assessments were conducted. Comments included the following:

We have talked about doing some kind of a survey with our nurse staff to identify what they need, because there is no really formal process in place.

Each of the individual programs do [sic] their own internal assessment.

Each supervisor decides if that assessment is enough or if we need to do another assessment.

Training needs beyond the required employee trainings are identified by supervisors at annual performance evaluations.

Finally, all TRAIN administrators reported that the lack of a systematic training approach, a clear budget for training, and a single person owning or overseeing training resulted in problems obtaining funding for training. Comments included the following:

A budget established for generalized training of all employees…would be helpful, but we don’t have that now.

If I identify a training need, I go to the director and ask for funding…but there is no regularly established budget for all of the agency [training].

Everybody is in a little silo and everybody gets their funding in silos and they do their own thing.

Training Is Undervalued

Of the 7 TRAIN administrators, 5 cited leadership changes as a major challenge for workforce training. Depending on the philosophy of each new administration, the level of priority assigned to training varied. Although the level of priority sometimes increased, all changes created an inconsistent approach to training. Comments included the following:

Three different commissioners in 2 years, each one with different priorities; that is a barrier: competing priorities and lack of consistent leadership.

Leadership is a barrier. You have to be able to inspire people to want to try to improve themselves, and you need an environment in which that happens. And that has not existed in our agency for a long time.

The change of management has helped because there has been a positive influence and direction…we are starting to see some results.

Our new director is very committed to employee and workforce development.

TRAIN administrators and/or performance improvement managers at 3 of the 7 agencies reported that training at their agency was perceived as a burden or inconvenience rather than as a tool for progress. They expressed a desire to see a cultural shift that would reverse this perception. Comments included the following:

A cultural shift would encourage training as a professional development tool and not as a burden.

We need to carefully plan training, but we first need to do a culture shift in the agency to appreciate and support training.

It is just that culture, until [training] becomes routine…is a work in progress.

About 56% of the staff reported [in a survey that] the agency not [being] fully in support of training is a high or moderate barrier to accessing training.

To help make training a valued commodity, TRAIN administrators and performance improvement managers called for their organizational leadership to have higher expectations of and more buy-in to training. Comments included the following:

[We need] leadership buy-in, to engage staff to train others in their area of expertise; and [buy -in] from staff and course providers to use TRAIN for all courses.

We need buy-in from management and leadership to expect training evaluation.

We are trying to get buy-in; if [only] there was greater value placed on employee training!

[We need] greater value placed on employee training.

Make a decision that education and training is important, and when you hire somebody you can look at the assessments and evaluate the outcomes and put together a group of people who can help make some decisions about what is the next step. We don’t do that.

[Leadership should promote] not only why it’s good for the agency to use TRAIN, but [explain] how it benefits the center, how it benefits course providers, supervisors, and end users.

Accreditation Preparation Is Positively Affecting Training

In 2013, all agencies represented by the study participants had already applied for accreditation or were planning to apply for accreditation through PHAB. TRAIN administrators and performance improvement managers reported that the preparation for accreditation, particularly through the work they were doing to meet the PHAB domain 8 accreditation requirement,⁷ was changing the perception of training within their organizations. Comments included the following:

Accreditation is having a huge, huge impact. A lot of the TRAIN language has been moved into department policy. It has elevated the priority of training both locally and at the state level in a major way.

Accreditation is the impetus for us to be doing the things we should be doing anyway.

Accreditation has been important because it is forcing more awareness and more use of TRAIN department-wide than ever before.

[The impact] has been huge; accreditation forces you to look at everything that you do in a kind of microscopic lens and forces you to look at best practices.

[Accreditation] has put the emphasis back on training and employee development.

Accreditation is going to impact [training] in a positive way.

TRAIN administrators and performance improvement managers reported that workforce development plans were not routinely being created in the past. However, the move toward accreditation was subsequently triggering conversations and collaborations within agencies in support of workforce development. Comments included the following:

I think [the workforce development plan] is going to bring out the need for more and more of TRAIN and how it would benefit [the agency].

The conversation around workforce development has people now looking at what [training] is going to look like.

After 100 years, those conversations [about training] are just beginning…in the past it was mostly driven by grants.

[Accreditation has been] the impetus to pull multi-stakeholder groups to develop a workforce development plan [and to] do the training needs assessment.

Discussion

The public health community faces numerous public health challenges, including infectious disease outbreaks, climate change, health inequities, the opioid crisis, the obesity epidemic, health care reform, and gun violence. As a result, the need for a well-trained public health workforce has never been greater. The results of our study indicate that workforce training at state health departments has been inadequate but point to opportunities for improvement.

For some time, state health departments have used unfunded, decentralized training models in which training has occurred in an unorganized way, been largely unscheduled, and relied primarily on staff members who volunteered their time. Although TRAIN has been available to TRAIN-affiliated departments as a robust tool for training, our analysis suggests that it has not been used to deliver all training at these agencies. Furthermore, we found that agency TRAIN administrators, who were most prepared to oversee the use of TRAIN, seemed to be poorly positioned within organizations to successfully advocate for the support and resources needed to increase the use of TRAIN.

We observed that most training at these state health departments has been optional and TRAIN has been sporadically used for training, reporting, or categorization of the core competencies. Indeed, it appears that workforce development efforts at these health departments have been neither consistently discussed nor adequately funded. Nevertheless, we also found that the public health agency accreditation movement has opened a new dialogue in these agencies about the importance of workforce training. However, a strategy has yet to be implemented in most agencies to systematically develop, implement, and evaluate agency-wide training. Yet, based on our analysis, TRAIN has the support of both TRAIN administrators and performance improvement managers as a valuable tool to help with these processes. That said, using TRAIN as a central component of a new training strategy would likely require additional financial and human resources, which may be challenging.

Despite resource limitations, leaders could affect the training landscape of these organizations’ health department workforce. They could centralize training oversight; become training promoters, consumers, and role models; allocate a training budget; support the creation of a training policy; and mandate the use of TRAIN for all staff development and training. Leaders could also take advantage of the momentum created by preparation for accreditation and organizationally align the responsibilities of TRAIN administrators and performance improvement managers in creating agency-wide workforce development plans. Finally, leaders could create staff member buy-in to training and build an organizational culture in which training is highly valued and perceived as a tangible, professional benefit rather than an unwelcome duty.

Limitations

This study had several limitations. First, only 7 of the 23 agencies participated in the study. Although the sample size was small, the 7 agencies represented 270 588 unique active TRAIN learners in 2014, the equivalent of 36% of TRAIN users nationally that year. In addition, the total population of the 7 states in our study that were using TRAIN was 16.7% of the total population of all 28 states using TRAIN. Second, given that TRAIN-affiliated agencies determine their own training policies (concerning how TRAIN is used, which staff members have TRAIN accounts, and how often training reports are generated), the results of this descriptive study may not be generalizable to all state health departments. Nonetheless, the agencies participating in the study were a broad mix of new and seasoned TRAIN affiliates, with 1 to 10 years of experience using TRAIN, and they represented a wide spectrum of TRAIN learner populations, ranging from 10 000 to more than 100 000 unique learners.

Conclusions

To begin addressing the training barriers reported in this study, state health departments could identify a “training director,” assign responsibility for agency-wide training, and allocate a budget. An appropriate candidate could be a senior staff member skilled in training and professional development and with the authority to build buy-in to increase training value. This training director could set minimum training standards policy and require that all training be delivered via TRAIN. Furthermore, this staff member could work in concert with the TRAIN administrator and staff member involved in public health accreditation. Most importantly, however, the training director should be in a high level of the organization, to either report to the director’s office or an agency-wide branch of the state health departments, to ensure ongoing support and effectiveness.