A Review of Public Health Training Needs Assessment Approaches: Opportunities to Move Forward

Abstract

Context:

Assessing training needs of the public health workforce is crucial for creating professional development opportunities to improve knowledge, competence, and effectiveness of this workforce.

Dissemination:

Regional Public Health Training Centers (RPHTCs) assess workforce training needs and deliver training based on identified needs. To determine training priorities, several needs assessment surveys have been administered by RPHTCs and national public health member organizations.

Evaluation:

This study identified the types of training questions being asked to public health practitioners in the various assessment surveys implemented by RPHTCs and national membership organizations. Although the surveys measured similar overarching constructs, multiple approaches with limited consistency were used to measure training needs.

Discussion:

Although successful in responding to the needs of their targeted constituents, the limited consistency among survey types makes generalization of findings difficult. Disseminating common metrics and aggregate survey findings would increase efficiency in determining workforce training needs and developing targeted training.

A capable and qualified workforce is a hallmark of a strong public health infrastructure.^1,2 Unlike many other fields, public health includes a range of disciplines that often require different levels of training and professional preparation. However, despite the large number of schools and accredited programs of public health and the thousands of students receiving degrees from these programs each year,³ overall, only 17% of the state and local health department workforce has any type of formal public health education, and even among public health science staff, only 30% have a formal public health degree.⁴

The governmental public health workforce has access to training resources that can be completed through certification programs, short courses, e-learning,^5,6 conferences, and workshops. In addition, on-the-job training opportunities such as the Public Health Associate Program⁷ and the Epidemic Intelligence Service⁸ provide hands-on experience that serves as a foundation for public health careers. However, as the public health mission becomes more complex and its service delivery landscape evolves, the 21st-century workforce needs a timely, responsive, and robust skill set that employers could support through ongoing professional development opportunities.

Sustaining a competent public health workforce requires assessing gaps in skills and knowledge and the availability of relevant training to fill those gaps. For nearly 2 decades, the US Health Resources and Services Administration (HRSA) has supported focused training for the public health workforce. In 1999, HRSA established the Public Health Training Center to assess workforce training needs and provide complimentary training. In 2014, this model was restructured into a collaborative partnership known as the Public Health Learning Network (PHLN) to improve the public health system by strengthening the technical, scientific, managerial, and leadership competence of current and future public health professionals.⁹ The PHLN comprises 10 Regional Public Health Training Centers (RPHTCs), each located in a US Department of Health and Human Services region, 40 local performance sites, and the National Coordinating Center for Public Health Training (NCCPHT) housed at the National Network of Public Health Institutes.⁹ Each RPHTC assesses the training needs of the public health workforce in its region by delivering different surveys specific to each of its constituents. Using the core competencies for public health professionals developed by the Council on Linkages between Academia and Public Health Practice (CoL),¹⁰ RPHTCs offer competency- and distance-based training specific for the public health workforce at a national level.¹¹ The PHLN provides public health training across all US states and territories. During the 2014–2015 academic year, the PHLN delivered approximately 1700 unique public health-related courses to approximately 87 500 practitioners working in specialties that included behavioral health, community health, nursing, medicine, dentistry, health education, and emergency preparedness.¹²

Other nongovernmental public health organizations exist in the United States, including public health member organizations. These organizations, such as the Association of State and Territorial Health Officials (ASTHO) and ASTHO’s Affiliate Council and Peer Network, represent public health staff and provide support functions to their members, including training needs assessments targeting specific disciplines or professions. ASTHO also surveys members regarding public health workforce training needs. The Public Health Workforce Interest and Needs (PH WINS) survey, the largest assessment of the public health workforce, was led in fall 2017 by ASTHO in collaboration with the de Beaumont Foundation; a previous similar effort was conducted in 2014.¹³ PH WINS included a separate training needs assessment module focused on crosscutting workforce training needs.

Although the literature establishes widespread support for generating data about workforce demographics, defining skills and competencies across disciplines, and expanding high-quality training based on the identified needs and gaps,^14–18 the public health system lacks a systematic framework for evaluating the workforce capacity required to ensure effective delivery of essential public health services.¹⁹ More specifically, it lacks a uniform process for determining and prioritizing public health workforce training needs.^{9,12–14,20,21} To identify a base on which to build standardized assessment tools, an assessment and document review was conducted to characterize training needs assessment approaches used by RPHTCs and public health member organizations. Results of that endeavor are presented in this report.

Approach

The study aim was to identify types of training questions being asked to public health practitioners in the various assessment surveys they receive. Having reviewed the literature and information collected through large surveys,^13,22 we elected to explore assessments from the following 2 major groups: RPHTCs and national membership organizations, because both had a regional or national focus. We requested training needs assessment tools (ie, surveys) developed or used by RPHTCs during fiscal years 2014–2016. We also identified surveys used by national member organizations during 2016. These surveys were largely collected through online searches of publicly available Web sites, although a limited number were collected through requests to individual membership organizations.

RPTHC surveys were reviewed, systematically abstracted in fall 2016, and categorized into the following 2 groups: those designed to assess the needs from potential trainees and those designed to elicit feedback from leaders, supervisors, or managers about staff training needs.

On the basis of content review of surveys used by RPHTCs, we identified 8 survey domains, including background and employment (eg, position or role), demographics (eg, sex or education), use of CoL core competencies, use of other competency frameworks, technology capacity and access, training modality and preferences, topics of interest, and other (eg, barriers and facilitators). The total number of questions (ie, items) included in each survey was identified, and items were categorized into one of the 8 domains. Item analyses regarding the use of core competencies, training modality and preferences, topics of interest, and respondents’ background were performed descriptively.

National membership organization surveys were examined through a similar process. Content reviews of these instruments indicated a prominent focus on training needs and related competencies (ie, the other 6 domains identified in surveys used by RPHTCs were largely not included). Because of this, surveys from national membership organizations were categorized into 2 areas: use of CoL core competencies framework and use of other frameworks.

Data were managed and analyzed using Microsoft Excel (Microsoft Corporation, Redmond, Washington). The RPHTC assessment was part of a larger evaluation and standardization effort across the PHLN, led by NCCPHT, and reviewed and approved by the University of Southern Maine’s Institutional Review Board. The Centers for Disease Control and Prevention reviewed this study for human subjects protection and deemed the work to be nonresearch.

Results

A total of 24 surveys were reviewed, including 14 used by 6 RPHTCs and 10 used by national membership organizations. Overall, 6 surveys were designed exclusively to assess the needs of potential trainees, and 15 were designed to elicit feedback from leaders, supervisors, or managers about the needs of their staff. Three surveys were used with both audiences.

The 14 RPHTC surveys reviewed contained 1373 items; surveys ranged from 17 to 282 items in length (mean = 98 items; median = 70 items). All RPHTC surveys included items within the background domain, and the majority included items regarding the CoL core competencies (86%), topics of interest (64%), and training modalities or preferences (57%) (Table 1). Approximately 88% of all RPHTC survey items reflected the CoL core competency framework. The CoL core competency framework was used more than any other competency-based approach among RPHTC surveys. Twelve (86%) RPHTC surveys incorporated this framework.

TABLE 1.

Items Analysis by Survey Domain Across Regional Public Health Training Center Needs Assessment Surveys (N = 14)

Survey Domains	Number of Items Across All Surveys	Percentage of Surveys With Items Within a Domain
Background (eg, position or role)	100	100
Competencies—Council on Linkages	1009	86
Topics of interest	33	64
Training modality or preferences	32	57
Demographics (eg, sex or education)	36	50
Technology capacity or access	26	43
Other (eg, barriers or facilitators)	53	43
Competencies—Other framework or items	84	29
Total	1373	...

The CoL core competency framework use varied greatly across RPHTC surveys. For example, some surveys assessed competency domains (n = 4), some assessed individual levels of competencies within these domains (n = 7), and others assessed both (n = 3). In addition, 13 surveys used the 2010 version of the CoL core competency framework and 1 survey applied the 2014 version. Among surveys that included items at the domain level, response options focused on perceived importance of these domains, need for training, and respondents’ confidence in performing certain skills. Inclusion of competency-specific items revealed variability in their selection, ranging from all items, across all domains, and all 3 tiers to a subset of selected or abbreviated competencies with no designated tiers.

The majority of surveys included response options with a scale ranging from 3 to 5 points. Examples of specific items and response options abstracted at the domain- and competency levels are presented in Supplemental Digital Content Appendix A (available at http://links.lww.com/JPHMP/A437).

Six (43%) RHPTC surveys assessed modality and preferences for training. Preferences included type of training course that “best fits in your schedule,” likelihood of participation in specific types of training, level of preferences for specific types of delivery methods, preferences for distance education, preferred days and times for classroom-based training, language needs, and perceptions regarding the most useful type of informal learning and perceived value of mentoring programs.

Survey items addressed similar topics, but they did so in different ways, measuring usefulness, preferences, value, and likelihood of participation. Four surveys also included a “check all that apply” option. No consistent approach for capturing the type of training preference was identified. Two surveys asked participants to identify their level of preference (eg, most preferred); 1 asked respondents to pick the best training modality based on 5 choices; 1 focused on 5 types of training and asked respondents about their likelihood to participate; 1 assessed the type of informal learning that was perceived as “most beneficial”based on a scale of 1 to 6, with six being the lowest; 1 measured the usefulness of specific training “activities” (eg, attending informal webinars); and the remaining surveys included “check all that apply” response options to assess preferred timing of training (eg, early morning), day, and modality.

Nine (64%) RPHTC surveys included items designed to assess public health topics of interest. Response options included the following: select up to 5 topics, check all that apply, select 3 content areas, list 3 useful topics, select 1 chronic condition, and rate their interest in a training program during the next 2 years based on a set of practice areas.

All RPHTC surveys included 1 or more items used to assess the following: job category or classification, length of employment, supervisory responsibility, competency-related tier, primary role, discipline of trainee, and job titles. A range of job classification approaches was used from a comprehensive list, with 185 classifications aligned with the Bureau of Labor Statistics to a list of 14 agency-specific job titles.

The 10 national membership organization surveys reviewed contained 571 items (Table 2). Surveys ranged from 21 to 123 items in length (mean = 61 items; median = 71 items). Among the 10 of these surveys, 6 used the CoL core competency framework. Three surveys based their training needs assessment using other competency-based frameworks, and 1 survey was a framework seemingly unrelated to the 10 Essential Public Health Services framework²³ but closely tied to the CoL core competency framework. These surveys also included items regarding analytical or assessment skills (129 items), leadership and systems-thinking skills (102 items), communication skills (82 items), and specialized skills or knowledge (79 items) (Table 2). Within these competency domains, the most frequently assessed skills were associated with evaluation (42 items); specialized laboratory skills (27 items); certification, safety, and compliance (25 items); program development and management (23 items); and development and maintaining of partnerships (23 items).

TABLE 2.

Competency Domains and Items Captured From Abstraction of National Member Organization Training Needs Assessments (N = 10)^a

Domain or Item	Sum of Item Count
Analytical or assessment skills	129
Evaluation	42
Research design	18
Data analysis	17
Data management	15
Surveillance	12
Advanced quantitative methods	8
Translation and dissemination	8
Needs assessments	8
All other	1
Communication skills	82
Communicating accessibly	17
Communicating persuasively	16
Specialized injury	13
All other	11
Marketing and media relations	9
Conflict management and negotiation	6
Risk communication	5
Active listening	5
Community dimensions of practice skills	44
Develop and maintain partnerships	23
Facilitation	9
All other	7
Planning	5
Cultural competency	12
General cultural competency	11
All other	1
Financial planning and management skills	27
Financial management	17
All other	10
Leadership and systems thinking skills	102
Business skills	13
Ethics	13
Leadership style	12
Professional development	9
Epidemiology	8
All other	7
HR management	7
Training assessment	6
Team management	6
Performance evaluation and improvement	6
Change management	5
Critical thinking	5
Introspection	5
Policy development or program planning skills	70
Program development and management	23
All other	17
Systems thinking	16
Policy development	14
Public health sciences skills	26
Social determinants of health	7
Public health knowledge	7
All other	6
Evidence-based practice identification	6
Specialized	79
Specialized laboratory	27
Certification, safety, and compliance	25
Specialized nursing	9
Emergency preparedness	7
All other	6
Specialized maternal and child health	5
Grand total	571

Abbreviation: HR, human resources.

^aItems with fewer than 5 counts are aggregated into “All other.”

Discussion

Determining the training needs of the public health workforce is a crucial process for prioritizing and guiding workforce development efforts and resources. RPHTCs and national member organizations are working to assess the training needs of the public health workforce and fill those gaps by facilitating relevant, accessible quality training. Each training center contributes expertise in particular skills-based training topics and subject areas. Together, they comprise the nation’s most comprehensive source of public health training and support.

The majority of RPHTCs and several member organizations have developed and used training needs assessment tools to provide skills-based training in communities across the United States. Our findings indicate that although many of the surveys measure similar overarching constructs, limited consistency exists in the method these constructs are assessed. The majority of surveys reviewed used the CoL core competency–based framework. However, despite a consensus set of skills for the broad practice of public health, substantial differences were identified in how skills are assessed across surveys. For example, some surveys assessed the importance of each competency, others assessed relevancy of skills, need for training, or confidence in applying skills, and different surveys assessed capability among 1 or more of the 3 CoL tiers of public health professionals. Our findings also indicate that although a few surveys focused solely on assessing competency domains, others assessed all competencies or a subset (often modified) within all domains. Furthermore, both the 2010 and 2014 CoL core competency–based framework versions were used, and differences were identified in the method response options were captured (eg, use of scales, rankings, or dichotomous responses).

Although the individual adaptations in the use of the CoL core competency framework might be attributable to the challenges of adequately capturing the full scope of public health practice skills, our findings indicate that opportunities for data aggregation and description of national or regional workforce training needs are scarce. Although only recently large-scale training needs assessments of the public health workforce have been conducted,¹³ these needs assessments are not being conducted on a regular basis and are not standardized. Workforce development resources are limited and implementing scalable solutions is imperative, which underscores the need for standardized processes to evaluate and measure workforce training needs across organizations.

Although each survey serves its purpose and responds to the needs of its targeted constituents, limited consistency among survey types makes transferability of the collective data generated challenging. However,these individual RPHTC and national member organization efforts could be designed to deliver a more consistent and generalizable survey that is relevant to national, regional, state, local, and discipline-specific needs. Data-based decision making is an essential element of continuous quality improvement, and standardizing the method to ask questions will help assess the efficiency and effectiveness of training needs assessments. For example, based on the experience of RPHTCs and national member organizations implementing this type of surveys, stakeholders might consider developing and disseminating a toolkit that offers a standardized set of core questions and suggestions for implementation.

Developing a standardized set of questions that provide valid, usable results requires thought, planning, logistical support, time, and stakeholders’ consensus. To ensure that set of items collect reliable and valid information, conducting psychometric testing and cognitive interviews might be a necessary step to ensure that items are interpreted and capture the information as intended. Using a standard construct for assessing training needs would allow for building a common data set and for aggregation, comparison between regions or constituents being surveyed, and improved generalizability. This standard construct could possibly serve as the basis of a standard national assessment tool. Substantial stakeholder engagement, including state and local health departments and national organizations representing these groups, is a requisite of such an approach.^10,21 Among the most substantial challenges is the need to assess a set of skills that can translate to multiple competencies or other frameworks. Although the CoL core competency framework is a common approach, our findings indicate that different organizations conceive of core training needs differently.

Proposing a Path Forward

Survey burden and fatigue are a real phenomenon in public health.⁴ Continuing to expect usable response rates if multiple, similar surveys are sent to the same groups of persons is not feasible. Furthermore, investing limited resources conducting noncomparable research on the same topic across the United States is counterproductive and needs to be recognized as a burden both to those conducting the assessments and to participants.

To move forward, we propose to harmonize and align training needs assessments efforts and reduce survey burden for public health practitioners. To be more efficient, a tripartite model could include national, regional, and discipline-specific efforts complementing each other to a better understanding of workforce training needs, replacing competitive and repetitive survey efforts. In this model, a national training needs survey could be fielded regularly (ie, approximately on a biennial basis) and results made available at the regional level for implementation of targeted approaches or trainings. These surveys would measure abilities and gauge needs across a wide field of crosscutting competencies and training needs and remain relatively consistent over time to promote comparability. They would not address the full complement of training needs, rather an agreed-upon subset based on an individual’s perceived need. Furthermore, to coordinate efforts and decrease survey fatigue, the survey could be part of a larger data collection effort currently established (eg, PH WINS). Conversely, discipline-specific training needs assessments can be conducted by targeting national membership organizations and associations. These organizations are best positioned to assess discipline-specific needs and membership desires and how to provide training to meet those needs. In our proposed model, RPHTCs would focus on addressing the gaps between assessments and training delivery on the basis of needs identified through national and discipline-specific efforts. While a study such as PH WINS can identify broad needs in the field (eg, communication, policy development, or data analysis), RPHTCs are well suited to conduct more detailed assessments based on a subset of core questions (qualitative and quantitative) aligned with the identified needs. This approach would help clarify and prioritize needs and develop training to address these concerns. A model of standardized and aligned approaches to assessing the workforce training needs will support comparability of data over time, support efficiency and effectiveness of training development and delivery, and decrease competing and redundant surveys.

Strengths and Limitations

Our study is the first to describe the different training needs assessment approaches used by RPHTCs and national member organizations and provides recommendations for developing a uniform process to evaluate and measure workforce training needs across organizations. This review included an in-depth analysis of a wide-ranging list of surveys in place to measure public health workforce training needs nationwide. This process was limited to surveys made available by RPHTCs and national membership organizations. The review did not include agency- or state-specific assessments or a review of qualitative efforts, nor did our approach evaluate the various types of needs assessment activities (eg, qualitative and quantitative) and their utility, value, or levels of participation. Finally, our review did not include an item- or domain-level analysis to determine response rates, psychometric properties, or other features of the survey design and administration among the various tools that might help guide the selection of a common set of metrics.

Implications for Policy & Practice.

• Training needs assessments of the public health workforce is crucial for proper operations management, planning, and creating relevant professional development opportunities designed to improve professional knowledge, competence, and effectiveness.

• Multiple individual approaches and surveys are currently being used to assess the training needs of the public health workforce. Although successful in responding to the needs of their targeted constituents, the limited consistency in survey types and approaches makes generalization of findings difficult.

• To facilitate quality improvement and aid in their training needs assessments and workforce development efforts, stakeholders, including state and local governmental public health, can engage in developing and disseminating a core set of survey items.

• A coordinated, consensus-based effort is essential to effectively describe and address local, state, and national public health training needs. Implementing a common set of metrics and routinely disseminating aggregate findings would likely create greater efficiency in determining the training needs of the workforce and aligning those needs with appropriate training.