Curriculum Mapping and Needs Assessment to Inform the Training of Health Policy Leaders

Abstract

Purpose

To identify synergies and gaps in knowledge, skills, and attributes identified by health policy leaders and create a summary measure of congruence with the Health Policy Leadership Fellowship Program curriculum.

Methods

We mapped the Health Policy Leadership Fellowship Program curriculum to the most highly ranked knowledge, skills, and attributes identified through the Health Policy Leaders’ Training Needs Assessment survey.

Results

Overall, the Health Policy curricular elements had the highest percentage of congruence with the needs assessment Knowledge elements (>60%). The lowest levels of congruence (<30%) occurred most frequently within the Attribute elements.

Conclusions

Mapping an existing program’s content and elements to needs perceptions from professionals practicing in the field may help to both inform and evaluate an existing program’s ability to attract and meet the needs of target learners. While needs assessments have traditionally been used to help develop programs, this study also demonstrates their application as a process evaluation tool when mapped to existing programs’ curricular elements

Background and Purpose

Curriculum Mapping and Needs Assessment Tools

Curriculum mapping and needs assessments are tools used throughout education and training processes. The goal of curriculum mapping is to give a broad overview of a specific curriculum1 as well as ensure a link between coordinated curricular content and desired program outcomes in a transparent and accessible manner.2 Curriculum mapping has been used across numerous educational settings3 including kindergarten – 12th grade education,4 higher education,5 and medical and health care training.6 Needs assessments are used to identify gaps in specific learning or performance.7 Like curriculum mapping, needs assessments have been used in a range of settings including interprofessional faculty development programs,8 community adolescent school-based health initiatives,9 and juvenile justice systems.10

The process of educational and training program development is often driven by group think, based on the planning group’s shared perceptions. This may or may not be grounded in the best available evidence. Ideally, programs should be developed based on educational evidence and the best available science. For example, Bloom’s Taxonomy of Educational Objectives, foundational work in the educational literature, delineated learning objectives by cognitive (knowledge), affective (attitudes), and motor/behavioral (skills).11 In intentional program development, program elements are built with cognitive, affective, and/or behavioral objectives linked to desired learning outcomes. Typically, in the program development process, needs assessment tools are used early in the process to identify needs and gaps. Then, the program’s desired outcomes, learning objectives, and curricular content are established. Finally, curriculum mapping tools are used after a curriculum is developed to map program elements and outcomes in a coordinated fashion.

Health Policy Leadership Fellowship Program

Heiman et al12 detailed the development of the Satcher Health Leadership Institute’s Health Policy Leadership Fellowship Program (HPLF) at Morehouse School of Medicine and highlighted the overarching program goal of preparing health leaders to take leadership roles promoting policies and programs to reduce health disparities and advance health equity. The 10-month HPLF brings cohorts of four to six multidisciplinary post-doctoral professionals together for: didactic sessions, applied learning experiences, leadership experiences, and policy experiences. Table 1 includes the HPLF curricular content highlighting the topic areas covered in the HPLF as well as the number of sessions or experiences within each topic area. The curricular topic areas and learning objectives have remained consistent from 2010 to 2018, with adjustments made to scheduling based on presenters’ availability. The practicum and policy trip experiences have varied slightly each year based on the fellows’ individual interests and each cohort’s collective interests.

Table 1. Health policy leadership training curriculum alignment: content covered within each topic area.

Health Policy (21)	Sexual Health (10)	Health Systems (14)	Leadership (16)
Health Policy Models & Overview	Sexual Health Models	Health Outcomes	Leaders & Leadership
Health Policy Process	Sexual Health Communication	Health of the Nation	Organizational Culture & Group Dynamics
Health Policy Analysis (2)	Lifespan Approach to Sexual Health	Determinants of Health	Your Leadership Style
Politics of Health	Sexual Health Values, Attitudes,& Beliefs	Public Health Overview	Communication
Health Reform & Health Equity	Sexual Health Disparities	Prevention	Organizational Structure & Strategic Planning
Public Health, Law & Civil Liberties	Maternal Child Health	Social Determinants of Health	Introduction to Sustainability
Medical Legal Partnership	LGBT Health	Health Care Organization	Government Grants
Health in All Policies	Government Recognition of Sexual Health	Health Care Finance	Program Planning & Evaluation
Health Policy Briefs	Sexual Health Policy Overview	Health Care Quality	Media Training
Policy Brief Presentations	HIV/STD	Primary Care & the Health Safety Net	SHLI Approach to Health Leadership
Health Policy Case Studies: Tobacco		New Models of Care: PCMH & ACOs	Diversity & Leadership
Health Policy Case Studies: Obesity	Health Disparities and Health Equity (9)	Health Impact Assessments	Leadership & Health Disparities
Health Policy Case Studies: HIV/STD	Health Disparities & Health Equity	Geospatial Analysis & Public Health	Informing Policy
Health Policy Case Studies: Health Equity	Health Determinants & Health Disparities	Prevention Effectiveness/ Economic Eval.	Ethical Leadership
Health Policy & the Media	Cultural Competence		Public Speaking
Health Advocacy Panel	Racism & Health Equity	Behavioral Health (8)	Sustainability
Navigating Federal Health Policy	IOM Unequal Treatment Report	Behavioral Health Overview
State Health Policy	Disabilities & Disparities	Behavioral Health Disparities	Other Components (7)
State Budget & Policy	Human Rights & Health Equity	Behavioral Health Promotion & Illness Prev.	Sexual Health Practicum
State Health Policy Implementation	Incarceration & Disparities	Behavioral Health & Primary Care Integration	Behavioral Health Practicum
Health Policy Presentations	Disparities Research Panel	Substance Use Disorders & Addiction	Conferences
		Mental Health Policy	Day at Capitol
		Mental Health Treatment Models	DC Policy Trip
		Global Mental Health	Faculty Development Program
			Mentoring

Assessment of Health Policy Leaders’ Training Needs

In 2013, we developed and administered the Health Policy Leaders’ Training Needs Assessment Tool to better understand the training needs of practicing health policy leaders.13 The tool divided items into sections on the needed knowledge, skills, and attributes for health policy leaders. Participants ranked items based on perceptions of their individual training needs as well as the training needs of health policy leaders in general. The participants who completed the tool were people who self-identified as leaders engaged in health policy work. Respondents included early, mid and later career professionals ranging in age from 18 to 65+ years old and located across the United States. More than half of the respondents had been involved in health policy for >15 years. It is possible that some of the HPLF alumni completed the tool as we recruited participants through numerous health policy networks.

Purpose

To assess our training program alignment with needs prioritized by health policy leaders, we mapped the curriculum of the fellowship program to the most highly ranked knowledge, skills, and attributes identified from a survey of health policy leaders. This innovative approach provided a formative evaluation of the fellowship program that we have not seen documented elsewhere in the education and training literature. Using this process, we hoped to identify synergies as well as gaps in knowledge, skills, and attributes identified by practitioners and leaders in health policy and create a summary measure of congruence between respondents’ feedback and the HPLF curriculum.

Methods

In this IRB-approved study, we mapped all the curricular elements in the seven main HPLF topic areas (Health Policy; Sexual Health; Health Disparities and Health Equity; Health Systems; Behavioral Health; Leadership; and Other Components [Table 1]), to the top five knowledge, skill, and attribute areas identified by participants in the Health Policy Leaders’ Training Needs Assessment.13 While the Needs Assessment included an extensive number of areas, the mapping process focused on the five most highly ranked knowledge, skill, and attribute areas. For a more in-depth explanation of the Needs Assessment rationale, methodology, and outcomes, see Understanding Health Policy Leaders’ Training Needs.13

The mapping process included several phases, similar to the process of qualitative data coding. First, members of the study team reviewed all elements of the HPLF and grouped the program content into the seven main topic areas based on the learning objectives for each program element (Table 1). Next, the study team identified the highest-ranking elements from the Health Policy Leaders’ Training Needs Assessment study to determine parameters for mapping feasibility. The top five elements in the knowledge, skill, and attribute dimensions ranked as needed for health policy leaders in general, proved feasible for the mapping process. Next, two team members independently mapped the HPLF content to the needs assessment elements. Then, two team members met three times to clarify the final mapping of the HPLF content to the needs assessment elements. Finally, two other team members reviewed the mapped data and, quantified and prioritized areas of congruence across the two data sets.

Microsoft Excel was used to compute summary congruency measures between the HPLF curricular content and the Health Policy Leaders’ Training Needs Assessment content and create visual data output. We used descriptive statistics, including frequencies and percentages to calculate congruency across data sets.

Results

The HPLF curricular content was categorized into seven main a priori topic areas by the HPLF team who designed the program. Within the seven main topic areas, Health Policy had 21 elements, Sexual Health had 10 elements, Health Disparities and Health Equity had 9 elements, Health Systems had 14 elements, Behavioral Health had 8 elements, Leadership had 16 elements, and Other Components had 7 elements (Table 1).

Knowledge Dimension

The top five knowledge elements identified by the needs assessment were: breadth/scope of health policy; dimensions of good policy; intersection of policy and politics; local/state/federal legislative processes; and policy analysis. When mapping the needs assessment knowledge elements to the HPLF curricular elements (Table 2), the highest level of overall congruence occurred within the Health Policy area (70.5%) and the lowest level of overall congruence occurred within the Leadership area (12.5%).

Table 2. Knowledge congruence.

	Knowledge dimensions (intersections within dimensions) (%)
	Breadth/scope of health policy, n (%)	Dimensions of good policy, n (%)	Intersection of policy and politics, n (%)	Local, state & federal legislative processes, n (%)	Policy analysis, n (%)	Overall congruence, (%)
Health systems	14 (100)	8 (57)	0	0	0	31.4
Health policy	21 (100)	16 (29)	20 (95)	9 (43)	8 (38)	70.5
Health disparities and health equity	9 (100)	7 (78)	4 (44)	0	0	44.4
Sexual health	9 (90)	3 (30)	4 (40)	2 (20)	3 (30)	42
Behavioral health	7 (88)	1 (13)	3 (38)	1 (13)	1 (13)	32.5
Leadership	2 (13)	2 (13)	4 (25)	0	2 (1)	12.5
Other components	5 (71)	2 (29)	5 (71)	2 (29)	2 (29)	45.7

Skills Dimension

The top five skill elements identified by the needs assessment were: engaging decision-makers; engaging stakeholders; garnering political support, effective communication; and critical thinking. When mapping the needs assessment skills elements to the HPLF curricular elements (Table 3), the highest level of overall congruence occurred within the Health Policy area (41.9%), closely followed by the Leadership area (41.3%). The Health System as well as Health Disparities and Health Equity areas had the lowest overall congruence (20%). Lower levels of congruence occurred across items in the Skills elements when compared to the Knowledge elements.

Table 3. Skills congruence.

	Skills dimensions (intersections within dimensions) (%)
	Engaging decision- makers, n (%)	Engaging stakeholders, n (%)	Garnering political support, n (%)	Effective communication skills, n (%)	Critical thinking skills, n (%)	Overall congruence, %
Health systems	0	0	0	0	14 (100)	20
Health policy	6 (29)	5 (24)	7 (33)	5 (24)	21 (100)	41.9
Health disparities and health equity	0	0	0	0	9 (100)	20
Sexual health	1 (10)	1 (10)	5	2	10 (100)	38
Behavioral health	1 (13)	1 (13)	0	0	8 (100)	25
Leadership	4 (25)	4 (25)	2 (13)	7 (44)	16 (100)	41.3
Other components	1	1	1 (14)	0	6 (88)	25.7

Attributes Dimension

The top five attribute elements identified by the needs assessment were: integrity; open mindedness; passion; perseverance; and professionalism. When mapping the needs assessment attribute elements to the HPLF curricular elements (Table 4), the highest levels of overall congruence occurred within the Leadership area (42.5%) and Other Components area (37.1%) and the lowest levels of overall congruence occurred within the Health System (0%), Behavioral Health (0%), and Health Policy (1.9%) elements.

Table 4. Attributes congruence.

	Attributes dimensions (intersections within dimensions) (%)
	Integrity, n (%)	Open mindedness, n (%)	Passion, n (%)	Perseverance, n (%)	Professionalism, n (%)	Overall Congruence, %
Health systems	0	0	0	0	0	0
Health policy	0	1 (5)	0	0	1 (5)	1.9
Health disparities and health equity	0	0	0	1 (11)	0	2.2
Sexual health	0	2 (20)	0	2 (20)	0	8
Behavioral health	0	0	0	0	0	0
Leadership	10 (63)	4 (25)	7 (44)	5 (31)	8 (50)	42.5
Other components	2	1 (14)	1 (14)	3 (43)	6 (86)	37.1

When assessed in its entirety (Figure 1), the Health Policy elements had the highest percentage of congruence with the knowledge elements (>60%). A medium level of congruence (31%-60%) occurred more frequently for Leadership, Sexual Health, and Other Component elements across the knowledge, skill, and attribute elements; while the lowest levels of congruence (<30%) occurred most frequently within the attribute elements.

Figure 1. Health policy leadership training curriculum alignment: congruence within each topic area.

Discussion

A dearth of information exists on the process of developing health policy training programs,14 as well as metrics for evaluating their objectives, processes, outcomes, and impacts. To the best of our knowledge, this study is the first to report on using curricular mapping and needs assessment tools as evaluation methods to map an academic, health policy leadership training program curriculum to needs assessment data from professionals practicing in the field. Wang et al15 used the two tools in the nonprofit management sector to help inform internal curriculum revamping. However, their process focused on a nonprofit management certificate program and the program’s students and alumni, not a comparison to professionals working in the field. It is possible that others in education, corporate and business spaces have used these tools in practice, but these findings have not been published in the academic literature.

Often in educational programs, curricula focus on knowledge content with little attention paid to developing the less tangible professional attributes of learners. Therefore, it is not surprising that the most congruence occurred between Health Policy and knowledge elements and the least amount of congruence occurred within the attribute elements. Knowledge elements are easier to teach as well as test in learners. Attribute elements require more frequency and intensity in training and are more challenging to test in learners. In addition, as reflected by Barrie,16 there are differing perceptions about incorporating attributes into training curricula and whether they can be taught.

It is also not surprising that Leadership scored relatively high in both the skills and attributes elements given the nature of the HPLF program and professional (and socio-demographic) characteristics of the participants of the needs assessment. It was surprising to see Health Disparities and Health Equity, Sexual Health, and Behavioral Health with the least congruence within the skills and attributes elements, which may be a reflection of the unique framing, values, and perspectives of the HPLF compared with those of health policy professionals practicing in the field.

When developing the HPLF program, intentional discussions occurred about including knowledge, skills, and experiences for developing health policy leaders committed to eliminating health disparities and advancing health equity. However, there was far less discussion and intentionality around working to address attributes as well as attitudes across the seven HPLF topic areas.

Limitations

The HPLF program is a 10-month intensive health policy leadership training program. To date, attention has focused on health policy and leadership content as well as knowledge and skills. Since no degree is offered, we have the flexibility to customize curricular elements to meet the needs of the fellows, which other programs may not. This customization may bias how curricular elements are categorized. In addition, the knowledge, skills, and attributes rankings are based on the perceptions of a self-selected group of health policy leaders practicing in various areas of health policy at a defined point in time. It is possible that perceptions may change over time, particularly as the health policy environment shifts. As such, while the curriculum mapping and needs assessment tools may be used as evaluation methods for other training programs, the findings from this study have limited generalizability to other health policy leadership training programs.

Conclusions

We learned several lessons in this study that may be helpful for those considering utilizing this methodology to evaluate current and future programs. First, in order to ensure methodological consistency, it is key to operationalize curricular concepts and terms. Utilizing an iterative process among team members participating in the mapping process is helpful, especially when making distinctions in the curricular cognitive (knowledge), affective (attitudes), and behavioral (skills) content. Additionally, when constructing a team for a mapping process it is helpful to have at least one participant who is not intimately familiar with the content of the program being mapped to raise questions that may be otherwise missed.

Mapping an existing program’s content and elements to needs perceptions from professionals practicing in the field may help to both inform and evaluate an existing program’s ability to attract and meet the needs of target learners. While needs assessments have traditionally been used to help develop programs, this study also demonstrates their application as a process evaluation tool when mapped to existing programs’ curricular elements. Future research on use of this methodology as a process evaluation tool in programs outside of health policy training and the HPLF program is needed.