Developing Competencies for Training Practitioners in Evidence-Based Cancer Control

Abstract

Background

Competency-based education allows public health departments to better develop a workforce aimed at conducting evidence-based control cancer.

Methods

A two-phased competency development process was conducted that systematically obtained input from practitioners in health departments and trainers in academe and community agencies (n = 60).

Results

Among the 26 competencies developed, 10 were rated at the beginner level, 12 were intermediate, and 4 were advanced. Community-level input competencies were seen as beginner level, whereas policy-related competencies were rated as advanced.

Conclusion

While adaptation to various audiences is needed, these competencies provide a foundation on which to build practitioner-focused training programs.

INTRODUCTION

The need for a stronger commitment to evidence-based interventions in cancer control is highlighted in the US National Cancer Institute’s (NCI) Strategic Plan for Leading the Nation¹ where 7 of 8 strategic objectives includes a call for more widespread adoption of scientifically-proven interventions. Two concepts are fundamental to achieving an evidence-based approach to cancer control. First, we need scientific information on the programs and policies that are most likely to be effective in controlling cancer (i.e., evidence-based decision making).^2–4 An array of effective interventions is now available from numerous sources including the Guide to Community Preventive Services,⁵ the Guide to Clinical Preventive Services,⁶ and Cancer Control P.L.A.N.E.T..⁷ Next, dissemination of effective cancer control interventions must occur more effectively at state and local levels.⁸ State and local public health departments are in key positions to control cancer because of their ability to assess a public health problem, develop an appropriate program or policy, and assure that programs and policies are effectively delivered and implemented.^{9, 10}

Within state and local agencies, an adequately trained workforce is essential to success in cancer control.¹¹ However, there appears to be a widening gap between the skills necessary to reach cancer control goals, and the actual skill set of the public health workforce.¹² Competency-based education is rapidly becoming a norm in all levels of education in the United States,^13–15 and it applies well to training cancer control practitioners^a. Formally, a competency is defined as a cluster of related knowledge, attitudes, and skills that affects the major part of one’s job and can be measured against well-accepted standards and improved through training.¹⁶ Competency sets are used both to guide curriculum development and credentialing processes.^{15, 17–19}

In this paper, we present findings from a two-phased competency development process in which we systematically obtained input from practitioners in state and local health departments and trainers in academe and community agencies. Our overall goal was to create a set of competencies for evidence-based cancer control to guide curriculum development for practitioner-focused training efforts.

METHODS

Competency development

In phase one, an initial, general list of competencies on evidence-based decision making was assembled from numerous sources including on-going training courses in evidence-based public,^{20, 21} findings from a recent project disseminating the US Community Guide,^{22, 23} the NCI Using What Works trainings,²⁴ and competencies for training in public health.^{25, 26} We relied on several guiding principles for competency development:

• Making decisions based on the best available scientific evidence (both quantitative and qualitative research);

• Using data and information systems systematically;

• Applying behavioral science theory and program planning frameworks;

• Conducting sound evaluation;

• Engaging the community in assessment and decision making; and

• Disseminating what is learned to key stakeholders and decision-makers.

A draft list of 56 competencies was compiled. The project team evaluated the competencies in five iterative rounds of review: identifying redundancies and missing elements, and seeking comprehensiveness. General competencies were adapted to make them more specific for our cancer control topic (i.e., obesity and cancer prevention). This review process resulted in a list of 26 competencies (Table 1), which were further tested in a card sorting process.

Table 1.

Competencies in evidence-based cancer control, 2007

Title	Domaina	Levelb	Competency
1. Community input	C	B	Understand the importance of obtaining community input before planning and implementing evidence-based interventions to prevent obesity.
2. Knowledge of relationship between obesity and cancer	E	B	Understand the relationship between obesity and various forms of cancer.
3. Community assessment	C	B	Understand how to define the obesity issue according to the needs and assets of the population/community of interest.
4. Partnerships at multi-levels	P/C	B	Understand the importance of identifying and developing partnerships in order to address obesity and cancer with evidence-based strategies at multi-levels.
5. Developing a concise statement of the issue	EBP	B	Understand the importance of developing a concise statement of the obesity and cancer issue in order to build support for it.
6. Grant writing need	T/T	B	Recognize the importance of grant writing skills including the steps involved in the application process.
7. Literature searching	EBP	B	Understand the process for searching the scientific literature and summarizing search- derived information on obesity prevention.
8. Leadership and evidence	L	B	Recognize the importance of strong leadership from public health professionals regarding the need and importance of evidence-based public health interventions to prevent obesity.
9. Role of behavioral science theory	T/T	B	Understand the role of behavioral science theory in designing, implementing, and evaluating obesity-related interventions.
10. Leadership at all levels	L	B	Understand the importance of commitment from all levels of public health leadership to increase the use of evidence-based obesity-related interventions.
11. Evaluation in ‘plain English’	EV	I	Recognize the importance of translating the impacts of obesity programs or policies in language that can be understood by communities, practice sectors and policy makers.
12. Leadership and change	L	I	Recognize the importance of effective leadership from public health professionals when making decisions in the midst of ever changing environments.
13. Translating evidence-based interventions	EBP	I	Recognize the importance of translating evidence-based interventions to prevent obesity to unique ‘real world’ settings.
14. Quantifying the issue	T/T	I	Understand the importance of descriptive epidemiology (concepts of person, place, time) in quantifying obesity and other public health issues.
15. Developing an action plan for program or policy	EBP	I	Understand the importance of developing an obesity plan of action which describes how the goals and objectives will be achieved, what resources are required, and how responsibility of achieving objectives will be assigned.
16. Prioritizing health issues	EBP	I	Understand how to choose and implement appropriate criteria and processes for prioritizing program and policy options for obesity prevention.
17. Qualitative evaluation	EV	I	Recognize the value of qualitative evaluation approaches including the steps involved in conducting qualitative evaluations.
18. Collaborative partnerships	P/C	I	Understand the importance of collaborative partnerships between researchers and practitioners when designing, implementing, and evaluating evidence-based programs and policies for obesity prevention.
19. Non-traditional partnerships	P/C	I	Understand the importance of traditional obesity prevention partnerships as well as those that have been considered non-traditional such as those with planners, department of transportation, and others.
20. Systematic reviews	T/T	I	Understand the rationale, uses, and usefulness of systematic reviews that document effective obesity interventions.
21. Quantitative evaluation	EV	I	Recognize the importance of quantitative evaluation approaches including the concepts of measurement validity and reliability.
22. Grant writing skills	T/T	I	Demonstrate the ability to create an obesity-related grant including an outline of the steps involved in the application process.
23. Role of economic evaluation	T/T	A	Recognize the importance of using economic data and strategies to evaluate costs and outcomes when making public health decisions related to obesity and cancer prevention.
24. Creating policy briefs	P	A	Understand the importance of writing concise policy briefs to address obesity and cancer using evidence-based interventions.
25. Evaluation designs	EV	A	Comprehend the various designs useful in obesity program evaluation with a particular focus on quasi-experimental (non-randomized) designs.
26. Transmitting evidence- based research to policy makers	P	A	Understand the importance of coming up with creative ways of transmitting what we know works (evidence-based interventions for obesity prevention) to policy makers in order to gain interest, political support and funding.

^aC = community-level planning; E = etiology; P/C = partnerships & collaboration; EBP = evidence-based process; T/T = theory & analytic tools; L = leadership; EV = evaluation; P = policy.

^bB = beginner; I = intermediate; A = advanced.

Card sorting

In the second phase of the project, we administered a card sorting exercise among practitioners and trainers in cancer control. Card sorting is a technique that explores how people group and prioritize items,^27–29 allowing us to develop a curriculum focusing on key competencies.

Participants for card sorting were drawn from multiple sources: practitioners were selected from two mid-sized state health departments and a county health department; trainers were identified from an ongoing evidence-based public health course,^{20, 21, 30} the partnership program of NCI’s Cancer Information Service,³¹ and the CDC- and NCI-funded Cancer Prevention and Control Research Network.³² Within each group, a list of possible participants was enumerated based on experience in the delivery of cancer and other chronic disease programs and expertise in training of practitioners.

There were four steps in the card sorting process. We first defined a competency as “a complex combination of knowledge, skills and abilities; demonstrated by individuals and necessary for them to perform their job functions at a high level.” Each competency was typed on a separate card. We then set out a scenario in which the respondent read each competency card and decided if the knowledge/attitude/skill should be designated as ‘Beginner’ (basic), or ‘Advanced’ (higher level) training (i.e., level of difficulty). Each person ended up with a ‘Beginner’ and an ‘Advanced’ stack of cards. Within each of the two stacks, respondents were then asked to categorize each competency as a low, medium, or high priority (i.e., perceived priority). We also provided blank cards so respondents could write out any competencies that they deemed missing from our list. Card sorting was conducted from July through November 2007 in both a group and individual format to accommodate people’s schedules. Group card sorting (n = 32) took 45–50 minutes per group and individual card sorting (n = 28) took 20–55 minutes per respondent. After several rounds of recruitment, 100% cooperation was obtained from our target audience (n = 60).

Analyses

Descriptive analyses were conducted to summarize all variables. Bivariate relationships were examined using chi-square or ANOVA, depending on the type of data. Summary data were displayed graphically in bar graphs to compare: beginner versus advanced categories, low versus medium versus high priorities, and ratings by trainers versus trainees. For each proportion, a 95% confidence interval was calculated.³³ When the percentage beginner versus advanced was statistically the same as 50%, the competency was designated as intermediate. Scatter plots were created using the two scores for each competency: the percentage advanced score and the percentage high priority score. For the scatter plots, lines were drawn at the 33.3% and 66.7% advanced score (X axis) to indicate beginner, intermediate and advanced categories. Lines were also drawn at the 33.3% and 66.7% high priority score (Y axis) to indicate low, medium and high priority.

RESULTS

Participants in the card sorting included a range of job titles, with most respondents serving as program managers/administrators or health educators (Table 2). Respondents were equally distributed across three intervals for agency longevity. Most participants had graduate education, with 68% holding a masters degree in public health or another discipline. Among respondents, 63% were potential trainees and 37% had experience as trainers. Eighty-five percent of respondents worked in obesity and/or cancer prevention. The groups of trainees and trainers differed on two characteristics: job title and highest degree held.

Table 2.

Participants in card sorting for evidence-based cancer control, 2007

Characteristic	All respondents n (%)	Trainees n (%)	Trainers n (%)	p-valuea
Job Title
Program manager/administrator	26 (43.3)	17 (44.7)	9 (40.9)	0.028
Health educator	16 (26.7)	14 (36.8)	2 (9.1)
Epidemiologist	2 (3.3)	0 (0)	2 (9.1)
Multiple job titlesb	8 (13.3)	4 (10.5)	4 (18.2)
Other job titlesc	8 (13.3)	3 (7.9)	5 (22.7)
Years working in agency/organization
<5 years	19 (32.2)	10 (27.0)	9 (40.9)	0.626
5 to <10 years	19 (32.2)	13 (35.1)	6 (27.3)
10+ years	21 (35.6)	14 (37.8)	7 (31.8)
Highest degree held
Bachelors	11 (18.3)	8 (21.0)	3 (13.6)	0.002
Masters (other than public health)	30 (50)	23 (60.5)	7 (31.8)
Masters (public health)	11 (18.3)	5 (13.2)	6 (27.3)
Doctorate	6 (10)	0 (0)	6 (27.3)
Other	2 (3.4)	2 (5.3)	0 (0)
Role in training
Trainee	38 (63.3)	--	--	--
Trainer	22 (36.7)
Working in the field of either obesity or cancer?
Yes	51 (85)	31 (81.6)	20 (90.9)	0.281
No	9 (15)	7 (18.4)	2 (9.1)

^ap-value for chi-square testing difference between trainees and trainers

^bMultiple job titles included: program manager/administrator and program planner; program manager/administrator and division/bureau head; program manager/administrator and health educator; health educator and local project coordinator.

^cOther job titles included: program coordinator; project coordinator; outreach program manager; academic project director; partnership program planner.

Among the competencies, 10 were rated at the beginner level, 12 were intermediate, and 4 were advanced (Figure 1). There was considerable variation in level of difficulty ratings across competency. For example, the competency on community input (#1) was rated as beginner by 93% of respondents whereas the competency on transmitting evidence-based research to policy makers (#26) was rated as beginner by only 14% of respondents. The 26 competencies covered 8 domains (Table 1). Several patterns emerged across domain and difficulty level. Community-level input competencies were seen as beginner level. Competencies in evaluation and evidence-based processes were likely to be rated as intermediate. The two policy-related competencies were rated as advanced.

Figure 1.

Rating of competencies by level, 2007

Bivariate relationships were examined between the variables in Table 2 and ratings for level of difficulty (beginner, intermediate, advanced) and perceived priority (low, medium, high). There were few significant differences across the categories in Table 1. For difficulty level ratings, those with less than 5 years experience rated competencies as more advanced (p = 0.022). Those who worked in obesity or cancer tended to rate competencies at a more advanced level (p = 0.62).

The data also varied by level of difficulty (beginner, intermediate, advanced) and perceived priority (low, medium, high) (Figure 2). Four competencies (circled in Figure 2) had difficulty levels rated significantly differently by trainers versus trainees; these were: translating evidence-based interventions (#13), qualitative evaluation (#17), non-traditional partnerships (#19), and systematic reviews (#20). Among these four competencies, only non-traditional partnerships was more likely to be rated as an advanced skill among trainers than among trainees whereas the other three were deemed more advanced among trainees.

Figure 2.

Competencies according to ratings for level and priority, 2007

Circled: competencies are those with widest discrepancy between trainers and trainees

1: Community input

2: Knowledge of relationship between obesity and cancer

3: Community assessment

4: Partnerships at multi-levels

5: Developing a concise statement of the issue

6: Grant writing need

7: Literature searching

8: Leadership and evidence

9: Role of behavioral science theory

10: Leadership at all levels

11: Evaluation in ‘plain English’

12: Leadership and change

13: Translating evidence-based interventions

14: Quantifying the issue

15: Developing an action plan for program or policy

16: Prioritizing health issues

17: Qualitative evaluation

18: Collaborative partnerships

19: Non-traditional partnerships

20: Systematic reviews

21: Quantitative evaluation

22: Grant writing skills

23: Role of economic evaluation

24: Creating policy briefs

25: Evaluation designs

26: Transmitting evidence-based research to policy makers

Using open-ended methods, a few additional competencies were identified from two or more respondents. These involved two concepts: 1) the need to explain the importance of evidence-based approaches and related definitions and 2) the importance of building trust and respect when conducting community-based interventions. Several related issues were identified that were related more to implementation of curriculum than to the content (e.g., the importance of understanding and using principles of adult learning).

DISCUSSION

As cancer control has become a key component of day-to-day public health practice over the past two decades,^{11, 34–36} the need for practitioners knowledgeable in evidence-based approaches has grown.³ Our study identified and prioritized a set of 26 competencies that are likely to be important in improving the delivery of cancer control interventions. By including ratings for level of difficulty (beginner to advanced) and perceived priority (low to high), this competency set provides a foundation for practitioner training programs.

Both researchers^{35, 37} and practitioners^{11, 38} have identified cancer control training as a high priority. Yet, implementing competency-based training requires focused and sustained efforts. Weed and Husten³⁷ summarized the training-related issues according to two questions: “Who should be trained?” and “What should be learned?”. The answer to the “who” question for a public health agency is likely to include persons from key disciplines (e.g., epidemiology, health promotion, public information, administration) who are involved in delivering cancer control programs or who encourage others to be trained. Most people working in cancer control within public health settings do not have formal training in public health.^{30, 39} The “what” question can be largely answered by the competencies that we identified and rated. A third question, “how” is an additional consideration that includes attention to mode of delivery (e.g., in-person, web-based trainings).

There are several implications from our study that should be taken into account in development and delivery of training:

• In conducting trainings that span beginning to advanced levels, it is helpful to target approaches to appropriate objectives to the level of difficulty. Beginning competencies may be largely cognitive; those at the intermediate level may apply existing tools and build basic skills; advanced competencies may teach in-depth skills that seek to make a person an expert in a certain area.

• For most competencies, there was close agreement between trainers and trainees for ratings on level of difficulty and perceived priority. When there was disagreement between trainers and trainees, trainees were more likely to rate a competency as advanced. Trainers should not assume that what seems “basic” to them is also “basic” to trainees. A skill assessment before delivery of training should help in targeting curricula to the correct level.

• It is increasingly clear that policy intervention holds great promise for cancer control.^{40, 41} From our study, policy-related competencies were the most advanced and may therefore require focused attention to adequately train the workforce. It may be important to identify some “basic” policy skills that in turn help a person build up to the more advanced policy competencies. This may call on training programs to move beyond typical public health training to include skills such advocacy, policy analysis, health communication, and negotiation.

• While both competencies for community-level planning (nos. 1 and 3) were deemed as beginner level, they also were considered highest priority. This suggests that these skills are essential for moving to intermediate and advanced levels.

Although not directly addressed in our study, implementation of training to address these competencies should take into account principles of adult learning. These issues were recently articulated by Bryan et al.⁴² and include the need: 1) to know why the audience is learning; 2) to tap into an underlying motivation to learn by the need to solve problems; 3) to respect and build upon previous experience; 4) for learning approaches that match with background and diversity; and 5) to actively involve the audience in the learning process.

While our study provides useful information, it also has limitations. Our findings are based on self-reported data and were obtained from a convenience sample of trainees and trainers. It is possible that a larger, more geographically-dispersed and professionally-diverse sample would yield different results. For example, training programs may need to be developed specifically for minority researchers and practitioners to adequately address health disparities.^{43, 44} In addition to individuals employed in state and local health departments, other agencies such as the American Cancer Society and the NCI’s Cancer Information Service play important roles in cancer control. It will be important to track the implementation of training programs in multiple venues. The scenario in our study focused on obesity and cancer. This approach should be validated in a range of cancer control (e.g., screening) and other public health topics (e.g., diabetes, infectious diseases). It is likely that the level of maturity for a public health topic will influence competency needs.

There are several logical next steps for efforts to build the workforce of cancer control practitioners. To better understand the “how” behind these competencies, qualitative, in-depth interviews with potential trainees should help in defining the optimal training approach, mode of delivery, and venue. Numerous training programs^{3, 24} and analytic tools^{5, 7, 45–47} for evidence-based public health practice are already available. It would be helpful to map our competencies against what is already available. A general gap in the literature is the lack of published evaluations on training programs.³⁰ More consistent and systematic evaluation should be conducted as new programs are developed.

In summary, our development process identified a manageable set of cancer control competencies, rated by level of difficulty and perceived priority. While adaptation to various audiences is needed, this group of competencies provides a foundation on which to build practitioner-focused training programs.