Evaluation of the Physical Activity and Public Health Course for Researchers

Abstract

Background

Since 1995, an 8-day Physical Activity and Public Health Course for Researchers has been offered yearly in the United States.

Methods

In 2013, an evaluation quantified time that fellows spent in different course offerings, surveyed fellows on course impact, documented grant funding, and identified fellow participation on leading physical activity-related journals.

Results

The number of fellows that attended the course ranged from 20–35/year. Fellows who participated in the web survey (n=322) agreed that the course: met their expectations (99%), had a positive impact on the physical activity research or practice work they did (98%), and helped increase their professional networking in the field (93%). Following the course, 73% of fellows had further contact with course faculty and 71% had further contact with other fellows. From the National Institutes of Health, 117 grants were awarded to 82 fellows (21% of eligible fellows). Out of 14 journals reviewed, 11 had at least one fellow on their staff as editor, associate editor, or editorial board member.

Conclusion

The Physical Activity and Public Health Course for Researchers helps address a training need by providing instruction and building capacity in the US and abroad for conducting research on physical activity and public health.

Introduction

Physical activity is an important modifiable behavior that can impact multiple chronic disease risk factors (e.g., obesity, hypertension, hypercholesterolemia) and outcomes (e.g., cardiovascular diseases, type 2 diabetes, and certain cancers), yet many youth and adults do not meet current guidelines for physical activity.^1–3 Because of this, lack of physical activity is considered an important public health challenge of the 21^st century.^{4, 5}

To help better train researchers to address this problem, the Physical Activity and Public Health Course for Researchers began in 1995, through the Prevention Research Center at the University of South Carolina, in collaboration with the Centers for Disease Control and Prevention (CDC) Division of Nutrition and Physical Activity (now known as the Division of Nutrition, Physical Activity, and Obesity). Since then, the 8-day course has been offered yearly, rotating among eastern and western United States (US) venues. It is planned, implemented, and taught by national and international faculty. Researchers apply to attend the course as fellows, with acceptance based on professional credentials, experience, and potential to enhance public health research and practice. A complementary practitioner course began one year later, in 1996, and is offered simultaneously to physical activity practitioners. Additional background on the courses is available elsewhere.⁶

The goal of the research course is “to enhance the public’s health by expanding the nation’s capacity for conducting research on the health implications of physical activity and the promotion of physical activity in populations” (http://www.sph.sc.edu/paph/). The objectives of the course are to expand a fellow’s knowledge of the relationships between physical activity and chronic disease outcomes, accepted methods for measurement of physical activity, appropriate research designs for the study of physical activity in populations, current methods for promotion of physical activity in individuals and populations, and specific applications of the aforementioned knowledge areas to studies of physical activity in special populations, including ethnic minorities, women, children, and older people. The course also seeks to enhance a fellow’s ability to identify important research issues pertinent to physical activity and health, identify sources of funding to support research on physical activity and health and to design and develop research grant applications.

Each year, the researcher and practitioner courses are evaluated by the current class of fellows at the beginning and end of the course. In 2012–2013, we collected information from participants across time to evaluate the impact of the courses on building public health capacity for physical activity and on shaping the physical activity and public health careers of fellows since taking the courses. The evaluation was also conducted to help improve future course offerings. This manuscript focuses specifically on the evaluation of the research-based course.

Methods

For the overarching evaluation of the Physical Activity and Public Health Course for Researchers, we documented (i) the time that fellows spent in course offerings, (ii) fellows’ perspectives on course impact, (ii) subsequent grant funding from the National Institutes of Health (NIH) and Robert Wood Johnson Foundation Active Living Research Program (RWJF/ALR) to fellows, and (iv) fellow participation on leading journals that publish on physical activity and public health. Each of these evaluation components is described in more detail. Data collection for the study was approved by the Office of Budget and Management (OMB) #0920-0864. The procedures for all components involving human subjects were approved by the Institutional Review Board at the University of North Carolina – Chapel Hill.

Course Content

We collected course syllabi from 1995–2012, and categorized offerings by day, session, and topical category. Total course time offered was calculated with and without including overlapping sessions (i.e., when fellows could choose from different sessions offered simultaneously) for each year.

Survey of Fellows

A database on fellows was obtained from the course administrator. The database included name, contact information, and year of course attendance. The database also included course observers, defined as those allowed to audit the course who did not meet post-graduate degree entrance criteria and who may not have participated in all course activities.

A web-based survey was created and pilot tested internally. The final version contained 28 questions, including an initial question that allowed the fellow to decline or consent to the survey. The survey asked how the course helped the fellow in their career and about subsequent contact with course faculty and fellows. There were 14 statements about the course with response options of strongly agree, agree, disagree, and strongly disagree. These response options were collapsed into two levels for analysis: agree or disagree. The survey also contained descriptive questions about the fellows, including the year they attended the course, if they were a US or international fellow, current and previous work experience, membership in the National Society of Physical Activity Practitioners in Public Health (now known as the National Physical Activity Society or NPAS), length of time in the physical activity field, education level and discipline, age, gender, race, and ethnicity.

In April 2013, all research fellows and course observers that attended the course from 1995 to 2011 were invited to complete the survey using Qualtrics^™ (version 43,874, Provo, UT). We did not invite the 2012 class, since not enough time had passed since course completion. An initial email was sent through Qualtrics^™ to all fellows for whom we had an email address. The email explained the purpose of the course evaluation and included an individualized, embedded link to the survey. This individualized link allowed only one survey completion per link, preventing a fellow from completing the survey multiple times.

Fellows had the option to decline the survey and not be contacted again. If the fellow did not respond to the first email, a follow-up email was sent as a reminder. If the fellow did not respond to the second email and had a telephone number available, then a phone call was made but no voicemail was left. If the fellow could not be reached on the first phone call attempt, a second follow-up phone call was made and a voicemail was left if necessary. Following this, a final contact attempt was made by mail if a mailing address was available.

The completed questionnaire responses were exported from Qualtrics^™ to SAS (version 9.3; Cary, NC) for analysis. Open ended, textual responses were reviewed separately. Codes were created for the open ended questions, and frequency and proportion of the responses that fit the criteria for each code were reported. For the 4-level agreement items, differences by course year (always categorized as 1995–2001, 2002–2006, 2007–2011) were explored using the Pearson’s chi-squared test (significance set at p<0.05).

NIH Grant Funding

To identify and characterize NIH grants that fellows received following course attendance, in October 2012 we searched the public-use NIH Research Portfolio Online Reporting Tool: Expenditures and Results (RePORTER) for awards related to physical activity and sedentary behavior between fiscal years 1995–2012. The award type “new” was selected to exclude competing renewals, noncompeting renewals, and grant revisions. We also excluded training grants, construction grants, research and development contracts, interagency agreements, and intramural research. Only the US and territories were chosen as geographic locations. The NIH funding search strategy did not capture postdoctoral fellowships or grants in which the fellow was a principal investigator of a subcontract or a co-principal investigator but not listed as the primary contact. Thus, the search under-represented all NIH grants obtained by fellows. The search also included fellows who may not be eligible to apply for NIH funding, such as those working in governmental positions.

Exercise and sedentary medical subject heading terms were used to conduct the search, including the singular and plural forms of aerobic exercise, exercise, physical activity, physical exercise, sedentariness, sedentary, sedentary behavior, sedentary lifestyle, and sedentary time. From the NIH RePORTER search, we extracted every project by a fellow that attended the course from 1995–2011. If a fellow’s grant was missing funding information, we used the NIH Awards by Location and Organization website (http://report.nih.gov/award) and searched for the grant using the project number, fiscal year, and organization (university). We extracted funding per project for each fellow. Projects funded before a fellow attended the course were reported separately. To compare overall patterns of the search strategy for all recipients to the fellow recipients, the numbers of projects were contrasted between all recipients (including fellows) compared to only the fellows.

Active Living Research Funding

ALR is a national program of the RWJF, founded in 2001 to support and share research on environmental and policy strategies to promote daily physical activity for children and families across the US (http://activelivingresearch.org). The program administered its first grant in 2002. To determine the RWJF/ALR grants awarded to fellows, a database was obtained from the group in August 2013 and was used in tandem with the list of fellows from 1995–2011. We report total funding and average funding per project, excluding projects that were funded before the fellow attended the course. We did not include RWJF projects funded outside of their ALR program.

Journal Participation

To identify journals that publish physical activity-related research, we searched the Web of Science on September 11, 2013. Limiting the search to the past five years and to only articles or reviews, we searched using the same terms as for the NIH funding search. Meeting abstracts, proceedings papers, and editorial material were excluded. From the search, 75,947 articles and 7,091 reviews were captured. From the 83,038 citations, the Web of Science generated a list of journals by article count. From the top 100 journals that published the 83,038 citations, we selected journals that published studies on physical activity and public health. Concurrent with the search, a listing of editorial board members, associate editors, and editors-in-chief were extracted from each journal’s website and compared to the fellow list, representing current positions.

Results

Course Content

From 1995–2012, the average number of training hours was 59.3 for the 8-day course. Because several of these hours represented overlapping sessions (i.e., two sessions offered simultaneously), fellows could only spend on average 53.0 hours in instruction during the course. Grant development and funding opportunities had the highest average class time offering (10.0 hours), followed by instruction on special populations (6.3 hours), reviewing articles in a journal club format (6.0 hours), and interventions (5.8 hours) (Table 1). Public health research, health outcomes, and environment/policy topics, as well as consultations, all had averaged over 3 hours. The most noticeable change in course offerings occurred in 2003, when the total hours spent on grants and funding decreased from 12.5 hours (2002) to 5.8 hours (2003). That same year, reductions in time spent on grants as a group activity was replaced with faculty consultations. Consultations increased from 0 hours (2002) to 9.0 hours (2003). These individual consultations were used to discuss and refine research questions and methods, as well as grant development.

Table 1.

Topics covered during the Physical Activity and Public Health Course for Researchers, averaged from 1995–2012

Topic	Mean hours including overlapping sessions
Grants and funding	10.0
Special populations	6.3
Journal club	6.0
Interventions	5.8
Consultations	5.1
Public health research	4.2
Health outcomes	3.9
Environment and policy	3.4
Introduction and closing	3.0
Presentations	2.6
Measurement	2.4
Special topics	2.2
Dose response	2.0
Exercise physiology	0.9
Guidelines	0.8
Research interests	0.5
Physical inactivity and sedentary behavior	0.5

Survey of Fellows

From 1995 through 2012, 507 fellows completed the Physical Activity and Public Health Course for Researchers, ranging from 20 (in 1998) to 35 (in 2011 and 2012). These yearly totals included 22 course observers who attended in the year 2000 or later. Among 507 fellows, 78 (15%) were international, representing 16 different countries outside of the US. The countries represented most often were Australia (n=20), Canada (n=14), and United Kingdom (n=11).

Out of 507 fellows that attended the course from 1995 to 2012, n=35 were excluded from the web survey because they attended the course in 2012 (not enough time had gone by to assess impact of the course) and 2 were deceased. Among 470 eligible fellows, 322 completed the survey, 8 partially completed the survey and were excluded, and 9 declined the survey. There was no current contact information for 2 fellows and 24 were unconfirmed (defined as an unconfirmed / missing email and phone number, along with a missing address or returned letter). The response rate among fellows with a confirmed contact was 73% (calculated as (322/(470-2-24)). Fellows who attended the course in more recent years had a higher response rate than earlier years (69% 1995–2001; 61% 2002–2006, 87% 2007–2011). When expanding the unconfirmed contact definition to also include those who did not respond to the letters that were mailed and not returned, the overall response rate was 84% (calculated as (322/(470-2-85)).

The 322 survey respondents represented the full range of course years, 14% were international fellows, and 4% were course observers (Table 2). While taking the course, the predominant non-mutually exclusive work positions were described as university or college employee (72%), physical activity researcher (70%), and/or postdoctoral fellow (25%). At the time of the survey, 92% were presently employed, 78% were at a college or university, and 63% considered themselves physical activity researchers. Fellows were educated in public health including epidemiology (43%), exercise physiology and kinesiology (32%), medical (20%), and behavioral science (19%) (percentages were not mutually exclusive).

Table 2.

Descriptive characteristics of research fellows who participated in the web survey (class of 1995–2011; n=322)

	N	%	Missing
Year Attended Course (based on course records)			0
1995–2001	106	33.0
2002–2006	83	25.8
2006–2011	133	41.3
Location
US	276	85.7	0
International	46	14.3
Course Observer	14	4.3	0
Worked During Course	309	96.3	1
Work Currently	294	92.2	3
Work Position During Course (not mutually exclusive)			3
University or college employee	228	71.5
Physical activity researcher	224	70.2
Postdoctoral fellow	81	25.4
State employee	60	18.8
Federal employee	43	13.5
Physical activity practitioner	27	8.5
Did not work during course	12	4.1
Current Work Position (not mutually exclusive)			24
University or college employee	231	77.5
Physical activity researcher	187	62.8
Postdoctoral fellow	18	6.0
State employee	40	13.4
Federal employee	40	13.4
Physical activity practitioner	23	7.7
Currently does not work	25	7.8
National Physical Activity Society Member	16	5.1	5
Length in Physical Activity Field			4
<1 year	11	3.5
1 to <3 years	2	0.6
3 to <5 years	22	6.9
5 to <10 years	16	5.0
10 to <15 years	86	27.0
15 or more years	95	29.8
Education			4
Master’s Degree	10	3.1
Doctoral Degree	308	96.9
Age			9
18–29 years	4	1.3
30–39 years	108	34.5
40–49 years	4	1.3
50–59 years	107	34.2
60–69 years	69	22.0
70+ years	21	6.7
Gender			0
Female	218	67.7
Ethnicity			12
Hispanic or Latino	18	5.8
Race			14
White	271	84.2
American Indian/Alaskan Native	4	1.3
Black or African American	18	5.8
Asian	16	5.2
Native Hawaiian or Other Pacific Islander	0	0.0
Other	5	1.6
Since attending the course, have you recommended the course to others?			4
Yes	303	95.3
If there was a refresher course or sequel course, would you be interested in attending it?			4
Yes	240	75.5

Almost all (95%) fellows responding to the survey recommended the course to others and 76% would be interested in attending a refresher or sequel course. More than 90% of fellows agreed or strongly agreed that the course met their expectations, positively impacted the physical activity research or practice work they did and helped in: developing their physical activity related research questions, increasing their professional networking in the field, preparing higher quality grant proposals in physical activity research, making research or practice collaborations with other physical activity professionals, and conducting higher quality physical activity related interventions and/or programs (Table 3). When asked if the course helped increase their leadership role in the physical activity profession, agreement averaged 67% and ranged from 60%–75% over time (p=0.04). The percent agreement was higher in the early years (75% 1995–2001; 69% 2002–2006; 60% 2007–2011). When asked if the course helped them integrate a focus on health disparities into their work, 58% agreed. No other differences by course year were identified for the items listed in Table 3 (p>0.05).

Table 3.

Agreement to statements about the course by research fellows reported on the web survey (class of 1995–2011; n=322)

The Physical Activity and Public Health Course…	Agree		Disagree		Not Applicable or Missing*
	N	%	N	%	N
…met my expectations.	315	98.7	4	1.3	3
…had a positive impact on the physical activity research or practice work that I do.	306	97.8	7	2.2	9
…helped me develop my research question for physical activity research.	297	96.1	12	3.9	13
…helped me increase my professional networking in the field.	293	92.7	23	7.3	6
…helped me in preparing higher quality grant proposals in physical activity research.	256	91.4	24	8.6	42
…helped me with research or practice collaborations with other physical activity professionals.	284	91.3	27	8.7	11
…helped me conduct higher quality physical activity related interventions and/or programs.	244	89.7	28	10.3	50
…helped me in preparing higher quality manuscripts in physical activity research.	261	85.6	44	14.4	17
…helped me conduct higher quality evaluations of physical activity related interventions and/or programs.	236	85.2	41	14.8	45
…had a positive impact on other work that I do, outside of physical activity research or practice.	244	80.8	58	19.2	20
…helped me prepare physical activity-related reports or dissemination materials.	212	80.6	51	19.4	59
…helped me identify funding resources for physical activity research and interventions.	216	74.0	76	26.0	30
…helped increase my leadership role in the physical activity profession, such as by serving on advisory boards, professional organization committees, manuscript reviewer, or consultations.	200	67.3	97	32.7	25
…helped me integrate a focus on health disparities into my work.	164	58.2	118	41.8	40

Note: The response options were “strongly agree” and “agree” (collapsed into “agree”), “strongly disagree” and “disagree” (collapsed into “disagree”), and “not applicable”.

^*Three fellows did not answer this section of the survey.

Since attending the course, 73% of fellows had any professional contact with course faculty and 71% had any professional contact with other course fellows. These percentages did not vary by year (p>0.05 for both faculty and fellows). The most common reason for specific contact with faculty and fellows, based on the items queried, was to collaborate on grant applications, followed by manuscript review or collaboration (Table 4). Two specific examples of faculty contact displayed significant differences by fellow enrollment year (p<0.05 for both): no collaboration on a manuscript (45% 1995–2001; 60% 2002–2006; 65% 2007–2011) and no assistance with evaluation (67% 1995–2001; 75% 2002–2006; 88% 2007–2011). Two specific examples of fellow contact displayed significant differences by fellow enrollment year (p<0.05 for both): no assistance with an intervention or program (63% 1995–2001; 57% 2002–2006; 74% 2007–2011) and no assistance with evaluation (68% 1995–2001; 65% 2002–2006; 84% 2007–2011). No other significant differences were identified by cohort year for either faculty or fellow contact activities in Table 4 (p>0.05).

Table 4.

Research fellow responses regarding contact after the course with faculty and other fellows reported on the web survey (class of 1995–2011; n=322)

Since the course, have you had contact with any course (faculty/fellow) for professional reasons…	Faculty Contact			Other Fellow Contact
	N	%	Missing*	N	%	Missing*
…to collaborate on a grant application?			4			5
Yes, and it probably would not have happened without the course	93	29.2		95	30.0
Yes, and it probably would have happened without the course	49	15.4		27	8.5
No	176	55.3		195	61.5
…to review a manuscript?			8			5
Yes, and it probably would not have happened without the course	73	23.2		87	27.4
Yes, and it probably would have happened without the course	61	19.4		36	11.4
No	180	57.3		194	61.2
…to collaborate on a manuscript?			5			5
Yes, and it probably would not have happened without the course	68	21.5		92	29.0
Yes, and it probably would have happened without the course	61	19.2		41	12.9
No	188	59.3		184	58.0
…to assist with an intervention or program?			6			7
Yes, and it probably would not have happened without the course	68	21.5		71	22.5
Yes, and it probably would have happened without the course	32	10.1		36	11.4
No	216	68.4		208	66.0
…to assist with an evaluation?			6			7
Yes, and it probably would not have happened without the course	37	11.7		55	17.5
Yes, and it probably would have happened without the course	32	10.1		27	8.6
No	247	78.2		233	74.0

^*Four fellows did not answer this section of the survey.

In addition to the list provided in Table 4, fellows could write in other ways contact was made. For faculty contact, the most common “other experiences” included employment and professional development (n=12), involvement with trainings, presentations, conferences, or seminars (n=11), and advising and mentoring (n=8). For fellow contact, the most common “other experiences” included expert panels and committees (n=12), employment and professional development (n=11), social reasons (n=11), involvement with trainings, presentations, conferences, or seminars (n=11), and consulting (n=8).

Fellows were asked to consider their work-related physical activity accomplishments and to describe whether the course helped them with those accomplishments. Overall, 250 fellows wrote in a response and these were coded into broad non-mutually exclusive categories (Supplementary Table 1). The most common responses related to networking, grants and funding, increasing knowledge, research questions and ideas, collaborations and partnerships, and career and professional development.

Approximately one-third (35%) of fellows marked “yes” as to whether changes were “needed in the course to meet the challenges ahead for physical activity researchers and practitioners?” For those that answered affirmatively, 90 participants provided written suggestions for potential changes, which were coded into several broad categories as follows (participants comments could fall into more than one category):

• n=60 on course topics: most common responses included analysis of physical activity data, sedentary behavior, emerging technologies, innovative study designs, alternative funding sources, environment and policy, and health disparities.

• n=11 on course structure: responses included more interaction and less lecture, sequel course for people at a more advanced stage, alternative presentation formats, and more international focus.

• n=7 on faculty: responses included adding junior faculty, international faculty, and to broaden the expertise of the faculty.

• n=6 on networking: responses included providing more networking opportunities after the course and expanding the listserv to include funding information.

• n=5 on grants: responses included providing best practices for grant writing, more critiques on proposals, and practicing as a grant reviewer.

• n=5 on cross-collaboration: all responses remarked on better integration between the research and practitioner-based course.

The other responses were too few to categorize into meaningful groups.

NIH Grant Funding

The search identified 82 fellows (21% of those eligible to apply for funding), with a total of 117 funded grants with a median funding of $281,814 (Supplemental Table 2). Fifty-seven (49%) awards were Research Project grants, followed by Exploratory/Developmental grants (27%), and Small Research grants (11%). The remainder of the grants made up the final 13%. For each course year from 1995–2011, an average of 5 US fellows (range 0 to 9) received NIH funding since completing the course. The proportion of fellows that received NIH grants was higher among earlier participants (1995–2001) and less among those who participated later (2002–2011). There were 17 projects funded to 17 fellows before they attended the course, averaging 1 fellow per year with prior NIH funding.

Overall, 4,510 physical activity projects were identified with the search from fiscal years 1995 to 2012, with 3,748 of them having funding data available. For all physical activity projects, a total of $1,441,520,536 was awarded with an average of $384,611 per award (median $300,600). This average award amount for all physical activity projects was lower than the average amount received by fellows ($476,335; median $271,662). However, when omitting one large grant for $13,320,021 received by one fellow, the average award amount for fellows reduced to $365,613 (median $269,943).

The fellow projects contributed to 3% of the total projects found in the search. From fiscal years 2000–2012, half of the fiscal years (2000, 2001, 2003, 2005, 2006 and 2011) had an average award amount per project lower for fellows compared to all physical activity projects found in the search. For the other years (2002, 2004, 2007–2010, 2012), the average award amount per project was higher for fellows compared to all physical activity projects found in the search.

Of the projects led by fellows, 38 (32%) were funded by the National Cancer Institute (NCI), 25 (21%) were funded by National Heart, Lung, and Blood Institute (NHLBI), 17 (15%) were funded by National Institute of Diabetes and Digestive and Kidney Disease (NIDDK), 15 (13%) were funded by National Institute of Child Health and Human Development (NICHD), and 6 (5%) were funded by the National Institute for Nursing Research (NINR). Nine remaining institutes funded 16 (14%) fellow projects. The overarching search found that the percent of physical activity related projects funded by NCI (11%) were lower as compared to the distribution of fellow funding (32%). Also, the overarching search yielded a broader array of funding institutes based on the physical activity and sedentary behavior search terms (n=38).

Active Living Research Funding

From 2002–2012, RWJF/ALR funded 238 projects with 189 different investigators. Among these projects, 26 were funded to fellows from the 1995–2011 courses (Supplemental Table 3). In total, 21 fellows (5% or 21/400) were funded, with some fellows receiving more than one grant. On average 2 (range 0 to 4) new RWJF/ALR-funded projects were funded to fellows each year.

Journal Representation

Out of 14 journals included in the analysis, 11 had at least one fellow currently on their editorial staff (editor, associate editor, or editorial board) (Supplemental Table 4). Among the positions, 1 fellow was a journal editor, 11 were associate editors, and 30 were on editorial boards (with some fellows contributing to more than one journal). The Journal of Physical Activity and Health had the highest representation with 12 fellows (40%) involved on their 30 person team. Other journals with high representation included Research Quarterly for Exercise and Sport (14% fellows), International Journal of Behavioral Nutrition and Physical Activity (10% fellows), and Journal of Aging and Physical Activity (10% fellows).

Discussion

In the mid-1990’s, several federal and non-federal physical activity experts recognized the need to develop an intensive course to train researchers on physical activity and public health. In 1995, the Physical Activity and Public Health Course for Researchers was launched and offered yearly since then. This paper describes the overarching evaluation of the course, to examine the course impact and provide feedback to improve future course offerings. The evaluation indicated that almost all fellows reported that the course had a positive impact on their physical activity and public health research and that they valued the professional networking opportunities and connections made with faculty and other fellows which have led to successful collaborations that continue long after the training course ends. Moreover, almost all fellows responding to the web-based survey have recommended the course to other colleagues.

The goal of the research course, to expand capacity to conduct research on the promotion of physical activity in populations and on the health implications of physical activity, may be exemplified by funding that the fellows received. One major component of the course included consultations in which faculty met with fellows, in part to give opportunities to discuss grant ideas. Additionally during the course, from 1995–2012, an average of 10 hours was allocated to learn about multiple funding sources and grant writing. This was our motivation to search for NIH and RWJF/ALR funding procured by fellows. Since attending the training, 82 fellows have conservatively leveraged over $55 million from NIH to fund physical activity projects aimed at health outcomes, (e.g. bone, cancer, cardiovascular diseases, diabetes, and mental health), environment and policy, genetics, lifestyle and behavior, media and technology, overweight and obesity, and physical activity measurement. We found that the proportion of fellows that received NIH grants was higher in the earlier years and less in the later years, which is to be expected since fellows from later classes had less cumulative time to compete for NIH grant funding following course completion. The 82 funded fellows represented 21% of those eligible, indicating that most fellows may not go on to receive NIH funding. However, the true proportion is almost certainly higher than our estimate, since we included some fellows that may be ineligible to apply to NIH funding and knowing that the grant search under-counted NIH grants. The RWJF/ALR program funded 21 fellows (5%), some of whom also received NIH funding. It should be acknowledged that the evaluation of funding leaves several questions unanswered that could be addressed in future evaluations. We did not collect funding data from other potential sources, identify what years the fellows may have been eligible to submit NIH grants, and collect which funding sources specifically fellows applied for or received.

The course was conceived to be dynamic with changing times, needs, and state-of-the-art evidence. In this regard, the survey of fellows identified areas that could use greater focus, including expanding more on issues of health disparities as applied to the field. Open-ended responses described emerging course topics to include in future iterations of the course. Suggestions included a greater focus on emerging technologies to assess physical activity and use of social marketing or new media to promote physical activity. Related to the issue of staying current, approximately three-fourths of respondents agreed that they would be interested in attending a refresher or sequel course, an idea for course administrators to consider. Several fellows suggested ways to make the course more interactive, such as relying on fewer lectures, using methods consistent with active learning styles, and including more junior and international faculty. In conjunction with the research course, as mentioned in the introduction, a shorter practitioner-focused course is offered at the same time in part to enhance interaction between the two groups. Several fellows recommended more integration across the two courses, particularly in light of the importance of translating research to practice.

The web survey documented high contact between fellows and faculty following the course (73% of fellows had any professional contact with course faculty and 71% had any professional contact with other course fellows). The survey indicated continued contact on topics related to manuscripts, grants, interventions, and evaluations, as well as professional consultation, and that many of these interactions would not have happened without the course. Several time trends were identified for interactions between fellows and faculty indicating more contact from earlier cohorts. This may be explained by the fact that earlier cohorts had more time to accumulate and report interactions. Survey results showed that the percent of fellows agreeing that the course helped them increase their leadership role in the profession was lower in later years. We conjecture that the course may have been more helpful in developing leaders in the field because there were fewer of them. In more recent years, there are more fellows that have training prior to attending the course and thus may not agree that the course contributed to increasing their leadership role.

In addition to the impact of the course documented from this evaluation, other impacts have been made. For example, in 2011 a Journal of Physical Activity and Health supplement that focused on fellow research was published. The editors received 43 abstracts from the invitation, 20 were invited to submit full papers, and 17 peer-reviewed research papers were ultimately published.⁷ This course (as well as the companion practitioner course) has also contributed to other international courses on physical activity and public health that modeled their offerings on the US course. These courses were created starting in 2004 by the CDC, the International Union for Health Promotion and Education, and the Center for the Study of Physical Fitness Laboratory of Sao Caetano de Sul, in response to the global challenges of a pandemic of physical inactivity and increasing chronic diseases.⁸ The most recent estimates document 44 international physical activity courses in 20 countries including in North America, South America, Africa, Asia, the Middle East, and the Pacific.⁸ In the last 9 years, more than 3,000 professional have been trained at these courses.

Fellows attending the research course could provide feedback immediately before, during, and immediately after the course. A longer-term evaluation of new fellows could be added, such as at 1-year following course completion, to obtain their perspectives after returning to work and query if and how they applied what they learned. At this mark, fellows may also have ideas for other course content to address either in the current or a refresher courses. Periodically, longer-term evaluation could continue to examine whether the course is positively impacting the field and promoting career growth among its fellows. Several fellows requested more networking opportunities following the course, and this could contribute to a further lasting impact of the course, as well as maintain contacts for future evaluation. Moreover, the inclusion of a comparison group, such as comparable researchers who did not apply or participate in the course, would strengthen the evaluation findings.

Limitations

The evaluation of the course is subject to several other limitations. First, the web survey was retrospective in nature and relied on recall, which may have been more difficult for fellows further removed from course completion. Though they were asked about course offerings, we would not expect fellows to be aware of any changes in offerings over time. Second, the web survey response rate may have introduced selection bias if respondents differed from all potential fellows. Third, we cannot causally link course attendance to acquiring funding, journal positions, or employment. These analyses serve merely as indicators of success. It is possible that well trained and highly productive researchers gravitate to the course and would be successful despite the course. Even so, these researchers found the course a valuable continuing education experience and recommended the course to their peers. Fourth, the journal representation analysis relied on the accuracy of the journal’s website. It is not known how up-to-date the listings were, but they were all reviewed in the same month and year. This review queried only 14 journals and may exclude other journals that fellows serve on. Moreover, it represents current positions and not past positions that fellows may have had. Fifth, the NIH funding search strategy did not capture postdoctoral fellowships or grants in which the fellow was a principal investigator of a subcontract or a co-principal investigator but not listed as the primary contact. Also, the terms used to define the search may still have missed grants that included physical activity. Thus, the search under-represented all NIH grants obtained by fellows. It also included fellows that may not have been eligible to apply for certain external funding.

Conclusion

The Physical Activity and Public Health Course for Researchers began in 1995 in order to fill a gap related to a need to train researchers in two separate disciplines, physical activity and public health. At the time, health professionals were typically trained as physical activity/exercise scientists or as public health researchers. There was little understanding of common interests, scientific language, training, and opportunities to collaborate across disciplines. The course helped to build a bridge between the “physical activity” and “public health” fields and supported the emergence of what is now commonly referred to as a single field of “physical activity and public health”. The course has been sustained, largely to meet demand for training in this field. After 18 years of existence, there are typically more fellows applying to the course than the course organizers can accept. Along with the findings from this evaluation of the course impacts, the continued demand remains high which suggests that there remains a need for this this type of continuing education training course.

As others have indicated,⁹ professional development to train researchers in the field of physical activity and public health continues to be needed. The Physical Activity and Public Health Course for Researchers helps address this need by providing intense instruction over a short time period, building capacity in the US and abroad for conducting research on physical activity and public health. Some fellows from the course continued on to receive national funding for related research, obtain editorial journal positions, and followed up with both their fellow peers and course faculty after course completion. This evaluation also identified areas for improvement to meet future challenges in the field, particularly related to grant funding. Regular overarching evaluation of the course, particularly with a comparison group, is recommended to help contribute to its continued success.