The University–Public Health Partnership for Public Health Research Training in Quebec, Canada

Abstract

Enhancing effective preventive interventions to address contemporary public health problems requires improved capacity for applied public health research. A particular need has been recognized for capacity development in population health intervention research to address the complex multidisciplinary challenges of developing, implementing, and evaluating public health practices, intervention programs, and policies. Research training programs need to adapt to these new realities.

We have presented an example of a 2003 to 2015 training program in transdisciplinary research on public health interventions that embedded doctoral and postdoctoral trainees in public health organizations in Quebec, Canada.

This university–public health partnership for research training is an example of how to link science and practice to meet emerging needs in public health.

Effective public health systems require highly trained and competent professionals, a constantly updated knowledge base, and interventions adapted to contemporary challenges.^1–3 Transdisciplinary approaches, cross-sectoral partnerships, and consideration of context and complexity are key to intervention design, implementation, and evaluation.^4–6 Research capacity is needed at the intersection of science, policy, and practice in public health,^1,7,8 as exemplified by the growing movement of academic health departments in the United States that aims to improve links between academia and public health practice.^9–11

Population health intervention research has been defined as the use of scientific methods to develop knowledge about program or policy interventions operating within or outside the health sector that have the potential to affect the distribution of risk factors or health determinants in a population.¹²

Highly trained public health researchers are needed who are able to study the upstream determinants of health and the complex pathways through which various determinants influence population health.^2,8,13,14 Training must prepare future researchers to work in multidisciplinary settings and teams and to communicate effectively with stakeholders from different sectors of society.^{3,5,6,13,15–18}

We developed a novel transdisciplinary research training program through sustained university–public health partnerships. The 13-year program aimed to shift the center of gravity of training from universities to formal public health organizations (PHOs) used as research training laboratories.

PROGRAM DESCRIPTION

We created the Quebec Transdisciplinary Research Training Program in Public Health Interventions: Promotion, Prevention and Public Policy (the 4P Program) in 2003. It was one of several initiatives of the Quebec Population Health Research Network¹⁹ to build research capacity, and it aimed to train highly qualified researchers to conduct intervention research within or in close collaboration with the public health system. It was funded through 2015 by the Canadian Institutes of Health Research²⁰ and the Quebec Population Health Research Network.

Our objectives were to enable future researchers to

1. undertake research careers relevant to public health interventions;

2. conduct research in applied settings such as PHOs as well as health care, policy, and decision-making milieus;

3. adopt a broad transdisciplinary approach to research;

4. conduct studies in partnership with different interest groups;

5. understand and integrate the ethical aspects of public health research; and

6. share research results with stakeholders.

ELIGIBILITY AND SELECTION CRITERIA

Entering PhD students and postdoctoral fellows in disciplines relevant to public health and registered in a Quebec university were eligible to apply to our annual fellowship competition. To be eligible the candidate had to submit a research project that was developed in partnership with and approved by a formal PHO and the PHO had to agree to host and integrate the candidate in an existing public health team. The research had to be transdisciplinary, relate to public health interventions (programs, policies, services), be set within a population health framework, and be anchored in explicit PHO needs. A committee of 4P Program mentors reviewed submitted applications on the basis of explicit selection criteria (see the box on this page).

Selection Criteria for PhD Student or Postdoctoral Fellow Admission to the 4P Program: Quebec, Canada, 2003–2015

Relevance of the proposed research to public health interventions

Project approved by the host public health organization

Demonstrated interest of applicant for population health intervention research

Scientific quality and originality of the proposed research

Potential to affect health on a population level and to address health inequity

Quality of the academic dossier of the applicant

Quality of letters of recommendation

Commitment of the public health mentor and host public health organization to the trainee

Note. 4P Program = Promotion, Prevention and Public Policy Program.

Successful candidates received a stipend that was renewed yearly upon assessment of the trainee’s progress for a total of three years of funding for PhD students and two years for postdoctoral fellows. We defined adequate progress as satisfactory progress according to trainee university expectations, achievement of PHO placement objectives, participation in the supplemental curriculum, and peer-reviewed publications and presentations.

PROGRAM ORGANIZATION AND DELIVERY

We developed the 4P Program objectives, content, and curriculum; recruited the university and PHO mentors; selected the trainees; and implemented the program in collaboration with mentors from PHOs and occasional guest faculty members. A history of preexisting collaborations among several organizations facilitated partnerships between universities and PHOs. Formal signed contracts between the 4P Program, PHOs, and individual trainees defined the responsibilities and commitments of each in regard to the program objectives. McGill University managed the program and allocated funds for trainee bursaries. The overall budget was approximately CAN $400 000 per year and included the salary of a full-time PhD-level program director who oversaw all aspects of the 4P Program, a half-time secretarial support person, trainee bursaries (minimum CAN $21 000 for PhD students and CAN $45 000 for postdoctoral fellows), educational material and seminar organization (about CAN $30 000 per year), monthly travel to seminars and attendance at two annual public health conferences for networking (about CAN $10 000 per year per trainee), and an external evaluation consultant. Total cost per trainee was approximately CAN $80 000 per year.

Fifty-five university researchers and public health practitioners cosupervised trainees (one university mentor and one PHO mentor for each trainee, in complementary roles). Mentors came from a wide range of fields, including physicians with specialization in public health and preventive medicine (n = 24), epidemiology (n = 6), psychology (n = 6), program evaluation (n = 5), political science (n = 3), sociology (n = 3), and others (n = 8; nursing, toxicology, geography, anthropology, nutrition). Teams of mentors were often from different disciplinary backgrounds, thus facilitating the development of new, interdisciplinary partnerships (e.g., sociology and public health, economics and epidemiology). Disciplinary mixing was further accentuated by the initial undergraduate background of trainees. In all, mentors and trainees were directly associated with 37 different university departments.

The program comprised two complementary pillars. The first was the integration of trainees into a PHO for the duration of their research training under the supervision of a public health mentor. PhD students were expected to spend three days per week after their comprehensive examination in the PHO; for postdoctoral fellows, this was four days per week. The trainees were embedded into public health teams to experience frontline public health practice and to develop working relationships with practitioners and an understanding of their needs and how public health problems are conceptualized, researched, and addressed and how decisions on interventions are made and implemented in PHOs.

The second pillar was a curriculum complementary to university curricula that promoted a transdisciplinary approach to conducting population health intervention research. The curriculum comprised 16 domains (Table A, available as a supplement to the online version of this article at http://www.ajph.org) addressed in monthly daylong, in-person seminars for all trainees and mentors to discuss key aspects and emerging issues of public and population health research. University faculty, PHO practitioners, and trainees contributed to teaching and learning activities. The entire curriculum was covered over two years. This seminar series included theoretical and practical elements and promoted the acquisition of core competencies, including

1. foundational knowledge of public health,

2. transdisciplinary research in public health,

3. intervention research ethics,

4. development of research networks and partnerships,

5. knowledge translation, and

6. research career management.

Seminars were held on a rotating basis in the different PHOs to maximize trainee exposure to a variety of settings.

To ensure that our fellows acquired these competencies, we adopted a pedagogical approach on the basis of self-evaluation of needs, establishment of personal learning objectives, and formal learning contracts specifying the respective responsibilities of fellows, the 4P Program, and mentors. We monitored progress by means of a semiannual assessment and feedback. Twice a year, trainees and mentors completed a short trainee progress assessment form, which included information on the frequency of encounters between trainees and PHO mentors and comments from trainees and PHO mentors and academic supervisors on the progress toward achieving learning objectives, the satisfaction with the mentoring relationship, and suggested improvements to any aspect of the program.

PROGRAM EVALUATION

We assessed achievement of program objectives using an internal–external and mixed-methods evaluation model as well as emergent program theory²¹ (Figure A, available as a supplement to the online version of this article at http://www.ajph.org). In 2014, former trainees and mentors completed an online survey (response rates were 61% and 75%, respectively). Questions pertained to the contribution of the program to enhancing a broader vision of public health interventions, an openness to other disciplines, and an understanding of ethics, using 5-point Likert scales (1 = not at all to 5 = extremely). Former trainees also answered questions on skills for interacting and working with people from other domains, creating research partnerships, and sharing knowledge as well as on current employment status and research activities. An independent evaluator conducted qualitative telephone interviews with mentors and former trainees to assess their views of the program. (Details of the interviews and the surveys [available in French only] are available as supplements to the online version of this article at http://www.ajph.org.)

In addition, we used other data sources: (1) in-depth qualitative interviews of each trainee at the end of training, (2) self-administered questionnaires, (3) curriculum vitae data, and (4) administrative data. We applied descriptive and univariate analyses to quantitative data (the t test for pre–post score change and comparison across groups) and thematic analyses for qualitative data. We used SPSS version 20 (IBM, Somers, NY) for our statistical analyses.

PROGRAM IMPACT ON TRAINEES

We held 15 competitions for admission to the program, assessed 181 candidates, and admitted 63 trainees between 2003 and 2014. Trainees came from 31 disciplines (37% social sciences, 25% health sciences, 22% natural sciences, 8% humanities, 8% others). They were registered in seven universities and conducted research in 10 different PHOs (at local, regional, and provincial levels). Trainees’ research themes covered a wide selection of population health issues in many areas and drew on a range of approaches and methods (Table 1; trainees’ research projects are also available as supplements to the online version of this article at http://www.ajph.org). Twenty-nine percent of trainees conducted research targeting vulnerable populations and an additional 15% focused on health equity. Participatory research approaches were increasingly used in later cohorts (13% of all projects in 2003–2009 vs 42% in 2009–2014; P < .02). Nearly two thirds (64%) of trainees’ research work involved a range of individual or organizational partners outside academia, such as health professionals, community members, program managers, community organizations, municipalities, and, to a lesser extent, the private sector.

TABLE 1—

4P Program Trainee Research Projects (n = 62): Quebec, Canada, 2003–2015

Research Projects	No. (%)
Focus
Understand the social and ecological determinants of health and conditions of risks and identify targets for interventions	21 (34)
Evaluate the effectiveness and efficiency of programs, policies, and services	15 (24)
Inform the development and scaling-up of effective programs, policies, and services	10 (16)
Facilitate and improve implementation of programs, policies, and services	9 (15)
Focus on theoretical, methodological, and ethical problems in population health intervention	7 (11)
Areas
Prevention	25 (40)
Health promotion	21 (34)
Organization of health care services	12 (19)
Public policies	4 (6)
Methods
Mixed methods	26 (42)
Quantitative	24 (39)
Qualitative	12 (19)

Note. 4P Program = Promotion, Prevention and Public Policy Program. One trainee had incomplete information.

In exit interviews, trainees said that the program increased their understanding of public health and of intervention research in public health. Of the trainees, 55% attained all of their learning objectives, and the remaining 45% attained 90%. Learning objectives included research career development (80%), professional network development (73%), transdisciplinary intervention research (68%), knowledge translation (52%), and public health concepts and system knowledge (52%).

Online survey results indicate that trainees acquired a broad transdisciplinary and ethical vision of research through the program. Trainees and mentors gave mean ratings of, respectively, 3.8 (SD = 1.1) and 3.9 (SD = 0.8) out of 5 (1 = not at all to 5 = extremely) to questions about the extent to which the program seminars broadened trainees’ perspectives on public health interventions, and 3.6 (SD = 1.1) and 4.0 (SD = 0.8) for the extent to which the seminars enhanced openness to other disciplines. Trainees gave a mean rating of 3.7 (SD = 1.0) for the extent to which the seminars enhanced interaction skills with colleagues from other disciplines (mentors were not surveyed about this). The mean rating for the extent to which the seminars enhanced trainees’ understanding of ethical issues in intervention research was lower (3.3; SD = 1.1). Internships and mentoring also contributed to the development of these competencies, particularly for interacting (3.6; SD = 1.1) and working (3.4; SD = 1.1) with people from other disciplines. Internships and mentoring also favorably enhanced trainees’ skills in knowledge sharing with communities (3.2; SD = 1.2) and decision-makers (3.2; SD = 1.0).

These results are consistent with an analysis of the self-assessment questionnaires of competencies that showed increases of at least one point (P < .05; 5-point scale) between trainees’ entry and exit assessments for each category: (1) general knowledge of public health and population health intervention research, (2) research skills, and (3) knowledge translation and exchange skills. Attitudes toward applied public health work also improved (0.6; P < .05). These results converge with former trainees’ description of the knowledge and skills they reported acquiring in qualitative interviews. However, skills for conducting research in partnership with different interest groups were only moderately enhanced by participating in the program according to online survey results (mean rating of 2.2; SD = 1.5) and qualitative interviews.

Trainees published 244 (four per trainee) peer-reviewed articles, 352 abstracts (six per trainee) at national and international conferences, and 200 reports (three per trainee), including public health and policy documents directly related to the research conducted during their 4P Program training. Fifty-seven percent of all peer-reviewed articles were coauthored with at least one public health mentor. The mean number of peer-reviewed publications, as first or coauthor, increased from 1.9 (SD = 1.9; median = 2; range = 0–6) per trainee at program entry to 4.9 (SD = 3.5; median = 4.5; range = 0–12) at the end of the program (P < .05). Each trainee also engaged in 10 knowledge transfer activities on average for mixed audiences composed of researchers, practitioners, and decision-makers.

The great majority of former trainees embarked on research careers relevant to public health interventions. In 2016, 55% of all former trainees held an academic position and 10% a position within a PHO; 21% were completing their training while planning a research career in population health; 10% were involved in research management or administration.

Qualitative interviews and surveys with mentors and trainees highlighted challenges including the difficulty of adapting research questions to the priority needs of PHOs, the incomplete integration of ethics during trainees’ internship, and the limited opportunities to strengthen skills needed to develop and sustain research partnerships with interest groups outside formal PHOs. We also received few public policy applicants to the program.

CONCLUSIONS

We trained 63 young researchers over a decade who mostly continue to conduct research in partnerships with PHOs. The program created a training environment that was conducive to transdisciplinary and intersectoral research in partnership with the public health community.

The program could have been improved by

1. aligning better with the needs of partner organizations,

2. clarifying the roles of public health mentors and academic supervisors,

3. integrating intervention ethics in PHO placements,

4. developing research partnerships with different interest groups, and

5. linking with political science graduate programs.

The 4P Program experience adds to the literature on the partnership between academia and PHOs.^{10,11,22–24} It is an example of how to successfully link science and practice to meet emerging needs in public health.

We demonstrated the feasibility, and relevance, of a transdisciplinary research training program on the basis of a university–PHO partnership. The 4P Program enhanced the research skills in population health interventions and the abilities of trainees to navigate the complex interactions between research and public health practice. Key program components included the strong links between research and practice, the transdisciplinary focus of the curriculum, and recurrent opportunities for interaction and knowledge exchange with peers, experts, and practitioners from different disciplines and different areas of expertise. This type of program may prepare future researchers to become leaders capable of advancing innovative public health solutions and effecting significant changes in population-level health outcomes.

Ensuring the long-term viability of such training programs remains a major challenge to building sustainable research capacity within the public health system. Although most funding for the 4P Program ended in 2015 because the Canadian Institutes of Health Research terminated its funding of research training programs, the Quebec Population Health Research Network plans networking activities for young investigators to maintain a community of interest in applied public health research. In addition, at least one university is currently planning a transdisciplinary and applied doctoral program in public health on the basis of our experience, and the Canadian Institutes of Health Research has announced an applied training program in health services and policy research. Sustained funding and university–public health partnerships are critical for preparing the research leaders of the 21st century, increasing public health research capacity, and accelerating progress toward better population health.