Advancing the Science and Practice of Dissemination and Implementation in Health: A Novel Course for Public Health Students and Academic Researchers

Dissemination and implementation (D&I) science—and the closely related fields of knowledge translation, scale-up/spread, and quality improvement^1–4—is critically important for bridging the research-to-practice gap and accelerating the movement of evidence-based practices, programs, and interventions into real-world settings to have an impact on population health. Indeed, D&I science has witnessed substantial growth during the past few decades, with increasing interest among researchers, practitioners, funding agencies, and delivery systems in health care, public health, and global health. In an effort to build capacity in this relatively nascent discipline, notable training programs have been developed (e.g., U.S. National Institute of Health's [NIH's] Training Institute for Dissemination and Implementation Research in Health,⁵ U.S. Department of Veterans Affairs [VA] Enhancing Implementation Science,⁶ NIH/VA Implementation Research Institute,⁷ and the Knowledge Translation Summer Institute⁸). Moreover, core competencies in the science and practice of knowledge translation have been proposed,⁹ and frameworks for training health professionals in D&I science have been developed.¹⁰ Former and current trainees have contributed by offering advice to those interested in pursuing research careers in D&I.^11–13

Many of these training programs, however, are focused predominantly—or even exclusively—on training the next generation of researchers; although a necessary and important goal, comparatively fewer programs focus simultaneously on building research and practice capacity among current academic researchers and future public health practitioners (e.g., students in schools of public health). Moreover, many of the programs that do exist tend to be in the form of training workshops held away from participants' home institutions and local colleagues. Although such an approach may be effective for building capacity across settings, it is less desirable for cultivating a critical mass within a single institution, which may be especially important given the interdisciplinary nature of D&I science. Such an approach may also be problematic for those who do not have flexible schedules and/or cannot devote an entire week to attend an offsite training program. Finally, programs tend to be individually focused; as such, less emphasis is placed on cultivating skills at the dyadic or team level, which requires the simultaneous engagement of both researcher and practitioner perspectives.

To complement existing training programs, and to expand the breadth and depth of available training opportunities, an innovative course to simultaneously teach public health students and academic researchers in D&I science was developed, implemented, and evaluated. This article describes the course in terms of content, format, participants, and the collaborative learning project; presents findings from the online course evaluation; and suggests viable strategies for adapting and implementing this course at other academic institutions.

METHODS

Course content

The course was designed to provide a broad overview of D&I science; key topic areas are listed in the Figure. Syllabi from related courses, which were collected by the instructor through colleague referral and requests for information, provided guidance for the course content. A draft of the course syllabus was sent to several experts for review; feedback was incorporated into the final version. Supportive materials (e.g., Microsoft^® PowerPoint presentations and lists of resources) from various training programs^5–8,10 were collected and supplemented the instructor's lectures and discussion topics, as appropriate. Chapters from the textbook Dissemination and Implementation in Health¹⁴ comprised most required readings; articles were also assigned as appropriate.

Figure.

Weekly topics in the Dissemination and Implementation in Health course, University of Alabama at Birmingham School of Public Health: fall 2012 and fall 2013 semesters

D&I = dissemination and implementation

Course format

The three-credit, graduate-level course, “D&I in Health,” was offered as an elective during the fall 2012 and fall 2013 semesters by the University of Alabama at Birmingham (UAB) School of Public Health, Department of Health Behavior. The course met in person twice weekly for 75 minutes each time. In addition to didactic lectures, classroom activities included viewing online, audio-recorded presentations (e.g., Dr. Ross Brownson, “Policy Dissemination Research: Are We Making Legislation or Sausage?;”¹⁵ and Dr. Amitabh Chandra, “The Right Way to Spend on Healthcare”¹⁶) and discussing pertinent resources (including, but not limited to, Grid-Enabled Measures Dissemination and Implementation Initiative,¹⁷ Implementation Network,¹⁸ Agency for Healthcare Research and Quality Innovations Exchange,¹⁹ NIH Conference on the Science of Dissemination and Implementation,²⁰ VA Enhancing Implementation Science conference,⁶ and American Board of Internal Medicine's Choosing Wisely^® campaign²¹).

Course participants

Course participants included (1) graduate students formally enrolled in the course and (2) academic researchers who expressed interest in learning more about D&I science and were willing to follow course procedures. Participating faculty were not formally enrolled in the course. A total of 24 students enrolled in the course: 13 students in the fall 2012 semester and 11 students in the fall 2013 semester. Students were recruited via flyers posted in the UAB School of Public Health. Eligible students included those enrolled in the master's in public health program or the doctoral Health Education/Health Behavior program; enrollment was also open to graduate students from other health-related programs pending instructor approval.

A total of 19 faculty researchers participated in the course: eight in the fall 2012 semester and 11 in the fall 2013 semester. Academic researchers were recruited via UAB-wide e-mail distribution lists and announcements through the Implementation Network (http://www.-implementationnetwork.com), an electronic newsletter that provides late-breaking information about D&I science to approximately 2,000 researchers, practitioners, and policy makers worldwide. Eligible faculty included those who e-mailed the instructor expressing interest in learning more about D&I science, completed an application form (e.g., name, position, department, institutional affiliation, and information about an evidence-based practice or program they were interested in disseminating and/or implementing), and were considered a good fit for the course by the instructor.

Collaborative learning project

The collaborative learning project mechanism—developed specifically for this course—was designed to (1) challenge students to apply D&I concepts to real-world situations; (2) create an efficient structure for academic researchers to learn more about D&I science without sitting in a classroom or attending an offsite, weeklong training; and (3) afford both students and academic researchers the opportunity to learn from one another in an “all teach, all learn”²² collaborative process consistent with principles of adult learning theory.^23–25 To accomplish these objectives, the course instructor paired each student with a participating academic researcher at the beginning of the semester (in a few instances, however, two students were paired with one academic researcher or one student was paired with two academic researchers). Students and faculty were expected to meet in person or virtually (e.g., via conference call, WebEx, or Microsoft Meeting) for approximately one hour weekly throughout the semester. During these meetings, and as part of the collaborative project, students were expected to teach their faculty partners about D&I concepts they learned in class. Academic researchers, in turn, were expected to teach their student partners about the evidence-based practice or program and associated content area that would be the focus of the project.

During the semester, the student-faculty dyads collaborated to develop research ideas for disseminating and/or implementing the evidence-based practice or program. Students summarized these plans and research ideas as part of their final course paper, and were encouraged (but not required) to share a copy of their final paper with their faculty partner. Per course guidelines, faculty were encouraged, but not required, to develop and expand these ideas further, perhaps leading to a grant application. Examples of project topic areas included, but were not limited to, telehealth to implement screening guidelines in primary care clinics, overuse of broad- vs. narrow-spectrum antibiotics among pediatric hospitalists, evidence-based oral health-care program delivered in nursing homes, cognitive-behavioral therapy for insomnia in college counseling centers, and physical activity programs for urban senior centers. Specific research ideas associated with these topic areas are not provided to maintain confidentiality and respect intellectual property rights.

Course feedback

Course feedback was obtained from participating students and academic researchers during the fall 2012 and fall 2013 semesters. After students' grades were submitted—to reduce actual or perceived conflict or coercion—the instructor sent an e-mail to students and faculty inviting them to complete a brief, online course evaluation. E-mail reminders were sent approximately one and two weeks after the initial invitation. The online survey assessed participants' demographic information and overall course evaluation; separate evaluations were used for students and faculty. Items were adapted from other course assessments^26–29 and measured using a five-point Likert scale (where 1 = strongly disagree and 5 = strongly agree). Sample items included, “Before taking this course, I had very little knowledge of D&I,” “Working on the collaborative learning project really helped me better understand D&I concepts,” “I am more confident in my ability to effectively implement practices in health settings after taking this course” (students only), and “I feel more confident in my ability to conduct D&I-related research after participating in this course” (faculty only).

Participants were characterized by their demographic information, and descriptive statistics were calculated for course evaluation items. Results were stratified by type of participant (i.e., student or faculty researcher) and collapsed across the fall 2012 and fall 2013 semesters. Positive aspects of the course (if any; open-ended response option) and common suggestions for improving the course (if any; open-ended response option) are presented in summary form.

RESULTS

Participant characteristics

Table 1 summarizes demographic characteristics of the 23 students and 18 faculty researchers who completed the online evaluation, collapsed across the fall 2012 and fall 2013 semesters. Note that all but two individuals (one student and one faculty) who participated in the course completed the evaluation. Most participating students were white (n=11), female (n=22), and primarily affiliated with the Department of Health Behavior in the UAB School of Public Health (n=21). Students were on average 27 years of age (standard deviation [SD] = 5), and most students (n=12) reported 1–5 years of public health experience (data not shown). Most participating faculty were white (n=12), half were female (n=9), and the average age was 45 years (SD=11). Faculty were represented across the spectrum of academic rank, including assistant professors (n=5), associate professors (n=3), and professors (n=5). Many held a primary appointment in a school of medicine (n=8) and most were affiliated with UAB (n=14).

Table 1.

Demographic characteristics of participating students (n=23) and faculty (n=18) in the Dissemination and Implementation in Health course, University of Alabama at Birmingham School of Public Health: fall 2012 and fall 2013 semesters

^aRespondents were able to select all that applied.

^bAssesses highest educational attainment at time of data collection (i.e., before graduation from master of public health program)

SD = standard deviation

UAB = University of Alabama at Birmingham

Course evaluation

Table 2 displays descriptive statistics for the course evaluation, stratified by student or faculty respondent and collapsed across both semesters. Among students, evaluation items with the highest average scores indicating agreement with the statement (1 = strongly disagree, 5 = strongly agree) included, “I would recommend this course to other students” (mean [M] = 4.86, SD=0.34); “I was able to apply what I learned through readings, lectures, and discussions to my D&I project” (M=4.73, SD=0.44); and “I really enjoyed taking this course” (M=4.69, SD=0.47). Among faculty, evaluation items with the highest average scores included, “I would recommend participation in this course to other faculty members” (M=4.44, SD=0.78) and “I was very interested in learning about D&I from my student(s)” (M=4.44, SD=0.78).

Table 2.

Evaluation from participating students (n=23) and faculty (n=18) in the Dissemination and Implementation in Health course, University of Alabama at Birmingham School of Public Health: fall 2012 and fall 2013 semesters

^aResponse options included 1 = strongly disagree, 2 = disagree, 3 = neither agree nor disagree, 4 = agree, 5 = strongly agree.

SD = standard deviation

D&I = dissemination and implementation

Positive comments.

Students and faculty had the opportunity to comment on the most valuable aspects of the course (if any; open-ended response option). Among students, common themes focused on the collaborative learning project, which simultaneously allowed students to work with expert faculty (e.g., “Collaborating with faculty”) and provided an opportunity to integrate research with practice (e.g., “Applying class materials to a real-life project was the best way for me to learn,” “The practical experience of working on the project,” and “Being able to apply knowledge from the classroom setting to an ongoing project was really valuable”). Among faculty, common themes centered on the collaborative learning project as an opportunity to learn more about D&I science (e.g., “Knowledge gained about D&I and the importance of it,” “Exposed to new literature,” and “Virtually taking a D&I class that I do not have access to at my institution”); work with talented graduate students (e.g., “Learning from bright, committed students who were receptive to feedback and committed to excellence”); and apply D&I concepts directly to one's own area of expertise (e.g., “Learning how to better apply D&I concepts to my ongoing project ideas,” and “Opportunity to discuss D&I principles in a context directly related to work that is important to me”) (data not shown).

Suggestions for improvement.

Students and faculty also had the opportunity to provide suggestions for improving the D&I in Health course (if any; open-ended response option). Among students, common suggestions for improvement included course logistics (e.g., provide copies of lecture slides, improve the final examination, and provide more opportunities for project feedback) and greater lag time between the beginning of the course and regular meetings for the collaborative learning project (e.g., “More time between the start of the course and when we meet with our faculty partners” and “Somehow providing for a longer period of learning the concepts before beginning projects”). Among faculty, common suggestions for improvement included an orientation meeting prior to the beginning of the course (e.g., primer on D&I science; group meeting with all participating faculty), clearer expectations for the collaborative learning project (e.g., contract between students and faculty, assignments for faculty), and the opportunity to attend lectures (either in person or videotaped) (data not shown).

DISCUSSION

We developed and evaluated a novel course to simultaneously teach public health students and academic researchers about D&I science. Participating students and faculty reported learning more about D&I science, enjoying participating in the course, and welcoming the opportunity to learn from one another via the “all teach, all learn” collaborative project. Among students, the most valuable aspects of the course included the emphasis on both research and practice and the opportunity to apply concepts to a real-world scenario through the collaborative learning project; among faculty, the most valuable aspects included the opportunity to work with students on the project and learn more about D&I science.

Future iterations may consider implementing this course in other academic institutions throughout the United States, perhaps adapting it for undergraduate (vs. graduate) students. Likewise, the course may be adapted to involve different stakeholders as partners in the collaborative learning project, such as program managers at community-based organizations, health-care professionals in private practice, and/or policy makers from local or state health departments. Similar to the University of California, San Francisco program for training health professionals in D&I,¹⁰ schools of public health may also consider developing a certificate program in D&I science, which may include D&I-related courses (e.g., Evidence-Based Public Health, Public Health Policy) and/or D&I-focused internships as an ongoing capacity-building partnership between academia and local health organizations.

Limitations

This study was subject to several limitations. To reduce social desirability bias, we collected the data anonymously rather than confidentially; however, this methodology precluded us from being able to examine responses at the student-faculty dyadic level to determine the level of agreement of ratings on the collaborative learning project. Additionally, our assessment of knowledge gained may be biased because we did not conduct a pre-post knowledge assessment of D&I science; to the best of our knowledge, however, such a measure does not exist. Future work is needed to develop, validate, and administer instruments to assess D&I knowledge gained through course participation. Finally, additional research is needed to evaluate the effectiveness of the course via long-term outcomes, such as D&I-focused grants submitted by faculty and/or D&I-related job placement among students.

CONCLUSION

This novel course was designed to expand the breadth and depth of training opportunities available in D&I science for public health students and academic researchers. Participating public health students and academic researchers learned more about D&I science, enjoyed the unique structure of the collaborative learning project, and would recommend the course to others, providing evidence to support broader adoption, adaptation (as needed), and application of this course to other academic institutions. Although additional research is needed to assess long-term outcomes, the evaluation data indicate that this course is a feasible, acceptable, and effective mechanism for simultaneously teaching public health students and academic researchers about D&I science, with the ultimate goal of accelerating research findings into delivery settings to have a greater impact on population health.