What Predicts Dissemination Efforts Among Public Health Researchers in the United States?

Abstract

Objectives

We identified factors related to dissemination efforts by researchers to non-research audiences to reduce the gap between research generation and uptake in public health practice.

Methods

We conducted a cross-sectional study of 266 researchers at universities, the National Institutes of Health (NIH), and CDC. We identified scientists using a search of public health journals and lists from government-sponsored research. The scientists completed a 35-item online survey in 2012. Using multivariable logistic regression, we compared self-rated effort to disseminate findings to non-research audiences (excellent/good vs. poor) across predictor variables in three categories: perceptions or reasons to disseminate, perceived expectation by employer/funders, and professional training and experience.

Results

One-third of researchers rated their dissemination efforts as poor. Many factors were significantly related to whether a researcher rated him/herself as excellent/good, including obligation to disseminate findings (odds ratio [OR] = 2.7, 95% confidence interval [CI] 1.1, 6.8), dissemination important for their department (OR=2.3, 95% CI 1.2, 4.5), dissemination expected by employer (OR=2.0, 95% CI 1.2, 3.2) or by funder (OR=2.1, 95% CI 1.3, 3.2), previous work in a practice/policy setting (OR=4.4, 95% CI 2.1, 9.3), and university researchers with Prevention Research Center affiliation vs. NIH researchers (OR=4.7, 95% CI 1.4, 15.7). With all variables in the model, dissemination expected by funder (OR=2.0, 95% CI 1.2, 3.1) and previous work in a practice/policy setting (OR=3.5, OR 1.7, 7.1) remained significant.

Conclusions

These findings support the need for structural changes to the system, including funding agency priorities and participation of researchers in practice- and policy-based experiences, which may enhance efforts to disseminate by researchers.

Significant investment is put toward research in public health. Ideally, the findings of this investment should improve public health practice.^1–5 Unfortunately, there is a lingering gap between research and practice.^6–10 Glasgow and Emmons identified barriers to translation of research findings into practice, such as research programs not being “packaged or manualized” for ease of implementation nor including adequate information for adapting implementation materials.⁶ The barriers have been attributed to a lack of fit between interventions and the settings in which they are implemented, as well as the information provided by the researcher and that needed by the practitioner tasked with implementation.^6,11,12 The lack of fit stems, in part, from a lack of emphasis among researchers on creating resources aimed at practitioners to ensure that the products of their research are disseminated.^6,13

Many factors may influence the extent to which researchers disseminate their findings to non-research audiences.^8,11,13–15 These factors include lack of capacity among practitioners to apply research evidence, lack of practice experience among academics, lack of priority in the published literature for effectiveness and dissemination research, and, importantly, limited funds for dissemination.¹³ Another influential factor could be a lack of priority on dissemination for individual researchers and their academic settings.¹⁴ This disconnect could also stem from broader, system-wide factors such as funding structures, which contribute to less-than-optimal dissemination.^14,16,17

Because numerous barriers to dissemination exist at multiple levels, there is also a gradient of options to enhance and encourage such efforts (e.g., designing for dissemination, training in dissemination, and implementation research) that fall along a spectrum of efforts to promote and enhance the capacity of researchers to disseminate findings.^14,17,18 The feasibility of these options, particularly regarding those at structural and setting levels, is accordingly variable. It is therefore important to identify malleable factors within the system to suggest feasible approaches to enhancing dissemination efforts. These factors are both at the individual researcher level, such as having had practice-based work experience, and at the system level, including funding opportunities. Few studies have explored factors associated with research dissemination efforts in large samples that cross settings. We aimed to identify factors related to public health researchers' self-rated excellence in dissemination of research findings to non-research audiences.

METHODS

Sampling

We conducted a PubMED search for the 12 journals with the highest impact factors in the category “public, environmental, and occupational health” as the first step of the sampling process.¹⁹ Additional details about the search as well as a list of journals searched can be found elsewhere.²⁰ The search aimed to identify a set of lead authors who could participate in the survey. Using the author's affiliation, we identified 100 intramural National Institutes of Health (NIH) investigators and 91 Centers for Disease Control and Prevention (CDC) investigators. Next, a random sample of 200 names was drawn from the 2,738 researchers not affiliated with NIH or CDC. Then, using the NIH RePORTER database (an electronic tool for searching NIH-funded research projects), we identified 57 NIH extramural grantees in dissemination and implementation research. One hundred additional names were randomly selected from among the 335 researchers affiliated with CDC's Prevention Research Centers (PRCs) Program, identified in a separate search of each PRC's website. These sources resulted in an initial pool of 548 investigators, of whom 488 were considered valid; cases of e-mail address failure, unknown duplicates, and individuals who were deceased or disabled were excluded.

Questionnaire

The survey was initially informed by a similar study conducted by Wilson and colleagues²¹ in the United Kingdom (UK) and contained 35 items. Additional details about questionnaire development can be found elsewhere.²⁰

Data collection

Surveys were completed online using Qualtrics^® survey software.²² Additional details about data collection can be found elsewhere.²⁰ The overall response rate was 54% (n=266/488). Response rates were 61% among university researchers, 41% among CDC scientists, and 38% among NIH scientists. The survey took a median of 11 minutes to complete. Data were collected from January to March 2012.

Measures and data analysis

We explored distributions for each predictor variable across the dependent variable, self-rated effort to disseminate research findings to non-research audiences (i.e., “Overall, how do you rate your efforts to disseminate your research findings to non-research audiences?”, with response options of excellent, good, adequate, poor, and not sure). Those who responded “not sure” to this question (6.4%, n=17) were excluded from the final analyses after determining their exclusion did not alter results. We explored several categories of predictor variables. First, perceptions or reasons to disseminate included items assessing whether dissemination is an obligation of researchers and the importance to their own work and that of their department to disseminate. A second category of items addressed whether researchers perceived that dissemination of their research findings was expected by their employer and by their funders. Finally, questions addressed the work, training, and experience of the researchers, including characteristics such as whether the researchers had ever worked in a practice or policy setting where their research might be applicable and at what type of institution they were employed (e.g., university, NIH). We used multivariable logistic regression to determine odds ratios for excellent/good vs. poor self-rated dissemination efforts.

RESULTS

Table 1 shows the univariate and bivariate descriptive frequencies and percentages of self-rated dissemination efforts and each predictor variable. When asked to rate their own efforts to disseminate their research findings to non-research audiences, 28% of respondents reported excellent/good dissemination, while the largest group (33%) rated its dissemination efforts as poor. The distribution of dissemination efforts among those who saw dissemination as important was roughly evenly distributed for all three variables; however, for those viewing dissemination as less important, effort was skewed farther toward adequate or poor. Approximately half (52%) of respondents felt that funders expected them to disseminate their findings, and, among these respondents, 36% rated their dissemination efforts as excellent/good; this pattern was similar for employer expectations.

Table 1.

Characteristics of public health researchers' background in dissemination and self-rated efforts in dissemination: U.S., 2012

^aPercentages may not total 100 because not all respondents answered each question.

^bNo = neither agree nor disagree/disagree/strongly disagree; yes = strongly agree/agree

^cNo = somewhat/not important; yes = very important/important

^dPopulation science: behavioral science/health education, biostatistics, economics, environmental health, epidemiology, global health, health policy, health services management, health services research, maternal and child health, nutrition, public health, and social work; medical science: psychology, medicine, and nursing; basic science: biology, chemistry, and genetics

^eResearchers at universities, not affiliated with CDC's PRC Program

^fResearchers affiliated with CDC's PRC Program

^gExamples include nonprofit organizations, think tanks, and health departments.

PRC = Prevention Research Centers

NIH = National Institutes of Health

CDC = Centers for Disease Control and Prevention

Self-rated dissemination efforts differed by work experience. Only 15% of researchers who had never previously worked in a practice or policy setting rated their efforts as excellent/good; however, 35% of researchers who had previously worked in a practice or policy setting self-rated their efforts as excellent/good. Among different workplace settings, university researchers with a PRC affiliation had the highest percentage rating their dissemination efforts as excellent/good (45%), and poor self-ratings for dissemination efforts were highest among NIH researchers (44%).

When we explored differences using multivariable regression, we found that many factors were significantly related to whether a researcher rated him/herself as excellent/good vs. poor (Table 2). Researchers who agree or strongly agree that they are obligated to disseminate their findings were nearly three times as likely as those who disagree or strongly disagree to rate their dissemination efforts as excellent/good (odds ratio [OR] = 2.7, 95% confidence interval [CI] 1.1, 6.8). Those who thought that dissemination was important or very important for their department were significantly more likely than those who thought dissemination was not important for their department to rate their dissemination efforts as excellent/good (OR=2.3, 95% CI 1.2, 4.5). Researchers reporting that dissemination of their findings was expected by their employer (OR=2.0, 95% CI 1.2, 3.2) or their funder (OR=2.1, 95% CI 1.3, 3.2) and those who have worked in a practice/policy setting (OR=4.4, 95% CI 2.1, 9.3) were significantly more likely to rate their dissemination efforts as excellent/good compared with researchers reporting no expectations. Researchers' workplace setting was also related to self-rated dissemination efforts. Compared with those at NIH, university researchers with PRC affiliations were significantly more likely to report excellent/good dissemination efforts (OR=4.7, 95% CI 1.4, 15.7).

Table 2.

Odds ratios for the relationship between self-rated excellence in dissemination and characteristics related to importance, expectation, and experience among public health researchers: U.S., 2012 (n=263)

^aNo = neither agree nor disagree/disagree/strongly disagree; yes = strongly agree/agree

^bNo = somewhat/not important; yes = very important/important

^cPopulation science: behavioral science/health education, biostatistics, economics, environmental health, epidemiology, global health, health policy, health services management, health services research, maternal and child health, nutrition, public health, and social work; medical science: psychology, medicine, and nursing; basic science: biology, chemistry, and genetics

^dResearchers at universities not affiliated with CDC's PRC Program

^eResearchers affiliated with CDC's PRC Program

^fExamples include nonprofit organizations, think tanks, and health departments.

OR = odds ratio

CI = confidence interval

Ref. = reference group

NIH = National Institutes of Health

PRC = Prevention Research Centers

CDC = Centers for Disease Control and Prevention

When all variables were included in the model, several predictors remained significant. When comparing excellent/good with poor self-rated effort, two factors were significant: whether dissemination was expected by the funder (adjusted OR [AOR] = 2.0, 95% CI 1.2, 3.1) and whether or not the respondent had ever worked in a practice setting (AOR=3.5, 95% CI 1.7, 7.1). Researchers reporting both expectation to disseminate by their funder as well as experience working in a practice or policy setting were significantly more likely to report excellent/good dissemination efforts than those with neither of these characteristics (AOR=3.2, 95% CI 1.6, 6.4) (data not shown).

DISCUSSION

Poor dissemination efforts were reported by 33% of the U.S. researchers who responded to this survey. Several factors were significantly related to dissemination efforts, including perceiving dissemination as an obligation, feeling that it is expected by the researcher's employer or his/her funder, and feeling that it is important to the work of the researcher's department. The strongest predictors of effective dissemination were having ever worked in a practice or policy setting and being a university researcher with a PRC affiliation.

These findings fit into the context of other studies that have explored researcher perspectives on -dissemination. Using a tool similar to our survey, a study of a population of researchers in the UK by Wilson et al.²¹ found that most respondents (84%) rated their current research dissemination activities as either good or adequate, but <1% rated them as excellent. We found that 60% of respondents rated their dissemination efforts as either excellent/good or adequate. In the Wilson et al. study, only 10% of their sample rated their efforts as poor, while 33% of respondents in our study rated their efforts as poor. Nedjat et al. also showed that passive strategies, such as publication of articles in peer-reviewed journals, hold a greater share of knowledge transfer activities than active strategies (e.g., targeted training of research users, materials tailored to various audiences) among researchers at the Tehran University of Medical Sciences.²³

These findings support common perceptions within the field that dissemination is difficult.^14,17,24,25 A potential cause of this difficulty is the lack of structure to facilitate dissemination of research findings. Colditz et al.¹⁴ have suggested that structural changes, similar to those needed to ensure behavior change in individuals, are needed in academia, the community, and funding agencies. Such a structure provides a model for researchers to disseminate their work. Central to this model is the role of funders. Activities such as using other available communication channels can be difficult, as such work is often unfunded.²⁶ The impact of funders can be seen in the data from respondents affiliated with the PRC program. The PRCs have a core function related to dissemination;²⁷ in our sample, PRC-based researchers reported stronger efforts than researchers in other settings.

Aside from a lack of funding to conduct such activities, funding agencies are often not structured to provide support for research dissemination. In the survey by Wilson et al., only 39% of respondents reported that funding agencies also provided advice or support, while 51% did not receive this support.²¹ When asked about the type of support and advice from funders, researchers predominantly reported receiving advice on the structure, length, content, and style of the final report and related outputs. Additional sources included press office/release support, media training, financial support for stakeholder workshops and meetings, facilitating meetings with relevant policy makers, and the production of patient/lay information leaflets. Wilson et al. concluded, “Funders should be encouraging their grant holders to adopt a more structured and theoretically informed approach to their research dissemination at the grant application stage.”²¹ In our study, we found that funder expectations were significantly related to self-rated effort to disseminate, providing further support for the potential influence funding agencies can have on the efforts researchers make in disseminating their work. This finding is particularly relevant, as funding agencies have begun to give special attention to issues related to dissemination and research translation.¹⁶

Another often-cited problem is the differing priorities for researchers, practitioners, and policy makers who may use the research results.^2,17,24,28 Although models and frameworks for dissemination exist,²⁹ it is not clear how often these models are being applied. Application of these models is likely to be enhanced via partnerships that focus on dissemination. For example, Newton et al.³⁰ queried basic and applied researchers about modes of dissemination. Among applied scientists, there was a greater focus on research that incorporates external validity (e.g., social and political factors) and the needs of research end users.^31,32 This type of output may result from collaborative partnerships between academics and non-academics. Such partnerships encourage research that transfers to practitioners and that addresses the needs of end users in the health-care system. They also recognize the importance and time required to develop working relationships with research end users, which facilitate the application of research results. Facilitation of such partnerships can also come from previous work experience in a practice or policy setting where the researchers' work might be applicable.

We found a strong association between having worked in such a practice or policy setting and self-reported efforts to disseminate findings. Prioritizing hiring researchers who have practice-based experience or encouraging time for practice experiences for researchers may impart in researchers a greater priority for dissemination as well as developing stronger university-practice partnerships. Training programs for researchers (e.g., the annual five-day Training Institute for Dissemination and Implementation Research in Health developed by NIH²⁵) provide another way to build dissemination capacity.

The bulk of translational research has focused on either researchers or practitioners. However, these groups see their roles in the dissemination and implementation of research into practice differently, and neither embraces the role as its own.³³ Researchers may not be ideal disseminators of their work, and it is likely not feasible for many practitioners to scour the literature for effective programs. An important role may exist for individuals and/or organizations that can bridge this gap and facilitate the handoff of information. As audience segmentation is an important component of social marketing, identifying the proper dissemination channel is also of significance.^17,28 Some models for dissemination include knowledge brokers,^34,35 and other efforts have incorporated strategies such as peer educators, education facilitators, and train-the-trainer programs with varying degrees of success.^7,36,37 Further, there are now online sources, such as Frontiers in Public Health Services & Systems Research, which provides free, shortened summaries of research articles targeted at practitioners. This system also links the reader to journals, where complete articles can be retrieved. Models such as these may fit better with the context in which many researchers see themselves, allowing them to focus on developing research to be included in systematic reviews and practice guidelines to be disseminated through appropriate channels.

Limitations

This study had several limitations. First, the data were self-reported, with the focus of this analysis on researchers' perception of their own success in dissemination. There are other ways to operationalize researchers' success in dissemination, such as adhering to principles of designing for dissemination (e.g., how often research summaries are produced for non-research audiences).²⁰ Researchers with high expectations for dissemination may be more aware of how they fall short of meeting such expectations; therefore, they may rate themselves lower relative to scientists with lower expectations, even if they are objectively making greater efforts at dissemination. For example, it may be that NIH researchers have higher expectations for dissemination, as they are at a government agency; therefore, they may have rated themselves more harshly. However, when we explored the importance and expectation variables by work institution, this group did not consistently rank these factors higher than researchers at other institutions.

Additionally, we did not assess reliability of the measures. The sample selected may also limit the generalizability of the findings, as part of the sample was drawn from high-impact journals; as such, we may have omitted important, practice-oriented journals with a significant focus on dissemination. Another potential source of bias in the sample stems from identification of PRC and non-PRC researchers. Researchers from PRCs were selected through an online search separately from university researchers, who were identified through NIH RePORTER and PubMed. However, both groups were sampled from comprehensive lists. Our response rate (55%) was similar to other studies surveying researchers about dissemination: 50% for Wilson et al.,²¹ 60% for Newton et al,³⁰ and 74% for Nedjat et al.²³ Further, the response rate was especially low (38%) among NIH researchers, indicating a potential for response bias. It is possible that researchers willing to complete a survey about dissemination are those more interested in the topic; therefore, this sample may have overestimated the prevalence of such efforts in the population of researchers.

CONCLUSIONS

In the context of national efforts to encourage research dissemination,³⁸ these findings support the idea that structural changes to the system, including funding agency priorities and participation of researchers in practice- and policy-based experiences, may enhance efforts to disseminate by researchers. Enhanced dissemination to non-research audiences will also require experimentation with creative new approaches (e.g., social media, online communities of practice) to better bridge the gap between discovery and application.