Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2020 Dec 13.
Published in final edited form as: Psychol Serv. 2019 Jun 13;17(Suppl 1):93–97. doi: 10.1037/ser0000364

State Mental Health Agency Officials’ Preferences for and Sources of Behavioral Health Research

Jonathan Purtle 1,*, Félice Lê-Scherban 2, Katherine Nelson 1, Paul T Shattuck 3, Enola K Proctor 4, Ross C Brownson 5,6
PMCID: PMC6908783  NIHMSID: NIHMS1031670  PMID: 31192673

Abstract

State mental health agencies (SMHAs) are integral to public behavioral health service systems. As such, senior-level officials within SMHAs are important targets for advocacy and dissemination of behavioral health research findings. Evidence-informed decision making in SMHAs can potentially be enhanced by developing summaries of behavioral health research (e.g., policy briefs) that reflect SMHA officials’ information preferences, but knowledge about these preferences is lacking. An exploratory study was conducted with the aims of characterizing senior-level SMHA officials’ preferences for behavioral health research and describing where they turn for this research when making policy decisions. A cross-sectional, web-based survey of senior-level SMHA officials (one per state) was conducted in March-May 2017 (n= 43, response rate=84%). The features of behavioral health research that SMHA officials identified as “very important” most frequently were research being relevant to state residents (93.0%), providing data on cost-effectiveness (86.0%) and budget impact (81.4%), and being presented concisely (81.0%). The primary sources that SMHA officials turned to for behavioral research when making policy decisions were professional organizations (79.1%), SMHA agency staff (60.5%), and university researchers (55.8%). Compared with state legislators’ responses to the same survey questions, results suggest that senior-level SMHA officials and legislators have similar preferences for behavioral health research but turn to different sources for this research. Advocates and researchers who seek to promote evidence-informed decision making in SMHAs should consider developing policy briefs that are concise, provide state-level prevalence data about behavioral conditions, contain economic evaluation data, and should disseminate these materials to multiple sources.

Senior-level officials of state mental health agency (SMHAs) are an important target audience for advocacy and the dissemination of mental health and substance abuse research (hereafter referred to as behavioral health) (National Research Council, 2009; Kirmayer, Kronick, & Rousseau, 2018; National Academies of Sciences & Medicine, 2016). Each year, SMHAs fund approximately 8,500 behavioral health providers and serve an estimated 7.3 million individuals in public systems, a disproportionality large proportion of whom are low-income individuals and children (Substance Abuse and Mental Health Services Administration, 2017). SMHAs also coordinate behavioral health programs across state agencies and serve as a primary source of behavioral health content expertise for the state legislature (National Association of State Mental Health Program Directors, 2012). Given the influence of SMHA officials on public behavioral health services and systems, it is important for their decisions to be informed by research evidence. Unfortunately, use of research evidence and investment in evidence-based treatments by SMHAs has declined between 2001 and 2012 (Bruns et al., 2015). Only 29 states have a formal definition of “evidence” to inform behavioral health policy and program decisions and just 16 of these states have definitions for multiple tiers of evidence (Pew-MacArthur Results First Initiative, 2017).

Although there are numerous barriers to the use of research evidence in state agencies (e.g., organizational structures, politics) (Oliver, Innvar, Lorenc, Woodman, & Thomas, 2014), evidence-informed decision making in SMHAs can be improved by tailoring summaries of behavioral health research (e.g., policy briefs) to reflect SMHA officials’ specific information preferences (Kreuter, Farrell, Olevitch, & Brennan, 2013; Petkovic et al., 2016). However, as shown in a 2015 systematic review (Williamson, Makkar, McGrath, & Redman, 2015), little is known about the preferences for behavioral health research among SMHA officials or US state policymakers more broadly (Hogan, 2015; Purtle, Brownson, & Proctor, 2016; Purtle, Dodson, & Brownson, 2016; Purtle, Peters, & Brownson, 2016).

What is known about policymakers’ preferences for behavioral health research comes from a 2017 multi-modal survey (post-mail, e-mail, telephone) of 475 US state legislators (i.e., elected policymakers), which was limited to actual legislators and not their staff (Purtle, Dodson, Nelson, Meisel, & Brownson, 2018). The survey found that the features of behavioral health research that legislators perceived as most important were the inclusion of information about cost-effectiveness and budget impact and brief, concise materials. The survey also found that the primary sources that legislators’ turned to for behavioral health research were legislative staff, SMHAs, behavioral advocacy organizations, and legislative trade associations. Although this survey provides guidance about how to most effectively disseminate behavioral health research to state legislators, it is unclear whether the results are generalizable to administrative policymakers in SMHAs. Compared to legislators, administrative policymakers typically have more specialized knowledge in specific areas (e.g., behavioral health in the case of SMHAs) and have distinct preferences for research evidence (Bogenschneider & Corbett, 2011; Bogenschneider, Little, & Johnson, 2013; Brownson et al., 2011; Jones & Louis, 2018).

This Brief Report presents the results of an exploratory survey that sought to inform how behavioral health research can be more effectively disseminated to SMHA officials. The aim of the study was to characterize SMHA officials’ preferences for behavioral health research and describe where they turn for this research when making policy decisions. The survey of SMHA officials, in addition to the aforementioned legislator survey (Purtle et al., 2018a), was part of a larger study focused on understanding how behavioral health research can be more effectively disseminated to US state policymakers (Purtle, Lê-Scherban, Shattuck, Proctor, & Brownson, 2017).

METHOD

Data Collection

A cross-sectional, web-based survey of senior-level SMHA officials was conducted in March-May 2017. A contact database containing the name and e-mail address for the SMHA director in every state and the District of Columbia (one director per state/District) was created using information on the National Association of State Mental Health Program Directors (NASMHPD) website (https://www.nasmhpd.org/content/nasmhpd-rosters). We focused on SMHA directors because they typically have the most decision making authority within the agency. Each director was sent up to five personalized e-mails from the project’s Principal Investigator with a link to the web-based survey (created in Qualtrics). NASMHPD sent an email to SMHA directors endorsing the survey when survey recruitment began. The director of each SMHA was invited to complete the survey, but recruitment e-mails stated that another member of the SMHA’s senior-level staff was permitted to complete the survey on behalf of the agency.

The survey instrument was reviewed by NASMHPD staff and cognitively pre-tested through telephone-based interviews with six former SMHA directors (identified through NASMHPD) to ensure that the wording of questions was clear and that response options were sufficiently comprehensive (Collins, 2003). The survey was completed by the director/senior- level staff of 43 SMHAs (response rate= 84%), which is consistent with other surveys of SMHA officials (Bruns et al., 2015; Stewart, Marcus, Hadley, Hepburn, & Mandell, 2018). Institutional Review Board approval was obtained.

Measures

SMHA officials’ preferences for behavioral health research and sources of behavioral health research were measured using items adapted from Bogenschneider’s studies of US state policymakers (Bogenschneider & Corbett, 2011; Bogenschneider, Little, & Johnson, 2013). All questions were asked in specific reference to “mental health/substance abuse” (i.e., behavioral health) research. The features of behavioral health research that SMHA officials perceived as most important were measured by asking respondents to indicate “how important would it be, if at all” for “mental health/substance abuse research [they receive]” to have seven features (e.g., “is presented in a brief, concise way,” “provides data on budget impact”) on a five-point Likert scale (1= not important, 5= extremely important) (Table 1). Consistent with how these items were used with state legislators (Purtle et al., 2018a), responses were dichotomized such that ratings of 1, 2, and 3 were coded as “not very important” and ratings of 4 and 5 were coded as “very important.” The items were dichotomized to allow for the responses of SMHA officials to be compared with those of state legislators and because dichotomization provided the most concrete guidance about the features of behavioral health research that are most important to consider when disseminating evidence to SMHA officials.

Table 1.

Preferred Features of and Sources for Behavioral Health Research among Senior-level Officials in State Mental Health Agencies, 2017, N= 43

n %
Features of behavioral health research rated as “very important”a
 Relevant to state residents 40 93.0
 Provides data on cost-effectiveness 37 86.0
 Provides data on budget impact 35 81.4
 Presented in a brief, concise way 34 81.0
 Tells a story of how an issue affects residents of my state 31 73.8
 Presents implications that are politically feasible 27 64.3
 Delivered by someone I know or respect 17 40.5
Sources turned to for behavioral health researchb
 Professional organizations (e.g., NASMHPD) 34 79.1
 SMHA Agency staff 26 60.5
 University researchers 24 55.8
 SMHAs in other states 12 27.9
 Mental health/substance abuse societies (e.g., APA) 7 16.3
 Advocacy organizations (e.g., NAMI) 7 16.3
 Federal organizations (e.g., SAMHSA)c 4 9.3
 Industry (e.g., insurance or pharmaceutical companies) 1 2.3
Open in a new tab

Note. SMHA= state mental health agency, NAMI= National Alliance on Mental Illness, NASMHPD = National Association of State Mental Health Program Directors, APA= American Psychological Association, SAMHSA= Substance Abuse and Mental Health Services Administration.

a

Importance rating of 4 or 5 on 5-point Likert scale

b

Participants were asked to choose up to 3 out of 9 sources. No participants indicated that they did not know where to turn to for behavioral health research.

c

Identified through open-ended response option

The primary sources from which SMHA officials seek behavioral health research were assessed by asking respondents to indicate “who they would turn to” if they “were going to seek out mental health/substance abuse research to make a policy decision” and instructing them to select up to three sources from a list of nine options (e.g., behavioral health professional societies such as the American Psychological Association, university researchers) including an open-ended response option and an option to indicate that they did not know where to turn for behavioral health research.

At the end of the survey, information was obtained on the position of the respondent within the SMHA (i.e., director or other senior-level staff) and the number of years they have worked at the SMHA (i.e., <1 year, 2-3 years, 3-5 years, 6-9 years, ≥10 years). The modal category of years worked at the SMHA was ≥10 years and this variable was dichotomized as ≥10 (yes, no). Univariate descriptive statistics were produced and bivariate analyses (i.e., χ2 and Fisher exact tests) were conducted.

RESULTS

Forty-nine percent of surveys were completed by a SMHA director of and 51% were completed by another senior-level staff member of the SMHA (e.g., deputy commissioner, chief-of-staff). Forty-eight percent of respondents had worked at the SMHA for ≥10 years and 52% had worked at the SMHA for <10 years. Chi-square and Fisher exact tests showed no statistically significant (p< .05) differences between SMHA directors versus other senior-level staff, or between SMHA officials who had worked at the SMHA for ≥10 years versus <10 years, in any behavioral health research preference or primary source of behavioral health research.

The features of behavioral health research that SMHA officials rated as “very important” most frequently were research being relevant to state residents (93.0%), providing data on cost-effectiveness (86.0%) and budget impact (81.4%), and being presented concisely (81.0%) (Table 1). Conversely, the features that were rated as “very important” least frequently were behavioral health research having politically feasible implications (64.3%) and being delivered by someone who is known as respected (40.5%).

The entities that SMHA officials most frequently identified as their sources for behavioral research when making policy decisions were professional organizations (e.g., NASMHPD) (79.1%), SMHA agency staff (60.5%), and university researchers (55.8%). Only 27.9% of SMHA officials identified SMHAs in other states as a source. Just 16% percent of SMHA officials identified behavioral health professional societies (e.g., the American Psychological Association) and behavioral health advocacy organizations (e.g., the National Alliance on Mental Illness) as sources to which they would turn.

DISCUSSION

This study characterized the behavioral health research preferences of senior-level officials in SMHAs and described where they turn for behavioral health research when making policy decisions. In doing so, the study offers concrete guidance about how behavioral health advocates and researchers might more effectively disseminate research to SMHA officials and promote evidence-informed decision making in SMHAs. The results of our SMHA official survey are particularly instructive when compared with those of the 2017 survey of state legislators in which identical questions about research preferences were asked (Purtle et al., 2018a). To compare the preferences of these two types of state policymakers, we conducted post-hoc analyses using χ2 and Fisher exact tests to determine if there were significantly significant differences in the proportion of SMHA officials versus state legislators who identified each feature of behavioral health research as very important and each source as place they would turn to for this research.

By and large, SMHA officials and state legislators appear to have similar preferences for behavioral health research. Nearly identical proportions of SMHA officials and legislators identified data on cost-effectiveness (86.0% versus 82.0%, χ2= 1.70, p= .192), budget impact (81.4% versus 82.0%, χ2= 0.470, df= 1, p= .495), and research being presented in a brief, concise way (81.4% versus 82.0%, χ2= 0.179, df= 1, p= .672) as “very important.” This suggests that evidence use in SMHAs could be improved by adding economic evaluation components to behavioral health services and policy research and concisely communicating the results. Practical guidance for conducting economic evaluations of behavioral health interventions has been published (French, Anderson, & Bradley, 1994; Shearer & Byford, 2015; Shearer, McCrone, & Romeo, 2016) and data on SMHA spending across different program areas are readily available (Substance Abuse and Mental Health Services Administration, 2017). This information could be used to convert effect estimates from behavioral health services and policy research into state- specific budget impact estimates. The Washington State Institute for Public Policy’s Benefit-Cost Results database provides an example of how economic evaluations of behavioral health interventions can be translated into public sector cost estimates (http://www.wsipp.wa.gov/BenefitCost).

Relevance to state residents was the feature of behavioral health research that was most frequently identified as “very important” by SMHA officials (93.0%), but this feature was identified as “very important” fourth most frequently by state legislators (75.6%). Post-hoc analysis revealed that this difference was statistically significant (χ2= 9.27, df= 1, p= .002). SMHA officials’ preference for locally relevant behavioral health research is consistent with the findings that a relatively small proportion (27.9%) of SMHA officials reported turning to other states for research. A qualitative study of state child welfare agency officials’ uses of research related to psychotropic medication prescribing also identified local relevance as a very important feature of research, especially when conceptualizing the issue as a problem that needed to be addressed (Hyde, Mackie, Palinkas, Niemi, & Leslie, 2016). Advocates should consider using state-level data about the prevalence of behavioral health conditions to tailor research summaries for SMHA agency officials in different states (Brownson et al., 2011).

While SMHA officials and state legislators appear to have fairly similar preferences for behavioral health research, the sources they turn to for it are quite different. The proportion of SMHA officials reporting that they would turn to a SMHA professional association (e.g., NASMHPD) was twice that of the proportion of legislators reporting that they would turn to one of their respective legislative professional associations (e.g., National Conference of State Legislatures) (79.1% versus 38.0%, χ2= 29.39, df= 1, p <.001 in post-hoc analysis). This finding suggests that NASMHPD and similar associations are important intermediary organizations that should be targeted in the dissemination of behavioral health research.

The proportion of SMHA officials reporting that they would turn to university researchers was also substantially higher than that of state legislators (55.8% versus 27.0%, χ2=18.00, df= 1, p <.001 in post-hoc analyses). This difference could be a result of the fact that most SMHAs have a program evaluation partnership with a university (NRI, 2017) and these partnerships might foster positive relationships between SMHA officials and university researchers, while many legislators rarely interact with and do not trust researchers (Brownson, Royer, Ewing, & McBride, 2006; Gollust et al., 2017). The higher proportion of SMHA officials turning to universities could also be a reflection of the fact that SMHA officials and university researchers both have specialized knowledge about behavioral health, unlike most legislators. Thus, SMHA officials and university researchers might be able to engage in more technical dialogues about policy issues (Jones & Louis, 2018).

Limitations

The current study has a number of limitations. First, the survey was limited to senior-level SMHA officials (one recruited per state). Although these officials have the most decision-making authority within SMHAs, the survey results are not generalizable to staff at different levels within SMHAs who also make policy decisions. Second, relatedly, the survey was limited to SMHA officials and results are not generalizable to senior-level officials in other state agencies that address behavioral health issues (e.g., Medicaid, child welfare, insurance). Third, although the survey response rate was consistent with that of other surveys of SMHA officials (Bruns et al., 2015; Stewart et al., 2018) and high by standards for survey of US state policymakers (Purtle et al., 2017), the sample size was small because the total population of SMHA directors (i.e., 51) is small.

Fourth, survey questions focused on discrete information preferences for, and sources of, behavioral health research and did not assess broader constructs related to evidence-informed decision making—such as research use practices or the perceived credibility of research from differences sources (Hu et al., 2018; Purtle et al., 2016; Zardo & Collie, 2014). Fifth, the survey collected very little information about the individual characteristics of SMHA officials (e.g., highest level of education, political party affiliation, ideology) and such characteristics have been found to be associated with the behavioral health research preferences of state legislators (Purtle et al., 2018a; Purtle et al., 2018b). Future research should explore whether the individual characteristics of SMHA officials are associated with their for research information. This could potentially inform the tailoring of dissemination materials for SMHA officials with different characteristics.

CONCLUSIONS

Advocates and researchers who seek to promote evidence-informed decision making in SMHAs should consider developing dissemination materials that are concise, provide state-level data about the prevalence of behavioral conditions, contain cost-effectiveness information, and highlight state-specific estimates of budget impact. To help ensure that these materials reach senior-level SMHA officials and inform their policy decisions, materials should be disseminated to multiple sources, specifically professional organizations (e.g., NASMHPD) and SMHA staff. Future research should test the effects strategic dissemination of behavioral health research on SMHA evidence use practices.

REFERENCES

  1. Bogenschneider K, & Corbett TJ (2011). Evidence-based policymaking: Insights from policy-minded researchers and research-minded policymakers: Routledge. [Google Scholar]
  2. Bogenschneider K, Little OM, & Johnson K (2013). Policymakers' use of social science research: Looking within and across policy actors. Journal of Marriage and Family, 75(2), 263–275. [Google Scholar]
  3. Brownson RC, Dodson EA, Stamatakis KA, Casey CM, Elliott MB, Luke DA, … Kreuter MW (2011). Communicating evidence-based information on cancer prevention to state-level policy makers. Journal of the National Cancer Institute, 103(4), 306–316. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Brownson RC, Royer C, Ewing R, & McBride TD (2006). Researchers and policymakers: Travelers in parallel universes. American Journal of Preventive medicine, 30(2), 164–172. [DOI] [PubMed] [Google Scholar]
  5. Bruns EJ, Kerns SE, Pullmann MD, Hensley SW, Lutterman T, & Hoagwood KE (2015). Research, data, and evidence-based treatment use in state behavioral health systems, 2001–2012. Psychiatric Services, 67(5), 496–503. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Collins D (2003). Pretesting survey instruments: An overview of cognitive methods. Quality of life research, 12(3), 229–238. [DOI] [PubMed] [Google Scholar]
  7. French MT, Anderson DW, & Bradley CJ (1994). A conceptual framework for the economic evaluation of substance abuse interventions. Evaluation and Program Planning, 17(4), 409–418. [Google Scholar]
  8. Gollust SE, Seymour JW, Pany MJ, Goss A, Meisel ZF, & Grande D (2017). Mutual distrust: Perspectives from researchers and policy makers on the research to policy gap in 2013 and recommendations for the future. INQUIRY: The Journal of Health Care Organization, Provision, and Financing, 54, 0046958017705465. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Hogan M (2015). Using evidence to influence policy and improve practice. Psychiatric Services0, 66(8), 769–769. [DOI] [PubMed] [Google Scholar]
  10. Hu H, Allen P, Yan Y, Reis RS, Jacob RR, & Brownson RC (2018). Organizational supports for research evidence use in state public health agencies: A latent class analysis. Journal of Public Health Management and Practice. doi: 10.1097/PHH.0000000000000821. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Hyde JK, Mackie TI, Palinkas LA, Niemi E, & Leslie LK (2016). Evidence use in mental health policy making for children in foster care. Administration and Policy in Mental Health and Mental Health Services Research, 43(1), 52–66. [DOI] [PubMed] [Google Scholar]
  12. Jones DK, & Louis CJ (2018). Using evidence to inform state health policy making: Lessons from four states comparing Obamacare and infant mortality. Journal of Health Politics, Policy and Law, 43(3), 377–399. [Google Scholar]
  13. Kirmayer LJ, Kronick R, & Rousseau C (2018) Advocacy as key to structural competency in psychiatry. JAMA Psychiatry, 75(2), 119–120 [DOI] [PubMed] [Google Scholar]
  14. Kreuter MW, Farrell DW, Olevitch LR, & Brennan LK (2013). Tailoring health messages: Customizing communication with computer technology: Routledge. [Google Scholar]
  15. National Academies of Sciences & Medicine. (2016). Ending discrimination against people with mental and substance use disorders: The evidence for stigma change: National Academies Press. Washington, D.C. [PubMed] [Google Scholar]
  16. National Association of State Mental Health Program Directors. (2012). Too significant to fail: The importance of state behavioral health agencies in the daily lives of americans with mental illness, for their families, and for their communities. Accessed on June 29, 2018 at: https://www.nasmhpd.org/sites/default/files/Too%20Significant%20To%20Fail%287%29.pdf.
  17. National Research Council, (2009). Preventing mental, emotional, and behavioral disorders among young people: Progress and possibilities: National Academies Press. Washington, D.C. [PubMed] [Google Scholar]
  18. NRI. The State Profile System. (2017). Accessed on June 29, 2018 at: http://www.nri-incdata.org/.
  19. Oliver K, Innvar S, Lorenc T, Woodman J, & Thomas J (2014). A systematic review of barriers to and facilitators of the use of evidence by policymakers. BMC Health Services Research, 14(2). [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Petkovic J, Welch V, Jacob MH, Yoganathan M, Ayala AP, Cunningham H, & Tugwell P (2016). The effectiveness of evidence summaries on health policymakers and health system managers use of evidence from systematic reviews: A systematic review. Implementation Science, 11(1), 162. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Pew-MacArthur Results First Initiative. (2017). How States Engage in Evidence-Based Policymaking A national assessment. Accessed on June 29, 2018 at: http://www.pewtrusts.org/en/research-and-analysis/reports/2017/01/how-states-engage-in-evidence-based-policymaking.
  22. Purtle J, Brownson RC, & Proctor EK (2016). Infusing science into politics and policy: The importance of legislators as an audience in mental health policy dissemination research. Administration and Policy in Mental Health and Mental Health Services Research, 1–4. [DOI] [PubMed] [Google Scholar]
  23. Purtle J, Dodson EA, & Brownson RC (2016). Uses of research evidence by State legislators who prioritize behavioral health issues. Psychiatric Services, 67(12), 1355–1361. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Purtle J, Dodson EA, Brownson RC (2017). Policy dissemination research. In Brownson RC, Colditz GA, & Proctor EK (Eds.). (2017). Dissemination and implementation research in health: translating science to practice. Oxford University Press. [Google Scholar]
  25. Purtle J, Dodson EA, Nelson K, Meisel ZF, & Brownson RC (2018a). Legislators sources of behavioral health research and preferences for dissemination: Variations by political party. Psychiatric Services, appi. ps. 201800153. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Purtle J, Lê-Scherban F, Shattuck P, Proctor EK, & Brownson RC (2017). An audience research study to disseminate evidence about comprehensive state mental health parity legislation to US State policymakers: Protocol. Implementation Science, 12(1), 81. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Purtle J, Lê-Scherban F, Wang X, Shattuck PT, Proctor EK, & Brownson RC (2018b). Audience segmentation to disseminate behavioral health evidence to legislators: An empirical clustering analysis. Implementation Science, 13(1), 121. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Purtle J, Peters R, & Brownson RC (2016). A review of policy dissemination and implementation research funded by the National Institutes of Health, 2007–2014. Implementation Science, 11(1), 1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Shearer J, & Byford S (2015). The basics of economic evaluation in mental healthcare. BJPsych Advances, 21(5), 345–353. [Google Scholar]
  30. Shearer J, McCrone P, & Romeo R (2016). Economic evaluation of mental health interventions: a guide to costing approaches. Pharmacoeconomics, 34(7), 651–664. [DOI] [PubMed] [Google Scholar]
  31. Stewart RE, Marcus SC, Hadley TR, Hepburn BM, & Mandell DS (2018). State adoption of incentives to promote evidence-based practices in behavioral health systems. Psychiatric Services, 69(6), 685–688. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Substance Abuse and Mental Health Services Administration. (2017). Funding and Characteristics of Single State Agencies for Substance Abuse Services and State Mental Health Agencies, 2015. Accessed on June 29, 2018 at: https://store.samhsa.gov/shin/content/SMA17-5029/SMA17-5029.pdf.
  33. Williamson A, Makkar SR, McGrath C, & Redman S (2015). How can the use of evidence in mental health policy be increased? A systematic review. Psychiatric Services, 66(8), 783–797. [DOI] [PubMed] [Google Scholar]
  34. Zardo P, & Collie A (2014). Predicting research use in a public health policy environment: Results of a logistic regression analysis. Implementation Science, 9(1), 142. [DOI] [PMC free article] [PubMed] [Google Scholar]

RESOURCES