Social Workers’ Orientation Toward the Evidence-Based Practice Process

Abstract

Objectives:

This study assesses social workers’ orientation toward the evidence-based practice (EBP) process and explores which specific variables (e.g. age) are associated.

Methods:

Data were collected from 341 Dutch social workers through an online survey which included a Dutch translation of the EBP Process Assessment Scale (EBPPAS), along with 13 background/demographic questions.

Results:

The overall level of orientation toward the EBP process is relatively low. Although respondents are slightly familiar with it and have slightly positive attitudes about it, their intentions to engage in it and their actual engagement are relatively low. Respondents who followed a course on the EBP process as a student are more oriented toward it than those who did not. Social workers under 29 are more familiar with the EBP process than those over 29.

Conclusions:

We recommend educators to take a more active role in teaching the EBP process to students and social workers.

Since its introduction into the social work profession by Gambrill (1999) and Macdonald (1998), evidence-based practice (EBP) has become increasingly influential in social work, particularly in English-speaking countries such as the United Kingdom, the United States, Canada, and Australia. Also, in many northern European countries, including the Netherlands, social workers are now increasingly being urged by policy makers to engage in EBP. For the last decade or so, the Dutch government, local authorities, and funding bodies are demanding more accountability and effectiveness in social work, which has led to increasing attention for EBP as a means of professionalization in social work (Steyaert, Van Den Biggelaar, & Peels, 2010). In 2008, the Dutch Ministry of Health, Welfare, and Sport commissioned the 5-year project effective social interventions to improve the use of research knowledge in social work. This national project was launched by Movisie (the Netherlands Centre for Social Development) to encourage and facilitate the use of research knowledge in social work practice by making available at no cost information about the effectiveness of frequently used social interventions in an online database. Also in 2008, the Dutch Ministry of Education, Culture and Science decided to fund a new master of social work (MSW) program for two universities of applied sciences which aims to create new professionals who focus on the effectiveness of interventions and accountability of the profession (HBO-raad/Vereniging Hogescholen, 2006; Van Pelt, 2011).

Despite these initiatives of policy makers and educators to improve and facilitate EBP in social work, the extent of social work practitioner acceptance of and engagement in EBP is still cause for skepticism, based on the criticisms of social workers and scholars regarding the feasibility and usefulness of EBP in social work. For example, the Nederlandse Vereniging van Maatschappelijk Werkers ([NVMW], National Association of Social Workers) in its professional profile mentions that there is very little opportunity in the Netherlands to choose interventions with scientific evidence of their effectiveness, because a lot is still to be done in this field (NVMW, 2011). A similar conclusion was reached by Dutch researchers who had worked on a project starting with the intention of applying EBP in a study of the interventions used by community workers. They concluded that it seemed unlikely that social work in the Netherlands was ready for ‘a strict method of accountability like EBP’ (Potting, Sniekers, Lamers, & Reverda, 2010).

In a previous literature study, we found that one of the major issues in the acceptance and the implementation of EBP in the Netherlands is the suspicious attitude (or sometimes even aversion) of social workers themselves (Van der Zwet, Beneken genaamd Kolmer, & Schalk, 2011). Another barrier we found was the inconsistent way EBP is used in Dutch literature. Descriptions are often not clear or transparent, which possibly leads to misperceptions of EBP (and possibly to misplaced criticism or distrust regarding EBP) and obstructs its adoption by social workers and organizations. Also a lack of research skills appeared to be an important barrier to the implementation of EBP in social work.

Recognizing that the EBP process can be successfully implemented in social work only if social workers believe it is both important and feasible, the current study assesses social workers’ views about the EBP process and implementation of the EBP process (Parrish & Rubin, 2012). It focuses on the extent to which Dutch social workers are familiar with the EBP process and feel capable of engaging in the EBP process, accept the EBP process, view that process as feasible, intend to engage in the EBP process, and actually engage in that process. Social workers’ views about the EBP process and use of the EBP process have never been investigated in the Netherlands. The present study adds to the existing literature by assessing Dutch social workers’ orientation toward the EBP process. Furthermore, this study explores whether specific variables (level of education, age, prior courses in EBP as a student and prior continuing education in EBP as a practitioner) are associated with the level of orientation toward the EBP process. Identifying such variables can be helpful in suggesting ways to improve practitioner acceptance and implementation of the EBP process (Rubin & Parrish, 2010). The rationale for investigating the differences in orientation toward the EBP process between the levels of education is the notion that social workers with a master’s degree are perhaps more oriented toward the EBP process than those with only a bachelor’s degree, as they are likely to have had (more) courses about research methods and EBP. The rationale for investigating the differences in orientation toward the EBP process between the different age-groups is the notion that younger social workers, as they are more likely to have had courses in EBP as a student, are perhaps more oriented toward the EBP process than older social workers who were educated in the pre-EBP era. The notion that social workers with prior education in EBP are perhaps more oriented toward the EBP process than those without was the rationale for investigating this association. In the following two sections, we will discuss which definition of EBP was adopted in this study and provide more information on the context of social work in the Netherlands.

Definition of EBP

This study does not focus on “evidence-based practices”; instead, it focuses on the EBP process. Social work literature often fails to distinguish between EBP as a decision-making process and EBPs. Descriptions of EBP in social work literature differ greatly, ranging from the broad definition as envisioned by its originators to narrow, fragmented views (Gambrill, 2011; Gray, Joy, Plath, & Webb, 2013). Recently, several authors have therefore argued that it is important to distinguish the singular term, the EBP “process,” from the plural term, evidence-based practices (or more correctly, empirically supported treatments or empirically supported interventions [ESIs]; Parrish & Rubin, 2012; Thyer & Myers, 2011). The latter refers to interventions for which there is consistent scientific evidence showing that they improve client outcomes (Drake et al., 2001). In contrast, the EBP process has been defined by its founders as a process that involves “the integration of best research evidence with clinical expertise and patient values” (Sackett, Straus, Richardson, Rosenberg, & Haynes, 2000, p. 1). This process involves the following five steps: (1) formulating an answerable practice question; (2) searching for the best research evidence; (3) critically appraising the research evidence; (4) selecting the best intervention after integrating the research evidence with clinical expertise and client characteristics, preferences, and values; and (5) evaluating practice decisions (Straus, Richardson, Glasziou, & Haynes, 2005). The EBP process acknowledges the importance of both clinical expertise and client characteristics/values, along with the consideration of the best available evidence when making practice decisions (Rubin & Parrish, 2011).

Social Work in the Netherlands

In the Netherlands, social workers are professionals who are active in social and community work in a broad sense. Professionals employed in social welfare and social services organizations offer community work, social work, youth work, debt counselling, welfare assistance, shelter for the homeless, social work with the elderly, day care, and support for refugees and asylum seekers. The National Association of Social Workers (NVMW) has a professional register, a professional code, and disciplinary rules; however, social workers are not required to register. Furthermore, as the Dutch government is cutting down on the funding of the social welfare and social services organizations, which are dependent on this funding, organizations are confronted with reorganizations, reductions, and budget cuts. Because of this, the flexible shell of young professionals decreases and the workforce in organizations is aging.

Before 2009, social work education in the Netherlands had only Intermediate Vocational Education and Higher Vocational Education (bachelor-level programs). Twelve of the 14 bachelor-level programs have specific educational programs for Maatschappelijk Werk en Dienstverlening (Social Work), Sociaal Pedagogische Hulpverlening (Social Educational Care Work), and Culturele en Maatschappelijke vorming (Cultural and Social Development) and two of them have broad bachelor programs. The MSW program was developed to respond to the need for a level of education and experience that exceeded the bachelor level (HBO-raad/Vereniging Hogescholen, 2006). The goal of the MSW program is to improve the quality of professional practice (social practice development) through the professionalization of senior social workers (with at least 3 years of experience in practice; Van Pelt, 2011). Now, there are four universities of applied sciences in the Netherlands, offering an MSW program. As MSW programs are relatively new in the Netherlands, there are still only a few social workers with an MSW degree.

Method

Sample

A convenience sample was obtained (N = 341), consisting of social workers employed in 22 social work organizations in various parts of the Netherlands. In 2012, a total of 61,500 professionals were working in social welfare and social services organizations in the Netherlands. We used two different strategies to approach respondents in order to try to maximize the response rate.

Our main strategy to find respondents was through the MOgroep (national sector association for social welfare and social services) as approximately 80% (577 organizations) of the social welfare and social services organizations are affiliated. Approximately 20% of the Dutch social work organizations are not members of the MOgroep, there is, therefore, some bias associated with this recruitment strategy. As this is the largest professional body representing social work organizations in the Netherlands, the MOgroep did, however, provide access to a large number of social work organizations and offered the most appropriate and accessible source of participants. The MOgroep announced the study by sending an e-mail to its social work organization members to invite CEOs to sign up their organizations for the study. A second strategy we used to get respondents was through Verdiwel (professional association of CEOs of social welfare and social services organizations). According to Verdiwel’s Policy Officer K. Neefjes (personal communication, March 13, 2014), approximately 50% of the national social welfare and social services turnover are represented by organizations affiliated to Verdiwel. Furthermore, most of the organizations that are affiliated are relatively large and most of them are early adopters. As smaller organizations and laggards are mostly not affiliated, there is some bias associated with this recruitment strategy. Verdiwel sent an e-mail to its members (70 CEOs) to inform them about the study and to invite them to participate.

This resulted in 22 organizations agreeing to participate in the study (of which 16 were recruited via the MOgroep). CEOs were given three options for participation in order to encourage them and to maximize the response rate. Thirteen of the organizations provided the researchers with a list of e-mail addresses after which the researchers sent the 630 social workers an invitation including a personal link to the online survey. Due to privacy reasons, six organizations were not willing to provide a list of e-mail addresses. In those cases, the organizations themselves sent an invitation directly to the social workers (338), which included an open link to the online survey provided by the researchers. To be able to monitor the response rate, the organizations informed the researchers about the number of social workers they had invited. Three CEOs did not want to burden all of their social workers because they were under time pressure already. Instead the CEOs selected a few social workers and provided their e-mail addresses to the researchers. One organization randomly selected the participants (15) and the other participants (9) were not randomly selected. The researchers sent the social workers (24) an invitation with a personal link to the online survey.

Data Collection

The survey was conducted online. There were two online follow-ups. The original EBP Process Assessment Scale (EBPPAS; Rubin & Parrish, 2011), which includes five subscales and 45 items, was used to measure social workers’ views about the EBP process and implementation of the EBP process. This scale is the first validated instrument to assess practitioners’ views on the EBP process and was developed with the intention of measuring indicators of practitioner orientation toward the EBP process in both surveys and evaluative efforts (Parrish & Rubin, 2011). It was developed and validated in the United States by Rubin and Parrish (2011) to specifically tap into practitioner (and student) views regarding the EBP process in contrast to EBPs.

As reported by Rubin and Parrish (2011), the EBPPAS has an excellent internal consistency, with a reported Cronbach’s α coefficient of .94. The original EBPPAS includes the following five separate subscale constructs: (1) familiarity/self-efficacy with the EBP process (10 items), (2) attitudes toward the EBP process (14 items), (3) perceived feasibility of the EBP process (5 items), (4) intentions to engage in the EBP process (8 items), and (5) actual self-reported EBP behaviors (8 items; Rubin & Parrish, 2011). All 45 items are followed by a 5-point Likert-type scale and 10 of the items convey negative responses about EBP and are reverse scored. The 45 items can be added up to get a composite score assessing the extent to which social workers are oriented to the EBP process. Higher scores indicate a more favorable response in each section and for the overall scale. In the current study, however, we removed 1 item from the attitudes subscale because it had a negative corrected item total correlation.

The questionnaire included the EBPPAS and 13 background/demographic questions. The EBPPAS (see Rubin & Parrish, 2011) was translated into Dutch separately by the researcher (RvdZ) and a translator (with a master of arts [MA] in English language and literature and an MA in translation studies). They compared both translations and agreed upon the best translation. Because the researcher and translator did not find many differences between their translations, they decided it was unnecessary to translate the questionnaire back to English to confirm a correct translation. As EBP is a loanword also used in the Dutch language, there were no problems translating it into Dutch. The background/demographic questions asked about self-reported familiarity with the EBP process, prior courses as a student in the EBP process, prior continuing education as a practitioner in the EBP process, age, gender, years of practice in social work, the number of employees working in their organization, field of practice, current position, years in current position, highest level of education, and years since earning that degree. The online survey was tested with a convenience sample of social workers in order to identify and address any problems. This study was not subject to an institutional review board. In the Netherlands, only studies that are medical–scientific studies, and studies in which persons are subject to procedures and/or are imposed to a way of behaving need to be approved by the Dutch Medical Research Involving Human Subjects Act.

Data Analysis

The Statistical Package for Social Sciences (SPSS) 19.0 was used to run descriptive statistics and to assess internal consistency. Internal consistency was assessed using Cronbach’s α. To assess whether the sample was representative for the entire Dutch population of social workers with regard to age and gender, we used one-sample χ² tests. Furthermore, as the self-reported familiarity was skewed positively to a large extent, we used a Mann–Whitney U test to compare the self-reported familiarity means of the respondents who completed the survey (N = 341) to the 192 nonrespondents who started the survey but did not complete it.

One-way between-group analyses of variance (ANOVAs) were conducted to examine the impact of the following four variables on the overall EBPPAS score: level of education, age, prior courses in EBP as a student, and prior continuing education in EBP as a practitioner. We used one-way between-group ANOVAs to investigate the impact of age, prior courses in EBP as a student, and prior continuing education in EBP as a practitioner on each of the five subscale scores. However, in some cases we needed to run a Welch’s test because the homogeneity of variance assumption was violated. To provide an indication of the magnitude of the effect, an effect size was calculated using eta square (η²; .01 = small; .06 = moderate; .14 = large). To minimize the chance of a Type 1 error, we conducted post hoc analyses using the Tukey’s honest significant difference (HSD) test and the Games–Howell post hoc procedure (in those cases where the homogeneity of variance assumption was not met).

Results

Response Rate and Sample Characteristics

Overall 992 social workers were invited to participate in this study, 565 social workers started the questionnaire and 373 social workers completed the questionnaire. Since we wanted to examine practitioners’ attitudes, we excluded the respondents who reported working in management or policy and research departments. This resulted in a sample of 341 social workers, providing a 34.4% response rate. The 341 social workers who completed the questionnaire also completed all scale items as it was not possible to skip questions.

Table 1 displays the background characteristics of the final sample used for the data analyses. The sample was primarily female (75.7%) and the mean age of the sample was 43 years. The largest proportion of respondents was 50 years of age or older (38.4%), 22.9% was 40–49 years old, 22.3% was 30–39 years old, and only 16.4% was 29 years or younger. The majority (86.8%) of the sample reported no familiarity or very little familiarity with EBP. Only 10.9% of the social workers in the sample reported having taken courses in EBP as a student and an even smaller percentage of social workers (8.8%) reported having taken continuing education in EBP as a practitioner. Social workers in the survey sample reported a mean of 10 years in their current position and a mean of 14 years of practice in social work. The largest proportion of respondents worked with adults (55%, n = 186). Other respondents worked with youth (19%, n = 66), specific vulnerable groups (12%, n = 42), elderly (11%, n = 38), and other (3%, n = 9).

Table 1.

Sample Characteristics and Background Variables (N = 341).

	M	SD
Age	43.37	11.89
Years of practice in social work	14.32	10.30
Years in current position	10.23	8.99
	n	%
Gender
Male	83	24.3
Female	258	75.7
Age-groups
<29	56	16.4
30–39	76	22.3
40–49	78	22.9
50>	131	38.4
Self-reported familiarity with EBP process
None	139	40.8
Very little	157	46.0
Quite a bit	45	13.2
Prior courses as a student in EBP process
Yes	37	10.9
Don’t know	64	18.8
No	240	70.4
Prior continuing education in EBP process
Yes	30	8.8
Don’t know	37	10.9
No	274	80.4
Field of practice
Youth	66	19.4
Adults	186	54.5
Elderly	38	11.1
Specific vulnerable groups	42	12.3
Other	9	2.6

Note. EBP = evidence-based practice; SD = standard deviation.

Internal Consistency

In the current study, the internal consistency for the entire 45-item scale was excellent, with a Cronbach’s α coefficient of initially .917. The familiarity/self-efficacy subscale had an excellent α of .92, and the intentions and behavior subscales had good αs above .80. The attitudes subscale initially had an α coefficient of .77. The shorter perceived feasibility subscale had a lower α of .68. However, as mentioned before, in the attitudes subscale we found that Item 4 (“Practitioners who engage in the EBP process show greater concern for client well-being than practitioners who do not engage in EBP.”) had a negative corrected item total correlation. This means this item was measuring something different from the scale as a whole. Therefore, we decided to delete Item 4 from the attitudes subscale. As shown in Table 2, this resulted in an improved Cronbach’s α coefficient of .816 for the attitudes subscale. The removal of Item 4 also improved Cronbach’s α coefficient for the entire scale (.919).

Table 2.

Coefficient α, Mean Score, Standard Deviation, and Per-Item Mean for Entire Scale and Subscales.

Scale	Coefficient α	Mean Score	SD	Per-Item Mean
Orientation (44)	.919	128.11	17.08	2.9
Familiarity (10)	.916	30.89	6.87	3.1
Attitude (13)	.816	41.50	4.36	3.2
Feasibility (5)	.675	14.79	2.52	3.0
Intentions (8)	.907	22.46	5.32	2.8
Behavior (8)	.912	18.47	6.25	2.3

Note. SD = standard deviation. N = 341.

Sample Representativeness

As there is no national data available on all social welfare and social services organizations, we used data of the organizations that are affiliated to the MOgroep to assess the degree of representativeness. The participating organizations were located in 8 out of the 12 Dutch provinces (see Table 3). The four provinces in the north of the Netherlands were not represented in the sample. This might be explained by the fact that there are probably fewer organizations in these provinces, as only a small percentage of the MOgroep members are located in these four provinces (H. Bijker, personal communication, May 15, 2014). Although social workers from the eight provinces in the sample may be unlike social workers from the four provinces that are not represented in the sample, we know of no reason to suppose that their attitudes toward and engagement in the EBP process should be different from those in the other four provinces. However, Table 3 shows that the relatively large organizations, with a large number of employees, are overrepresented, while the relatively small organizations are underrepresented. With regard to the type of practice, the organizations appear to be representative.

Table 3.

Profile of Participating Organizations and MOgroep Members.

	Sample	MOgroep Members
	%	%
Provinces
Drenthe	0	2.7
Flevoland	0	2.5
Friesland	0	3.6
Gelderland	9.1	13.3
Groningen	0	4.4
Limburg	9.1	4.8
Noord-Brabant	13.6	12.1
Noord-Holland	36.4	15.3
Overijssel	9.1	6.7
Utrecht	9.1	11.4
Zuid-Holland	9.1	20.8
Zeeland	4.5	2.5
Number of employees
<51	36.4	61.5
51–100	27.3	16.7
101–200	22.7	10.4
>200	13.6	11.5
Type of practice
Social welfare	63.6	69.3
Social services	22.7	19.8
Social care and shelter	4.6	6.4
Other	9.1	4.5

With regard to age and gender, the sample was representative of the entire Dutch social worker population. The one-sample χ² test showed that there was no statistically significant difference (χ² = 5.1, p = .17) in the proportion of respondents that fall into the various age categories of the sample (see Table 1) and the entire population of Dutch social workers (<29: 16.2%; 30–39: 20.6%; 40–49: 28.2%; and >50: 35%). There was no statistically significant difference (χ² = 1.2, p = .27) in the proportion of males and females of the sample (male: 25% and female: 75%) and the entire population of Dutch social workers (male: 27% and female: 73%).

Furthermore, because the respondents are perhaps more familiar with the EBP process than the nonrespondents, we also compared the self-reported familiarity mean scores. The Mann–Whitney U test showed a significant difference in self-reported familiarity scores (Z = −2.69, p = .01). Social workers who did complete the survey (n = 373) had an average rank of 295.13, while nonrespondents (who did not complete the survey; n = 192) had an average rank of 259.43. This indicates that with regard to self-reported familiarity, the sample is probably not representative of the larger social worker population.

Social Workers’ Orientation Toward the EBP Process

The mean score for the entire scale was 128.11, which suggests a slightly lower per-item mean (2.9) than the midpoint of 3 (on a Likert-type scale of 1–5, higher scores reflect a more favorable response; see Table 2). The standard deviation was 17.08, suggesting that the scale allows for detection of variability. The five subscales had per-item means ranging between 2.3 and 3.2, with the behavior subscale lowest (per-item mean = 2.3) and the attitudes subscale highest (per-item mean = 3.2). The attitudes subscale (per-item mean = 3.2) and the familiarity/self-efficacy subscale (per-item mean = 3.1) have higher per-item means compared to the intentions subscale (per-item mean = 2.8) and the behavior subscale (per-item mean= 2.3; see Table 2). These results indicate that although the respondents are slightly familiar with the EBP process and have slightly positive attitudes about it, their intentions to engage in the EBP process and their actual engagement in the EBP process are relatively low. To benchmark Dutch social workers’ orientation toward the EBP process, we compared our findings to a US study. Rubin and Parrish (2011) found a higher per-item (3.3) mean for the entire scale. Furthermore, in the U.S. study, the subscales had per-item means ranging between 2.9 and 3.5, with the behavior subscale lowest (per-item mean = 2.9) and the familiarity/self-efficacy subscale highest (per-item mean = 3.5).

Approximately 5% of the respondents (strongly) agreed with the question, “I know how to skillfully apply the steps of the EBP process,” which is included in the familiarity/self-efficacy subscale. Approximately 50% of the respondents (strongly) agreed with the question, “I understand how to evaluate the outcomes of my practice decisions” (familiarity/self-efficacy subscale). However, it is conceivable that this item was interpreted by some respondents to mean any type of practice evaluation (perhaps including unsystematic evaluations based on subjective judgments; Parrish & Rubin, 2012). When asked whether they agreed that “EBP helps to improve clients’ outcomes” (attitudes subscale), approximately 32% of the respondents (strongly) agreed. Approximately 15% of the respondents (strongly) agree that “The judgment of esteemed colleagues or supervisors offers a better basis than research evidence for improving practice effectiveness” (attitudes subscale). Approximately 7% of the respondents (strongly) agreed with the question, “I have enough time to engage in the EBP process” (perceived feasibility subscale) and approximately 9% of the respondents (strongly) agreed with the question, “I have enough access to the research literature to engage in EBP” (perceived feasibility subscale). Approximately 14% of the respondents reported that they “intend to read about research evidence to guide my practice decisions” often or very often and approximately 7% of the respondents reported that they “intend to engage in all steps of the EBP process” often or very often (intentions subscale). Approximately 11% of the respondents reported “reading about research evidence to guide my practice decisions” often or very often and approximately 1% of the respondents reported “engaging in all steps of the EBP process” often or very often (behavior subscale).

Education

We found no statistically significant difference in the overall EBPPAS scores between the five levels of education (see Table 4). We found a statistically significant difference in the familiarity/self-efficacy subscale scores for the five different levels of education, F(4,336) = 2.5, p = .04, but none on any of the other four subscales. Despite reaching statistical significance, the actual difference in mean scores between the groups was quite small. The effect size, calculated using η², was .03. The mean score on the familiarity/self-efficacy subscale for social workers with intermediate vocational education (M = 34.09, SD = 4.20) was significantly different from the mean score of social workers with higher vocational education (M = 30.55, SD = 6.60).

Table 4.

Mean Differences Between Different Levels of Education on Scale and Subscale Scores.

	IVE	HVE	HVE/Ma	AE/Ma	Other	F	η2
	(n = 34)	(n = 259)	(n = 16)	(n = 16)	(n = 16)
	M (SD)	M (SD)	M (SD)	M (SD)	M (SD)
Familiarity	34.09a (4.20)	30.55a (6.60)	29.00 (9.30)	2.19 (11.05)	30.31 (6.58)	2.5*	.03
Attitude	40.71 (4.05)	41.44 (4.28)	42.31 (5.04)	44.25 (5.80)	40.56 (2.99)	2.2	.03
Feasibility	15.15 (2.83)	14.80 (2.41)	4.38 (2.19)	13.88 (3.98)	15.31 (1.89)	1.0	.01
Intentions	24.47 (5.33)	22.29 (4.91)	23.81 (6.90)	21.38 (6.97)	20.63 (7.19)	2.2	.03
Behavior	20.35 (7.02)	18.46 (5.95)	17.44 (7.12)	16.31 (6.75)	17.81 (7.64)	1.4	.01
Orientation	134.76 (16.94)	127.54 (15.95)	126.94 (22.60)	128.00 (25.55)	124.63 (17.90)	1.6	.02

Note. IVE = intermediate vocational education; HVE = higher vocational education; HVE/Ma = higher vocational education/master; AE/Ma = academic education/master; HSD = honest significant difference; SD = standard deviation. Means sharing a common subscript are statistically different at the .05 level according to the Tukey’s HSD procedure.

*p < .05.

Age

Respondents were divided into four groups according to their age (Group 1: <29; Group 2: 30–39; Group 3: 40–49; and Group 4: 50 and above). There was a statistically significant difference in EBPPAS scores for the four age-groups, F(3,337) = 2.9, p = .04 (see Table 5). Despite reaching statistical significance, the actual difference in mean scores between the groups was quite small. The effect size, calculated using η², was .03. However, post hoc comparisons using the Tukey’s HSD test did not indicate that the mean score of the groups was significantly different (see Table 5).

Table 5.

Mean Differences Between Age-Groups on Scale and Subscale Scores.

	Group 1: <29		Group 2: 30–39		Group 3: 40–49		Group 4: 50>
	(n = 56)		(n = 76)		(n = 78)		(n = 131)
	M	SD	M	SD	M	SD	M	SD	F	η2
Familiarity	33.25ab	6.31	31.49	6.36	29.78a	7.13	30.21b	7.02	3.6*	.03
Attitude	42.16	3.97	41.76	4.33	41.50	4.81	41.06	4.25	.9	.01
Feasibility	14.32	2.90	15.43	2.53	14.76	2.52	14.65	2.28	2.5	.02
Intentions	22.07	4.64	23.18	5.05	22.69	5.94	22.07	5.36	.9	.01
Behavior	19.71	5.63	19.55	6.35	17.87	6.33	17.66	6.29	2.5	.02
Orientation	131.52	6.70	131.42	7.37	126.60	8.22	125.64	5.98	2.9*	.02

Note. HSD = honest significant difference; SD = standard deviation. Means sharing a common subscript are statistically different at the .05 level according to the Tukey’s HSD procedure.

*p < .05.

We found a statistically significant difference in the familiarity/self-efficacy subscale scores for the four different age-groups, F(3,337) = 3.6, p = .01, but none on any of the other four subscales (see Table 5). Despite reaching statistical significance, the actual difference in mean scores between the groups was quite small. The effect size, calculated using η², was .03. The mean score on the familiarity/self-efficacy subscale for social workers under 29 (Group 1; M = 33.25, SD = 6.31) was significantly different from the mean score of social workers between 40 and 49 (Group 3; M = 29.78, SD = 7.13) and from the mean score of social workers of 50 and above (Group 4; M = 30.21, SD = 7.02).

Courses on the EBP Process as a Student

We also investigated whether there was a difference in orientation toward the EBP process between social workers who reported having followed a course on the EBP process as a student (Group 1), social workers who reported that they did not know whether they did (Group 2), and social workers who reported not having followed a course on the EBP process as a student (Group 3; see Table 6). We found a statistically significant difference in the overall EBPPAS for the three groups, F(2,338) = 12.17, p = .00. The actual difference in mean scores between the groups was moderate. The effect size, calculated using η², was .07. The mean score on the overall EBPPAS was significantly different for the social workers who reported having followed a course on the EBP process as a student (Group 1; M = 139.76, SD = 15.26) compared to social workers who reported not having followed a course on the EBP process as a student (Group 3; M = 125.74, SD = 17.46) and compared to social workers who reported that they did not know whether they did (Group 2; M = 130.28, SD = 13.39).

Table 6.

Mean Differences on Scale and Subscale Scores for “Prior Course in the EBP Process as a Student.”

Prior Course as a Student in EBP?	Group 1: Yes		Group 2: Don’t Know		Group 3: No
	(n = 37)		(n = 64)		(n = 240)
	M	SD	M	SD	M	SD	F	η2
Familiarity	34.27a	5.72	32.08b	4.71	30.06ab	7.32	9.26***	.04
Attitude	44.08ab	4.83	41.39a	3.49	41.13b	4.37	7.67***	.04
Feasibility	14.03	3.11	15.27	2.20	14.79	2.48	2.87	.02
Intentions	25.05ab	3.57	22.63a	4.70	22.02b	5.59	9.69***	.03
Behavior	22.32ab	5.18	18.92a	5.16	17.75b	6.45	11.63***	.05
Orientation	139.76ab	5.26	130.28b	3.39	125.74a	7.46	12.17***	.07

Note. EBP = evidence-based practice; HSD = honest significant difference; SD = standard deviation. Means sharing a common subscript are statistically different at the .05 level according to the Tukey’s HSD or the Games–Howell procedure.

***p < .001.

Subscale analyses showed a statistically significant difference in the familiarity/self-efficacy subscale scores for the three groups, Welch’s F(292.79) = 9.26, p = .00 (see Table 6). Despite reaching statistical significance, the actual difference in mean scores between the groups was quite small. The effect size, calculated using η², was .04. The Games–Howell post hoc procedure indicated that the mean score on the familiarity/self-efficacy subscale for social workers who reported not having followed a course on the EBP process as a student (Group 3; M = 30.06, SD = 7.32) was significantly different from social workers who reported having followed a course on the EBP process as a student (Group 1; M = 34.27, SD = 5.72) and from social workers who reported that they did not know whether they did (Group 2; M = 32.08, SD = 4.71). There was also a statistically significant difference in the attitudes subscale scores for the three groups, F(2,338) = 7.7, p = .00 (see Table 6). Despite reaching statistical significance, the actual difference in mean scores between the groups was quite small. The effect size, calculated using η², was .04. Post hoc comparisons indicated that the mean score on the attitudes subscale for social workers who reported having followed a course on the EBP process as a student (Group 1; M = 44.08, SD = 4.83) was significantly different from social workers who reported that they did not know whether they did (Group 2; M = 41.39, SD = 3.49) and from social workers who reported not having followed a course on the EBP process as a student (Group 3; M = 41.13, SD = 4.37). There was no statistically significant difference in the perceived feasibility subscale scores for the three groups. The Welch test showed a statistically significant difference in the intentions subscale scores for the three groups, Welch’s F(296.40) = 9.69, p = .00 (see Table 6). Despite reaching statistical significance, the actual difference in mean scores between the groups was quite small. The effect size, calculated using η², was .03. Games–Howell post hoc procedure indicated that the mean score on the intentions subscale for social workers who reported having followed a course on the EBP process as a student (Group 1; M = 25.05, SD = 3.57) was significantly different from Group 2 (M = 22.63, SD = 4.70) and from social workers who reported not having followed a course on the EBP process as a student (Group 3; M = 22.02, SD = 5.59). The Welch test showed a statistically significant difference at the p < .05 level in the behavior subscale scores for the three groups, Welch’s F(2,89.27) = 11.63, p = .00 (see Table 6). The actual difference in mean scores between the groups was small to medium. The effect size, calculated using η², was .05. Games–Howell post hoc procedure indicated that the mean score on the behavior subscale for Group 1 (M = 22.32, SD = 5.18) was significantly different from for Group 2 (M = 18.92, SD = 5.16) and from Group 3 (M = 17.75, SD = 6.45).

Given the finding that social workers under 29 (Group 1) appeared to be more familiar with the EBP process than social workers over 40 (Groups 3 and 4) and the finding that social workers who reported having followed a course on the EBP process as a student are more oriented toward the EBP process than social workers who did not, we wondered whether young social workers were perhaps more familiar with the EBP process because they have had more prior exposure to EBP during their education than older social workers who have been educated in the pre-EBP era. We found that 28.5% of the social workers under 29 (Group 1) in our sample reported having followed a course on the EBP process as a student, while this was only 10.5% of the social workers between 30 and 39 (Group 2), 10.3% of the social workers between 40 and 49 (Group 3) and even 3.8% of the social workers of 50 and above (Group 4). These findings indicate that young social workers have indeed had more prior exposure to the EBP process during their education than older social workers.

Continuing Education as a Practitioner

Finally, we investigated whether there was a difference in orientation toward the EBP process between social workers who reported having followed prior continuing education on the EBP process as a practitioner (Group 1), social workers who reported that they did not know whether they did (Group 2), and social workers who reported not having followed prior continuing education on the EBP process (Group 3; see Table 7). We found a statistically significant difference on the overall EBPPAS for the three groups, F(2,338) = 3.6, p = .03. Despite reaching statistical significance, the actual difference in mean scores between the groups was quite small. The effect size, calculated using η², was .02. However, post hoc comparisons using the Tukey’s HSD test did not indicate that the mean score of the groups was significantly different, although one of the pairs was a borderline case; Group 1 (M = 134.60, SD = 17.19) was almost significantly different from Group 3 (M = 126.94, SD = 17.11; p = .05).

Table 7.

Mean Differences in Scale and Subscale Scores for “Prior Continuing Education in the EBP Process as a Practitioner.”

Prior Continuing Education in EBP?	Group 1: Yes		Group 2: Don’t Know		Group 3: No
	(n = 30)		(n = 37)		(n = 274)
	M	SD	M	SD	M	SD	F	η2
Familiarity	32.00	5.05	31.68	6.07	30.67	7.14	0.8	.00
Attitude	43.80a	5.52	42.16	4.04	41.16a	4.18	5.6**	.03
Feasibility	14.73	2.84	15.19	2.03	14.75	2.54	0.5	.00
Intentions	23.73	4.77	22.86	4.20	22.27	5.50	1.1	.01
Behavior	20.33	5.99	19.68	5.93	18.10	6.29	2.5	.01
Orientation	134.60	7.19	131.57	5.50	126.94	7.11	3.6*	.02

Note. EBP = evidence-based practice; HSD = honest significant difference; SD = standard deviation. Means sharing a common subscript are statistically different at the .05 level according to the Tukey’s HSD procedure.

*p < .05. **p < .01.

Subscale analyses showed a statistically significant difference in the attitudes subscale scores for the three groups, F(2,338) = 5.6, p = .00 (see Table 7). Despite reaching statistical significance, the actual difference in mean scores between the groups was quite small. The effect size, calculated using η², was .03. Post hoc comparisons indicated that the mean score on the attitudes subscale for social workers who reported having followed prior continuing education on the EBP process as a practitioner (Group 1; M = 43.80, SD = 5.52) was significantly different from social workers who reported not having followed prior continuing education on the EBP process (Group 3; M = 41.16, SD = 4.18). Group 2 (M = 42.16, SD = 4.04) did not differ significantly from Group 1 or from Group 3. We found no statistically significant difference on any of the other four subscales (familiarity/self-efficacy, perceived feasibility, intentions, and behavior) for the three groups.

Discussion and Applications to Social Work

This is the first study in the Netherlands to measure the levels of social workers’ orientation toward the EBP process and implementation of the EBP process. The primary finding of this survey is that, despite initiatives by policy makers and educators to improve EBP, Dutch social workers in this sample have a relatively low overall orientation toward the EBP process. They are slightly familiar with the EBP process and have slightly positive attitudes about it, but their intentions to engage in the EBP process and their actual engagement are relatively low. When comparing the results of this Dutch study to a US study, Dutch social workers have a lower overall orientation toward the EBP process than respondents from the United States. In both studies the behavior subscale had the lowest score of the five subscales, but the U.S. study had a higher per-item mean (2.9) on the behavior subscale than the Dutch study (2.3). These results are not surprising, as the U.S. sample was obtained from both social workers and students from four areas that were selected because they had prestigious graduate social work education programs that integrated EBP into the curriculum. Social work educators in the United States have been optimistic for a longer time than those in the Netherlands about the usefulness of EBP. This optimism has resulted in the initiation of innovative approaches in various places in the United States to educate students in the EBP process and provide continuing education on the EBP process to social workers (Rubin & Parrish, 2010). Also, a number of American graduate social work programs have adopted EBP as their unifying conceptual framework (Thyer & Myers, 2011). To our knowledge, there have been no similar initiatives to educate students in the EBP process in the Netherlands. Moreover, because the MSW program in the Netherlands is new and thus there are only a few social workers with a master’s degree yet, the Dutch sample contained only 16 social workers with a master’s degree.

This study also explored whether specific variables were associated with the level of orientation toward the EBP process, as these variables can be helpful in suggesting ways to improve practitioner acceptance and implementation of the EBP process. The strongest effect size (the effect size was medium) was identified in a comparison of social workers who reported having followed a course on the EBP process as a student with those who reported not having followed a course on the EBP process as a student: the former are more positively oriented toward the EBP process than the latter. Furthermore, their actual engagement in the EBP process is higher (the effect size was small to medium). They are also more familiar with the EBP process, have more positive attitudes about it, and their intentions to engage in the EBP process are higher; however, these differences were not large. We found no difference with regard to their perceived feasibility of implementing EBP in the real world.

Furthermore, we found that social workers under 29 were more familiar with the EBP process than social workers over 40, although this difference was not large. Given this finding, and the finding that the largest percentage of social workers that followed a course on the EBP process as a student was found in the youngest age-group, we believe it is likely that young social workers are more familiar with the EBP process because they have had more prior exposure to EBP during their education than older social workers. Furthermore, social workers who reported having prior continuing education on the EBP process as a practitioner have more positive attitudes about the EBP process than social workers who reported not having followed prior continuing education on the EBP process, although this difference was not large. We found no statistically significant difference in the overall orientation toward the EBP process between the five levels of education. However, in four of the five groups, the number of respondents was low since most social workers in our sample had a bachelor’s degree (higher vocational education). This may have affected the result. As mentioned before, the low number of social workers with an MSW degree in the sample is not surprising, because only a few social workers have an MSW degree. An unexpected and interesting finding was that social workers with intermediate vocational education were more familiar with the EBP process than social workers with higher vocational education, although this difference was small. It is difficult to explain why this is the case. It might be that there is a difference in the extent to which EBP is taught in intermediate vocational education and higher vocational education. Another explanation could be the occurrence of self-selection bias, as social workers with intermediate vocational education may have been more inclined to not respond to the survey, if they were not familiar with EBP, than social workers with higher vocational education. However, since it is not clear why this finding emerged, additional research is needed to explain and interpret this effect.

There are certain limitations to be considered in interpreting our findings. We were able to obtain a relatively large total number of respondents (N = 341), providing a 34% response rate, because we used two different strategies to approach respondents. However, it should be taken into account that the findings are based on only 0.5% of a total population of 61,500 social professionals. Furthermore, the results may be limited by a self-selection bias as we were not able to draw a random study sample because we were not given direct access to the members’ lists of the MOgroep and Verdiwel. Although the sample was representative for the entire Dutch population of social workers with regard to age and gender, it is conceivable that organizations that agreed to participate in the study and respondents may have been more oriented to the EBP process than nonresponders. This concern is underscored to some extent by our finding that the group of social workers who did complete the survey was more familiar with the EBP process than the group of social workers who did not complete the survey. This suggests that the sample is probably not representative of the larger practitioner population with regard to self-reported familiarity and that the Dutch population of social workers is probably less familiar with the EBP process than the social workers in the sample.

However, despite the aforementioned limitations this study provides preliminary evidence that Dutch social workers are not much oriented toward the EBP process and suggests implications for social work education, practice, and research. In discussing these implications, it is important that we acknowledge our caution in generalizing our findings to the entire population of Dutch social workers. Our sample was probably more familiar with the EBP process than the entire population of Dutch social workers. Furthermore, in our sample large organizations were overrepresented and small organizations were underrepresented, therefore the findings are most applicable to large organizations.

As only 11% of the respondents reported “reading about research evidence to guide my practice decisions” often or very often and approximately 1% of the respondents reported “engaging in all steps of the EBP process” often or very often, there is a need for improvement. Although it might be considered encouraging that approximately 32% of the respondents (strongly) agreed that “EBP helps to improve clients’ outcomes,” only approximately 5% of the respondents (strongly) agreed with the question, “I know how to skillfully apply the steps of the EBP process.” Our results show that having followed courses in EBP is associated with a higher overall level of orientation toward the EBP process. Social workers who followed courses in EBP were more familiar with EBP, had more positive attitudes about it, had more intentions to engage with the EBP process, and were more engaged with the process. Therefore, we recommend universities of applied sciences (and universities) to take a more active role in teaching students (both bachelor and master) and social workers the principles of the EBP process. Educators should be aware however that there was no difference in the perceived feasibility of implementing EBP in the real world. The results from this survey show that social workers see insufficient time and lack of access to research literature as barriers to EBP implementation in practice. Educators should address these feasibility issues, for example, by teaching methods for finding evidence efficiently. In addition, respondents’ skepticism about the feasibility of implementing EBP in practice may be an accurate assessment of the barriers that hamper EBP implementation. This is underscored by findings from a review of empirical studies examining the implementation of EBP in the human services literature that identified several barriers to EBP implementation, such as inadequate agency resources dedicated to EBP, lack of time and access to research evidence (Gray et al., 2013). Addressing these barriers in practice might improve the perceived feasibility to engage in the EBP process and also improve EBP implementation.

In the meantime, current efforts to improve EBP implementation may want to focus especially on social workers who are already familiar with the EBP process. Our study suggests that social workers who followed courses in EBP as a student and social workers under 29 tend to be more familiar with the EBP process. Familiarity, as measured by the EBPPAS, is an indication of social workers’ self-efficacy in using the EBP process. Bender, Altschul, Yoder, Parrish, and Nickels (2014) have argued that it is possible that higher self-efficacy in the ability to use the EBP process will lead to successful implementations of the EBP process. In other words, social workers who feel more confident in their knowledge of the EBP process may be better equipped to apply it in practice. Therefore, it is encouraging that both social workers who followed courses in EBP as a student and social workers under 29 have greater familiarity in the EBP process. However, as mentioned before, the flexible shell of young professionals decreases and the workforce of organizations is aging. Therefore, it might be useful for social work organizations’ human resource departments to take this into consideration. For example, when they are hiring new staff, they might want to select social workers who are familiar with the EBP process.

In order to evaluate the impact of these efforts, it would be interesting to repeat this study in the future and use the findings of the current study for comparison. Future studies should aim to enhance methodological rigor through random sampling and monitoring of reasons for nonresponse to overcome the threats to external validity inherent in this initial study. Furthermore, social work research should assess Dutch MSW students’ level of orientation toward the EBP process. Given the aim of the MSW program to create new professionals that focus on the effectiveness of interventions and accountability of the profession, it is conceivable that MSW students are more oriented toward EBP than, for example, social workers with only a bachelor’s degree. Therefore, we recommend that future research be directed at universities of applied sciences to investigate to what extent MSW students are actually being educated in the EBP process and to evaluate the impact of the MSW programs (preferably with a pretest–posttest design) on MSW students’ orientation toward the EBP process.