Abstract
Using a cluster-randomised design, this study analyses the effects of a government-administered skill training programme for social workers in Norway. The training programme aims to improve social workers' professional competences by enhancing and systematising follow-up work directed towards longer-term unemployed clients in the following areas: encountering the user, system-oriented efforts and administrative work. The main tools and techniques of the programme are based on motivational interviewing and appreciative inquiry. The data comprise responses to baseline and eighteen-month follow-up questionnaires administered to all social workers (n = 99) in eighteen participating Labour and Welfare offices randomised into experimental and control groups. The findings indicate that the skill training programme positively affected the social workers' evaluations of their professional competences and quality of work supervision received. The acquisition and mastering of combinations of specific tools and techniques, a comprehensive supervision structure and the opportunity to adapt the learned skills to local conditions were important in explaining the results.
Keywords: Evidence-based, skill training, social work, randomised, experimental, cluster-randomised, professional competences
Introduction
The requirements for contemporary social work of being effective and empowering simultaneously increase the need to investigate and understand how professional competences develop. In Norway, social workers within a nationwide welfare-to-work programme, the Qualification Programme, addressed the need for increased competences regarding the close follow-up to which clients were entitled. Accordingly, the Norwegian Directorate of Labour and Welfare developed and implemented a skill training programme: the Comprehensive, Methodological and Principle-based Approach (CMPA). They also commissioned an independent evaluation of the programme. The aim of the implementation and evaluation was to improve social workers' skills, develop evidence-based methods for social work and increase the knowledge of what ‘works’ within social work practices (Labour and Welfare Administration, 2011; Malmberg-Heimonen, 2015).
The CMPA programme can be understood as a marker of the development of evidence-based practice in social work and, consequently, within the context of the general academic turn of social work into an academic discipline. This development is characterised by a need to understand knowledge production, including evidence-based research and practice. Sackett et al. (1996, p. 71) define evidence-based practice as the ‘conscientious, explicit, and judicious use of current evidence in making decisions about the care of individual patients’. It rests on a paradigm, which maximises opportunities to help clients and avoid harm, but it also derives from public resource limitations, which impose pressure to document efficiency. The latter perspective can be criticised for building on the notion of practitioners and clients as rational actors, which may reduce social work to an instrumental activity (Webb, 2001). Neo-liberal ideologies and perspectives of managerialism may also result in a market view of welfare services and the individualisation of social problems (Wiggan, 2012).
On the other hand, it is fundamental to evidence-based practice that expertise from the field should be acknowledged. While some scholars have demonstrated the importance of guidelines, standardised work methods and specific interventions, others argue that the reflexive and relational competences of the professional is essential (Messer and Wampold, 2002). Critics emphasise that evidence-based practice takes insufficient account of practitioner competences, clients' values and wishes (Johnson and Austin, 2006). Thus, the complexity of the discipline calls for a broad understanding of what evidence-based practice is, how it relates to professional competences and how it influences social workers' skills and everyday practice.
The government-administered CMPA skill training programme seeks to improve social workers' professional competences by systematising and improving follow-up work in the welfare-to-work Qualification Programme. Using a cluster-randomised research design with an eighteen-month follow-up, this study evaluates the effects of the CMPA on social workers' assessments of their professional competences, working alliance, the quality of work supervision received and the three main areas of the programme (i.e. encountering the user, system-oriented efforts and administrative work). As the CMPA programme focuses on increasing professional competences as well as specific skills for follow-up work, we expect it to have a positive effect on measured outcomes.
The CMPA skill training programme in a Norwegian social work context
In 2006, the Norwegian government introduced an extensive reform initiative, the aim of which was to streamline and improve Norway's welfare services based on a ‘one-stop shop’ principle. The reform integrated social welfare, social security and employment services into one joint unit, the Norwegian Labour and Welfare Administration. To prevent poverty and social exclusion among the long-term unemployed, the Labour and Welfare Administration also implemented the Qualification Programme. The programme is regarded a new context for Norwegian social work; social workers are expected to conduct a close and comprehensive follow-up that should be tailored to the needs, preferences and limitations of the individual client. It is a full-time programme lasting up to two years, which focuses on increasing qualifications, motivation and self-efficacy (Andersen and Skinnarland, 2011; Helgøy et al., 2010).
In order to meet social workers' need for increased competences regarding the follow-up work within the Qualification Programme, the Norwegian Directorate of Labour and Welfare developed and implemented the CMPA. The CMPA programme consists of four joint seminars (for a total of nine days) and a comprehensive supervision structure carried out at the local work place. It is described as ‘innovative’ because it introduces a model consisting of a new structure and a common, systematic approach for social workers' follow-up work. It also attempts to improve their abilities for more accurate and tailor made social work. Therefore, the model recognises the complex, locally situated nature of social work and that clients' needs vary, allowing for pragmatic adjustments. A more comprehensive description of the programme is to be found in Malmberg-Heimonen (2015).
According to the CMPA programme (Figure 1), the follow-up work should cover three essential areas: encounters with users, focusing on the relationship between the social worker and the client; system-oriented efforts, focusing on work with collaborating partners (e.g. the client's social network, collaborators in welfare services and the labour market); and administrative work, which includes charting, planning, coordination and evaluation of services (Labour and Welfare Administration, 2011; Malmberg-Heimonen, 2015).
Figure 1.
The main areas of the CMPA skill training programme
Source: The Norwegian Labor and Welfare Administration, 2011
CMPA's methodological tools are based on motivational interviewing (MI) and appreciative inquiry (AI). MI offers an important way of working with resistance, as it involves the use of active listening skills (Markland et al., 2005; Prochasca and Di Clemente, 1983). Due to its anti-oppressive nature, MI is important in the context of social work (Watson, 2011). AI encourages individuals to adopt a positive, constructive approach in order to support organisational changes. Its relevance to social work is that it challenges the problem-focused approach often dominant in the field (Dematteo and Reeves, 2011). In the skill training programme, MI relates especially to the first area, encounters with users, while AI relates to the second area, system-oriented efforts.
To ensure local implementation of the CMPA programme, a three-level work supervision structure was implemented. Previous research has demonstrated that, in order to translate learned knowledge into practice, long-term work supervision with individual feedback and coaching after workshops or courses are essential (Miller et al., 2004; Heaven et al., 2006; Madson et al., 2009). Within CMPA, the first level of work supervision consists of the CMPA team leaders at the local Labour and Welfare offices who supervise social workers in the Qualification Programme. The second level is the county CMPA representatives who supervise the CMPA team leaders. The third level is the resource group at the Directorate of Labour and Welfare, which supervises the county-level CMPA representatives and, if needed, representatives at other levels. Social workers were to attend supervisory sessions with the CMPA team leaders every fourteen days. Local CMPA implementation was further supported by two booster seminars in which social workers discussed their experiences, and by a two-day seminar for office leaders.
Professional competences in social work
Professional competences are acquired through both formal education and workplace experience. The concept of competence has been criticised (O'Hagan, 1996), as it may result in a technical and bureaucratic approach to social work practice that is contrary to a holistic perspective emphasising critical reflection and autonomy, suitable for an organisational framework in which strategies and structures are valued more than individualised and person-focused development. These criticisms are similar to those related to evidence-based practice within social work, as pointed out in the introduction section. However, the criticism depends on how ‘competence’ is defined; the concept does not have one unique definition, but different meanings depending on perspective and epistemological traditions. It has for instance been argued that a rationalist, positivistic perspective, measuring competences defined as certain qualifications, would not take into account the actor's perspective (Garavan and McGuire, 2001). A phenomenological approach, however, represents an alternative perspective where the socio-cultural context and self-assessments are recognised.
Nygren (2004, p. 151) defines professional competence as ‘relevant expertise that enables people to master one or more academic tasks assigned to the profession in relation to specific claims’. Professional competence should be interpreted as an individual response to the specific requirements in various contexts and practices. This ‘operational expertise’ emerges only when it is realised in concrete actions in a particular practice, which is defined as ‘competence to act’. Competence to act appears, first, as generalised potential competence and, second, as context-specific realised competence (Nygren, 2004). Work supervision is crucial for the development of competences both as regards learning and the process of linking new knowledge to potential use. Work supervision should be carried out interactively, focusing dialogue and relationships (Cohen, 2004).
The concept of competence is interrelated with knowledge production. Gibbons et al. (1994) separate between Mode 1 and Mode 2 types of knowledge production. In Mode 1, knowledge is defined and resolved by the scientific community whereas, in Mode 2, knowledge is produced in interaction with practice. Gray and Schubert (2012) argue that, within social work, the concept of ‘evidence-based’ should encompass both knowledge modes. Gambrill (2007) pinpoints ‘evidence-informed’ as a more convenient concept for the knowledge production within social work, leaving room for clinical experience and judgements of practitioners and clients. As regards applied social research, Davies et al. (2008) suggest knowledge interaction as an appropriate description of the engagement of multiple players with diverse sources of knowledge.
Data and methods
A cluster-randomised design was used to evaluate effects of the CMPA skill training programme. While the Labour and Welfare Administration was responsible for developing and implementing CMPA, independent researchers were responsible for the evaluation. Social workers in experimental and control groups responded to baseline and follow-up questionnaires that consisted of items related to knowledge and skills that are important for professional practice, as well as items concerning their competences within the Qualification Programme. The evaluation also produced qualitative data in the form of interviews and observations. Additionally, effects of CMPA for clients were documented through questionnaires, administrative data and interviews (Malmberg-Heimonen, 2015; Malmberg-Heimonen and Tøge, 2015; Natland and Hansen, 2014). However, these data are not utilised in this study.
Recruitment and randomisation
Fifty of the largest Labour and Welfare offices nationwide were invited by the Labour and Welfare Administration to participate in the project. The Administration informed the leaders of these offices of the requirements for participation, namely organisational readiness and participation in research, while the researchers informed them about the cluster-randomised design. Of the fifty offices invited, eighteen decided to participate. A common reason for declined invitations was that the offices had recently undergone major organisational changes. Clusters (offices) instead of individual social workers were randomised because the skill training programme was implemented at the office level, and randomising individual social workers would have increased the risk of contamination and problems of programme implementation (Bloom, 2005; Ivers et al., 2011; Malmberg-Heimonen, 2015).
Before randomising the participating offices, all 103 social workers who were working in the Qualification Programme, either full-time or part-time in addition to other tasks, received the T1 questionnaire and the consent form. Of these, 99 responded, for a response rate of 96 per cent of the initial population of social workers. The T1 questionnaire focused on documenting the follow-up work, especially concerning encountering the user, system-oriented efforts and administrative work. The questionnaire also documented social workers' professional competences, working alliance and motivations for and experiences of follow-up work in the Qualification Programme. In addition to the standardised WAI scale (Working Alliance Inventory Scale), we developed measures that fitted the goals of the CMPA programme but at the same time were general and suitable for social workers in the control group. Measures were developed in conjunction with programme developers, practitioners and researchers in the field and were tested in a pilot study involving twenty-four social workers. Factor analyses were conducted on the various scales and poorly functioning items were omitted from the final questionnaire.
Of the ninety-nine social workers who completed the first questionnaire, 78 per cent were women and 22 per cent men. The average age of social workers was 42.9 years. In terms of education, 69 per cent had a bachelor's degree and 28 per cent a master's degree; only 3 per cent had less education than a bachelor's degree. Regarding experience, 72 per cent of the social workers had previous experience with similar follow-up work, having worked in the Qualification Programme for an average of fifteen months.
After the social workers had completed the first questionnaire, the eighteen Labour and Welfare offices were randomised into experimental and control groups. Nine offices were randomised to the experimental group and nine to the control group. Social workers from the experimental group offices began their skill training, while social workers from the control group offices continued with business as usual. The CMPA resource group from the Directorate arranged the training over four seminars and a total of nine days between 10 May and 15 September 2011; homework assignments were completed between seminars. After the seminars ended, the programme continued with CMPA supervision at the workplace. Social workers from the control group offices received work supervision as ‘business as usual’, which often consists of reflections and discussions of concrete challenges in their daily work.
Table 1 compares office-level and social worker-level information between the experimental and control group conditions at T1 (baseline). Although the randomisation was successful, the levels of T1 working alliance and T1 professional competences were significantly lower among social workers in offices randomised to the experimental group than in control group offices. Consequently, the baseline values of variables showing significant differences will be controlled for when determining the final effects of the CMPA skill training programme.
Table 1.
The success of randomisation based on information from offices (n = 18) and the T1 questionnaire for social workers (n = 99)
| Variables | Experimental group offices, M (SD) | Control group offices, M (SD) | p-value |
|---|---|---|---|
| Office-level administrative data, n | 9 | 9 | |
| Number of participants in Qualification Programme, 2010 | 86.4 (48.78) | 67.4 (28.78) | 0.333 |
| Number of completed programmes, 2010 | 29.9 (17.23) | 21.7 (21.73) | 0.388 |
| Number of participants attaining employment, 2010 | 9.7 (7.85) | 10.0 (10.50) | 0.943 |
| Questionnaire for social workers, n | 54 | 45 | |
| Gender (% women) | 82 | 73 | 0.332 |
| Relevant courses (% yes) | 53 | 47 | 0.538 |
| Experience of similar work (% yes) | 72 | 72 | 0.966 |
| Average months worked in the Qualification Programme | 15.4 (9.92) | 14.3 (9.25) | 0.558 |
| Frequency of supervision latest month (T1) | 0.87 (1.44) | 1.60 (3.67) | 0.182 |
| Professional competence, T1 (6–30) | 20.5 (3.46) | 21.9 (3.11) | 0.046 |
| Working alliance, T1 (12–60) | 45.0 (4.78) | 47.5 (3.82) | 0.008 |
| Quality of work supervision, T1 (2–10) | 5.5 (1.87) | 5.4 (1.98) | 0.810 |
| Encountering the user, T1 (4–20) | 14.4 (2.52) | 15.2 (2.09) | 0.104 |
| System-oriented efforts, T1 (4–20) | 14.8 (2.33) | 15.4 (1.88) | 0.194 |
| Administrative work, T1 (3–15) | 12.0 (1.77) | 12.3 (1.63) | 0.458 |
Follow-up and attrition
The follow-up questionnaire was sent to social workers within experimental and control conditions eighteen months after T1, which was one year after the CMPA skill training programme had ended. The second questionnaire repeated measures from the first one. In addition, social workers from offices randomised to the experimental group were asked to evaluate the CMPA skill training programme. Of the ninety-nine social workers who had completed T1, eighty-two responded to T2, for a response rate of 83 per cent. Among social workers from the experimental group offices, the response rate was 75 per cent, while it was 86 per cent for their counterparts in the control group offices. The most common reason for not responding was employee turnover.
Analyses of attrition generally indicated no significant differences between T2 respondents and non-respondents in terms of gender, age, education, relevant courses, work experience or time working in the Qualification Programme. However, T2 respondents generally reported somewhat higher means on T1 measures of administrative work and professional competences than did those who did not complete the T2 questionnaire. Attrition analyses based on experimental condition indicated that T2 respondents in the experimental group scored somewhat higher on the administrative work scale than did T2 non-respondents (p = 0.011), while no such difference was found in the control group respondents. None of the other variables displayed skew attrition based on experimental condition.
Measures
Background variables were measured using standard survey questions. Gender was measured as 1 = male and 2 = female. Age was measured as the age of the respondent in 2011. Education was assessed as 1 = elementary school or high school, 2 = bachelor's degree or equivalent and 3 = master's degree, equivalent or higher.
Previous participation in relevant courses and experience of work similar to that involved in the Qualification Programme were assessed using a dichotomous variables, namely 0 = ‘no’ or 1 = ‘yes’. In addition, the number of total months working in the Qualification Programme was assessed. Frequency of work supervision received was assessed at T1 and T2 by the sequences of supervision during the latest month prior to responding.
Professional competences were assessed using a six-item scale. The items were: ‘I have sufficient professional competence to help participants reach their goal of a job’, ‘I have sufficient competence to help participants improve their quality of life’, ‘I have sufficient competence to help participants reach their goals of activity’, ‘I have sufficient knowledge of the labour market in our municipality to help participants get employment’, ‘I have received training that gives me competence in my work with Qualification Programme participants’ and ‘In our office, social workers in the Qualification Programme are highly competent’ (Cronbach's alpha = 0.81). The response options for each item range from 1 = never to 5 = always.
Working alliance was measured using the Working Alliance Inventory Scale, short revised version (Hatcher and Gillaspy, 2006; Horvath and Greenberg, 1989). The scale consists of a twelve-item measure focusing on the alliance between the social worker and the service user, especially concerning the dimensions of goals (‘The participant and I work towards goals that we have agreed on’), tasks (‘We agree on what is important for the participant to work on’) and bonds (‘I think the participant likes me’). The response options for each item range from 1 = never to 5 = always. The total scale ranges from 12 to 60 and is treated as one dimension in the present study (Wykes et al., 2013). The reliability of the scale is high (Cronbach's alpha = 0.86).
Quality of work supervision received was measured using the following items: ‘The supervision I get at the office helps me maintain and develop skills that are important to me as a social worker in the Qualification Programme’ and ‘I receive supervision during training on specific skills that are important in the follow-up work’. The response options for each item range from 1 = never to 5 = always (Cronbach's alpha = 0.72).
Finally, the three areas of the CMPA skill training programme were determined using principal-axis factoring with varimax rotation (Table 2). The response options for each item range from 1 = never to 5 = always. The encountering the user scale consists of four items, measuring the use of MI-related techniques (Cronbach's alpha = 0.85). The system-oriented efforts scale also consists of four items, focusing on whether and to what extent the social worker collaborates with the participant's network (private and professional) and to what extent the participant has an active role in this collaboration (Cronbach's alpha = 0.79). The administrative work scale was assessed based on three items, measuring the amount and degree of documentation (Cronbach's alpha = 0.76). The explanatory variances for the three dimensions were 38.2 per cent for encountering the user, 18.8 per cent for the system-oriented efforts and 14.9 per cent for the administrative work.
Table 2.
Main areas of the CMPA skill training programme measured using principal factor analysis with varimax rotation
| Area | M (SD) | Factor |
||
|---|---|---|---|---|
| 1 | 2 | 3 | ||
| Encountering the user | ||||
| When encountering the participant, I ask open questions | 4.1 (0.62) | 0.76 | 0.20 | –0.13 |
| When encountering the participant, I summarise the discussions under way | 3.7 (0.76) | 0.83 | 0.08 | 0.12 |
| In discussions with the participant, I reflect on his or her thoughts and feelings | 3.4 (0.78) | 0.80 | 0.03 | 0.10 |
| When the participant talks about changing something in his or her life, I explore this further | 3.6 (0.67) | 0.63 | 0.21 | 0.09 |
| System-oriented efforts | ||||
| I think that co-operation with other actors (e.g. collaborators in welfare services and the labour market and private network) functions well | 3.6 (0.64) | –0.06 | 0.69 | –0.01 |
| In co-operating with other actors, we manage to work towards a common goal | 3.6 (0.59) | 0.17 | 0.85 | 0.15 |
| The participant's understanding of his or her situation is important | 4.1 (0.72) | 0.29 | 0.60 | 0.23 |
| The participant has an active role in the collaborative meetings | 3.8 (0.80) | 0.29 | 0.60 | 0.19 |
| Administrative work | ||||
| I systematically document the follow-up work | 4.1 (0.69) | –0.01 | 0.00 | 0.89 |
| I continuously document the follow-up work | 4.1 (0.62) | –0.00 | 0.15 | 0.75 |
| I write down and document the participant's wishes and needs | 3.9 (0.75) | 0.21 | 0.25 | 0.56 |
Note: The responses range from 1 to 5. Factor loadings of leading items on each dimension are shown in boldface type.
Analysis plan
The three areas of the CMPA programme—encountering the user, system-oriented efforts and administrative work—were identified by factor analyses (Table 2). Bivariate correlations were identified for background and T1 variables (Table 3). After initial statistical tests, we measured social workers' assessments of their work before and after the implementation of the skill training programme, based on unadjusted mean values (Table 4).
Table 3.
Means, standard deviations and bivariate intercorrelations for study variables (n = 83–99)
| Variables | M | SD | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1. Experimental condition (cont. = 0, exp. = 1) | 0.6 | 0.50 | – | |||||||||||||
| 2. Gender (1 = men, 2 = women) | 1.8 | 0.42 | 0.10 | – | ||||||||||||
| 3. Age | 43.3 | 10.08 | –0.18 | –0.38** | – | |||||||||||
| 4. Education (1–3) | 2.3 | 0.50 | –0.09 | 0.02 | –0.05 | – | ||||||||||
| 5. Relevant courses (1 = no, 2 = yes) | 1.5 | 0.50 | 0.06 | –0.03 | 0.00 | –0.13 | – | |||||||||
| 6. Experience of similar work (1 = no, 2 = yes) | 1.7 | 0.45 | 0.00 | –0.16 | 0.22* | 0.05 | –0.02 | – | ||||||||
| 7. Average months worked in the Qualification Programme | 14.9 | 9.59 | 0.06 | –0.22* | 0.27* | –0.13 | 0.39** | 0.12 | – | |||||||
| 8. Frequency of supervision latest month (T1) | 1.20 | 2.70 | –0.13 | 0.06 | –0.08 | –0.09 | 0.10 | –0.24* | –0.12 | – | ||||||
| 9. Encountering the user, T1 (4–20) | 14.8 | 2.36 | –0.17 | –0.20 | 0.28** | –0.10 | –0.14 | 0.28** | 0.10 | –0.13 | – | |||||
| 10. System-oriented efforts, T1 (4–20) | 15.0 | 2.16 | –0.12 | 0.09 | 0.07 | –0.10 | –0.19 | –0.09 | –0.12 | 0.18 | 0.37*** | – | ||||
| 11. Administrative work, T1 (3–15) | 12.1 | 1.71 | –0.12 | 0.02 | –0.07 | –0.20 | –0.16 | –0.19 | –0.14 | 0.05 | 0.14 | 0.32** | – | |||
| 12. Professional competences, T1 (6–30) | 21.1 | 3.37 | –0.21* | –0.18 | 0.32** | 0.14 | 0.07 | 0.14 | 0.46** | –0.10 | 0.33** | 0.26* | 0.07 | – | ||
| 13. Working alliance, T1 (12–60) | 46.1 | 4.54 | –0.28* | –0.07 | 0.29** | –0.08 | 0.18 | –0.01 | 0.26* | 0.05 | 0.54*** | 0.53 | 0.21 | 0.56*** | ||
| 14. Quality of work supervision, T1 (2–10) | 5.4 | 1.91 | 0.04 | –0.02 | 0.01 | 0.02 | 0.14 | –0.11 | 0.02 | 0.17 | 0.00 | 0.06 | 0.22 | 0.16 | 0.11 | – |
Note: * p < 0.05; ** p < 0.01; *** p < 0.001.
Table 4.
Social workers' self-assessments of their follow-up work before and after CMPA implementation, unadjusted
| Experimental |
Control |
|||||
|---|---|---|---|---|---|---|
| Variables | T1 (M) | T2 (M) | Mean difference (CI 95%) | T1 (M) | T2 (M) | Mean difference (CI 95%) |
| Professional competence | 20.4 | 23.3*** | 2.9 (1.74–4.05) | 21.8 | 22.2 | –0.4 (–1.84–0.98) |
| Working alliance | 45.1 | 47.8** | 2.7 (1.22–4.18) | 47.8 | 47.0 | –0.8 (–2.28–0.77) |
| Quality of work supervision | 5.9 | 7.0** | 1.1 (0.37–1.89) | 5.2 | 4.9 | –0.3 (–1.24–0.55) |
| Encountering the user | 14.5 | 15.6** | 1.2 (0.50–1.82) | 15.2 | 15.4 | 0.2 (–0.52–0.98) |
| System-oriented efforts | 14.7 | 15.1 | 0.4 (–0.36–1.17) | 15.4 | 14.9 | –0.6 (–1.32–1.50) |
| Administrative work | 12.01 | 12.5 | 0.4 (–0.27–1.13) | 12.4 | 12.7 | 0.3 (–0.30–0.96) |
Note: * p < 0.05; ** p < 0.01; *** p < 0.001.
The real effects of the skill training programme were determined using analyses of covariance. The statistical models control for experimental condition, T1 working alliance, T1 professional competences and T1 baseline predictor where applicable (Table 5). In order to evaluate the impact of cluster randomisation, we also applied a clustered general linear model. Table 5 reports both standard p-values and p-values adjusted for the clustered data structure. For transparency, Table 6 presents main trends for each cluster.
Table 5.
Effects of the CMPA skill training programme on social workers' competence; analyses of covariance with model-adjusted means
| Variables | Professional competence | Working alliance | Quality of work supervision |
|---|---|---|---|
| Grand mean, T1 (both groups) | 21.5 | 46.4 | 5.5 |
| Intervention group mean, T2 (CI 95%) | 23.6 (22.63–24.60) | 48.3 (47.07–49.55) | 6.9 (6.35–7.55) |
| Control group mean, T2 (CI 95%) | 21.3 (20.09–22.47) | 46.2 (44.74–47.68) | 5.2 (4.46–5.86) |
| Group difference p-value | 0.005 | 0.038 | 0.000 |
| Cluster adjusted p-value | 0.012 | 0.072 | 0.007 |
| Total model adj. R² | 0.13 | 0.30 | 0.29 |
| Variables | Encountering the user | System-oriented efforts | Administrative work |
| Grand mean, T1 (both groups) | 14.8 | 15.0 | 12.4 |
| Intervention group mean, T2 (CI 95%) | 15.8 (15.14–16.37) | 15.3 (14.69–15.85) | 12.6 (12.07–13.16) |
| Control group mean, T2 (CI 95%) | 15.0 (14.28–15.67) | 14.4 (13.65–15.07) | 12.6 (11.90–13.24) |
| Group difference p-value | 0.130 | 0.056 | 0.891 |
| Cluster adjusted p-value | 0.133 | 0.128 | 0.930 |
| Total model adj. R² | 0.36 | 0.27 | 0.11 |
Note: Analyses of covariance with T1 baseline predictor and covariates: experimental condition, T1 professional competence, T1 working alliance. Group means are based on T1 and T2 respondents (n = 61–65) and adjusted for all covariates and T1 baseline predictor. Grand means at T1 are adjusted for covariates. Group differences are reported by standard and adjusted p-values.
Table 6.
Effects of the CMPA skill training programme on social workers: office-level changes
| Cluster | n* | Professional competence | Quality of work supervision | Working alliance | Encountering the user | System-oriented efforts | Administrative work |
|---|---|---|---|---|---|---|---|
| Change | Change | Change | Change | Change | Change | ||
| Experimental | |||||||
| 2 | 4 | + | + | + | + | + | + |
| 4 | 4 | + | + | + | + | + | + |
| 5 | 9 | + | + | + | + | + | 0 |
| 9 | 3 | + | + | + | – | + | 0 |
| 10 | 6 | + | 0 | + | + | + | + |
| 13 | 9 | + | + | 0 | + | – | + |
| 16 | 2 | + | + | – | – | – | + |
| 17 | 4 | + | + | + | + | 0 | 0 |
| 18 | 2 | + | + | + | + | + | + |
| Control | |||||||
| 1 | 6 | 0 | – | 0 | + | + | + |
| 3 | 5 | – | + | – | + | 0 | + |
| 6 | 3 | + | – | + | 0 | 0 | – |
| 7 | 5 | + | 0 | 0 | + | + | 0 |
| 8 | 4 | – | + | – | 0 | – | + |
| 11 | 3 | + | + | + | 0 | 0 | 0 |
| 12 | 3 | – | – | – | – | – | + |
| 14 | 6 | 0 | + | – | – | – | + |
| 15 | 4 | – | – | 0 | 0 | – | 0 |
* T1 and T2 respondents, total n = 82.
All analyses were conducted using SPSS version 20.0. In the planning phase of the project, power analysis was conducted based on the Guittet et al. (2005) model accounting for ICC (intraclass correlation coefficient). Power analysis estimated that, with ten clusters in each arm, 286 clients would be needed per arm with an ICC of 0.005. This study has nine offices in each arm, with a total of ninety-nine social workers and 617 clients (the effects of CMPA on clients' employment outcomes have been analysed in Malmberg-Heimonen (2015) and Malmberg-Heimonen and Tøge (2015)) who were involved in the Qualification Programme. The Norwegian Data Inspectorate and Norwegian Social Science Data Services (case 25275) approved the study design. We have followed their requirements regarding processes of data anonymity and security.
Results
At the follow-up, the social workers from experimental group offices were asked to evaluate their experience with the CMPA skill training programme. The follow-up was conducted eighteen months after T1, approximately one year after finishing the CMPA seminars. Although the assessments varied somewhat between the four evaluated seminars, 92–97 per cent of the social workers reported that the skill training programme had been ‘somewhat useful’ or ‘very useful’ for their daily work in the Qualification Programme.
Of the social workers in the experimental condition, 83 per cent had received CMPA work supervision. The average amount of CMPA supervision received between January and September 2012 was 4.4 (SD = 5.2) sequences per social worker. In addition, 77 per cent of the social workers assessed that the CMPA supervision had been useful for the follow-up of Qualification Programme participants. When comparing the frequency of work supervision during latest month prior to T2, the results show that social workers from the experimental group had received on average 1.13 (SD = 1.27) sequences, while social workers from control group offices had received 0.52 (SD = 1.09) sequences. Although social workers from offices randomised to the experimental group had received work supervision more often, the difference is not statistically significant (p = 0.119).
Table 3 presents the means, standard deviations and bivariate correlations for the background and T1 variables of the study. Generally, the findings indicate fairly high positive correlations between encountering the user, system-oriented efforts, professional competences and working alliance. The duration of work in the Qualification Programme is also positively associated with social workers' assessments of their professional competences and working alliance, indicating that these assessments are generally more positive the longer social workers' experience of the Qualification Programme is. However, the correlation between the administrative part of the work and earlier work experience is negative, although not significant.
Table 4 shows social workers' unadjusted assessments at T1 and T2, presented by experimental condition. Generally, the findings indicate positive changes for social workers in the experimental condition, while no such effects are found for social workers from control group offices. Within the experimental group offices, significant positive changes were found for professional competences (p = 0.000), working alliance (p = 0.001) and quality of work supervision (p = 0.004). Of the three main areas within the programme, it was the area of encounters with users that showed positive changes within experimental group offices (p = 0.001), while there were no changes for system-oriented efforts or the administrative part of the work. For social workers from control group offices, there was a negative trend for several of the measures, including professional competences, working alliance, quality of work supervision and system-oriented efforts.
Table 5 presents the effects of the CMPA programme when controlling for T1 working alliance, T1 professional competences and T1 baseline predictor (where applicable). Findings indicate that the effects of the CMPA programme are related to professional competences (p = 0.002), working alliance (p = 0.016) and quality of work supervision (p = 0.001). Of the three main programme areas, only the measure of system-oriented efforts is significant (p = 0.025), while there is a non-significant but strong trend for the encountering the user measure (p = 0.077). There are no effects for the administrative work, however. Nevertheless, when we account for the clustering, the effects are significant for professional competences (p = 0.012) and quality of work supervision (p = 0.007). Additionally, there is a strong trend on relational alliance (p = 0.072). Thus, the main findings in this study are still valid after adjusting for the clustered data structure (Campbell and Walters, 2014).
Finally, Table 6 presents the trends for the various clusters in the study. For the measure of professional competences, results show a positive development for all clusters in the experimental group, but not in the control group. Of the clusters in the experimental group, eight of nine clusters report a positive development for quality of work supervision received and seven of nine clusters report a positive development for working alliance. Of the main three programme areas, seven of nine clusters report a positive development for encountering the user and six of nine clusters report a positive development for system-oriented efforts. Clusters in the control condition reveal no clear-cut pattern.
Discussion
Using a cluster-randomised design, this study evaluated the effects of a government-administered skill training programme, the CMPA programme, implemented in the Norwegian labour and welfare context. The main finding of the study is that the programme had significant positive effects on social workers' assessments of their professional competences and quality of work supervision received one year after participation in the programme (eighteen months after T1).
Several aspects of the programme may have contributed to the positive effects. The CMPA's learning environment was based on interaction rather than mere transfer of knowledge. This was underscored by the programme's recognition of social workers' existing competences, which may have influenced confidence and motivation to learn. The possibility for pragmatic adjustments facilitated implementation in the local context. These aspects may have contributed to a ‘locus of control’ (Tziner et al., 1991) as regards the learning context beyond the mere improvement of particular knowledge and skills. Further, the social workers received on-site training through supervision that was carried out within the CMPA context. Our findings indicate supervision as an important factor for competence development, as it secured the process of intermediating potential and context-specific competence. This can be interpreted as development of operational expertise, leading to new competence to act (Nygren, 2004). Also, within the context of evidence-based knowledge, on-site training through work supervision and the possibility to adapt the intervention to its context have been demonstrated to be important for the transfer of knowledge into practice (Fixsen et al., 2005).
Scholars have argued that the concepts of evidence-based practice and competence can be problematic within social work. Nevertheless, these findings demonstrate that it was the evidence-based, standardised methods and tools in conjunction with opportunities for critical reflection and pragmatic adjustment to daily practice, under supervision, that improved social workers' professional competence. Seen from this view, the implementation of CMPA can be understood as a practice that is evidence-informed rather than evidence-based (Gambrill, 2007).
Nevertheless, the limitations of the present research should be acknowledged when interpreting the findings. Only eighteen offices were included in the study. A higher number of offices would have increased the statistical power of the analysis. Thus, adjusting the statistical models for the clustered data structure and the main trends presented for each cluster increases the reliability of our findings. It is also important to note that the present results are based on social workers' subjective assessments and it can be questioned what these effects really represent. Although, the strength of this specific study is the cluster-randomised design estimating the effects of the CMPA programme, there is a need to also empirically understand the processes explaining these effects and, more generally, how social workers define key concepts as ‘competence’ and ‘quality’ within their practices. Consequently, data production designed for capturing effectiveness, complexities and processes should be emphasised, as it contributes to a deeper understanding of the development of professional practice within social work.
This study has implications for future social work practice and research. The findings emphasise the importance of high-quality supervision for the development of professional competences, including relational alliance. The results also demonstrate the role evidence-informed practices can have for the development of professional competences. In this study, we only applied quantitative data; however, a mixed-methods approach could further improve our knowledge, as it would contribute a more nuanced understanding related to the process of becoming a professional social worker.
Acknowledgements
This research was funded by the Norwegian Directorate of Labour and Welfare.
References
- Andersen R., Skinnarland S. (2011) ‘Nav-kontoret som lærende organisasjon’ [‘The Nav office as a learning organisation’], in Andreassen T. A., Fossestøl K. (eds), Nav ved et veiskille: Organisasjonsendring som velferdsreform [Nav in a Cross-Road: Organisational Change as a Welfare Reform], Oslo, Gyldendal Akademisk. [Google Scholar]
- Bloom H. S. (2005) ‘Randomizing groups to evaluate place-based programs’, in Bloom H. S. (ed.), Learning More from Social Experiments: Evolving Analytic Approaches, New York, Russell Sage Foundation. [Google Scholar]
- Campbell M. J., Walters S. J. (2014) How to Design, Analyse and Report Cluster Randomised Trials in Medicine and Health Related Research, Chichester, West Sussex, John Wiley & Sons. [Google Scholar]
- Cohen C. S. (2004) ‘Clinical supervision in a learning organization’, in Austin M., Hopkins K. M. (eds), Supervision as Collaboration in the Human Services: Building a Learning Culture, Thousand Oaks, SAGE Publications, pp. 71–84. [Google Scholar]
- Davies H., Nutley S., Walter I. (2008) ‘Why “knowledge transfer” is misconceived for applied social research’, Journal of Health Services Research, 133, pp. 188–190. [DOI] [PubMed] [Google Scholar]
- Dematteo D., Reeves S. (2011) ‘A critical examination of the role of appreciative inquiry within an interprofessional education initiative’, Journal of Interprofessional Care, 253, pp. 203–8. [DOI] [PubMed] [Google Scholar]
- Fixsen D. L., Naoom S. F., Blasé K. A., Friedman R., Wallace F. (2005) Implementation Research: A Synthesis of the Literature, Tampa, FL, University of South Florida. [Google Scholar]
- Gambrill E. (2007) ‘Transparency as the route to evidence-informed professional education’, Research on Social Work Practice, 175, pp. 553–60. [Google Scholar]
- Garavan T., McGuire D. (2001) ‘Competencies and the workplace learning: Some reflections on the rhetoric and the reality’, Journal of Workplace Learning, 134, pp. 144–64. [Google Scholar]
- Gibbons M., Limoges C., Nowotny H., Schwartzman S., Scott P., Trow M. (1994) The New Production of Knowledge, London, Sage Publications. [Google Scholar]
- Gray M., Schubert L. (2012) ‘Sustainable social work: Modelling knowledge production, transfer and evidence-based practice’, International of Social Welfare, 21 2, pp. 203–14. [Google Scholar]
- Guittet L., Giradeau B., Ravaud P. (2005) ‘A priori postulated and real power in randomized trials: Mind the gap’, BMC Medical Research Methodology, 525, pp. 5–25. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hatcher R. L., Gillapsy A. J. (2006) ‘Development and validation of a revised short version of the Working Alliance Inventory’, Psychotherapy Research, 161, pp. 12–25. [Google Scholar]
- Heaven C., Clegg J., Maguire P. (2006) ‘Transfer of communication skills training from workshop to workplace: The impact of clinical supervision’, Patient Education and Counselling, 603, pp. 313–25. [DOI] [PubMed] [Google Scholar]
- Helgøy I., Kildal N., Nilssen E. (2010) Mot en ny yrkesrolle i Nav? [Towards a New Professional Role in Nav], Report No. 1, Bergen, Norway, Uni Rokkansenteret, Stein Rokkan senter for flerfaglige samfunnsstudier. [Google Scholar]
- Horvath A. O., Greenberg L. S. (1989) ‘Development and validation of a revised short version of the working alliance inventory’, Journal of Counselling Psychology, 362, pp. 223–33. [Google Scholar]
- Ivers N. M., Taljaard M., Dixon S., Bennett C., McRae A., Taleban J., Skea Z., Brehaut J. C., Boruch R. F., Eccles M. P., Grimshaw J. M., Weijer C., Zwarenstein M., Donner A. (2011) ‘Impact of CONSORT extension for cluster randomised trials on quality of reporting and study methodology: Review of random sample of 300 trials’, British Medical Journal, 343, d5886. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Johnson M., Austin M. (2006) ‘Evidence-based practice in the social services’, Administration in Social Work, 303, pp. 75–104. [DOI] [PubMed] [Google Scholar]
- Labour and Welfare Administration (2011) CMPA Method Book, 3rd edn, Oslo, Norwegian Labour and Welfare Administration. [Google Scholar]
- Madson M. B., Loignon A., Lane C. (2009) ‘Training in motivational interviewing: A systematic review’, Journal of Substance Abuse Treatment, 361, pp. 101–9. [DOI] [PubMed] [Google Scholar]
- Malmberg-Heimonen I. (2015) ‘Social workers' training evaluated by a cluster-randomized study: Reemployment for welfare recipients?’, Research on Social Work Practice Advance Access published February 8, 2015, 1049731515569357.
- Malmberg-Heimonen I., Tøge A. G (2015) ‘Effects of individualised follow-up on activation programme participants' self-sufficiency’, International Journal of Social Welfare, Advanced Access published June 30, 2015, DOI: 10.1111/ijsw.12179. [Google Scholar]
- Markland D., Ryan R. M., Tobin V. J., Rollnick S. (2005) ‘Motivational interviewing and self-determination theory’, Journal of Social and Clinical Psychology, 246, pp. 811–31. [Google Scholar]
- Messer S. B., Wampold B. E. (2002) ‘Let's face facts: Common factors are more potent than specific therapy ingredients’, Clinical Psychology: Science and Practice, 91, pp. 21–5. [Google Scholar]
- Miller W. R., Yanhe C. E., Moyers T. B., Martinez J., Pirritano P. (2004) ‘A randomized trial of methods to help clinicians learn motivational interviewing’, Journal of Consulting and Clinical Psychology, 726, pp. 1050–62. [DOI] [PubMed] [Google Scholar]
- Natland S., Hansen H. C. (2014) ‘Kompetanseutvikling hos veiledere i kvalifiseringsprogrammet [Development of competence among social workers within the qualification programme]’, Fontene Forskning, 2, pp. 58–71. [Google Scholar]
- Nygren P. (2004) Handlingskompetanse: Om profesjonelle personer [Competence to Act: About Professionals], Oslo, Gyldendal Akademisk. [Google Scholar]
- O'Hagan K. (1996) ‘Social work competence: An historical perspective’, in O'Hagan K. (ed.), Competence in Social Work Practice, London, Jessica Kingsley Publisher, pp. 1–24. [Google Scholar]
- Prochasca J. O., Di Clemente C. C. (1983) ‘Stages and processes of self-change of smoking: Toward an integrative model of change’, Journal of Consulting and Clinical Psychology, 513, pp. 390–5. [DOI] [PubMed] [Google Scholar]
- Sackett D. L., Rosenberg W. M. C., Gray J. A. M., Haynes R. B., Richardson W. S. (1996) ‘Evidence based medicine: What it is and what it isn't’, British Medical Journal, 3127023, pp. 71–2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Tziner A., Haccoun R., Kadish A. (1991) ‘Personal and situational characteristics influencing the effectiveness of transfer of training improvement strategies’, Journal of Occupational Psychology, 642, pp. 67–177. [Google Scholar]
- Watson J. (2011) ‘Resistance is futile? Exploring the potential of motivational interviewing’, Journal of Social Work Practice, 254, pp. 465–9. [Google Scholar]
- Webb S. (2001) ‘Some considerations on the validity of evidence-based practice in social work’, British Journal of Social Work, 311, pp. 57–79. [Google Scholar]
- Wiggan J. (2012) ‘Telling stories of 21st century welfare: The UK coalition government and the neo-liberal discourse of wordlessness and dependency’, Critical Social Policy, 323, pp. 383–405. [Google Scholar]
- Wykes T., Rose D., Williams P., David A. S. (2013) ‘Working alliance and its relation to outcomes in a randomized controlled trial of antipsychotic medication’, BMC Psychiatry, 1328, pp. 1–7. [DOI] [PMC free article] [PubMed] [Google Scholar]