Co-production of an intervention to increase retention of early career nurses: Acceptability and feasibility

Abstract

Co-production is a process employed to solve complex issues, recognising the expertise of all stakeholders. This paper reports on co-production undertaken by nursing students, early career nurses and researchers as part of a larger study to design an intervention to increase retention of early career nurses. Mixed methods were used to evaluate the acceptability and feasibility of the co-production process in a UK university.

Data were collected prospectively, concurrently and retrospectively via interview and questionnaire, between April 2018 and January 2019.

Twelve co-production group members completed the questionnaire and six group members and facilitators were interviewed. Students and early career nurses reported personal benefit from participating; they developed and practised transferrable communication and problem-solving skills, believed they were able to make a difference, enjoyed contributing, found benefit from using the group as a reflective space and considered that co-production produced a credible intervention.

Findings indicated co-production equipped participants to function more effectively in their nursing roles; incorporating co-production into the development of future interventions may prove beneficial. The relative novelty of this approach, and the potential application of the findings to a diverse range of geographical and organisational settings, add to the utility of the findings.

Highlights

• •Co-production is acceptable and effective for designing nurse retention interventions 84.
• •Co-production helped develop skills that were transferrable to the clinical area.
• •Facilitators found co-production subject to competing demands and expectations.
• •Co-production should be considered to promote a credible retention interventions.

1. Introduction

The nursing workforce is a national and global priority, highlighting the importance of identifying reliable ways to encourage nurses to remain in their role and in the profession. The extent and impact of nurse turnover has been discussed extensively in the nursing literature as it affects many countries’ ability to fight disease and improve health. In England, more than 1 in 10 registered nursing posts in the NHS are vacant (Beech et al., 2019), and international research indicates that on average 10% of the nursing workforce seriously considers leaving either their current role or the profession (Heinen et al., 2013). The issue of retention is particularly topical in the wake of the COVID-19 pandemic, which has shone a spotlight on the importance of a sufficient and robust nursing workforce.

Although current nurse workforce issues are particularly acute, this is not a new concern and the international nursing literature is replete with examples of initiatives to reduce turnover and increase retention in high and medium-income economies. A large systematic review (Brook et al., 2018) explored interventions to increase retention and decrease turnover of early career nurses in a range of settings. Whilst a wide range of interventions were reviewed, many studies showed limited and inconsistent benefits. The review highlighted a paucity of evidence relating to the explicit participatory nature of work to design interventions to increase retention of nurses. What was missing from these studies was the voice of students and early career nurses regarding potentially beneficial solutions.

The increasingly popular approach of co-production is one way to access and incorporate those voices. Traditionally developed in relation to service improvement (Osbourne et al., 2016), co-production has been embraced as the cornerstone of public policy reform (Horne and Shirley, 2009) and is now referenced prolifically in grey, policy-orientated literature. In particular, the value of service user involvement in creating health services that are fit for purpose is increasingly recognised in UK policy (Potter et al., 2015) and mirrored in expectations of higher education institution healthcare programme development and delivery (Quality Assurance Agency for Higher Education, 2018; Nursing and Midwifery Council, 2018). The underpinning premise of co-production is that working in partnership results in more relevant, appropriate and sensitive outputs (Latif et al., 2017). Working with nurses and students to develop an intervention that aims to increase retention of nurses is more likely to result in a pragmatic and credible intervention (Henshall et al., 2018), that may also be more effective.

This paper reports on a co-production process undertaken by nursing students, early career nurses and researchers as part of a larger study to design an intervention to increase retention and decrease burnout of early career nurses. Those individuals potentially impacted by attrition from the nursing profession were key to the process, offering greater understanding of the value of co-production in similar future intervention development. The co-production approach used to design the intervention spanned both healthcare and higher education contexts. The relative novelty of this approach, and the potential application of the findings to a diverse range of geographical and organisational settings, add to the utility of the findings.

1.1. Background

There is no consistent definition in the literature but (Durose et al. 2017, P135) describe co-production as ‘joint working between people or groups who have traditionally been separated into categories of user and producer’.

Co-production originated in public service administration, with recognition that drawing on the expertise of a plurality of actors was a practical and reinvigorating solution to complex problems (Sorrentino et al., 2016). Whilst the construct of co-production is continually evolving (Sorrentino et al., 2018), the benefits outlined in the wider literature such as more sustainable change, greater translation of knowledge into practice and closing the theory-practice gap (Durose et al., 2011), are of relevance to clinical and higher education contexts. Although there is an inevitable blurring of boundaries, co-production differs from participatory research, which often seeks to disrupt the traditional power relationships and distinction between researcher and the researched (Higginbottom and Liamputtong, 2017). Co-production focuses on a commitment to working together, recognising the equity of all forms of knowledge, and the social capital of the participants (Voorberg et al., 2015). Participants in co-production recognise that although different parties may require different outcomes from the process, undertaking the journey together will result in a jointly owned product (Hickey, 2018). The shared experience may also leave a more enduring and beneficial legacy than traditional forms of service development.

Whilst co-production is a relatively new phenomenon in health education contexts, it promises to allow students as co-production group members to evolve beyond passive receivers of knowledge. A recognised challenge is the paradigm shift that students must undertake to see themselves as co-producers rather than participants (Vargo and Lush, 2004) and acknowledge the capital they bring in terms of experience and perspective.

Despite the importance that policy makers attach to co-production, a systematic review (Voorberg et al., 2015) identified a dearth of studies that addressed the outcomes and concluded that the process is primarily seen as a virtue in itself, with few external objectives given to legitimise the choice of process. Potential achievable objectives included increased effectiveness, efficiency, and service user satisfaction. Challenges included lack of awareness about co-production, preconceptions about service users’ ability to co-produce, service and time constraints, and developing, managing and utilising sometimes tense partnerships (Holland-Hart et al., 2019). Ultimately, knowledge is scarce about how co-production works in practice, in what context it is best utilised, or whether it is a superior approach to alternatives (Oliver et al., 2019).

This study evaluates the feasibility and acceptability of the co-production process, in order to gain insight into the value and utility of the process of co-production to the stakeholders and project objectives. The co-production group was facilitated by two researchers employed by a UK university and six group meetings were held in the university during spring 2018. The established co-production toolkit, Experience Based Co-design (EBCD), was used to guide the process (Point of Care Foundation, 2016). The toolkit consists of specified exercises and steps, which were implemented as described in Table 1 .

Table 1.

The Co-production process using Experience Based Co-design (EBCD).

Step	Activity
1	An early career nurse and research fellow were recruited to work together to facilitate the co-production group process.
2	Literature and archive film were searched to gain an understanding of what was happening in relation to early career nurse retention.
3	Staff from a partnership NHS organisation and staff from the university were interviewed about their experiences and opinions of early career nurse retention. Interviews were recorded on film, and archive film was collated.
4	The film was edited to incorporate new footage and archive footage, resulting in a 30-min film of themed chapters outlining the issues and evidence base.
5	Early career nurses and student nurses were invited to participate in the co-production group.
6	An early career nurse feedback event was held where the film was shown to stimulate discussion. Areas to focus on for intervention development were discussed.
7	A student feedback event was held where the film was shown to stimulate discussion. Areas to focus on for intervention development were discussed.
8	A joint student-early career nurse event was held to share experiences and agree areas for focus and further discussion to develop the intervention.
9	The co-production group met on 6 occasions over 3 months to develop the intervention, facilitated by the research fellow and early career nurse.
10	Once finalised, the intervention was shared with wider study team for comment and preparation for implementation.

2. Methods

Aim: The aim of the study was to implement and evaluate the co-production of an intervention to increase retention of early career nurses. The focus of the evaluation was to explore the experiences of the co-production group members and facilitators, their perceptions of the acceptability of co-production as a process, and the feasibility and value of co-production within the context of nursing and education.

Design: Using an overall evaluation research methodology, an explanatory sequential design was undertaken using mixed methods to provide a comprehensive view of the co-production process. A questionnaire was used to collect data on acceptability of the co-production process, followed by semi structured interviews with group members and facilitators, and reflective field notes, to explain and add context to the questionnaire data.

Sample/Participants: A purposive sample of all members and facilitators of the co-production group were invited to participate in the study. This included students undertaking adult or child pre-registration nursing programmes at a UK university, who were completing their placements at a specific partner NHS organisation, and early career nurses (within their first year following registration) working at the collaborating NHS organisation. Students were recruited to the co-production group from existing pre-registration nursing programmes and the early career nurses were identified through the preceptorship programme at the NHS organisation. All participants attended at least one co-production group session. The two facilitators were employed by the university as researchers, one was jointly employed by the NHS organisation as an early career nurse.

2.1. Data collection

The mixed methods approach incorporated three data collection methods: questionnaire data collection, semi structured interviews and reflective field notes.

Questionnaire data: Acceptability questionnaires based on the theoretical framework of acceptability (TFA) of healthcare interventions (Sekhon et al., 2017) were developed. The TFA comprises of seven constructs (Fig. 1 ). To evaluate acceptability, questions related to the seven constructs in the model and used a likert-type scale to collect data. Questionnaires were completed between April and June 2018, to explore changes over the duration of the co-production process. Question phrasing was changed for each data collection point to reflect the temporal nature of the process. The complete questionnaire is available as Supplementary Table 1, however, an example of the temporality can be seen in Question 6, relating to the construct of opportunity costs:

Fig. 1.

The Theoretical Framework of Acceptability (Sekhon et al., 2017), adapted to illustrate the seven constructs in relation to co-production.

Time point 1: Participating in this co-production group will interfere with my other priorities.

Time point 2: Participating in this co-production group is interfering with my other priorities.

Time point 3: Participating in this co-production group interfered with my other priorities.

Semi-structured interviews: interviews were conducted between December 2018 and March 2019 and explored experience of the process, including the challenges, benefits, and perceived effectiveness as a method for developing an intervention to increase nurse retention. Interviews with the co-production group members were conducted by telephone by a female academic researcher (JB) independent to the co-production process but integral to the larger project. Interviews with facilitators were conducted face-to-face by another independent female researcher (GL), who had no prior relationship with the facilitators or project.

Reflective field notes: Group facilitators provided reflective notes about their experience of the co-production group sessions, particularly how they felt as facilitators, what went well, what was challenging and how the process could have been improved.

Ethical considerations: Ethical approval was gained from the University Research Ethics Committee (reference: Staff/17–18/18), the Health Research Authority (21.06.18: IRAS ID: 245992) and by the collaborating NHS Trust Research and Development (02.07.18: R&D No: 012400) with respect to research capacity. Specific consideration was given to the voluntary nature of participation, the need for informed consent and respecting the anonymity of the participants.

2.2. Data analysis

Questionnaire data: descriptive statistics were reported as frequency of response choice for each question.

Semi-structured interview data and reflective field notes: thematic analysis followed the six phases outlined by Braun and Clarke (2006). The field notes and interview were integrated during analysis. Reflective field notes, facilitator and group member interview data were given equal weighting during analysis.

Validity and reliability/Rigour: The questionnaire was based on the TFA. The multi-component framework allows identification of the source of an acceptability issue and easier refinement of the co-production process (Sekhon et al., 2017). The questions were tested for both content and construct validity and amended to improve clarity and validity. Social desirability bias was mitigated through self-administration of the questionnaire and reassurance that answers would be kept confidential.

The interview guide was formulated following analysis of questionnaire data and decisions about which aspects required further explanation. One interviewer was a nurse (JB) and therefore had insight into the professional context of the group members, and one (GL) was an experienced social researcher and educator. Although there are some limitations to telephone interviews, such as not being able to read visual cues through body language, they were used as a pragmatic solution to contacting the geographically dispersed group members. The research team were reassured that the interviews elicited rich descriptions of the participants’ experiences.

Quantitative data analysis was conducted using SPSS version 25. Qualitative data analysis was conducted by two researchers independent to the co-production group. Each researcher analysed the qualitative data independently, themes and ideas were discussed, agreed and presented to one early career nurse from the co-production group for comment, who attended a steering group meeting for the wider project, and who agreed that the findings represented their experience of the process.

3. Results/findings

The co-production group consisted of 12 members, two male and 10 female. Two members were early career nurses and 10 were student nurses; 2 in the first year of their programme, 1 in the second year and 7 in the final year. The co-production group met 6 times with an average attendance of 5 members (maximum 8, minimum 3). First and second year students attended only the first meeting.

3.1. Acceptability questionnaires

Twenty-three acceptability questionnaires were completed, 12 at the beginning, 6 at mid-point and 5 at end point (Table 2 ).

Table 2.

Co-production acceptability questionnaire results by time point.

	N (%)
	Strongly disagree			Disagree			No opinion			Agree			Strongly agree
Question and TFA constructa	Start	Mid-point	End	Start	Mid-point	End	Start	Mid-point	End	Start	Mid-point	End	Start	Mid-point	End
1) I feel it is acceptable for me to take part in the co-production process (general acceptability)	0	0	0	0	0	0	0	0	0	2 (17%)	0	0	10 (83%)	6 (100%)	5 (100%)
2) I am enjoying taking part in the co-production process (affective attitude)	0	0	0	0	0	0	0	0	0	4 (33%)	1 (17%)	0	1 (17%)	5 (83%)	5 (100%)
3) It is taking effort for me to take part in the co-production process (Burden) (disagree = less effort)	0	0	2 (40%)	3 (27%)	1 (17%)	3 (60%)	4 (37%)	0	0	2 (18%)	3 (50%)	0	2 (18%)	2 (33%)	0
4) The co-production process is an effective way to design this intervention (perceived effectiveness)	0	0	0	0	0	0	0	0	0	4 (33%)	0	1 (20%)	8 (67%)	6 (100%)	4 (80%)
5) I value the co-production process as a way of designing an intervention (ethicality)	0	0	0	0	0	0	1 (8%)	0	0	6 (50%)	0	0	5 (42%)	6 (100%)	5 (100%)
6) Participating in this co-production group is interfering with my other priorities (opportunity costs) (disagree = less interference)	2 (17%)	1 (17%)	1 (20%)	8 (67%)	4 (67%)	4 (80%)	1 (8%)	0	0	0	0	0	1 (85%)	1 (17%)	0
7) I am confident that I am able to contribute to the co-production process (self-efficacy)	0	0	0	0	0	0	0	0	0	7 (58%)	2 (33%)	0	5 (42%)	4 (67%)	5 (100%)
8) I understand the co-production process and how it works (intervention coherence)	0	0	0	0	0	0	0	0	0	(58%)	2 (33%)	0	5 (42%)	4 (67%)	5 (100%)

^aPhrasing of questions was adapted as appropriate for the time point.

Participants were generally positive about the co-production process. Greatest change over time related to the constructs of burden (Q3) and opportunity costs (Q6) with respondents anticipating that participating in the co-production group would take effort and would interfere with other priorities. By the final session, participants indicated they disagreed or strongly disagreed that co-production required effort or interfered with their other priorities.

Participants increased their agreement over time that the co-production process was enjoyable (Q2), an effective way to develop the intervention (Q4), a process they personally valued (Q5), they were confident they could contribute (Q7), and they understood the process (Q8). The semi-structured interviews were used to explore the acceptability in greater depth, particularly around the constructs of affective attitude, burden and effectiveness of co-production.

3.2. Semi-structured interviews

Telephone interviews were carried out with 4 co-production group members; 1 early career nurse and 3 final-year students, who had qualified as nurses subsequent to the co-production group meetings. Face-to-face interviews were conducted with the group facilitators.

Thematic analysis of the interviews and the reflective field notes of the facilitators identified two overarching themes: co-production as a framework or philosophy; and the added value of co-production. The relationship between themes and subthemes is illustrated in Fig. 2 .

Fig. 2.

Themes and sub-themes identified from the interview and field notes data.

3.3. Co-production as a framework or philosophy

Data relating to co-production as a framework or philosophy included three subthemes: the effectiveness of co-production, being heard and being valued, and the influence of the higher education setting.

The effectiveness of co-production: facilitators and group members were unanimous in their belief in co-production as an effective method of designing an intervention. Participants saw the strength of co-production as the integrity of the end product, derived from the ideas and experiences of the group:

‘Yeah, definitely because it keeps it realistic and you're able to actually gather real opinions in order to help other people because then we've been through it and so we understand so we're able to … form a realistic intervention.’ (Participant 2)

The facilitators highlighted that co-production encouraged group members to be proactive; they were stimulated to think in a different way to develop the intervention. One group member suggested that co-production could be more effective if decision makers from partner organisations were also present as group members:

‘I think if you were to bring in people with a bit more of a status in the degree programme itself, the students will be thinking, you know what this is the opportunity. They are going to be sitting across the table from me, let me just shoot them a couple of questions.’ (Participant 1)

Group members and facilitators both stated that they would engage with the process again and would consider incorporating a co-production element into future research, although facilitators recognised it as a resource intensive process that may not be feasible in every research project.

Being heard and being valued: the facilitators were aware of the potential power differences between themselves as researchers, the qualified nurses and the students. Attempting to flatten this hierarchy, they integrated into the group:

‘And I just felt like one of them if that makes sense which is, the main point of the co-production is that we're on the same level and there isn't any hierarchy. And I really did feel I wanted to really have a bit of a gossip with them because that was because I'd experienced it as well.’ (Facilitator 1)

To a large extent this was effective, with participants reporting that they were able to talk freely, that their ideas or suggestions were welcomed and that their voices were being heard:

‘They facilitated the environment where everyone … felt comfortable opening their mouth and giving their opinion regardless of what it was. Even if it was disagreeing with something someone else might have said … We're all adults and I'm sure we can have a good conversation about something without getting too heated.’ (Participant 1)

Only one group member reported that they felt their opinions were not valued by the facilitators and stopped attending the group meetings. This group member was experiencing significant problems in clinical placement and felt it would be useful to share with the group, however the facilitators, conscious of time pressure, guided the conversation to areas they felt were more relevant to the project:

‘So, my voice was being, yeah, it was being heard but when it was not being valued was when I was a bit frustrated, but yeah it was being heard.’ (Participant 4)

‘Eventually and after treading carefully so as not to upset anyone, we offered that we would hang around after the session to have a moan about placements but if we could focus on solutions and interventions for now as we didn't want to run over time.’ (Field notes 28.06.18)

The facilitators acknowledged the process would have been very different had they not felt the need to focus on outputs, and this brought into sharp focus conversations that were perceived as not solution focused, inefficient, or going off topic. Not being in a position to ‘fix’ the issues in the way they would in their roles as academic or nurse, was one of the most difficult aspects of co-production.

The influence of the higher education setting: the facilitators attempted to minimise the impact of the higher education context on the process but acknowledged that as the meetings took place in a university, inevitably, the students associated them with learning. Both facilitators were tempted to resort to established techniques when working with the group members and language closely related to academic roles rather than co-production was evident in their interviews:

‘That's something that I was quite aware of, is trying not to be too lecturey, it's really, really hard though because … I'm used to the structure … I'm used to being in charge, I'm used to talking a lot.’ (Facilitator 2)

By the final group meeting the facilitators found a balance between their academic style and the philosophy underpinning co-production, encouraging group members to leave their desks, accepting the free-flow nature of the discussions and abandoning any attempts to encourage written exercises.

3.4. The added value of co-production

Participants felt there were unanticipated benefits to participating in a co-production group, both for the facilitators and the group members. These benefits were founded in their experience, specifically being part of a group and making a difference, developing transferrable skills, and enhanced health and wellbeing. This led to the conclusion that co-production was a salutogenic model, an approach that supports the relationship between health, stress and coping.

Making a difference: this was key to both facilitator and group member experience, as three of the group members also identified that helping the next generation of nurses was both a motivation to take part and a positive outcome of co-production:

‘I felt like that was a chance for us to speak out about some things that might be, that could help other people, other students, not go through the same thing, if that makes sense?’ (Participant 2)

The co-production meetings were mutually beneficial; students learned from early career nurses and early career nurses enjoyed the opportunity to reconnect with the students. Attending the group meetings was not perceived to be onerous, but were positive and enjoyable; participants cited interest in hearing other people's opinions and the outcome of the intervention implementation.

Transferrable skills: both facilitators and group members reported that they had developed new skills during the co-production process. Facilitators enhanced their group work skills, gaining confidence in working closely with students, and facilitating co-production:

‘I think actually running the coproduction groups helped with the listening, not interjecting, going with the flow, it's OK if it doesn't go according to plan, which it hasn't done many times, that sort of thing I think has been helpful from it.’ (Facilitator 2)

The group members identified that they had developed skills that they could use in their clinical roles, such as problem solving or thinking outside the box. They appreciated others’ opinions and the value of sharing ideas to reach a consensus or solution. In particular several group members identified enhanced communication skills allowing them to relate more effectively to patients:

‘Storming, just considering everybody's ideas. Although we use the same language, we use it differently to articulate ourselves and sometimes it's lost in translation and sometimes you just have to hear with a different ear … it's not just about hearing what your patient's saying, it's about hearing what they're not saying as well.’ (Participant 4)

Salutogenesis: comments from participants indicated that the experience of participating in a co-production process may in itself support psychological health and wellbeing, which is the essence of the concept of salutogenesis. They identified the value of sharing issues from clinical placement, validating their feelings and being reassured they were not alone with their experiences. The opportunity to voice their concerns and hear them interpreted from alternative perspectives was cathartic. One group member felt strongly that sharing personal challenges in clinical placement was central to the group's remit:

‘It was just I was going through personal issues within my placement, like I said, and I thought … if I voiced them in the coproduction, it could have been something that can help other people.’ (Participant 4)

The facilitators were challenged by the use of the group as a therapeutic space, acutely feeling the tension between the welfare of the group members and the remit of the project. They reported regularly guiding the focus of the group back to the narrow parameters of the terms of reference, possibly in juxtaposition to the collaborative nature of co-production:

‘But that was probably the worst part was saying, no we can't do that sorry … yeah, that was horrible to do because you know that they want to talk about some things and sometimes they just want to rant about certain things.’ (Facilitator 1)

The benefit to the group members of being able to discuss their current issues with empathetic and compassionately critical colleagues was not lost on the facilitators and the difficulty associated with interjecting was the issue raised most frequently by both of them.

3.5. How the qualitative data explained the questionnaire data

The questionnaire data indicated that co-production was a generally acceptable process. Perceptions changed positively over time about effort, enjoyment and opportunity costs related to co-production. As understanding about the process increased, so did the confidence of the participants that they could contribute and the process was effective.

The qualitative data was a rich source of insight into the influences at each of the temporal assessments of acceptability. The perception that the process was effective developed in parallel with participants recognising their developing skills and appreciating the opportunity to help future generations of nurses. Those interviewed explained how they came to understand the co-production process, value their own and others' contributions over time, and took the opportunity to gain personally from the experience, which impacted on their perception of opportunity costs. Their comments identified that they enjoyed hearing other's experiences and this increased their self-efficacy as they gained confidence with their own practice. The interview data also gave context to the challenges the facilitators faced trying to balance the needs of the group members with the needs of the project, how this may not always have been possible or desirable but ultimately influenced the experience of the group members.

4. Discussion

Engaging service users, students or public in the design of services or education intuitively has value, but co-production has been described as weakly conceptualised with little consensus about why we do it, what effects are being sought, or achieved (Oliver et al., 2019). This argument is compounded by the longstanding debate about the nature of knowledge and expertise (Durose et al., 2011). This study was designed to explore the experiences and perceptions of co-production group members with regard to acceptability and feasibility of the co-production process. The findings raised key points for discussion: first, the co-production process was generally acceptable and group members were unanimous in the opinion that they would like to participate in co-production again. Second, co-production in a higher education setting presented challenges related to group members’ perceptions and relationships that impacted on the feasibility of co-production to design an intervention. Third, co-production has the potential to offer unanticipated added value beyond that of the design of an intervention, both for group members and facilitators and for the integrity of the output.

Evaluation of acceptability identified practical challenges to participating in the co-production process. Resources and time restraints are recognised barriers to co-production (Holland-Hart et al., 2019; Oliver et al., 2019) and all data sets revealed concern about competing priorities. Once group members understood the concept of coproduction, the level of burden was perceived to be more acceptable and group members found ways to overcome the practical barriers to engagement. This suggests that clearer explanation, with examples of similar processes, at an earlier point would enhance the process.

The academic setting for meetings may have been an additional constraint to achieving the full potential of the process. The high value placed on academic knowledge influences both the degree of control taken by academics in collaborative partnerships and the likelihood of challenge to that decision-making offered by other stakeholders (McCabe et al., 2016), exacerbated in this case by the traditional student-academic hierarchy. Potentially, respectful relationships masked the limitations this imposed. Incongruence between the social boundaries of academic/researcher/facilitator, early career nurse and student in an academic institution may in reality prevent the blurring of boundaries that is espoused as key to the co-production process (Kirkegaard and Anderson, 2018). The distribution of knowledge and expertise across organisations and the significance of context in co-production is key (Ledger and Slade, 2015).

The facilitators struggled with the tension between allowing the group members time to benefit from the process and meeting the aim of the project. Co-production is not without cost; it is emotionally demanding and subject to competing demands and expectations. The skills required to manage co-production group dynamics may differ from those traditionally required or rewarded by higher education institutions (Flinders et al., 2016). Ideally, co-production offers equality and lack of hierarchy but in reality the demands of different stakeholders impact on the ability of the facilitators to value all contributions equally or to invest time in relationships and conversations that may not have guaranteed output (Oliver et al., 2019). These findings indicate that the pressure to design an intervention weighed heavily on the facilitators, potentially exacerbated by their awareness of and engagement in the evaluation of the co-production process. They juggled ethical and political complexity and tried to design an intervention that was ‘right’ for all stakeholders but found it impossible to utilise all the ideas from the group. In one instance this resulted in a group member feeling undervalued and unrepresented, a recognised potential cost to co-production (Boaz et al., 2018).

One explanation for the enthusiastic response from those interviewed may be their perceived personal gain from the process; an increased sense of self-efficacy and self-worth because at the end of the process they believed they were able to make a difference, help future generations of nurses, and develop communication and problem-solving skills useful for clinical practice. Crucially, they also understood the capital that they brought to co-production. The salutogenic nature of co-production is echoed in studies considering the benefits for service user involvement in participatory nursing research, with key contributory factors being the leadership of the process, clarification about their role, the structure of the meetings and being a member of a team (Mjøsund et al., 2018). The appreciation group members had of the facilitators’ styles justifies their commitment to respectful engagement (Flicker and Nixon, 2015) and genuine respect for group member opinions (Simpson et al., 2014). Given that the co-production process was implemented to design an intervention that would support newly qualified nurses, the benefit of using a salutogenic approach, maximising the potential of the relationship between health, stress and coping, was not lost. This approach has potential to be of particular value during periods of increased demand such as the current COVID-19 pandemic. In England, nursing student experience has been impacted considerably by their early incorporation into the nursing workforce and will influence their experiences as newly qualified nurses.

4.1. Strengths and limitations

The strength of this study lies in the novel approach to co-production, spanning boundaries between nursing practice and higher education. To our knowledge this is the first use of co-production to design this type of intervention making the findings relevant to a range of settings. The study represented all participant voices, offering a comprehensive perspective.

The mixed methods, explanatory sequential design further strengthens the credibility of the findings as the interview data contextualises the questionnaire data. We saw from the questionnaire data that the process became more acceptable to group members over time but that attendance was challenging. The use of interviews allowed us to triangulate with the questionnaire data and explore the context, which added significant value to the evaluation of the process. Providing opportunities for the facilitators and group members to reflect both during and after the process aligned with the principles of co-production, which advocate hearing and valuing multiple voices.

A potential limitation to the co-production process and the evaluation is the small number of participants. Although all group members attending the initial, mid-point and final group sessions completed the acceptability questionnaire, fewer attended the final group meeting. Those who found participation difficult may have decided not to continue to attend and may have had different views of acceptability of the co-production process than those who did attend, however, the interview data helped to explain the context of the lower attendance.

Ultimately, we were interested in the potential of co-production as an effective process at the interface between clinical and higher education settings. Our findings were limited by the scope of the study, as we are unable to report on the legacy of participants’ experience in relation to their professional practice. We would recommend further research to explore this area.

5. Conclusion

Co-production is a technique that attempts to include individuals not traditionally seen as ‘experts’ in the development of services or products. Our evaluation of this co-production process highlighted the potential for greater use of the strategy in the context of higher education and clinical nursing practice. Not only was an intervention designed using co-production, but the individual nurses and students who participated gained personally from the process and saw the end product as more credible. They developed and practised communication and problem-solving skills that transferred effectively to the clinical environment. They recognised and enjoyed being able to make a difference and found benefit from using the group as a reflective space. This added value of co-production has equipped and motivated the group members to function more effectively in their nursing roles and is a persuasive rationale for incorporating co-production into the development of future interventions.

The findings indicate that higher education institutes should work with clinical partners to develop capacity, knowledge and skills that would enable facilitation of co-production, including active listening, emotional literacy and understanding of power dynamics and strategies to equalise power relationships. This would acknowledge the unique advantages of this inclusive and effective process for the participants, for the end product, and for the enhancement of understanding and collaboration between the two organisations. Co-production has particular utility in relation to retention of early career nurses, as the findings indicate that inclusion of those most likely to leave the profession or their role will in itself engender a more positive perspective towards their professional activity. In the UK nursing context, commitment of early career nurses to the profession has potentially been impacted by student nurse experience during the emergency period, so it is imperative that this generation of nurses have the opportunity to work collaboratively to develop a solution.

CRediT authorship contribution statement

Judy Brook: Conceptualization, Methodology, Validation, Formal analysis, Investigation, Writing - original draft, Writing - review & editing, Visualization, Project administration, Funding acquisition. Leanne Aitken: Conceptualization, Methodology, Validation, Writing - review & editing, Funding acquisition, Supervision. Dr Julie MacLaren: Writing - review & editing, Funding acquisition, Supervision. Debra Salmon: Conceptualization, Methodology, Validation, Writing - review & editing, Funding acquisition, Supervision.

Declarations of competing interest

None.

Funding statement

This study was funded by the Burdett Trust for Nursing [Grant number: SB\ZA\101010662\253815].

Ethical approval details

Ethical approval was gained from the University Research Ethics Committee (reference: Staff/17–18/18), the Health Research Authority (21.06.18: IRAS ID: 245992) and by the collaborating NHS Trust Research and Development (02.07.18: R&D No: 012400) with respect to research capacity.

Appendix A. Supplementary data

The following are the Supplementary data to this article: