Increasing credibility in government assignments: an example from Sweden of stakeholder involvement by using Concept Mapping

Abstract

Background

The incidence of people diagnosed with cancer is increasing worldwide, in Sweden with approximately 65 000 new cases yearly. To enhance conditions for a continued good life, high-quality, accessible and equal cancer rehabilitation is required. In 2020 the National Board of Health and Welfare (NBHW) received a government assignment to investigate the conditions for the offer of cancer rehabilitation. This paper aims to highlight how the Concept Mapping (CM) methodology was strategically used by the NBHW for mutual learning to raise different perspectives and ‘voices’ of cancer rehabilitation.

Methods

The stepwise, mixed-method CM was used. A broad selection of relevant stakeholders was invited to complete the prompt: a good and equal cancer rehabilitation requires… the stakeholders were involved in brainstorming, sorting and grading data. The research group performed multi-dimensional scaling and hierarchical cluster analysis to provide a visual presentation of the results. Two digital seminars were held: one where stakeholders did online sorting and grading of data and one where results were presented and discussed.

Results

118 respondents from four stakeholder groups provided 489 original ideas. Six clusters from 67 unique ideas constituted the concept map, and a go-zone showing the 12 ideas rated as most important and feasible was obtained.

Conclusion

For the first time, the NBHW has used CM in investigative work to produce development proposals and improvements that the government can use as a basis for political decisions. The method showed great potential, by allowing for stakeholder involvement as co-creators in all steps, high participation and possible direct utilisation of results. A clear learning perspective was obtained, both from the NBHW and the involved stakeholders; hence, the method can be used in further assignments where improvement perspectives are requested.

WHAT IS ALREADY KNOWN ON THIS TOPIC

• Cancer diagnoses are rising globally, but with improved treatment, more people survive or live longer.

• To provide better conditions for a continued good life, high-quality, accessible and equal cancer rehabilitation is required.

WHAT THIS STUDY ADDS

• This study shows how the Concept Mapping (CM) methodology was strategically and successfully used by a Swedish authority for mutual learning to raise different perspectives and ‘voices’ of cancer rehabilitation.

• The study shows the effectiveness of CM in involving stakeholders in improving cancer rehabilitation services, identifying six key clusters and 12 actionable items, directly converted into a large engagement through the interactive workshops.

HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY

• The findings can guide future research in stakeholder involvement by authorities, improve patient-centred cancer rehabilitation practices and inform policies for equitable cancer rehabilitation access, using CM as a model for comprehensive feedback collection.

• Increasing credibility in government assignments—an example from Sweden of stakeholder involvement by using CM.

Background

In Sweden, approximately 65 000 people are diagnosed with cancer every year, with incidence expected to increase.¹ With constantly improving treatment methods, more people will also live with the disease for a long time or even be considered as chronic.² Everyday life for the individual coping with side effects from the treatment, as well as for people close to them, will be affected and/or altered.³ To provide better conditions for a continued good life, high-quality, accessible and equal cancer rehabilitation (CR) is therefore required, something that in Sweden is stated in legislation⁴ and regulated in national care plans. Cancer rehabilitation involves physical, social, psychological and existential aspects to help the person to return to life as before or to achieve the best possible quality of life (QoL).³ However, scientific evidence for the cost-effectiveness of CR is sparse and needs scrutinising.⁵ Further, despite a lot of knowledge in the field about what needs to be done, moving from knowledge to implementation has been proven more difficult. Especially when, as in CR, many actors and activities are involved, unwanted variation around patient access to rehabilitative interventions can arise, causing inequity within cancer care processes. However, CR measures³ make it possible to prevent other types of care needs, improve the outcome of cancer treatment and help people return to or enter the labour market. In 2020 the Swedish National Board of Health and Welfare (NBHW) received a government assignment to investigate the conditions for the offer of CR and to propose measures to strengthen the development and to create a better reach for patients and next of kin to take part in CR activities. Concept Mapping (CM) methodology⁶ was chosen as part of the assignment, since it has the potential to involve many concerned actors in both data generation and interpretation, as well as gathering data over large distances with relatively little effort from participants. Further, the possibility for the results to be instantly used for joint learning and identification of development areas as well as proposals for actionability seemed favourable to other methods, more commonly used by the authority.

The aim of this paper is to highlight how CM was strategically used by the NBHW for mutual learning to raise different perspectives and ‘voices’ of CR. Further, we wished to evaluate the future possibilities of using the CM method in upcoming authority assignments.

Methods

Ethics approval and consent to participate

This study did not involve the collection, processing or handling of sensitive or personal data. Thereby, it falls outside the scope of the Swedish Ethical Review Act (Etikprövningslagen 2003:460) and does not require ethical approval. All data was anonymised, and participants were already divided into the four groups described below before being asked to participate. The consent that was obtained from all the participants was informed, and all results are presented groupwise. CM is not a traditional survey, and only one prompt was used as an incomplete sentence to complete 2–5 times. Two control questions were asked: (1)on what level do you work with cancer rehabilitation? (Nationally, regionally, locally, digitally or cannot specify). (2) In what way do you have experience in cancer rehabilitation work? (I work/have worked clinically in patient care, I work/have worked with strategic development in healthcare, I work/have worked with development/improvement from a patient perspective, I work/have worked in the management/control of care). Further, all participants were told not to add any personal information, and it was specified what kind of information that would entail. For the following workshops, participants were asked to accept an invitation, and if they agreed, they were invited by e-mail. During the workshops, participants were informed that all data was anonymised and could not be traced back to them and agreed to participate by opening and completing the links for the sorting and rating tasks described below. Data generated in this study are available from the corresponding author on request.

Stepwise data collection and analysis: CM methodology

The NBHW assigned an expert group with specific knowledge about the subject to assist with the investigation. Additionally, a research group was formed with investigators from NBHW and method experts from the Regional Cancer Centre West and a Technical University. This group designed and performed the analysis and oversaw the part of the investigation described in this paper.

Patient and public involvement

To enable a broad selection of stakeholders’ voices to be heard (see below), we used the established method CM.^{6 7} Previously used for similar dissemination and implementation of evidence-based health programmes⁸ and policy documents,⁹ the method provides a clear visual map, created through both qualitative and quantitative analyses. CM is a stepwise, data-generating methodology and analysis, where stakeholders are involved as co-creators in all steps (see figure 1). Instead of asking questions with predefined answers, the stakeholders are asked to complete a sentence, a so-called prompt, from their own knowledge and experience. Each completed sentence is referred to as a suggestion, item or idea. For the government assignment, we chose the prompt: a good and equal cancer rehabilitation requires…

Figure 1. Flowchart of the process.

Using an open prompt allows suggestions that cover all possible angles. Originality rather than quantity is prioritised when deciding which items are included in the analysis, making a unique idea just as important as something mentioned by many. Further, the analysis is automatically validated by the stakeholders concerned. The results can be presented, discussed and used in dialogue with stakeholders from the visual concept map that shows identified clusters (figure 2), and if ideas are rated with respect to importance or feasibility, a go-zone showing those ideas identified as most prominent to address can be produced (figure 3). For this project, after being asked to complete a prompt 2–5 times through an online survey sent by e-mail, we invited the stakeholders to participate in two digital workshops, where an individual sorting and rating task took place. This enabled further co-creative learning for both the research group and the stakeholders.

Figure 2. Concept map with six cluster solution and percentage of original ideas. The boundary of cluster two has been adjusted for visual clarity.

Figure 3. Go-zone with the 12 highest scoring items for importance and feasibility. (Prompt to gather the ideas was ‘A good and equal cancer rehabilitation requires….’) Link text: Zoom for better visibility.

As the government assignment stipulated that a learning perspective should be included, the recruitment of participating stakeholders was crucial. Since it was a national investigation, reaching out to the whole country with its 21 regions, and CR taking place in various levels of organisation, providing a sample that was as representative as possible was a challenge. As mentioned above, trying to capture the perspectives of relevant stakeholders, in this case actors working professionally, or in an organised way with improving the availability of CR, we identified four ‘voices’ we wanted to be heard:

• Group 1: the organised patients’ voice (patients’ voice from patient organisations, networks, patients’ councils, support centres and support groups).

• Group 2: the professional voice (professionals from different levels working daily with cancer and medical rehabilitation, for example, nurses, physiotherapists, dietitians, social workers and staff from oncology units).

• Group 3: the voice of developers of knowledge (units working strategically with cancer rehabilitation, professional networks, development leaders and researchers).

• Group 4: the voice from management and quality assurance (politicians and managers responsible for the implementation and evaluation of cancer interventions).

The prompt was addressed from these four perspectives in order to provide a comprehensive view of what the prerequisites for providing a good and equal CR in Sweden are considered to be. The four voices were chosen for their strategic impact from different perspectives with the intention of also enabling learning between these voices as a part of the project. The vitally important voice of patients during or after treatment was addressed elsewhere in the assignment and not included in this project other than through group 1. Possible participants from each group were strategically selected, using the extensive network of the expert group assigned by NBHW, with additional snowball sampling.¹⁰ Great care was taken in making the sample as representative as possible, ie, ensuring a spread of stakeholders from the four groups as well as providing a national, regional and local perspective. The process is shown in figure 1 as a flowchart.

After the identification of possible participants, an e-mail was sent from NBHW with information about the project and an invitation to participate in the first step: the original brainstorming. From this, 478 ideas were gathered via an online survey link from an in-house NBHW tool. The stakeholders were also asked to take part in the two digital workshops, where other steps of analysis would take place. Next, the research group qualitatively cleaned the data and identified which of the 478 collected ideas could be seen as unique: preferably not more than 100 ideas.⁶ For this project, we ended up with 67 unique ideas. These de-identified ideas were then entered in software for the specific purpose of sorting and providing data for statistical analysis.¹¹

During the first digital workshop, stakeholders were given information about the project and progress so far. Each stakeholder was then provided with a link to the sorting tool¹¹ and given 1 hour to perform the sorting task, which involved digitally sorting the 67 items into groups as perceived by the individual and to suggest names for the groups they created. Each sorter decided for themself how many groups to create. Second, participants were provided with a new link where they were asked to rate the 67 items for importance and feasibility on a seven-grade scale (1-7). Some of the participants only completed the rating task (n=41), but not the sorting (n=34) due to practical work issues (see figure 1). All information about the 67 ideas can be found in an online supplemental file.

Between the two workshops, the results were analysed statistically using R software.¹² The sorting data was structured in a way that, for each pair of ideas, shows the number of times these ideas were sorted into the same group. Multidimensional scaling (MDS)⁶ was then used to convert the results to coordinates in two-dimensional space so that the distance between two ideas is small if they are frequently grouped together and large if they are rarely placed in the same group. Next, hierarchical cluster analysis (HCA)⁶ was used to separate the ideas into thematically similar groups or clusters. HCA results in a set of as many cluster solutions as there are ideas, that is, everything from a solution with one cluster (all ideas in the same group) to a 67-cluster solution (each idea in an individual cluster). The research team scrutinised several possible consecutive cluster solutions until agreeing on the one with most coherent content for each cluster, that is, where the ideas grouped together are not thematically very different. In a final quantitative step, the rating data was compiled, averaging importance and feasibility scores per idea for all raters overall as well as by the rating group (perspective). The importance and feasibility limits were defined as the 70th percentile in the respective dimension and the go-zone as all ideas with an average rating exceeding both the importance and feasibility limits. In this case, a concept map with six clusters was identified and a go-zone with 12 actionable items emerged from the ratings (figures2 3). This go-zone was seen as a collectively decided point of departure for further work.

For the second digital workshop, the stakeholders who, with one exception, had all been part of the first workshop were first presented with the final concept map and the go-zone (figures2 3). Additionally, they were shown which items ended up in a common go-zone and how the different perspectives/‘voices’ had rated the items (figures3 4). A discussion with reflection on the results took place. Second, the stakeholders were divided into four groups, mixed from the four original perspectives/‘voices’. Each group had a designated facilitator and a person taking notes and was given three of the items from the go-zone to discuss from two perspectives: who ‘owns’ the item and what can be done to address the item? Later, the four groups reconvened and shared their thoughts with the rest of the participants and the facilitators, reaching consensus on what actions were needed and by whom.

Figure 4. Group comparison for the go-zone for importance and feasibility.

As a last step, all original items were qualitatively assigned to the six clusters. This is not a part of the original CM process but has been done previously^{13 14} and was requested by the NBHW to add a dimension of frequency to the clusters (see figure 2). Finally, the results from the CM process were used to provide information to the report to the Swedish government, where suggestions were made on how to proceed with the work to create a better and more equal CR in Sweden.¹⁵ Altogether, the process took 4 months.

Results

From the stepwise data collection and analysis, a six-cluster concept map and a 12-item go-zone list were identified, visualised and presented in two digital workshops with stakeholders (figures2 3). The complete process is presented under the Methods section. The results can be seen as a collective compilation of the actors concerned in the development and implementation of CR in Sweden. Furthermore, it is possible to compare the results from both a meta-perspective as well as between the stakeholder groups to learn more. A synthesis of the clusters (figure 2) and the discussion from the workshops follows below:

Coordinated support and planning

This cluster emphasises the role of the contact nurse in supporting both the patient and their relatives. The importance of using validated instruments for needs assessment—resulting in a care plan developed in collaboration with the patient—is highlighted. It is evident that both medical and psychosocial needs should be addressed concurrently throughout the entire care continuum: before, during and after treatment, whether with curative or palliative intent. Clear delineation of responsibilities—knowing who does what and when—is essential, without placing excessive burden on the patient or their family.

Availability

This cluster addresses various aspects of availability. Digitalisation is seen as a facilitator for self-assessment, group-based training and follow-up of implemented interventions through interaction with healthcare systems. Research and development activities may also benefit from these advances. Information should be available in multiple languages, and socio-economic disparities must not affect access to services. Centralising expertise through designated centres and offering rehabilitation in residential settings are considered important. The flexibility of sick leave and insurance systems to accommodate varying needs is another relevant dimension of accessibility.

Coordination and structure

This cluster contains ideas related to the organisation of cancer rehabilitation services. There is a clear demand for defined leadership responsibilities, greater competence and system understanding. This includes the development of routines for coordination, tiered service levels and action plans tailored to the different levels and actors involved. Multidisciplinary approaches, established models and enhanced team-based collaboration are seen as prerequisites for effective communication between primary and specialised care.

Resources

Several types of resources are identified as necessary to meet national requirements, policies and goals. Adequate financial and human resources—as well as appropriate facilities—are essential. These resources should be available at multiple levels and ideally supported through national consensus or even state subsidies. The presence of diverse professional roles within rehabilitation teams is also emphasised as crucial.

Individual adaptation

The focus of this cluster is on helping individuals and their families manage their situations based on their unique circumstances. Early rehabilitation interventions are underscored as important, along with evidence-based, tailored information about treatment, side effects, sexual health and physical activity. The significance of peer support—particularly for young adults—and evaluating interventions from the patient’s perspective are also central themes.

Development of knowledge and long-term vision

This cluster calls for evidence-based, clearly defined and structured working methods integrated into diagnosis-specific care pathways. Knowledge development is expected to contribute to a long-term perspective on issues such as physical activity and web-based group rehabilitation. The involvement of former cancer patients in care development alongside professionals and training for employers on how to support employees affected by cancer are examples of integrative proposals that connect to several other clusters.

Meta-clusters

A closer analysis of the six clusters of the concept map, as shown in figure 2, displays how the contents of the clusters interact and shift focus from the structure of care to taking in the patients’ perspective to a greater extent. On the right side of the concept map are Cluster 3 (Coordination and structure) and Cluster 4 (Resources), where most suggestions concern the organisation and structuring of care to improve the provision of cancer rehabilitation. On the left side are Cluster 1 (coordinated support and planning) and Cluster 5 (individual adaptation), both of which focus more directly on the individual’s needs in rehabilitation. Cluster 2 (availability) links these two perspectives, while Cluster 6 (development of knowledge and long-term vision), though somewhat broad, contains proposals highlighting the need for sustainability and ongoing knowledge development to support equitable and high-quality cancer rehabilitation.

Finally, the additional frequency analysis, as previously mentioned not part of the traditional CM methodology, revealed that the largest proportion of original ideas (33.6%) could be attributed to Cluster 3 (coordination and structure). This is more or less double the proportion attributable to the second largest cluster (Cluster 1 coordinated support and planning, 16.5%). Altogether, the concept map provides a visual system perspective of the most prominent areas and how these relate to each other.

The go-zone reveals items/suggestions from all six clusters, neatly illustrating the complexity of the issue, as all aspects are almost equally important (figure 3). It is also clear that most items score higher for importance than for feasibility. The item clearly deemed by all stakeholder groups as most important was “A good and equal cancer rehabilitation requires that it is planned and executed in collaboration with the patient”, belonging to Cluster 1. Even if this may sound self-evident, it is of great importance that the whole system sees this as top priority, since it resembles person-centred care, a philosophy,¹⁶ often referred to in Swedish healthcare policy documents and even in legislation (ie, the Patients Act⁴). A closer study of the way the different groups rated the other items and clusters indicates a wider spread between the four stakeholder groups, but this could be addressed, as mutual learning took place during the mixed group discussions (figure 4).

Discussion

For the first time, the NBHW has used CM in investigative work to produce development proposals and improvements that the government can use as a basis for political decisions. This section highlights some important experiences that have emerged along the way.

First, the method provided the opportunity to involve different perspectives and ‘voices’ (the voice of the organised patient, the professional voice of care, the voice of developers of knowledge and the voice of management and quality assurance). Collecting all these perspectives in a structured way, simultaneously, became a learning experience. Second, this approach provided a higher response rate from stakeholders than previous investigations by the NBHW, engaged them and kept them informed about the project in a more transparent manner than otherwise possible.

The clear learning perspective included in the assignment made the NBHW decide to take this task further than before by testing the CM methodology, with the ambition not just to ask but to include stakeholders throughout the process to achieve mutual learning along the way. Additionally, the method could capture a much larger target audience than conducting individual interviews. Similar attempts to involve the whole system in developing guidelines¹⁷ and using regional cancer plans have been done,⁹ with good results, but considered time-consuming and difficult to organise. Consequently, having digital workshops during the pandemic also made it possible to reach and gather people from all over the country in a structured way, and the design setup has subsequently been used for similar projects. This can enable larger participation, by not demanding travelling and extensive use of time for stakeholders with full schedules.

By mixing the groups in the second digital workshop, the people representing different perspectives could interact and learn from each other. Placing representatives for the whole system in the same room (in our case digitally) and having clear ideas to discuss created a climate of learning and respect that can be hard to provide otherwise, thereby encouraging the utilisation of results straight away. The visualisation of both the clusters in the CM map (figure 2) and the go-zones from different stakeholder perspectives (figure 4) provided an easy-to-comprehend common ground for discussions. Feedback from the participants showed great interest in the methodology, and it was deemed to be a useful tool for moving from policy documents into action, similar to experience from a previous project by parts of the research group.⁹

The evaluation of the importance of different ideas that have emerged, along with assessing their feasibility, visualised differences that may be important to address for government decisions and the variations that exist in Sweden regarding the patient’s access to rehabilitated interventions and support. Hence, the learning was extended from both inter- and intra-organisational perspectives,¹⁸ something posed already in 2005 by Sutherland and Katz.¹⁹ They suggested that if key stakeholders were taken together as in a CM study, they would represent the organisation as a bounded unit, setting the stage for improved interaction between evaluation practice and organisational learning. Actively using and designing such learning opportunities can be seen as crucial for sustainability in development projects.

Using idea generation to collect input on improved cancer rehabilitation has led to learning at different levels. First, those affected (the patients) and their opportunity to influence were put in focus through the lens of those supposed to provide and improve CR. The workflow from recruitment to joint analysis included, in addition to the patient’s perspective, taking part in and reflecting on the responses and input of patient-related activities from various system levels that are involved in and provide the conditions for a good and equal CR. CM methodology has generated commitment and direct feedback from the participants to the authority in terms of the importance of involving the ‘voices’ and perspectives of those it affects (the patient) as well as those who are part of the responsibilities and decisions of various processes. Similarly, CM has been used to examine the perspectives of teachers in a Swedish nursing programme on future demands,²⁰ to evaluate the implementation and use of policy documents in cancer care,⁹ to evaluate education in quality improvement in healthcare,¹⁴ to generate knowledge for medical programmes¹⁷ and to explore the meaning and role of a social innovation for people affected by cancer.²¹ In all projects, CM was found useful for involving stakeholders in development. Similar results have been presented elsewhere,²² indicating a good fit for the method from a learning perspective.

One of the problems of having an equal offering of CR throughout the system and between different actors was visualised by the high frequency of ideas relating to coordination and structure. Just over 50%of the original ideas were directed towards how CR needs to be improved in these areas, both from a patient and a system perspective. Bringing the results back to actors and stakeholders in the workshops further enhanced stakeholder participation and provided a platform for reflection and action. The CM methodology was considered exciting and reliable by the participants, and feedback from the workshops was positive and will hopefully provide continuous work on the topics for improvement and ownership. Since this study, an updated national care plan for CR has been launched and gained a lot of interest within the field of rehabilitation.

Strengths and limitations

A difficulty that many authorities’ investigations struggle with is getting response from enough stakeholders. The data collection here consisted of only one prompt, thereby demanding less time from participating stakeholders, but still providing greater insights than if only using pre-defined questions. The digital solution of the workshops, together with the survey’s single question, contributed to respect for the respondents’ time and opportunities to participate. Even if great care was taken to provide a representative and diverse sample for inclusion, we may have missed some important aspects by not including the right stakeholders. However, we encouraged included stakeholders to suggest other persons they believed had important input, and the extensive networks provided by the regional cancer centres for CR ensured a wide spread of participants. A further disadvantage may be that the analysis phase of CM required the commitment and time of more investigators from NBHW than usual. Initially a problem, but in a longer perspective, several investigators have learnt the methodology and working process and can use it in new assignments where improvement perspectives are requested. The investigators’ participation also contributed to increased knowledge of the material in the further analysis and the drafting of the report but also increased opportunities to identify gaps in both their own investigation and other ongoing investigations. The authority’s collaboration with researchers and statisticians who use and develop the CM method is now established and available based on the authority’s future needs.

Conclusion

Government investments in Swedish healthcare usually do not take place directly from the national level to clinical practice but via various region-wide activities at an administrative level. Cancer rehabilitation assumes, like care in general, that the micro-systems in a complex system must function where knowledge and facts are used wisely and generate value for individual patients. Everyone who works in the micro-system needs to be committed to both performing their work and improving it, as well as collaborating as smoothly as possible. The way of working in the implementation and analysis of the assignment has highlighted both the importance of joint learning, as well as learning between different operations and goals of long-term sustainable solutions and improvements. An advantage is that the anchoring process of the proposals has begun early in the investigation process and is reflected based on its possibilities to be implemented. Such work is likely more successful since it allows for ownership rather than a top-down approach commonly seen by authorities. For the first time, the NBHW has used CM in investigative work to produce development proposals and improvements that the government can use as a basis for political decisions. The method showed great potential, by allowing for stakeholder involvement, high participation and possible direct utilisation of results. Placing representatives for the whole system in the same room (in our case digitally) and having clear ideas to discuss created a climate of learning and respect that can be hard to provide otherwise, thereby encouraging the utilisation of results straight away. The visualisation of both the clusters in the CM map and the go-zones from different stakeholder perspectives provided an easy-to-comprehend common ground for discussions. A clear learning perspective was obtained, both from the NBHW and the involved stakeholders; hence, the method can be used in further assignments where improvement perspectives are requested.

Data availability statement

Data are available upon reasonable request.