Researching two Compassionate Cities: study protocol for a mixed-methods process and outcome evaluation

Abstract

Background/Objectives:

Compassionate Cities are social ecology approaches that apply a set of actions, targeting a broad range of stakeholders, with the intention of renormalising caring, dying, loss and grieving in everyday life. While several initiatives have been described in the literature, a rigorous evaluation of their processes and outcomes is lacking. This article describes the protocol for a mixed-methods study to evaluate the development process and the outcomes of two Compassionate Cities in Flanders, Belgium.

Methods and Analysis:

We will use a convergent multiphase mixed-methods design, in which a combination of qualitative and quantitative data collection methods will be triangulated in the data analysis stage to capture both development processes and outcomes. Our design includes a quasi-experimental component of a quantitative outcome evaluation in both Compassionate Cities and two comparable control cities with no formal Compassionate City programme. Both Compassionate Cities will be co-created in collaboration with local stakeholders. A critical realism lens will be applied to understand how and why certain processes manifest themselves.

Discussion:

The creation of Compassionate Cities implies high levels of complexity, adaptivity, unpredictability and uncertainty. This requires various data collection methods that can be applied flexibly. A researcher taking on the role of active participant in the project’s development has several advantages, such as access to scholarly information. Reflexivity in this role is paramount to questioning where the ownership of the project lies. By applying a critical realism lens, we remain cautious about our interpretations, and we test the homogeneity of our findings through other forms of data collection.

Conclusion:

This is the first published study protocol to describe both a process and outcome evaluation of a Compassionate City project. By transparently describing our aims and data collection methods, we try to maximise information exchange among researchers and to inform others who desire to implement and evaluate their own initiatives.

Background

During the last century, healthcare delivery in the developed world evolved from being primarily community-provided to highly professionalised and institutionalised.^1,2 The same trend can be observed with regard to people facing serious illness and dying.^3,4 However, because of a rapidly growing proportion of people with complex care needs, the existing specialised and generalist palliative care might quickly reach its limits. Therefore, novel social-embedded approaches to the challenges of serious illness, dying, loss and grieving are needed to complement current professional services.^2,4,5

By empowering community members to perform caring and supporting tasks for their fellow citizens facing these challenges (e.g. by doing their shopping, keeping them company or helping them participate in everyday community life), professional services can more effectively focus on their core responsibilities.^6,7 Moreover, there is growing understanding that dying is predominantly a social experience with a medical component, as opposed to the other way around.^8,9 Hence, support by family, friends and others during these social experiences may substantially impact the quality of life as well as the quality of dying.^10,11

Public health programmes have the potential of enhancing or enlarging these social networks, which direct some responsibilities away from professional healthcare services. Compassionate Cities have been suggested as such public health responses to serious illness, death, dying and bereavement. By focusing on prevention, harm reduction and early intervention, they aim to reintegrate and normalise the end-of-life in everyday life.^5,12 Compassionate Cities are social ecology programmes that focus on creating a supportive environment around people performing caring tasks or experiencing illness, dying, loss and grieving. By facilitating the involvement of citizens in care delivery, Compassionate Cities typically target all sectors of society. They usually start by engaging city or town officials and using their influence to implement changes society-wide. Through the involvement of citizens in defining the actions needed to minimise harm and improve end-of-life care, a sustainable model of community-controlled palliative care is pursued.⁸

To do so, a series of activities is worked out that aim at educating, raising awareness, changing policy or strengthening networks (among other results) in the local society. This is generally realised through the involvement of a variety of groups coming from workplaces, schools, civil society organisations and others. From this follows that the development of Compassionate Cities depends highly on the chosen development approach as well as the unique social and cultural background of the involved people, organisations and communities. Because they are designed and realised in co-creation – where outcomes depend on local stakeholders’ input – Compassionate Cities might face high levels of unpredictability and adaptivity. Because of their aim to increase the citizens’ agency over their own health, certain outcomes pertaining to increases in knowledge, attitudes, awareness or skills regarding serious illness, caring, dying and grieving can be expected.⁵

Furthermore, the founding principles of Compassionate Cities can be traced back to the Ottawa Charter for Health Promotion which proposes five main domains of focus to realise health promotion: building healthy public policies, creating supportive environments, strengthening community action, developing personal skills and re-orienting healthcare services.¹³ The Healthy City movement set an example of operationalising this approach but neglected applying the same principles to illness, dying, loss and grieving.¹⁴ Consequently, outputs related to these domains are an important focus in this research project.

A recent systematic review has shown that several Compassionate Cities and Communities have been described in literature.¹¹ However, the review also demonstrated that most of the reported initiatives have not been thoroughly evaluated and that only a few of the evaluation studies include both a process and an outcome evaluation. Because Compassionate Cities aim at bringing about societal change, it is important to perform both process and outcome evaluations to provide insight into how Compassionate Cities are developed, to measure the actual impact that has occurred as a result of the implementation and to uncover the mechanisms that have led to the desired outcomes. Furthermore, the review showed that some evaluation studies reported positive results, but the methods of the studies were never described in enough detail for them to be replicated. Until now, only a handful of study protocols for the evaluation of Compassionate Cities or Communities have been published, and those that have been published all focus on the evaluation of a single activity or outcome within the initiative (e.g. changes in the patients’ quality of life when involving volunteers; evaluation of a model to develop networks of care around people) instead of on the project as a whole.^15–18 Publishing study protocols of studies, including both process and outcome evaluations, helps in better understanding the development, implementation and outcomes of Compassionate Cities.

A collaboration between two universities, two cities and various local stakeholders within those cities has recently resulted in the development of two Compassionate Cities in Flanders, Belgium. One is located in the semi-rural municipality Herzele (18,500 inhabitants) and the other in the highly urbanised city of Bruges (118,000 inhabitants). In this article, we describe the protocol for a mixed-methods study that will be performed to evaluate the process of development and the impact of these two Compassionate Cities in Flanders, Belgium.

Methods and analysis

Study design

This study follows a convergent multiphase mixed-methods design including a quasi-experimental component of a quantitative outcome evaluation performed over a period of time in both intervention cities and two control cities, as well as a process evaluation in both intervention cities.¹⁹ We apply a convergent multiphase mixed-methods design in which we perform qualitative data collections throughout the whole project, perform quantitative data collections (survey) once pre-implementation and once post-implementation and merge the information at the post-implementation stage to be able to describe and compare both Compassionate Cities (Figure 1). Contesting findings will be tested against literature to confirm or to challenge, or to build new knowledge on the study subject. For the analysis of all gathered data, we are guided by the Consolidated Framework for Implementation Research to regard the elements that were essential in the implementation of the Compassionate City initiatives.²⁰ This framework provides a structured arrangement of everything that contributed to the implementation and helps in explaining why something works or does not work in a certain situation. The development of the Compassionate Cities can be divided into four phases: case search, pre-implementation, implementation and post-implementation. Data collection runs continuously with quantitative and qualitative data collections running parallel at times (Figure 1).¹⁵ At the post-implementation stage, a cross-case comparison is done: results from both intervention cities and the results from the intervention cities with the control cities will be compared. Data coming from interviews, document analyses, focus groups, group discussions, observations, diaries, a network analysis and surveys will be collected independently and triangulated at the data analysis stage to formulate uniform answers to the research questions (Table 1).²¹ Both Compassionate Cities will be compared and emphasis will be put on how and why differences manifest themselves and on the mechanisms that have contributed to the results. Where applicable, Standard protocol items: Recommendations for interventional trials (SPIRIT) guidelines were followed when constructing the protocol for this study.²²

Figure 1.

Convergent multiphase mixed-methods design. Green refers to data from the intervention cities and blue from control cities. Dotted lines indicate the progress of time, and arrows indicate the data collection or analysis is performed based on the prior event. Data are collected throughout the four project phases and triangulated at post-implementation.

Table 1.

Study objectives and research questions with corresponding data collection methods.

Objective 1: To describe how both Compassionate Cities are co-created and what they entail (process evaluation)
Research questions	Corresponding data collection methods
	Structured weekly diaries	Semi-structured observations	Semi-structured interviews	Focus groups and group discussions	Document analysis	Most significant change technique	Pre–post structured survey
– How are the Compassionate Cities co-created with stakeholders in the two cities (describing the process of development)?	X	X	X	X	X	X	X
– What do the Compassionate City programmes entail, and what is their reach?		X			X
– What proportion of the citizens of the (future) Compassionate City knows their city has become a Compassionate City by the end of the research project? How did they discover this?							X
– What are the barriers and facilitators experienced by the stakeholders and by the community facilitator in developing and implementing a Compassionate City programme? How do they differ between the two cities?	X		X	X	X	X
– What explains the differences between the two cities when comparing the process of development and the experienced barriers and facilitators (by both stakeholders and the community facilitator)?	X		X	X	X
– Which stakeholders co-created the Compassionate Cities?		X
Objective 2: To evaluate the impact of the Compassionate City programmes on the citizens of the Compassionate Cities, on the stakeholders and on the organisations’ social networks at the organisational level (outcome evaluation)
Research questions	Corresponding data collection methods
	Pre–post structured surveya	Organisational network analysis	Most significant change techniquea	Health insurance data
– What is the impact of the Compassionate City programme on the citizens’ knowledge, attitudes, awareness, self-efficacy, intention to look for support and skills concerning life-limiting illness, death, dying, loss and bereavement?	X
– What is the impact of the Compassionate City programme on the citizens’ local community participation and their neighbourhood connections?	X
– Is the impact of the Compassionate City programme different between the two cities? What are these differences and how can they be explained?	X
– What is the impact of the Compassionate City programme on the organisations’ social networks related to end-of-life topics at an organisational level?
– What are the most significant changes experienced by the stakeholders in each Compassionate City, and to what extent are these changes alike or different in the two cities?		X	X
– What is the impact of the Compassionate City programme on the end-of-life health experiences of family carers and on the healthcare use of patients?				X

^aThe pre–post structured survey and the most significant change technique are primarily used as data collection tools for the outcome evaluation which explains why their description can be found under section ‘Data collection methods for outcome evaluation’.

Applied research paradigm

This research applies a critical realist paradigm. Considering the aspect of ontology, critical realists attempt to capture anything that is perceived to be real, relating to anything that produces observable effects. From an epistemological perspective, critical realism emphasises that an objective world exists independently of the observed reality which is inherently shaped and coloured by our subjective interpretation, imagination and language and is thereby perceived differently by different people. It thus follows that a final, objective truth and knowledge do not exist. Nonetheless, this reasoning argues that it is possible to work towards a closer understanding of the nature of reality. Therefore, critical realism does not rely solely on quantitative or qualitative data collection methods to describe the world, but instead balances between an objectivist approach, which captures the truth in numbers and facts, and a subjective approach, in which all knowledge is relatively debatable and differently interpreted. Thus, one does not simply determine the occurrence of changes, but tries to describe how and why (i.e. the context in which) these changes take place.²³

Researcher positioning

The main researcher takes on the role of active participant observer.²⁴ This means that he collects data and actively interacts with the stakeholders involved in the development of the Compassionate Cities, which leads to him being known by the projects’ developers. His participation consists of reporting research data to the stakeholders as well as actively participating in meetings. However, he does not provide direct input into the design and development of the two Compassionate Cities.

Study setting

As part of the project to Develop Capacity in Palliative Care Across Society (CAPACITY) in Flanders, Belgium, two cities were selected to become Compassionate Cities. Candidate cities provided a written motivation for their potential participation. Table 2 provides an overview of the criteria used, which resulted in the selection of a smaller municipality of approximately 18,500 inhabitants, Herzele, and one larger city of approximately 117,000 inhabitants, Bruges. Both cities face an ageing population. For each Compassionate City, a control city was selected that is comparable in size and level of urbanisation – Sint-Niklaas (80,000 inhabitants) and Gavere (12,000 inhabitants) – for the quantitative impact evaluation component.

Table 2.

Selection criteria for the potential Compassionate City.

Selection criteria
1. Number of inhabitants	<20,00075,000–125,000
2. Support and involvement of mayor and/or aldermen
3. Presence of a civil servant to support the researchers and a community facilitator
4. Confirmed willingness to provide (financial) means for relevant social actions
5. Willingness to develop a long-term policy related to the end-of-life
6. Prior experience with social change projects (e.g. dementia-friendly city and fair trade city)
7. Providing access to existing stakeholder coalitions from previous social change projects
8. Existing advice boards (e.g. youth parliament and cultural board)
9. The city area falls within one and the same primary care zone
10. The city/municipality is a member of a Flemish volunteer network

Development of the Compassionate Cities in co-creation with local stakeholders

Under the impulse of the End-of-Life Care Research Group of the Vrije Universiteit Brussel and Ghent University, each intervention city was invited to participate and form a leading coalition consisting of local professionals and politicians. A project manager will be appointed to coordinate the project and lead the meetings of the coalition. The leading coalition will select multiple topics of focus through which the Compassionate Cities will be developed. The Compassionate Cities will be developed in co-creation with local professionals, the city council and citizens.²⁵ The research group will appoint a community facilitator, with expertise in change processes, to facilitate the development of the two Compassionate Cities.

Participants

Five groups of participants who participate in the development of the Compassionate Cities can be distinguished: researcher, community facilitator, local project leads, local stakeholders and citizens. Which participants are targeted for which data collection method is presented in Figure 2.

Figure 2.

Data collection methods and corresponding targeted populations. The different arrow colours are to facilitate interpretation and have no other significance.

1. Researchers: the researchers will not provide direct input for the project’s development, but they may exercise an indirect influence by, for example, communicating scholarly information and research results to involved stakeholders.

2. Community facilitator: the community facilitator is a person appointed by the university with expertise in change processes, change management and group dynamics.

3. Local project leads: each Compassionate City will have a local project lead appointed by the city or municipality. This person has a mandate to lead the project.

4. Local stakeholders: local stakeholders can be anyone involved in the design and/or development of the Compassionate Cities. Stakeholders include representatives of the city council (e.g. aldermen and local politicians), life-stance organisations, healthcare services (including, but not limited to, palliative care), volunteer organisations, family care organisations, civil society organisations, educational institutions, workplaces, citizens and so on. Owing to the unpredictable and co-creative nature of Compassionate Cities, the types of potential stakeholders involved in the development is unlimited.

Data collection methods for process evaluation

Structured weekly diaries

The community facilitator will work with weekly diaries throughout the project to register how much time and on which activities she spent time. Diaries help in logging reflections shortly after the occurrence of events, which facilitates remembering momentary data such as feelings, emotions or social contexts. Thus, diaries are well suited for capturing details of time- and context-sensitive data, such as one-on-one meetings or specific actions.²⁶

Semi-structured observations

The semi-structured observations will be performed by the main researcher. The observations will be used to register qualitative information (e.g. what is said, which emotions are observed, nonverbal interactions) and quantitative information during any meaningful event in the development of the Compassionate Cities (e.g. how many people are present at the event, duration of the event). Anyone involved in the design of the Compassionate Cities, as well as the design or execution of social actions, can be the subject of an observation. Observations run continuously throughout the project. We will make use of a semi-structured template that will be filled out by the researcher and that contains the following variables: (1) information about the setting, (2) enumeration and description of the participants, (3) chronological event description, (4) description of physical setting and materials, (5) description of behaviours and interactions, (6) conversations, (7) self-reflections.²⁷ If possible, the participants will be asked for consent prior to the observation. The observation template can be consulted in Supplemental Appendix 1.

Semi-structured interviews, focus groups and group discussions

We will perform semi-structured interviews with the community facilitator, the cities’ project leads and local stakeholders. During the interviews, we will cross-check hypotheses, explore topics in depth, investigate the reach of the projects, inspect barriers and facilitators in the creation of the initiatives and gain insight into the strategy and reasoning behind developmental decisions. Interviews with the community facilitator and the projects leads will be held every 3 months and will be face-to-face or online.

Prior to the interview, the interviewee’s consent is requested. All interviews are audio recorded and transcribed nonverbatim. There is no specific timing for the focus groups or group discussions because at first, it is necessary to ensure a functional and trusted operational atmosphere to maximise the likelihood that the participants will voice their true opinions.

Document analysis

We aim to register policy changes on relevant themes and the effect of the Compassionate Cities on participating organisations. To do so, we will collect documentation drawn up or changed by stakeholders or organisations as a consequence of their participation in the Compassionate City project. We decided not to discriminate between types of documentation but instead to focus on all documentation with relevance to the research topics. These documents will be collected throughout the project.

Data collection methods for outcome evaluation

Pre–post structured survey

We will administer a structured survey to a random sample of the general population of both Compassionate Cities and both control cities, once before the development of the Compassionate Cities and once post-implementation. The aim of the survey is twofold: to evaluate the impact of the Compassionate Cities and to inform the leading coalitions about possible areas of focus (e.g. low scores on palliative care knowledge could warrant palliative care education). The concepts to be studied are (1) knowledge of palliative care; (2) attitudes towards the dying of others; (3) awareness about the existence of palliative care and how this awareness was obtained; (4) self-efficacy regarding individual emotional competence and competence in supporting others; (5) intention to look for support; (6) skills regarding advanced care planning, in supporting a carer and in finding information; (7) local community participation for general and palliative care–related themes; and (8) neighbourhood connections regarding palliative care–related themes.

We will apply a simple random sampling of citizens older than 15 years in both intervention and control cities. In each city, 1100 citizens will be selected with a total of 4400 citizens. Family carers, if registered in their city of residence, will be oversampled. The survey will be sent out on paper with the possibility to fill it out online. The Dillman total design method will be applied to increase our response rate in which a maximum of three reminder mailings are sent to nonresponders.²⁸ The names and addresses of citizens from the sampling frames are accessed only by civil servants of the city who are responsible for the random sampling and the mailing, with remote assistance from the research team. This assures that the researcher remains blinded and cannot link the collected participant information to the individual.

The survey is sent out once pre-intervention and once post-intervention. A difference-in-differences approach will be used for this purpose. The approach is applied in geographically distinct areas with comparable populations, for which, in our case, two out of four populations undergo an intervention and the others do not.¹⁹ Because of the assumption of trends running parallel between similar populations (i.e. societal changes that transcend their locality such as cultural trends or political changes), any changes observed in the Compassionate Cities that are not observed in the control cities can, with a certain level of probability, be attributed to the intervention. Furthermore, by considering societal changes that happen outside the Compassionate City programme, the method limits history bias.

Organisational network analysis

In our research project, the organisational network analysis shows professional connections related to serious illness, death, dying and bereavement made by organisations involved in the Compassionate Cities’ development. We will map the connections by using a stakeholder grid, which every stakeholder involved in the development of the Compassionate Cities and representing an organisation will be asked to fill out. The focus is on organisational connections relating to sickness, death, dying, mourning and care. The stakeholders will be asked to reflect on their current organisation’s connections and the connections before the project commenced. We will further explore the direction, content and importance of this connection. This will be repeated during the post-implementation phase to show any network changes as a consequence of participating in the project. These findings will be discussed per stakeholder to verify possible findings.

Most significant change technique

Through this technique, we aim to describe the most meaningful and important changes experienced by stakeholders involved in the creation of the Compassionate Cities.²⁹ It is important to look beyond the predefined outcomes because the unpredictable nature of Compassionate Cities leads to unpredictable outcomes. Furthermore, because this is a co-creational project, the meaningful outcomes realised by the stakeholders’ input can be different from, and bear different significance than, the ones described and pre-constructed by researchers. All stakeholders will be asked to participate. They will be requested to write down the most significant changes they experienced as a result of their participation in the development of the Compassionate Cities. These changes will pertain to a maximum of five domains of change, which will have been predefined by the city council. An open domain and a negative change domain will be added, which allow stakeholders to add stories that do not fit the pre-constructed domains, and that allow the researcher to collect negative experiences or weaknesses in the development process.

During group discussions, stakeholders will discuss their story and explain why this story is significant to them. The group discussions will be held without predefined questions and will be audio recorded, with one researcher acting as a moderator and a second researcher as an observer who fills out the observation template. The moderator may read a story out loud if the participant prefers to keep their story anonymous. During the discussions, participants can voice their opinions on the shared stories (e.g. how many people recognise this story, does anyone wish to react to the story). People higher in hierarchy – possibly local policy makers – will receive the anonymised stories and will be asked to select the most significant change story, along with a written motivation for their selection.

Routinely collected administrative data

Data collected by the Belgian Intermutualistic Agency will be used to compare the healthcare claims data from patients in the intervention cities against those in the control cities. To add strength to the assumption that trends run parallel between both intervention cities and their control cities, it will be possible, by making use of a controlled interrupted time series design over a period of multiple years, with multiple measuring points in time, to compare the number of home deaths, hospital deaths, emergency hospital admissions in the last month of life, intensive care unit admissions in the last month of life and official palliative care status.^30,31

Data analysis

Data from the structured weekly diaries will be analysed independently, but we choose to triangulate it with other data (e.g. by cross-checking registrations through interviews).²⁶ We will make use of the qualitative data analysis software NVivo for thematic analysis and will follow the six analysing steps as proposed by Braun and Clarke.³² This procedure is also followed for the semi-structured interviews, focus groups and group discussions. Recurring responses from interviewees will receive special attention.

The semi-structured observations will undergo a thematic data analysis using the NVivo software, and themes will be constructed inductively. Findings will be tested against existing theoretical literature, leading to confirmation or warranting additional research when the expected is not confirmed. Important findings will be cross-checked through interviews to prevent misinterpretation.²⁷ Quantitative data will be analysed for specific actions to study the reach of each Compassionate City.

Documents will be analysed according to content, type and number of documents, and they will be imported into NVivo for qualitative data analysis.

For the organisational network analysis, data will be analysed both quantitatively (by looking at the number of connections) and qualitatively (by looking at the type of connections that were formed). We will make use of social network analysis software (e.g. Gephi) and present the results in a graph.

Every story coming from the most significant change technique and every additional conversation will be transcribed using verbatim transcription. Then, the transcripts will be imported into NVivo and analysed using a thematic approach.

Data from the routinely collected administrative data will be collected and analysed for both Compassionate Cities, after which different phases (including pre- and post-implementation) can be compared.

Data management

All data collected that could lead to identification of the involved persons will be pseudonymised in external communications, including publications. Where possible, the researcher will request participant consent prior to collecting qualitative data. If large numbers of participants are subject to qualitative data collection (e.g. during a festival), the collection of consent is not necessary because no information that can lead to the individual’s identification will be collected. All collected data will be securely stored on an encrypted server of the Vrije Universiteit Brussel, which is only accessible to members of the research team. All paper surveys are stored in a locked cabinet. In accordance with the Good Clinical Practice guidelines, the electronic (raw) data (privacy-sensitive information or any other information that could lead to the identification of individual people) will be stored for 15 years. Audio files will be deleted as soon as they have been transcribed.

Discussion

This protocol describes a mixed-methods process and outcome evaluation of two Compassionate Cities in Flanders, Belgium. Published study protocols that include both process and outcome evaluations of Compassionate Cities are, to our knowledge, nonexistent. However, such protocols are very much needed to better understand the development, implementation and outcomes of Compassionate Cities and Communities. Such protocols increase transparency and inform other researchers in their choice of research methods, which can prove useful when studying the relatively recent and poorly understood phenomenon of Compassionate Cities.^11,33

Mixed-methods research designs combine the strengths of both quantitative and qualitative methods.²¹ Focusing solely on quantitative methods would undervalue the importance of contextual factors, would provide little insight into the applied processes and would ignore important process factors such as the extent of self-organisation, sustainability, participation, agency or reach. On the other hand, applying only qualitative methods may fail to encompass population-level changes or to generate representative results. Most importantly, a single evaluation method would be unsuitable for capturing the complex and adaptive nature of Compassionate City or Community interventions.

Typically, the outcomes of complex interventions are unpredictable in an environment where the response to an intervention can be difficult to predefine. This is largely due to synergies between different situational aspects, such as the types and backgrounds of the stakeholders involved, their intrinsic and extrinsic motivations and the availability of resources such as funding, time and manpower. Because co-creating Compassionate Cities depends on the input of the local stakeholders involved, and because these projects intervene in different complex levels of society (such as civil society organisations, places of worship and workplaces or schools), outputs and outcomes are highly unpredictable.^34,35 Therefore, we can assume that Compassionate Cities, just like any complex intervention, are predominantly self-regulatory as opposed to centrally planned, which enhances their capriciousness.³⁵

This points to the necessity of developing a mixed-methods design, which encompasses a wide range of adaptable data collection methods where the one may be more fitting than the other to grasp the changing environments and therefore data.^36–38 However, when choosing data collection methods prior to an intervention that is characterised by unpredictable outcomes, the limitations of the methods can appear when the outcomes are not (or not satisfactorily) captured. For example, a survey can indeed produce outcomes other than those that the study initially hoped for, which is inevitable when selecting outcomes prior to an intervention. Therefore, complex interventions warrant a mixed-methods data collection approach so that the chosen method matches the emerging outcomes (and not the predicted, envisioned or assumed outcomes).

Taking on the role of active participant in Compassionate City development projects (and in co-creation projects in general) as a researcher has several advantages. First, researchers have access to scholarly information from other initiatives that local developers do not have access to. This can inspire them to facilitate bilateral information exchange. Second, at times, it is preferred to share information: for example, when input from researchers is requested or when a situation demands an intervention that can positively influence the project (e.g. examples of specific compassionate actions). Third, the researcher can communicate preliminary results from data collections to designated stakeholders which can positively influence the development process. Fourth, active participation leads to the researcher being known to the developers, which facilitates access to research data.³⁹ However, being an active participant does not necessarily imply the researcher’s active involvement in, and decision-making about, the process of development. The focus of the researchers can predominantly remain on capturing the voices of the people involved in, and affected by, the project, which can lead to changes in the development process when the research results are presented to the project developers. Even when the researchers decide not to take on a more participative role, their mere presence in the development of the Compassionate Cities might influence people’s behaviour and thereby, the data collected. Therefore, it is paramount for researchers to stay reflective about their positioning and to consider where the ownership of the project lies and the possible influences they are willing to exercise on the development process.⁴⁰

By applying critical realism to this research project, we will remain reflective about our personal (and therefore subjective) interpretations, and we will cross-check what we observe to be real through other forms of data collection to confirm or contest the findings.^23,27 Furthermore, what is shared by people in groups is often likely to reflect desirable group behaviour, which calls for us to critically review our perceptions and interpretations. Remaining reflective is a challenging process, and thus, it is important to log personal reflections during observations and to be quick on the draw by, for example, using informal conversations to discuss recent findings. Apart from focusing on what causes the observed effect, the underlying mechanisms that may have contributed warrant equal consideration. This is where qualitative interviews, in which information is shared in a private environment, play a vital role in capturing the interviewees’ individual opinions and possible deviations from our individual interpretations.

Conclusion

In this protocol article, we described the first protocol for evaluating both the process and the outcomes of the development of Compassionate Cities using a set of mixed-methods data collection methods. With this study protocol, we aim to enhance transparency, which is important when studying phenomena that are as yet poorly understood and studied. At the same time, we present examples of the methods that can be used by other researchers and developers of prospective Compassionate Cities to evaluate their initiatives. Finally, we highlight the importance of researchers remaining reflexive about their role in, and possible influence on, the development and outcome of such initiatives, especially when the researcher is an active participant in the project.