Do Healthy Cities Work? A Logic of Method for Assessing Impact and Outcome of Healthy Cities

Abstract

In this article, we discuss an appropriate methodology for assessing complex urban programs such as the WHO European Healthy Cities Network. The basic tenets and parameters for this project are reviewed, and situated in the broader urban health tradition. This leads to a delineation of the types of questions researchers can address when looking at a complex urban health program. Such questions reach appropriately beyond traditional public health concepts involving proximal and distal determinants of health (and associated upstream, midstream, and downstream rhetoric). Espousing a multi-level, reciprocal pathways perspective on Healthy Cities research, we also adopt a distinction between impacts and outcomes of Healthy Cities. The former are value-driven, the latter intervention-driven. These approaches lead to the acknowledgment of a logic of method that includes situational and contextual appreciation of unique Healthy City experiences in a Realist Evaluation paradigm. The article concludes with a reflection of evaluation and assessment procedures applied to Phase IV (2003-2008) of the WHO European Healthy Cities Network and an interpretation of response rates to the range of methods that have been adopted.

Introduction

Methodology is the logic of methods. Elsewhere I have argued that establishing this logic of methods for research is not merely a philosophical reflection on inquiry systems or possibly a deeply ingrained preference for quantitative data over qualitative material.¹ Quite the contrary: a logic of methods, driving the choice for specific approaches to collect data and generate information and knowledge, should first and foremost be driven by the answers we seek. And the answers we seek are intimately connected to the questions we raise, and how they are raised by whom. This article aims to identify the types of questions asked, particularly as generated in the WHO European Healthy Cities Network. It will subsequently position the types of answers that can and should be provided. Although this has to involve a short history of public health and urban studies, the ultimate aim of this exercise is to arrive at a framework that would allow us to develop the methodology we need. Such a methodology would clarify what researchers, communities, urban planners, service providers, industry, and all those others that make cities such exciting places, ask of their Healthy City.

Healthy City Tenets

The Healthy Cities movement in the European Region of the World Health Organization is a heavily codified and institutionalized enterprise. In 2009 it embarked on its Phase V, and each of the earlier four developmental stages has had a set policy and program priorities. Phases were marked by requirements for designated cities (those local government administrations in direct contact with WHO) to develop and deliver specified policies, approaches, or planning documents. In the Phase II, for instance, cities signed up to produce City Health Plans based on City Health Profiles.² The subsequent Phase continued such a requirement for “new” cities entering the network, but committed “old” cities to transcend their Health Plans and produce City Health Development Plans. In the last phase that we can report evaluation findings on, these planning exercises have programmatically culminated in more specified, targeted, and yet also more strategic deliverables:

• Specific approaches to equity and the social determinants of health

• The development and implementation of Health Impact Assessments

• Targeted approaches to healthy aging

• The introduction and implementation of healthy urban planning processes in the operations of local government

• Targeted approaches to active living

These six elements for the Fourth Phase complemented the ongoing commitment of designated cities to

• Work through, develop, and sustain (intersectoral) partnerships

• Develop, implement, and publish city health profiles and city health development plans

• Ensure active participation by and empowerment of communities

• Engage in and support networks of Healthy Cities across the European region of WHO (and as a consequence, often beyond)

It has been observed that the mere resilience of hundreds, if not thousands, of Healthy Cities across different political systems, in often radically versatile socio-economic conditions, over at least two decades is a sign of the vibrant efficacy of this movement. As Dr. Agis Tsouros observed in his closing statement of the Zagreb conference, celebrating 20 years of the movement in Europe, “The replicability of this idea across the diversity of WHO member states is in itself evidence of its effectiveness.” However true this may be, tough questions remain to be asked of Healthy Cities. In a world where evidence of effectiveness regrettably still hinges on large-scale controlled trials with highly homogenized control and experimental groups, questions such as “Does it work; what is in it for the people; what political gain can be established?” challenge such conventional approaches.

One reason why such questions may be considered tough is that they deal with messy, wicked,³ complex, and multi-level issues. Another reason that these tough questions remain to be asked lies in the fact that the evolution of the modern scientific paradigm has favored dissection and fragmentation of problems over integration and whole-systems perspectives. On both counts, we should apply more rigorous and transparent analysis to reach an agreement on what is examined when we aspire to assess Healthy Cities. Looking at the complexity of Healthy Cities and its deliverables we should recognize that the Phase components mentioned above should all be considered pathways toward comprehensive urban health development, or Healthy Cities.

There have been a number of efforts to describe the elements of a Healthy City, and Lawrence⁴ starts with outlining the key characteristics of a city. Cities are characterized by:

• Centralization of resources and populations in a specific place

• Verticality of construction arrangements (be they for housing or other functional purposes)

• Related to the first two characteristics, concentration of resources (man-made, natural, and human) and waste products requires the effective urban management by authorities of flows of these within the city perimeter and between the urban and regional environment

• These characteristics in turn lead to a greater diversity, both in natural (ecosystemic) and in cultural terms, which potentially makes for greater opportunities for ecological, economic, and social well-being

• Information and communication are pivotal in the development and exchange of knowledge and innovation

• From their earliest stages, cities have relied on mechanization and technology for their smooth operation

• Political authority and governance systems must be in place for cities to sustain themselves

Reviewing how these and other characteristics would lead to health in urban environments, Hancock and Duhl proposed, at the launch of the “official” WHO Healthy Cities Project in 1986, the following 11 parameters for a Healthy City:⁵

1. A clean, safe, high-quality physical environment (including housing quality)

2. An ecosystem which is stable now and sustainable in the long-term

3. A strong, mutually supportive and non-exploitive community

4. A high degree of public participation in and control over the decisions affecting one’s life, health, and well-being

5. The meeting of basic needs (food, water, shelter, income, safety, and work) for all the city’s people

6. Access to a wide variety of experiences and resources with the possibility of multiple contacts, interaction, and communication

7. A diverse, vital, and innovative city economy

8. Encouragement of connectedness with the past, with the cultural and biological heritage, and with other groups and individuals

9. A city form that is compatible with and enhances the above parameters and behaviors

10. An optimum level of appropriate public-health and sick-care services accessible to all

11. High health status (both high positive health status and low disease status)

Lawrence⁴ believes that there are a number of prerequisites for effectively applying these parameters in the urban health reality. First, he states that “proactive policies and programs with sufficient monetary and non-monetary resources over the long term should complement remedial measures.” The mere strategic direction of the WHO Healthy Cities movement in Europe indicates that there is a conscious and conscientious drive towards such policies and programs.^6,7 Secondly, “intersectoral collaboration between scientists, professionals, policy decision-makers and representatives of community groups” should guide the activities of Healthy Cities. Elsewhere, we have published an inventory of approaches that can be employed to bridge the divide that some believe exists at the nexus between research, policy, and practice,⁸ and the value systems that Healthy Cities espouse through their commitment to the above 11 parameters appear to be an excellent starting point for such intersectoral collaboration. Thirdly, Lawrence⁴ identifies that “a shared definition of goals, priorities and resources allocation in order to achieve desired outcomes” through the establishment of evidence and outcome-driven partnerships for urban health is critical for Healthy City success. A fourth, and slightly troublesome prerequisite, is the development and maintenance of “systematic monitoring and feedback in order to develop a coordinated database and information about the specific characteristics of cities including the health and well-being of the population.” Although this has been a consistent endeavor of European Healthy Cities,^9,10 the variability and accessibility of such databases, including the generating capacity of local governments of such data, has been a significant challenge to their efforts. Traditional research approaches have not succeeded in developing a reliable and convincing base for evidence-based Healthy Cities; below we will propose a way forward. Lawrence^4,11 maintains that a holistic, integrated, and ecological interpretation of urban health is pivotal to advancing it substantially beyond the remedial and fragmented approach. The latter has become successful over the last century because of its simplistic, almost always mono-causal appeal: its main proponent in the biomedical paradigm, the germ theory, assumes that the presence of pathogens will lead to disease. A modern, socio-ecological model of human well-being, addressing social, economic, and health inequalities (in urban environments) states that there are multiple, complex, and often reciprocal pathways between health outcomes and their vast range of determinants. Figure 1 attempts to describe this systems-complexity for the Phase IV of the WHO Healthy Cities Project.

Figure 1.

The connection between the “11 Qualities Healthy Cities should strive for” (inner circle, Hancock and Duhl, 1986) and priority areas for cities designated into Phase Four of the European WHO Healthy Cities Project (outer circle).

The Politics of Causation—Why Messy Issues Need Complex Inquiry

The problem with a layered representation of reality as in Figure 1 is that it seriously challenges conventional wisdom and procedure in health and biomedical research.¹ The field is experiencing exponential hyperspecialization¹² and fragmentation,¹³ and as a consequence the demands on the types of questions asked, the nature of the answers provided, and the methods applied (in all, the methodology) seem to follow suit. Researchers tend to follow a reflex to cut up phenomena under study into disjointed elements, thus disabling them to see and interpret the whole. Often they choose research methods that are not suited to the issue under investigation¹⁴ as a result of this unwarranted dissection. And even though the opportunistic choice of method may result in the possibility to publish research in top peer-reviewed journals, in fact a type IV error (Table 1) may be committed. Clearly such an error leads to research results that disconnect the process of scientific inquiry from its relevance to practitioners, communities, and their governance structures.^8,15

Table 1.

Types of error^3,16–18

Error type	Characterization	Example
Error I	Hypothesis rejected while it should be accepted	An HIV test shows that an infected person does not have the virus
Error II	Hypothesis accepted while it should be rejected	A mammography showing “suspicious” cells identified as cancerous, while they are innocent
Error III	Answering the wrong question	Why are there more hospital beds in Healthy Cities?
Error IV	Answering the right question from the wrong paradigm	An investigation of Healthy City governance parameters, applying an experimental research design to political decision-making analysis

Nancy Krieger¹⁹ eloquently describes the fallacy of over-simplifying causality in health issues. She finds that (public) health researchers, in a probably unconscious attempt to mimic medical research, have adopted a rhetoric around determinants of health that does not do justice to the complex and reciprocal relations impacting on human well-being.

This discourse around determinants of health has been influenced quite powerfully by the “upstream”–“midstream”–“downstream” metaphor (sometimes complemented by an “in stream” component to denote community engagement and empowerment). The metaphor usually tells the story of a helpful soul on the bank of a river, witnessing a steady stream of drowning people being carried downstream. The good Samaritan jumps in time and again to rescue these victims (a “downstream” intervention), until she decides to walk upstream to see what is going on. Usually, the story will say something about a busy or ramshackle bridge from which the least powerful and resilient people fall (or are being pushed?) into the river. The “midstream” intervention would be to caution people that the bridge may be dangerous, or cynically, to teach people to swim. The “upstream” intervention would be to repair the bridge or replace it with a sounder version.

Downstream interventions are usually considered the domain of the clinical health professions, midstream (or in stream) interventions the prerogative of public health and health promotion people, and the upstream interventions fall in the realm of the politicians, decision-makers, and more generally, technocrats.

Again, the hyperspecialization and fragmentation of the professions has not done much good to the capacity of humankind to deal with complex (urban health) problems. In the “standard normal” model of public health and epidemiology this complexity tends to be dissected into proximal (downstream) and distal (upstream) factors, the former being amenable to individual action (medical intervention, lifestyle change), and the latter subject to institutional consideration (laws, rules, and design parameters). Krieger¹⁹ shows that this oversimplification even applies to modifications of this notion that account for the complexity and multi-level nature of most population health problems. With the introduction of a hierarchy of levels between the “most proximal” (gene-protein responses to intracellular and intercellular environments) and “most distal” (the health of our planet—and possibly beyond) the “causal potency” (and our potential to substantively improve human health) has not been changed.

The proximal–distal divide therefore does not seem extremely helpful, even when a hierarchy of levels of influence and action is introduced. The reason for this is that persistently, the rhetoric has ignored issues of power and co-existent multi-level interactivity between different functions of determinants of health.¹⁹ Only by recognizing this multi-dimensional co-existence in space and time in a universe that is determined by political econology^20,21 can the true re-integration of urban planning and public health occur.^22,23 Table 2 illustrates how one and the same public health issue (cigarette smoking) has two simultaneous semiotic meanings. Only by connecting these in health public policies²⁴ or health-in-all policies²⁵ can true advances in the promotion of human health be made.

Table 2.

Simultaneous meanings of a public health issue¹⁹

A cigarette is…
A combustible mass of tobacco leaves and additives whose burning smoke transports psychoactive and addictive chemicals (e.g., nicotine) and carcinogens deep down the respiratory tract to the innermost parts of the lung and its alveolar capillaries, thereby increasing risk of cancer, cardiovascular and pulmonary disease, and other smoking-related ailments.	A highly profitable product whose production, distribution, advertisement, and consumption involves relentless corporate marketing (including manipulation of ideologies involving freedom, class, gender, sexuality, race/ethnicity, and targeting of marginalized groups), government regulation and taxation, tobacco farmers and workers, land ownership, trade agreements, and international treaties.

Healthy Cities Impact and Outcome

The nature and perspective of the WHO European Healthy Cities Project matches closely the approach advocated by Krieger. The specific, yet non-hierarchical, focus on city health profiles, health development plans, partnerships, community partnership and empowerment, equity, Health Impact Assessment, urban planning, networking, active living, and healthy aging, resulting in action on the 11 parameters of a Healthy City, encompasses the integrative approach of co-existing levels, pathways of influence and causality, and issues of power.

Although this recognition in itself (1) makes Healthy Cities evaluation more profound and responsive to real needs, (2) seems to have the potential to avoid Type III and IV errors, and (3) would eventually yield results that significantly boost our understanding of urban health (and how to influence it), the associated methodology does not necessarily become that much easier to consider and develop. As I described elsewhere,¹ this is not surprising: single-dimensional problems with clear and transparent causal and final relations can be studied with fairly straightforward methods, but complex, multi-dimensional problems require sophisticated sets of interconnected and often innovative research methods. Above I stated that the question we ask ourselves determines our methodology. We have now established that Healthy City questions are exactly the multi-level issues that merit complex research approaches.

In order to describe this complexity a little more analytically, there is another important characteristic of Healthy Cities—and an implicit notion in Krieger’s approach—that we need to consider. Healthy Cities are not just about interventions that can be studied for their effectiveness. Healthy Cities more importantly embrace a value system for health that is a coherent, innovative urban health policy approach. Important as it is to evaluate the effectiveness of specific interventions, it is as important—if not more critical—to assess how the adoption of this value system would generate new and enhanced options for the actual implementation of such interventions. In fact, in the evaluation efforts for the Phase III and IV of the European Healthy Cities Project, the expressions of that value system (e.g., sustainability, equity, community participation, and empowerment) in innovative approaches to, for instance, healthy urban planning and health impact assessment, have been an explicit driving force of the Healthy Cities research agenda.

In order to distinguish the effects of the adoption of this value system from the more operational aspects and effects of specific interventions it makes sense to differentiate two types of effect measures: impact and outcome.

It is recognized that conceptual confusion exists on the nature, direction, and definition of these measures. Below we will define impact as the consequence (intended or unintended) of complex interactions, and outcome as the intended result of a consciously developed (regulatory, facilitative, or communicative) intervention. This perspective is embedded in solid empirical tradition, and yet another tradition claims virtually the opposite.^45,46

Impact is a result or change that has come about as the result of complex interactions between a range of factors, including Healthy City designation, but potentially much wider (multi-causality, such as socio-political change, economic up/downturns, extending or limiting the range of stakeholders, etc.) and is driven by priorities determined by the Healthy Cities value system. This definition is consistent with perspectives offered in the literature: the European Monitoring Center for Drugs and Drug Addiction defines impact evaluation as assessing the results beyond the achievement of objectives set (greater range of results which were not explicitly and previously planned).²⁶ This idea that “impact” refers to effects that transcend a specific focus is also central in the Organization for Economic Cooperation and Development perspective about “[t]he positive and negative, primary and secondary, long-term effects produced by a development intervention, directly or indirectly, intended or unintended.”²⁷ De Leeuw and Skovgaard found that the internal processes in cities leading to formal designation by WHO, that is, processes that develop and confirm adherence to the Healthy City value system, have to occur before such cities can start their active pursuit of “Healthy City interventions” in the WHO project.²⁸

An outcome, on the other hand, is a result or change that can be directly attributed to such an activity or intervention within a Healthy City (if intervention A, then result A_x). Again, this conceptualization is consistent with the literature. An influential public health and health promotion glossary²⁹ states that outcome evaluation refers to the consequent effect of a program on the health outcomes in populations, corresponding to the program goal or target. Or more succinctly: outcome evaluation assesses the result in terms of achievement of objectives set.²⁶

Healthy Cities Evaluation—Answering the Messy Questions

Summarizing the above, in Healthy Cities evaluation we are interested in multi-level complexities and impact and outcomes evaluations, reflecting the Healthy Cities “package” as well as specific interventions. In Figure 2 we present these challenges in a relatively simple 2 × 2 grid, filled with examples from Healthy Cities as derived from the Phase IV evaluation exercise.

Figure 2.

Levels and pathways of influence for determinants of health connected to impact and outcome parameters for Healthy Cities.

Obviously, this grid can be made infinitely more complex by specifying levels, and their complex interactions, as well as a detailed delineation of impacts and outcomes. On the other hand, a reasoned and valid focus on a specific level, appropriately linked through (evidence-based, or at least conceptually grounded) pathways to relevant other levels, and a conscientious and documented focus on either impact, outcome or both, would allow Healthy City researchers to frame their research approach.

An initial guide to assessing Healthy Cities suggests that the important questions for Healthy Cities are process questions³⁰; in an adaptation of this framework for the Third Phase of the European project, questions were further refined and adapted to reflect processes, changes, as well as results. This happened in the Monitoring, Assessment, Reporting, and Impact Assessment framework.¹ My purpose here is to situate the further evolution of evaluation efforts that have been undertaken by a group of 14 senior researchers for the assessment of Phase Four of the European Healthy Cities Project.

Lipp and Winters ask questions around the engagement of cities in partnership development and maintenance. The level addressed in their piece is of a higher aggregate, drawing on insights that intersectoral action (within government sectors, but particularly beyond these into private, non-governmental [NGOs] and quasi-governmental [QUANGOs] arenas) will enable cities to address health and its determinants more widely. The assumption here is that more inclusive, broader, and sustained partnerships will yield opportunities for health development that the public sector on its own cannot generate.

Webster addresses an aspect of Healthy Cities that has been integral to the Project since its inception: the City Health Profile as a critical tool of health information for all stakeholders in the Healthy City. The development and regular update of these City Health Profiles, it is assumed, is a higher-level precondition for policy making and intervention development.

Green assesses one of the intended consequences of these City Health Profiles: the City Health Development Plans. Again, such plans assume a complex causality in that they would drive more specific action for urban health; they would lead cities from an environment of “projectism”³¹ into a longer-term policy perspective.

Dooris and Heritage look at the nature of community participation and empowerment, not in terms of their effectiveness (which have been demonstrated by, for instance, Laverack³² and Wallerstein)³³ but in terms of the qualitative presence of elements of Davidson’s wheel of participation.³⁴ Again, their question addresses a higher-level complexity: engagement by Healthy Cities in participation and empowerment endeavors.

In Ritsatakis’ piece, the question that is addressed is whether European Healthy Cities have placed equity and the wider social determinants of health on their political and social agendas. The question asked here steps beyond the moral, ethical, or philosophical (on effectiveness or otherwise) as these have already been addressed adequately elsewhere.³⁵ But connecting the health equity and social determinants agendas with the broader Healthy City approaches and value system would, it is taken, lead to better capacity to develop equitable population health characteristics.

Another procedural question is asked in the paper by Ison. Health Impact Assessment (HIA) is a method used to assess the potential effects of a policy, program, or project on the health of a population. It has been defined as a critical tool for building Healthy City policies and plans. There is an increasing evidence base that HIA is an effective tool across the full range of social, health, and equity issues,³⁶ so Ison does not address the question whether HIA is effective, but rather clarifies that in connection with the other tenets of the European project, it assumes that the effectiveness of Healthy City elements will improve: again, this is a complex multi-level approach.

Green and Faskunger, in two essays on specific aspects of health (aging and active living, respectively), look at systems and processes that enhance the inclusion of these areas in urban health programs. In both cases the premise is that cities are supremely equipped to deal with systems and infrastructure development for aging populations and active living.

Barton and Grant retrace the re-integration of public health and urban planning (once inseparable children of the same parent)²³ in the healthy urban planning approach. The logical conceptual framework that has driven the emergence of healthy urban planning now dictates a complex, multi-level, and intersectoral endeavor that intricately links with other aspects of Healthy Cities.

Finally, Heritage and Green take a look at possibly the highest level of influence on health in European Healthy Cities: the networking among the 77 designated cities, and their own networking within national and regional European entities (totaling some 1,300 cities). Again, the merit and value of networking per se are uncontested because they have been established earlier^37,38 and the review addresses the complex and reciprocal pathways and functionalities that make up such networks.

Looking at these questions, it becomes obvious that evaluators of the WHO European Healthy Cities Project have moved beyond the “intervention outcome” issues (the upper right quadrant of Figure 2). In itself this is already a daring move, as so many health researchers still seem to be stuck in an earlier stage of problem solving: the epidemiological analysis of (relative) risk. However, as Jaime Lerner, a recognized leader in the global movement of ecological cities, stated on the occasion of the celebrations that for the first time in the history of humankind more people live in cities elsewhere³⁹: “Cities are not the problem; cities are the solution.” Positioning Healthy City research questions in the lower two quadrants of the grid suggests a particular methodology.

The Logic of Method in Healthy Cities Research

The higher-level questions that are being analyzed in the Fourth Phase evaluation are associated with complex pathways of (positive) health and disease. None of these questions, in fact, look at direct health impacts and outcomes. Rather, in the spirit of the report of the WHO Commission on Social Determinants of Health⁴⁰ they specifically address the unique systems determinants that impact on good governance of health and well-being matters in cities.^15,41

The logic of method therefore suggests that a comprehensive set of research instruments, including qualitative as well as quantitative approaches, would generate the types of data that can answer these questions. The development, pre-testing, and administration of these instruments shall be acutely cognizant of sociopolitical realities in urban governance.

One framework that can determine the composition of such a toolbox of research instruments is that of Realist Evaluation.^42,43 The core tenet of this framework is that each “program” (such as a Healthy City) has its own unique program theory that can be tested. The main question addressed in realist evaluation is therefore not whether a general theory or intervention works, but “What works for whom in what circumstances?” The realist evaluator understands causality in terms of underlying causal mechanisms generating regularities for the program. In a fairly simple intervention formula: outcome = mechanisms + context. This vision has also been espoused by the Theory-Based Evaluation (TBE) framework.⁴⁴ A more rigorous meta-analysis of the applicability of Realist Evaluation in a TBE framework should be undertaken to show how valid this approach would be beyond the European Healthy City context.

For those more familiar with the quantitative sciences, the valuable insight is that Realist Evaluation shares a logical similarity with Bayesian statistics. Unlike traditional statistics (in which each probability is presumed independent from temporal or cognitive causal or final relations), Bayesian statistics take into account the uniqueness of each probability situation by including a priori conditions that determine the distribution of outcomes a posteriori.

In terms of the methodology associated with realist evaluation and TBE the logic dictates the development and application of a range of methods for data generation that address the unique specifics and underlying causal mechanisms pertinent to the program under study. The mere accumulation of indicators and measures for (urban) health—albeit an important descriptor for health and policy problems urban governance systems are facing—is an insufficient process for demonstrating the strengths and weaknesses of particular actions in specific environments. This approach has been an integral part of review and assessment processes in the European Healthy Cities movement.¹

This logic of method therefore suggested a distinctive collection of research methods that was applied to the evaluation of Phase Four of the project:

• The Annual Reporting Template (ART) is a tool that has been applied since early stages of Phase Three of the project (1998). ART is a direct function of the designation process, and asks Healthy Cities to report on progress against designation criteria. For Phase Four, these specifically included the action areas in the outer circle of Figure 1. ART furthermore asks designated Healthy Cities to assess changes in their social, political, and economic context that have enabled them, or have prevented them, from achieving the goals they have set themselves. These changes can be at any governance level (be it at the neighborhood, community, province, state, or global plane).

• At the end of each Phase a “taking stock” General Evaluation Questionnaire (GEQ) is being developed in consultation with a scientific advisory group, but more importantly, driven by Healthy City coordinators and politicians. Although the GEQ follows the same general pattern for each Phase (and is, in some sense, therefore a validated research tool), its design allows a dynamic response to issues that may have emerged during the lifetime of the project. In the case of Phase Four, for instance, one of the first questions is “What is your proudest achievement?” enabling cities to highlight successes and innovations that would not have emerged through the application of a standardized (quantitative) research tool.

• Cities were invited to highlight their achievements, finally, through submission of case studies for presentation at the Zagreb International Healthy Cities conference (October, 2008). Evaluators had access to 168 such case studies for their analysis.

Finally, a word on response rates that form the basis of the findings in this volume. The dynamic, interactive nature of the above set of three research instruments, addressing such a diverse package of issues, presents a particular challenge in describing the nature of the response. For example, some cities reported major accomplishments in the areas of active living, but failed to address issues around urban planning. Conceptually, this is awkward as healthy urban planning has a direct impact (conceptually and practically) on active living parameters. Yet, in the methodology employed this is not surprising.

When offering the units under investigation (in this case, “Healthy Cities”) a multitude of ways to voice their issues, concerns, and successes, it is almost self-evident that the response is as varied as the research instrument employed. In our case, this led to the following response rates. On the GEQ and ART combined, the total response rate was 79.2% (or 61 out of 77 cities¹). However, on the different elements of these research tools, response rates ranged from an outlying low of 46 cities (on a question around resource allocation for Health Impact Assessments) to a cluster in the 55–60 range, and some questions asking specifics (e.g., city population) with a full response by 61 cities.

All of these response rates, however, can be considered excellent. In order to further ascertain that no response biases had occurred we engaged in (non-)response analysis (Figures 3 and 4). First, we wanted to know whether there had been any difference in rates between “old” cities (those that had been designated in Phase Three) and “new” cities (those that joined the project in Phase Four). One might hypothesize issues around “questionnaire exhaustion” or perhaps “eagerness,” and other phenomena, that would over- or under-represent either group. Figure 3 shows that this has not been the case. We also needed to make sure that geographically (and as an inference, in terms of governance systems) no single model dominated the response. Figure 4 shows that response rates compared between regions are indicating that no one region has significantly better response rates than others. Even the relatively larger non-response in Mediterranean cities falls entirely within the expected range; a response of 67.9% is quite satisfactory.

Figure 3.

Response to two research instruments for Phase Four (Annual Reporting Template ART and General Evaluation Questionnaire GEQ). ‘Old’ cities are those that were already designated by WHO at some point during Phase Three. ‘New’ cities have been designated during Phase Four (and may have been designated in Phases One and Two but not Phase Three).

Figure 4.

Response to two research instruments for Phase Four (Annual Reporting Template ART and General Evaluation Questionnaire GEQ). New Independent States are those that gained nationhood since 1989 (predominantly in Central and Eastern Europe). Mediterranean nations are those on the Mediterranean rim. Some Mediterranean and NIS states are Organization for Economic Cooperation and Development (OECD) members (e.g., Turkey or Slovenia). Such nations are—for the purpose of geographical response classification—not included as OECD nations.

Conclusion

Asking the “right” question in Healthy Cities assessments is not easy. In this article I have drawn attention to a number of issues that need to be considered when researching Healthy Cities. First, questions need to be appropriate, relevant, and appreciative of the unique situational context of Healthy City endeavors. These are part value-driven, and part oriented towards interventions. Each of these aspects should be researched, but each with their appropriate logic of method. In order to do this I introduced a distinction between impact (values) and outcome (intervention) research. Whereas a lot of research is accumulating mounting evidence on the effectiveness of interventions, there is a paucity of research into the adoption and consequences of Healthy Cities values. An appropriate paradigm that may be applied to the study of this area is that of Realistic Evaluation.