Summary.
Objectives:
To assess whether a community intervention on health related behaviour in deprived neighbourhoods was delivered as planned and the extent of exposure to the intervention programme.
Methods:
Data were gathered throughout the intervention period using minutes of meetings, registration forms and a postal questionnaire among residents in intervention and comparison neighbourhoods.
Results:
Overall, the intervention was delivered according to the key principles of a “community approach” although community participation could have been improved. Neighbourhood coalitions organized more than 50 health related activities in the neighbourhoods over a two-year period. Most activities were directed at attracting attention, providing information, and increasing awareness and knowledge, and at changing behaviours. Programme awareness and programme participation were 24% respectively 3% among residents in the intervention neighbourhoods.
Conclusions:
The process evaluation indicated that it was feasible to implement a community intervention according to the key principles of the “community approach” in deprived neighbourhoods. However, it is unlikely that the total package of intervention activities had enough strength and sufficient exposure to attain community-wide health behaviour change.
Keywords. Process evaluation, Community intervention, Health related behaviour, Deprived neighbourhoods
Zusammenfassung.
Prozessevaluation einer holländischen Gemeindeintervention zur Verbesserung von gesundheitsbezogenem Verhalten in sozial benachteiligten Wohngegenden
Untersuchungsziel:
Eine systematische Beurteilung der planmässigen Umsetzung der Intervention und der Inanspruchnahme des Programms.
Methoden:
Die Datenerhebung erfolgte während des gesamten Programms und schloss Sitzungsprotokolle, Anmeldeformulare und postalische Fragebögen an die Wohnbevölkerung ein.
Ergebnisse:
Insgesamt wurde die Intervention in übereinstimmung mit den Schlüsselprinzipien des “community approach” durchgeführt; jedoch hätte die Partizipation durch die Gemeinde verbessert werden können. Nachbarschaftsvereinigungen organisierten im Laufe von zwei Jahren über 50 gesundheitsbezogene Aktivitäten in den Wohngegenden. Die Mehrzahl der Aktivitäten bezogen sich auf die Erhöhung der Aufmerksamkeit, Informationsvermittlung, Bewusstseins- und Wissensvergrösserung, sowie Verhaltensänderungen. 24% der Wohnbevölkerung der Gemeinde, in der die Intervention durchgeführt wurde, wussten um die Durchführung des Programms, 3% nahmen daran teil.
Schlussfolgerungen:
Die Ergebnisse der Prozessevaluation weisen darauf hin, dass in sozial benachteiligten Wohngebieten Gemeindeinterventionen nach den Schlüsselprinzipien des “community approach” durchführbar sind. Allerdings ist es unwahrscheinlich, dass das Gesamtangebot an Interventionsaktivitäten genügend Dauerhaftigkeit und ausreichende Inanspruchnahme aufwies, um gemeindeweite Veränderungen im Gesundheitsverhalten zu bewirken.
Résumé.
Intervention visant à améliorer les comportements en lien avec la santé dans des communautés hollandaises défavorisées : évaluation de processus
Objectifs:
Evaluer si l’intervention a été menée comme sa planification le prévoyait et identifier le degré d’exposition à ce programme d’amélioration des comportements en lien avec la santé dans des communautés hollandaises défavorisées.
Méthodes:
Les données ont été rassemblées sur toute la durée de l’intervention : procès-verbaux des réunions, formulaire d’inscription, questionnaire postal auprès des habitants des communautés concernées et pas par l’intervention.
Résultats:
Globalement, l’intervention a été menée selon les critères d’une «approche communautaire», bien que le degré de participation de la communauté eût pu être amélioré. La communauté a organisé plus de 50 activités en lien avec la santé sur une période de deux ans. La majorité de ces activités avaient comme objectifs d’attirer l’attention, de fournir de l’information, d’augmenter la prise de conscience et les connaissances, ainsi qu’á modifier les comportements. Parmi les résidents des communautés recevant l’intervention, 24% connaissaient le programme, et 3% y avaient participé.
Conclusions:
L’évaluation de processus indique qu’il est possible de mettre en œuvre une intervention dans une communauté défavorisée selon les critères d’une «approche communautaire». Cependant, il est probable que l’ensemble des interventions n’a pas eu un impact suffisant et n’a pas permis une exposition propre à induire des changements de comportements dans la communauté entière.
In order to tackle health inequalities, the 30 big cities in the Netherlands are advised to follow the interventions and policy measures recommended by the Dutch advisory committee on socioeconomic inequalities in health (Mackenbach & Stronks 2002). Part of their recommendations was to adapt health promotion programmes to the needs of lower socioeconomic groups. Especially, community interventions were thought to be suitable for this purpose (Mackenbach & Bakker 2003).
One of the 30 big cities in the Netherlands, the city of Eindhoven funded a community intervention programme in two deprived neighbourhoods. The objectives of the programme “Wijkgezondheidswerk” (Dutch for “Working on Healthy Neighbourhoods”) were to improve health-related behaviour among adults aged 18 to 65 years. A community intervention trial with quasi-experimental design was used to assess the effectiveness of the 2-year intervention programme on (intermediate) outcomes of health-related behaviour. An impact evaluation (Kloek et al. in press), process evaluation and participatory action research among stakeholders (Weijters & Koelen 2003) were part of the evaluation study. In this article we report the results of the process evaluation that was conducted during the implementation phase of the intervention from September 2000 to September 2002.
Community interventions are complex interventions that are implemented at multiple levels, include multiple audiences and consist of multiple components, so that process evaluation becomes important in order to know the extent to which all intervention components are actually implemented. Furthermore, process evaluation can help explain why certain results were achieved (Baranowski & Stables 2000; Linnan & Steckler 2002). The purpose of this process evaluation is to examine whether the intervention programme was delivered as planned and to what extent the target population was exposed to the intervention programme. Before discussing the results of the process evaluation we continue with a brief history of the preparatory phase of the programme “Wijkgezondheidswerk”.
The programme “Wijkgezondheidswerk”
The Municipal Health Services (MHS) initiated the programme “Wijkgezondheidswerk” in 1999. The preparatory phase of the intervention from June 1999 to September 2000 was conducted as follows.
Beforehand, the MHS, as the lead organisation, assigned two deprived neighbourhoods (“Tivoli” and “De Bennekel”) to receive the intervention. Geographically, one of these neighbourhoods consisted of two smaller areas. Therefore, three comparison neighbourhoods were matched with the intervention neighbourhoods, using information on sociodemographic and health-related behaviour variables. The programme started in June 1999 with a meeting in which several professional organisations and local grassroots organisations brought up health issues that could be tackled in the neighbourhoods. Thereafter the MHS started to develop two neighbourhood coalitions in both neighbourhoods, which should plan and deliver the intervention programme. In January 2002, the implementation of intervention activities was postponed because of differences in opinion about the intervention goals and allocation of tasks between the neighbourhood coalitions and the MHS. The MHS was bound to its contract with the external funding agency and therefore had to focus on health related behaviour, while the other organisations preferred to implement activities more generally related to health. To solve these competing opinions, the power and control of the programme were given explicitly to the neighbourhood coalitions. Furthermore, an assessment of the capacity of manpower of all participating organisations contributed to a better collaboration between the lead organisation and the neighbourhood coalitions. In May 2000, the neighbourhood coalitions achieved an agreement on the intervention goals with the lead agency. The goals were related to Lalonde’s health fields (Lalonde 1974) and in the field of lifestyle both neighbourhoods chose nutrition, and especially fruits and vegetable consumption and physical activity as their main intervention topics. A third topic in “De Bennekel” was smoking and in “Tivoli” excessive alcohol consumption. The implementation of intervention activities started in September 2000. At this same moment the baseline data for the impact evaluation were collected from a random sample of 4800 residents in the intervention and comparison neighbourhoods aged 18–65 years. A follow-up of the baseline cohort was conducted in September 2002, after the end of the 2-year implementation phase.
The results of the impact evaluation showed that the intervention had no impact on outcomes of vegetable consumption, physical activity, smoking and alcohol consumption, or on intermediate outcomes (including knowledge, attitudes, self-efficacy expectations, awareness of one’s one behaviour, and stages of change) of vegetable consumption or physical activity. There was evidence of a small impact on fruit consumption and intermediate outcomes of fruit consumption including knowledge related to fruit and vegetable consumption, stages of change for fruit consumption, and self-efficacy expectations for fruit consumption.
Methods
Key process evaluation components
To describe the process evaluation data we used the key process evaluation components as defined by Linnan & Steckler (2002): (1) context or environment that may influence intervention implementation, (2) recruitment of participants on the individual and community level, (3) dose delivered by the intervention providers, (4) reach of the target audience, (5) dose received by the target audience, and (6) fidelity of intervention delivery. The measure of fidelity addressed whether the intervention was carried out according to important key principles of the “community approach”; participation of residents in planning and implementing the intervention, intersectoral collaboration of organisations from relevant sectors, tailoring to local needs and existing structures, incorporation of a social ecological perspective, use of multi-strategies and a broad lifestyle approach (van Assema & Willemsen 1998).
Data collection of process evaluation components
A representative of the MHS kept minutes of all community coalitions’ meetings. The minutes of the meetings were used to determine the number of meetings and members during intervention implementation. They were also screened for additional information on intervention activities, especially for activities that were not implemented. Registration forms were used by the MHS to record information on the intervention activities about content and duration of the activity, target groups, number of participants, successful and unsuccessful elements, and sustainability. In the postal survey that was conducted in September 2002 among the general population of the intervention and comparison neighbourhoods, questions were asked about programme awareness and programme participation. To measure programme awareness, we asked, “Do you know the ‘Wijkgezondheidswerk’ Project?”: yes, yes and I also recognize the projects’ logo, I am not sure, no. To measure programme participation, we asked, “Did you participate in activities of the ‘Wijkgezondheidswerk’ Project?”: yes, I am not sure, no. Furthermore, the survey included questions about awareness of and participation in several large-scale intervention activities, the monthly newsletter, and sociodemographic factors. The final survey sample comprised 1 929 respondents who completed both the baseline and follow-up surveys (attrition rate: 31%). Lost to follow-up did not differ between intervention and comparison neighbourhoods. To assess the context of the intervention we used available statistics of the municipality of Eindhoven (Anonymous 1999b; Anonymous 1999a; ten Caten 2003). To assess the dose delivered by the intervention providers and reach of the target audience we used data from the registration forms, and to assess the dose received by the target audience we used the data from the postal survey.
Data analyses
The data on the registration forms were entered in a computerized database. If applicable, additional information from the minutes was also added in this database. “Dose delivered” was computed by summing the number of activities across intervention topic and intervention period for each intervention neighbourhood. “Reach” was computed by summing the number of participants of the intervention activities across topic, objective and channel of communication for each intervention neighbourhood. “Dose received” was computed using the survey measures of programme awareness and participation for each intervention neighbourhood and the comparison neighbourhoods. Analyses were conducted using SAS version 8.2. Frequency distributions were calculated on all process measures in the survey. Multivariate logistic regression was used to identify sociodemographic and health behaviour correlates of intervention exposure and reading the newsletter.
Results
Context
Table 1 shows the socioeconomic characteristics of the assigned neighbourhoods for the community intervention. The deprived neighbourhoods showed a higher presence of ethnic minority groups, higher unemployment rates, higher rates of rented property, lower mean household income, and a higher presence of lower educated residents. Furthermore, a citywide health survey had shown that there were considerable differences in health and lifestyle between the deprived neighbourhoods and the non-deprived neighbourhoods in Eindhoven. Community change was induced at the neighbourhood level when the MHS started to mobilize relevant sectors around the problem of health in the neighbourhoods (Fig. 1). In 1999, the intervention programme started as a top-down programme with a priori defined goals. A further description of the early phases of intervention development can be found in the introduction of this article. During the implementation phase, the neighbourhood coalitions planned most intervention activities. The MHS was the initiator of almost one third of the intervention activities, almost half were initiated by other participating professional organisations and 20 % by either the residents themselves or the neighbourhood coalition. Not all participating organisations had enough manpower for intervention delivery. Therefore, main executers of intervention activities were the MHS (40 %), Social Work organisations (19 %), residents (16 %) and other professional organisations (25 %). The focus of most intervention activities was on lifestyle and not on one of the other determinants of health. The total budget for intervention implementation was 90 000 Euros.
Table 1.
Intervention neighbourhoods | Comparison neighbourhoods | Eindhoven | ||
---|---|---|---|---|
De Bennekel | Tivoli | |||
Number of residents | 6 654 | 1 764 | 8 332 | 199 896 |
Number of households | 2 927 | 766 | 3 781 | 86 203 |
Ethnic Origin | ||||
% Moroccan | 5 | 2 | 2 | 2 |
% Turkish | 8 | 14 | 17 | 4 |
% Surinamese/Antillean | 5 | 5 | 5 | 3 |
% Unemployment | 12 | 14 | 14 | 7 |
Housing | ||||
% Rented house | 80 | 95 | 82 | 63 |
% Private property | 20 | 5 | 18 | 37 |
Movement dynamics* | 1,11 | 0,89 | 1,36 | 1,00 |
Mean household income** | EUR 15 200 | EUR 14 200 | EUR 15 033 | EUR 18 100 |
% low household income < EUR 7 260 | 23 | 19 | 23 | 14 |
Educational level | ||||
% I (low) | 21 | 29 | 22 | 11 |
% II | 41 | 49 | 35 | 34 |
% III | 21 | 21 | 19 | 27 |
% IV(high) | 16 | 9 | 23 | 28 |
* movement dynamics in the city of Eindhoven are set to 1.00. If neighbourhood value >1.00 movement dynamics are above the city average
** standardized for household size and household composition
Recruitment
At the neighbourhood level the MHS recruited members from various professional organisations and the local grassroots organisation to form a neighbourhood coalition (Fig. 1). The coalition’s goals involved assessing the situation, deciding what action to take, and implementing the programme. Representatives of the MHS, community social work organisations, Municipal Task Force of Social Development, local grassroots organisation, a general practitioner and researchers participated in the neighbourhood coalitions. In the beginning eight organisations were involved in the coalition (see Tab. 2). After a merger of two community social work organisations, seven organisations were left in the second year of implementation. To ensure awareness of the programme at higher organisational levels of the participating professional organisations a community advisory board was established at the city level.
Table 2.
September 2000–September 2001 | September 2001–September 2002 | ||||
---|---|---|---|---|---|
Tivoli | De Bennekel | Tivoli | De Bennekel | Overall | |
Number of members** | 9 (8) | 9 (8) | 7 (7) | 7 (7) | - |
Number of meetings | 10 | 8 | 7*** | 10*** | 35 |
Nutritional activities | 3 | 3 | 2 (1) | 2 | 10 (1) |
Physical activity activities | 1 (1) | 5(1) | 2 (2) | 7 | 15 (4) |
Smoking activities | n.a. | 1 (1) | n.a. | 1 (1) | 2 (2) |
Alcohol activities | 0 | n.a. | 0 | n.a. | 0 |
Other activities related to health | 5(1) | 2 (1) | 3 (1) | 6 | 16 (3) |
Total number of activities | 9 (2) | 11 (3) | 7 (4) | 16 (1) | 43 (10) |
* in parenthesis the number of activities that were not started (in total N = 10) due to low participation rates (n = 6), implementation problems (n = 3) or bad weather conditions (n = 1)
** in parenthesis the number of organisations involved in the neighbourhood coalition
*** including one meeting organized with both neighbourhood coalitions together in January 2002
At the beginning of the project, community participation was limited to participation of a single member of the local grassroots organisation, but as the project progressed, the level of community participation also grew. At the end of the second year, the coalition had implemented several intervention activities that were proposed and organized by residents themselves. In nearly one third of the intervention activities, one or more volunteers were involved.
Participants of the intervention activities were recruited in various ways. Most activities were announced in the programs’ monthly newsletter and also on posters hanging outside the information shop of the local grassroots organisations. Smokers received a letter from their general practitioner to call for their participation in a quit-smoking course. Participants for school-based activities were recruited by means of a letter from their children’s’ teacher. Two activities used a door-to-door method to recruit participants.
Dose delivered
In total the neighbourhood coalitions planned and developed 53 activities of which 10 could not be implemented due to low participation rates of neighbourhood residents, implementation problems or bad weather conditions (Tab. 2). The coalitions implemented, 10 nutritional activities, 15 physical activity activities, 2 smoking activities, 16 activities that were related to health in general, and no activities directed at excessive alcohol consumption. Twenty were implemented in the first year of the implementation phase and 23 in the second year. The “De Bennekel” coalition implemented more activities (n = 27) than the “Tivoli” coalition (n = 16). Table 3 shows an overview of all implemented intervention activities (n = 43) by topic, objective and channel of communication. One third of the implemented activities were directed to attracting attention or providing information, one third was directed at increasing awareness or knowledge, and one third was directed at behaviour change. The most frequently used channels of information were face-to-face multiple session activities (courses with multiple sessions to improve health related behaviour) and one-off special events. Only one activity was directed at the neighbourhood environment and only one activity was evidence-based. (All intervention activities are listed by neighbourhood and topic in the appendix at the end of the article.)
Table 3.
Objective | Channel of communication | De Bennekel | Tivoli | ||
---|---|---|---|---|---|
Activities (n) | Participants (n) | Activities (n) | Participants (n) | ||
Nutritional activities | |||||
attention/information | special event | 2 | 115 | - | - |
knowledge/awareness | face-to-face: single session | 3 | 66 | 2 | 221 |
face-to-face: multiple session | - | - | 2 | 33 | |
behavior change | environmental | - | - | 1 | unknown |
Physical activity activities | |||||
attention/information | special event | 3 | 601 | 1 | 300 |
knowledge/awareness | face-to-face: single session | 2 | 44 | - | - |
behavior change | face-to-face: multiple session | 3 | 26 | 2 | 35 |
special event | 4 | 78 | - | - | |
Smoking activities | |||||
behavior change | face-to-face: multiple session | 2 | 13 | - | - |
Other activities related to health | |||||
attention/information | special event | 2 | 400 | 3 | 450 |
mass media* | 2 | 3 000 | 2 | 700 | |
knowledge/awareness | face-to-face: single session | - | - | 1 | 16 |
face-to-face: multiple session | 3 | 60 | 2 | 46 | |
behavior change | face-to-face: multiple session | 1 | 10 | - | - |
Total | 27 | 1413 | 16 | 1 101 |
* The distribution of the direct mail newsletter started in September 2000 and was continued in September 2001 in both neighbourhoods. The number of participants is equal to the number of households that received the newsletter each time (they are not added up to the total number of participants).
Reach
The intervention programme reached about 1 400 residents in De Bennekel and about 1 100 in Tivoli, respectively 21 % and 62 % of the total number of residents (Tab. 3). These numbers could not be corrected for double counts or participating residents from other neighbourhoods and they do not include the households that received the monthly newsletter. Thirty percent of the activities had between 11 and 25 participants. Several special events and activities in schools attracted large numbers of participants (n > 100). Almost half of the interventions were directed at adults in general, one third at adults and their children and there were several activities for ethnic groups (n = 5) or adults over 55 years old (n = 3). The easyto-read monthly newsletter included information on intervention activities to come or that took place and information on health related behaviour topics. Of the respondents who knew the “Wijkgezondheidswerk” programme, 82 % remembered the newsletters. Among the respondents who remembered the newsletters, 41 % reported that they had read the newsletter often/always, 43 % read it sometimes, and 13 % never read it. In general, respondents thought the newsletters were interesting and understandable. Multivariate logistic regression showed that age and ethnicity were independently associated with reading the newsletters. The odds ratios for younger respondents age 18–24 and 25–44 to have read the newsletter were 0.21 (95 % CI 0.08–0.60) and 0.50 (95 % CI 0.30–0.81) respectively when compared to that of respondents aged 45–66. The odds ratios for Dutch respondents was 1.75 (95 % CI 1.03–2.98) compared to respondents from other ethnic origin.
Dose received
Table 4 shows that between 69 %–71 % of the respondents of the postal survey in the intervention neighbourhoods were aware of one or more large-scale intervention activities of the programme and 11 %–13 % participated in one of these activities. In the comparison neighbourhoods, 11 % of the respondents were aware and 1 % participated in one or more large-scale activities in the intervention neighbourhoods. However, only 24 % of the respondents in the intervention neighbourhoods and 7 % of the respondents in the comparison neighbourhoods knew the programme by its name because of e.g. the monthly newsletter, information in the neighbourhoods’ information shop, other residents or the general practitioner. In the intervention neighbourhoods, 3 % of the respondents participated in activities of the programme “Wijkgezondheidswerk”. Multivariate logistic regression showed that neighbourhood and age were independently associated with participation in intervention activities. Residents from intervention neighbourhoods were more likely to participate than respondents from comparison neighbourhoods, but participation in Tivoli was also 2.89 (1.27–6.39) times larger than participation in De Bennekel. The odds ratios for respondents, age 18–24 and 25–44 years, were 0.20 (95 % CI 0.03–1.05) and 0.37 (95 % CI 0.15–0.88) respectively compared to that of older respondents, age 45–66 years.
Table 4.
Large scale intervention activities | Awareness | Participation | ||
---|---|---|---|---|
% (95% CI) | n* (95% CI) | % (95% CI) | n* (95% CI) | |
De Bennekel (N=713) | ||||
Overall: One or more activities | 70 (67–73) | 2 904 (2 776–3 032) | 13 (11–15) | 539 (445–633) |
Direct mail newsletter De Bennekel | ||||
A WGW newsletter to all 3,000 households | 42 (39–45) | 1 743 (1 605–1 880) | n.a. | n.a. |
Bennekel on the move | ||||
A WGW community event concerning physical activity | 42 (39–45) | 1 743 (1 605–1 880) | 4 (3–5) | 166 (111–221) |
Annual Bennekel event | ||||
Community event organized by local grassroots organization | 56 (53–59) | 2 323 (2 185–2 462) | 8 (6–10) | 332 (256–408) |
GALM intervention | ||||
A physical activity intervention for inactive 55+ adults** | 25 (19–31) | 147 (114–179) | 6 (3–9) | 35 (17–53) |
Opening of the Verhulst square in De Bennekel | ||||
Offical launch of new playing/sports field for children | 35 (32–38) | 1 452 (1 319–1 585) | 4 (3–5) | 166 (111–221) |
Tivoli (N=280) | ||||
Overall: One or more activities | 72 (67–77) | 827 (775–879) | 11 (8–14) | 126 (90–163) |
Direct mail newsletter Tivoli | ||||
A WGW newsletter to 700 households | 37 (32–42) | 425 (369–481) | n.a. | n.a. |
Tivoli health market | ||||
A WGW community event concerning healthy behaviors | 60 (55–65) | 689 (633–746) | 5 (3–7) | 57 (32–83) |
Tivoli on the move | ||||
A WGW community event concerning physical activity | 56 (51–61) | 643 (586–701) | 5 (3–7) | 57 (32–83) |
Comparison neighborhoods (N=829) | ||||
Overall: One or more activities | 11 (9–13) | 696 (582–809) | 1 (0–2) | 63 (27–99) |
Note 1 WGW = programme “Wijkgezondheidswerk”
Note 2 Distribution of awareness and participation is calculated only among residents of the neighbourhood where the event took place.
* extrapolated absolute number of participants based on the total number of residents in the age of 18–66 years old living in De Bennekel (N = 4 149), Tivoli (N = 1 149) or comparison neighbourhoods (N = 6 323)
** in respondents age 55+ (N = 586)
Fidelity
Table 5 lists the key principles of the “community approach” and whether they were carried out sufficiently or insufficiently. Most key principles were carried out sufficiently except for community participation and the use of an ecological perspective.
Table 5.
Key principles of community approach | Carried out | Remark | |
---|---|---|---|
+ | +/− − | ||
1. Participation of residents in planning and implementing the intervention | X | In the first year, participation of residents was mostly limited to the involvement of a single member of the local grassroots organisation in the neighbourhood coalition. In the second year, there was also involvement of residents in organising activities. Furthermore the neighbourhood coalition was more frequently contacted by residents to express their wishes or ideas on health related topics. | |
2. Intersectoral collaboration | X | Relevant professional organisations including the local grassroots organisations and the general practitioner participated in the neighbourhood coalitions. All organisations were involved in planning, developing and implementing the intervention. | |
3. Tailoring to local needs and structures | X | The programme “Wijkgezondheidswerk” was incorporated in the municipal action plans for deprived neighbourhoods, with an emphasis on the use of a “community approach” to promote health related behaviour. To achieve interaction between policy makers and the neighbourhood coalitions a community advisory board was established. | |
4. Incorporation of a social ecological perspective | X | The intervention used an ecological framework, targeting its efforts at the intrapersonal, interpersonal, environmental and community level. Most intervention activities were directed at the intrapersonal level. Organisational elements targeted the interpersonal and community level. Only one activity was directed at the environmental level. | |
5. Using multi-strategies | X | The health related activities were mostly organized on a small-scale. Some of these activities already existed, some were especially developed. Some activities existed already on the city level and were adapted to be implemented at the neighbourhood level. The effectiveness of most activities was not established. Only one large-scale activity, a movement programme for older adults was evidence-based (Stevens et al. 1999). Activities were only implemented if it was likely that they could be sustained. | |
6. Using broad lifestyle approach | X | The coalition plans for intervention implementation were not only directed to health related topics but also to the physical and social environment of the neighbourhood. The main goals that focused on health related behaviour were to increase awareness and consumption of fruits and vegetables and to increase moderate-intensity physical activity. Furthermore the coalitions defined goals for action on barriers in the social and physical environment and health care services. |
+ = sufficient, +/− = not sufficient/not insufficient, − = insufficient
Discussion
The programme “Wijkgezondheidswerk” was implemented according to most of the key principles of the “community approach”. Intervention delivery was less sufficient in its ecological reach with regard to the environmental level and in community participation, especially in the first implementation year. Most intervention activities were directed at attracting attention, providing information, and increasing awareness and knowledge. One third of the activities had the objective to change behaviour. Programme awareness and programme participation were 24 % respectively 3 % among residents in the intervention neighbourhoods. What are the lessons learned from this process evaluation?
The difference in goals and priorities found between the neighbourhood coalition and the MHS had two important implications. Firstly, single intervention components using intuitively reasonable methods to promote behaviour change satisfied the need of the neighbourhood coalitions. This approach may be viewed as being in opposition to the need of researchers for evidence-based decision making, to achieve an impact on health behaviour outcomes by using effective intervention components. Secondly, to measure programme effectiveness on health behaviour outcomes preferably all activities should be focused at the chosen health behaviours and target population of the impact evaluation. In practice, it appeared to be impossible to expect that the community coalitions would implement intervention activities under such strict conditions.
Furthermore, the neighbourhood coalition members shared knowledge, skills, expertise, perspectives and connections, but financial resources from outside the coalition paid most of the costs that were associated with intervention implementation and the MHS delivered most of the manpower to organize the activities. In this context, sustainability of such a community coalition will depend on new funding and sustained involvement of the collaborating organisations and in particular the MHS for their manpower.
According to the participatory action research the level of community participation depended on the role residents had in the project, good intersectoral collaboration, the phase of intervention delivery, and the kind of intervention activity (Weijters & Koelen 2003). Some tasks in the delivery of the intervention were just better suited to professionals and full control of community members on these tasks was not necessary. At the beginning of the programme community participation was limited to a single member of the local grassroots organisations, but as the programme progressed, the level of community participation grew.
The results of the impact evaluation showed that the programme “Wijkgezondheidswerk” only demonstrated weak evidence for a small intervention effect on outcomes of fruit consumption and no evidence for an intervention effect on other outcomes of health related behaviour (Kloek et al. in press). The insufficient strength of the total package of intervention activities is likely to play a role in the absence of a clear intervention effect. As mentioned earlier, most of the activities were based on intuitively reasonable methods and only one intervention component in the total package was evidence-based. Furthermore, two thirds of the activities of the intervention programme were directed at attracting attention, providing residents with information and increasing awareness and knowledge of health related behaviour, which are the first steps towards behavioural change. As a result, we may expect programme effects on knowledge, attitudes and beliefs. The intervention only showed a small impact on intermediated outcomes of fruit consumption including, knowledge, stages of change and self-efficacy expectations.
During the 2-year implementation period just one activity was focused on the neighbourhood environment. In one of the intervention neighbourhoods the local greengrocer’s shop advertised in the monthly newsletter with special offers. Although we do not know which intervention components contributed to the small effect on fruit consumption, this same intervention neighbourhood showed a higher increase in fruit consumption than the other intervention neighbourhood. Possibly, the attention for individual characteristics combined with changes in the environment attained synergy in promoting behaviour change. Furthermore, changes at the environmental level are difficult to achieve in a short time frame and they are also more difficult to achieve for health practitioners.
The intervention reached a considerable part of the general population in the intervention neighbourhoods, but probably lacked sufficient tailoring of intervention activities to reach all different segments within the community. Especially, men and younger residents were less often reached with the intervention activities. In addition, several events and activities were school-based, whereas the school population did not belong to the target population of the evaluation study. The monthly newsletter was important in reaching the residents of the intervention neighbourhood, although it was less often read by ethnic minority groups and younger residents. In practice, it was impossible to expect that the neighbourhood coalitions would implement only intervention activities for the chosen target population.
The self-reported awareness of large-scale intervention activities showed that a lot of respondents were aware that these activities were organized in their neighbourhood but most of them were unaware that these activities were part of the programme “Wijkgezondheidswerk”. Participation in the intervention activities was quite low; 3 % of the respondents were aware that they participated in one of the intervention activation. Given this low participation rates, behaviour change at the community level is not very likely.
Although there are a number of projects that evaluated a community intervention in a deprived setting, comparing the results of process evaluations remains difficult, as these are often not available, the setting is too different or other process measures were used. Best comparable are the process outcomes of the Coeur en Santé St-Henri programme (Paradis et al. 1995; O’Loughlin et al. 1999) in which also awareness and participation outcomes were measured. In this programme, like in our study, there was a large difference between awareness of and participation in intervention activities. Furthermore, their results also showed some contamination of the comparison neighbourhoods and their perceived barriers during implementation like the difficulties with sustained commitment of participating organisations were recognizable.
The process evaluation had several limitations. At this moment, there is some general consent about the key components of process evaluation, which were used to structure this process evaluation (Linnan & Steckler 2002). However there are no standardized methods available to measure these individual components. Furthermore, the data collection methods we used could not capture all potentially important process information. For example, the assessment of the “dose received” was only done in the follow-up survey among the general population. Assessment of the “dose received” for all implemented intervention activities among participants of the interventions would have given more information on the short-term effects of these single intervention components. Furthermore, some of our process evaluation measures are based on observation of documents and minutes concerning intervention development and implementation. Although this was done systematically, the results are subject to interpretation bias.
A last remark concerns the complexity of community interventions and the challenges they pose for evaluation. It is important that future investments should be made and judged within the reality of community intervention practice. Beforehand, practitioners and researchers should decide what are realistic and valid outcomes of a proposed community health programme. Are they interested in individual risk behaviour change or in the process of community change? The answer to this question will be clearly linked to the goals of the community intervention. Furthermore, they should decide a priori what the acceptable level of implementation would be. Is a community health programme successfully implemented if e.g. and intervention is implemented according to key principles of the “community approach” or should implementation objectives be directed at more specific goals for the key process evaluation components reach, dose delivered, dose received and fidelity. Measuring programme implementation is critical in avoiding a type III error (Schwartz & Carpenter 1999), i.e. evaluating a programme that has not been adequately implemented, and thus drawing incorrect conclusions about the effectiveness of a given intervention.
In conclusion, this process evaluation indicated that it was feasible to implement a community intervention according to the key principles of the “community approach” in deprived neighbourhoods. However, it is unlikely that the total package of intervention activities had enough strength and sufficient exposure to attain community-wide health behaviour change. The results from this process evaluation were useful to show what was planned and what was achieved and they helped to explain the results from the impact Evaluation. Furthermore, the data can contribute to the international exchanges of experiences with development, implementation and evaluation of community interventions in deprived settings.
Acknowledgement
This research project was supported by a grant from the Dutch Health Research and Development Council (ZonMw) (No. 22000045) and the study protocol was approved by the Medical Ethical Committee of the Catharina Hospital, Eindhoven, The Netherlands.
Apppendix 1 Overview of intervention activities in De Bennekel by topic (n = 31)
Intervention activity: description | ||||||||
---|---|---|---|---|---|---|---|---|
Initiator* | Executor | Volunteers | Goal | Target group | Channel of communication | Participants (n) | Costs | Implemented |
Nutritional activities | ||||||||
“Healthy sweets”: information on healthy sweets for parents of children at the daycare centre from a dietician and making healthy sweets in practice. | ||||||||
other | municipal health service | no | knowledge/awareness | adults and children | face-to-face; single session | 43 | EUR 550 | yes |
Nutrition project on primary school: during one week several lessons were focused on nutrition. Parents (n=4) received dietary information from a dietician | ||||||||
other | municipal health service | no | knowledge/awareness | adults and children | face-to-face; single session | 304 | EUR 290 | yes |
“Healthy salads”: information on healthy nutrition for parents of children at the daycare centre from a dietician and making healthy salads in practice. | ||||||||
other | municipal health service | no | knowledge/awareness | adults and children | face-to-face; single session | 19 | EUR 115 | yes |
Ethnic womens’ party: party for ethnic women where nutrition education information was available in the own language | ||||||||
collaborating organization | social work organization | yes | attention/information | adults (ethnic group) | special events | 70 | EUR 15 | yes |
Volunteers day: all volunteers of the local grassroots organization received a healthy present. | ||||||||
residents | residents | yes | attention/information | adults | special events | 45 | EUR 50 | yes |
Physical activity activities | ||||||||
GALM: physical activity intervention for inactive adults 55+ including 30 lessons of 1 hour. Recruitment by former participants. | ||||||||
other | other | yes | behavior change | older adults (55+) | face-to-face; multiple session | unknown | EUR 2 382 | yes |
“Aqua slim”: course of 12 swimming lessons and 6 dietary information lessons | ||||||||
lead organization | municipal health service | no | behavior change | adults | face-to-face; multiple session | 7 | EUR 0 | no |
Gymnastics class 2000/2001: weekly gymnastic lessons for women | ||||||||
collaborating organization | social work organization | no | behavior change | adults | face-to-face; multiple session | 12 | EUR 0 | yes |
Bicycle prize-giving: a new bike for one of the respondents of the questionnaire | ||||||||
lead organization | municipal health service | no | attention/information | adults | special events | 1 | EUR 750 | yes |
“On the way” 2001: swim instruction for children and information about swimming lessons for parents from non-Dutch origin | ||||||||
collaborating organization | social work organization | no | attention/information | adults (ethnic group) | face-to-face; single session | 20 | EUR 34 | yes |
Bennekel on the move: community event concerning physical activity | ||||||||
lead organization | municipal health service | yes | attention/information | adults and children | special events | 400 | EUR 4 550 | yes |
Gymnastics class 2001/2002: weekly gymnastic lessons for women | ||||||||
collaborating organization | social work organization | no | behavior change | adults | face-to-face; multiple session | 14 | EUR 0 | yes |
Walking tour winter 2001: walking event in the surroundings of De Bennekel, concluded with a meal. | ||||||||
neighborhood coalition | grassroots organization | yes | behavior change | adults | special events | 35 | EUR 82 | yes |
Indoor soccer activity: weekly soccer match for Maroccan men | ||||||||
residents | residents | no | behavior change | adults (ethnic group) | special events | 12 | EUR 390 | yes |
Residents’ quit smoking group: social support walking group for those who quited smoking in the Quit Smoking courses | ||||||||
residents | residents | yes | behavior change | adults | special events | 6 | EUR 775 | yes |
“On the way” 2002: swim instruction for children and information about swimming lessons for parents from non-Dutch origin | ||||||||
collaborating organization | social work organization | no | attention/information | adults and children | face-to-face; single session | 24 | EUR 35 | yes |
Walking tour summer 2002: walking event in the surroundings of De Bennekel, concluded with a meal | ||||||||
neighborhood coalition | grassroots organization | yes | behavior change | adults | special events | 25 | EUR 114 | yes |
Opening Verhulst square: official launch of new playing/sports field for children | ||||||||
collaborating organization | municipal task force | yes | attention/information | adults and children | special events | 200 | EUR 66 | yes |
Smoking activities | ||||||||
Course “Quit smoking” (1): course to stop smoking within a group under supervision of a teacher | ||||||||
lead organization | municipal health service | no | behavior change | adults | face-to-face; multiple session | 7 | EUR 1 001 | yes |
Course “Quit smoking” (2): course to stop smoking within a group under supervision of a teacher | ||||||||
lead organization | municipal health service | no | behavior change | adults | face-to-face; multiple session | 2 | EUR 0 | no |
Course “Quit smoking” (3): course to stop smoking within a group under supervision of a teacher | ||||||||
lead organization | municipal health service | no | behavior change | adults | face-to-face; multiple session | 6 | EUR 715 | yes |
Course “Quit smoking” (4): course to stop smoking within a group under supervision of a teacher | ||||||||
lead organization | municipal health service | yes | behavior change | adults | face-to-face; multiple session | 5 | EUR 0 | no |
Other activities related to health | ||||||||
Social map De Bennekel: a booklet including an overview of all health care organizations and professionals to be distributed among residents | ||||||||
lead organization | other | no | attention/information | adults | mass media | - | EUR 0 | no |
Direct mail newsletter De Bennekel 2000/2001: newsletter delivered 7 times during implementation phase to 3,000 households | ||||||||
lead organization | municipal health service | no | attention/information | adults | mass media | 3000 | EUR 504 | yes |
Health education 50+: course for professionals and volunteers on how to recognize health and psychosocial problems of older women | ||||||||
collaborating organization | other | yes | knowledge/awareness | professionals | face-to-face; multiple session | 12 | EUR 365 | yes |
Direct mail newsletter De Bennekel 2001/2002: newsletter delivered 10 times to 3,000 households | ||||||||
lead organization | municipal health service | no | attention/information | adults | mass media | 3000 | EUR 0 | yes |
Annual event De Bennekel: community event organized by social welfare and local grassroots organization | ||||||||
collaborating organization | social work organization | yes | attention/information | adults and children | special events | 400 | EUR 0 | yes |
Poster exhibition: exhibition of winning posters in the lifestyle poster competition | ||||||||
neighborhood coalition | municipal health service | no | attention/information | adults | special events | unknown | EUR 750 | yes |
Body Business: 8 lessons for children about body and health. Parents receive health information in the own language | ||||||||
other | social work organization | no | knowledge/awareness | adults and children | face-to-face; multiple session | 35 | EUR 275 | yes |
“Feeling good”: course including 6 lessons on how to recognize stress complaints and learn how to relax for women | ||||||||
collaborating organization | social work organization | no | behavior change | adults | face-to-face; multiple session | 10 | EUR 10 | yes |
“Just be healthy”: course for 55+ women to discuss questions/complaints/experiences of one’s own health | ||||||||
collaborating organization | social work organization | no | knowledge/awareness | older adults (55+) | face-to-face; multiple session | 13 | EUR 15 | yes |
* Initiator: lead organisation = Municipal Health Services, collaborating organisation = one of the professional organisations involved in the neighbourhood coalition, other = an organisation not involved in the neighbourhood coalition.
Appendix 2 Overview of intervention activities in Tivoli (n = 22)
Intervention activity: description | ||||||||
---|---|---|---|---|---|---|---|---|
Initiator* | Executor | Volunteers | Goal | Target group | Channel of communication | Participants (n) | Costs | Implemented |
Nutritional activities | ||||||||
“Healthy Lifestyle”: course of 3 lessons on healthy lifestyle and nutrition | ||||||||
lead organization | municipal health service | no | knowledge/awareness | older adults (55+) | face-to-face; multiple session | 23 | EUR 450 | yes |
Nutitrition information for mothers: 4 lessons from a dietician during mothers’ weekly meeting at the primary school of their children | ||||||||
lead organization | municipal health service | no | knowledge/awareness | adults | face-to-face; multiple session | 10 | EUR 285 | yes |
“Healthy Sweets”: children make healthy sweets under supervision | ||||||||
residents | grassroots organization | yes | attention/information | adults and children | face-to-face; single session | 200 | EUR 105 | yes |
Collaboration greengrocer’s shop: the local greengrocer’s advertizes in the newsletter with special offers | ||||||||
lead organization | municipal health service | no | behavior change | adults | environmental | unknown | EUR 0 | yes |
Breakfast project: children at the daycare center and their parents (n=21) have breakfast together. Parents receive information on healthy breakfasts from a dietician. | ||||||||
other | municipal health service | no | knowledge/awareness | adults and children | face-to-face; single session | 61 | EUR 255 | yes |
“Slim club”: lessons on how to get slim including physical activity and nutrition information for women | ||||||||
collaborating organization | municipal health service | no | behavior change | adults | face-to-face; multiple session | 0 | EUR 0 | no |
Physical activity activities | ||||||||
Weekly swimming in a group | ||||||||
residents | residents | no | behavior change | adults | face-to-face; multiple session | 15 | EUR 0 | yes |
“Aqua slim”: course of 12 swimming lessons and 6 dietary education lessons | ||||||||
lead organization | municipal health service | no | behavior change | adults | face-to-face; multiple session | - | EUR 0 | no |
Tivoli on the move (1): community event concerning physical activity | ||||||||
lead organization | municipal health service | yes | attention/information | adults and children | special events | - | EUR 115 | no |
Gymnastics class for Turkish women: weekly gymnastic lessons | ||||||||
residents | grassroots organization | no | behavior change | adults (ethnic group) | face-to-face; multiple session | 20 | EUR 0 | yes |
Tivoli on the move (2): community event concerning physical activity | ||||||||
lead organization | municipal health service | yes | attention/information | adults and children | special events | 300 | EUR 4 435 | yes |
Walking tour: initiative to mark a walking tour in the surroundings of Tivoli | ||||||||
neighborhood coalition | grassroots organization | yes | behavior change | adults | special events | - | EUR 0 | no |
Other activities related to health | ||||||||
Direct mail newsletter Tivoli 2000/2001: newsletter delivered 8 times to 700 households | ||||||||
lead organization | municipal health service | no | attention/information | adults | mass media | 700 | EUR 450 | yes |
“Health market”: community event concerning health related behaviors | ||||||||
lead organization | municipal health service | yes | attention/information | adults and children | special events | 200 | EUR 2 000 | yes |
“Growing older in The Netherlands: course for older Turkish men and women seperately in their native language | ||||||||
collaborating organization | social work organization | no | knowledge/awareness | adults (ethnic group) | face-to-face; multiple session | 30 | yes | |
Information market municipal task force: presentation of the program “Wijkgezondheidswerk” | ||||||||
collaborating organization | municipal task force | no | attention/information | professionals | special events | 100 | EUR 0 | yes |
Welcome’s gift: all new residents in the neighborhood receive a gift to attract attention to the program | ||||||||
residents | grassroots organization | yes | attention/information | adults | special events | 150 | EUR 65 | yes |
Large-scale community meeting: for this meeting relevant professional organizations were invited to discuss new priorities and possibilities concerning health related activities | ||||||||
neighborhood coalition | municipal health service | no | attention/information | professionals | special events | - | EUR 0 | no |
Direct mail newsletter Tivoli 2001/2002: newsletter delivered 10 times to 700 households | ||||||||
lead organization | municipal health service | no | attention/information | adults | mass media | 700 | yes | |
Social map Tivoli: a booklet including an overview of all health care organizations and professionals to be distributed among residents | ||||||||
lead organization | other | no | attention/information | adults | mass media | - | EUR 0 | no |
Health project on primary school: during three weeks several lessons are focused on the topics dental hygiene, nutrition and physical activity. Parents (n = 43) also receive information on these topics. | ||||||||
lead organization | municipal health service | no | knowledge/awareness | adults and children | face-to-face; multiple session | 123 | EUR 590 | yes |
Information on dental hygiene: during mothers’ weekly meeting at the primary school of their children | ||||||||
other | municipal health service | no | knowledge/awareness | adults and children | face-to-face; single session | 16 | EUR 36 | yes |
* Initiator: lead organisation = Municipal Health Services, collaborating organisation = one of the professional organisations involved in the neighbourhood coalition, other = an organisation not involved in the neighbourhood coalition.
References
- 1.Anonymous (1999a). Area analysis Bennekel. Rotterdam: Inicio Rotterdam.
- 2.Anonymous (1999b). Area analysis Tivoli. Rotterdam: Inicio Rotterdam.
- 3.Baranowski T, Stables G (2000). Process evaluations of the 5-a-day projects. Health Educ Behav 27: 157–166. [DOI] [PubMed]
- 4.Goodman RM (1998). Principals and tools for evaluating community-based prevention and health promotion programs. J Public Health Manag Pract 4: 37–47. [DOI] [PubMed]
- 5.Kloek GC, van Lenthe FJ, van Nierop PWM, Koelen MA, Mackenbach JP (in press). Impact evaluation of a Dutch community intervention to improve health-related behaviour in deprived neighbourhoods. Health & Place doi:10.1016/j.healthplace.2005.09.002 [DOI] [PubMed]
- 6.Lalonde M (1974). A new perspective on the health of canadians. Ottawa. http://www.hc-sc.gc.ca/hcs-sss/alt_formats/hpbdgps/pdf/pubs/1974-lalonde/lalonde_e.pdf
- 7.Linnan L, Steckler A (2002). Process Evaluation for Public Health Interventions and Research. An Overview. In: Steckler A, Linnan L (eds). Process Evaluation for Public Health Interventions and Research. San Francisco: Jossey-Bass: 1–23.
- 8.Mackenbach JP, Bakker MJ (2003). Tackling socioeconomic inequalities in health: analysis of European experiences. Lancet 362: 1409–1414. [DOI] [PubMed]
- 9.Mackenbach JP, Stronks K (2002). A strategy for tackling health inequalities in the Netherlands. BMJ 325: 1029–1032. [DOI] [PMC free article] [PubMed]
- 10.O’Loughlin JL, Paradis G, Gray-Donald K, Renaud L (1999). The impact of a communitybased heart disease prevention program in a lowincome, inner-city neighborhood. Am J Public Health 89: 1819–1826. [DOI] [PMC free article] [PubMed]
- 11.Paradis G, O’Loughlin J, Elliott M, et al. (1995). Coeur en santé St-Henri — a heart health promotion programme in a low income, low education neighbourhood in Montréal, Canada: theoretical model and early field experience. J Epidemiol Community Health 49: 503–12. [DOI] [PMC free article] [PubMed]
- 12.Schwartz S, Carpenter KM (1999). The right answer for the wrong question: consequences of type III error for public health research. Am J Public Health 89: 1175–1180. [DOI] [PMC free article] [PubMed]
- 13.Stevens M, Bult P, de Greef MHG, Lemmink KAPM, Rispens P (1999). Groningen Active Living Model (GALM): Stimulating Physical Activity in Sedentary Older Adults. Prev Med 29: 267–276. [DOI] [PubMed]
- 14.ten Caten H (2003). Deprived neighbourhoods on the move, part 1. Five years of developments in social structure and developments in housing stocks [in Dutch]. Eindhoven: Dienst algemene en publiekszaken, Afdeling bestuursinformatie en onderzoek.
- 15.Weijters J, Koelen M (2003). Working together on health in the neighbourhood. Participatory action research in Eindhoven [in Dutch]. Wageningen: Wageningen University.
- 16.van Assema P, Willemsen M (1998). Community projects [in Dutch]. In: Damoiseaux V, Molen van der HT, Kok GJ, eds. Health education and behavioural change. Assen: Van Gorcum: 398–417.