Abstract
Objective: To assess the net 5 year effects of intervention of a community based demonstration project, the Heartbeat Wales programme, on modifiable behavioural risks for prevention of cardiovascular disease.
Design and setting: Quasi-experimental design comparing results from two independent cross sectional population surveys conducted in 1985 and 1990 in Wales and a matched reference area in north east England.
Subjects: Random, stratified samples of people aged 18-64 years (18 538 in 1985 and 13 045 in 1990) in Wales and in north east England (1483 and 4534, respectively).
Intervention: A coordinated range of activities for heart health promotion in Wales entailing public education campaigns along with supportive policy and infrastructure change. In the reference area no additional community heart health promotion was planned, though considerable activity did take place, “contaminating” the reference area.
Main outcome measures: Fifteen self reported behavioural indicators relating to dietary choice, smoking, frequency of exercise, and weight.
Results: Positive changes (for health) in behavioural outcomes were observed among the population in Wales, including a reduction in reported smoking prevalence and improvements in dietary choice. There was no net intervention effect for the programme over and above observed change in the reference area.
Conclusions: No definite conclusions can be drawn concerning the efficacy of the programme in terms of behavioural outcomes. With hindsight, the difficulties of evaluating such a complex multifaceted intervention were underestimated. Further debate on the most appropriate methods for assessing the effectiveness of community based health promotion programmes is called for.
Key messages
Heartbeat Wales was set up in 1985 as a community based programme to demonstrate risk reduction for cardiovascular disease
Important changes were observed in modifiable risks for cardiovascular disease in Wales between 1985 and 1990
There was an unexpectedly rapid uptake of heart health promotion activity in the reference area
No net intervention effects were found for the programme over and above changes in the reference area
Improvements in methods of evaluation for community based health promotion programmes are required
Introduction
Cardiovascular disease remains one of the major causes of morbidity and premature mortality in the United Kingdom.1 During the 1980s a consensus evolved on the need to reduce this toll of ill health and death through population-wide preventive measures (see, for example, papers by the World Health Organisation2 and Rose et al3). The Welsh Office and the existing national agency for health education, the Health Education Council, agreed to establish a community based demonstration programme in Wales directed towards reducing modifiable behavioural risks for cardiovascular disease.
The programme was publicly launched in 1985 as Heartbeat Wales with three strategic aims: leadership—to coordinate, support, initiate, and monitor action at local and regional levels which would encourage improvements in modifiable behavioural risks for prevention of cardiovascular disease; demonstration—to stimulate, disseminate, and assist the development of strategies and programmes to promote health and prevent cardiovascular disease throughout the United Kingdom; and experimentation—to research, develop, and evaluate a range of new projects and initiatives for heart health promotion and provide feedback on their feasibility and impact.4
Heartbeat Wales drew on the experiences of other community based risk reduction programmes for cardiovascular disease, particularly those in Finland and the United States.5–8 The programme used a range of established health promotion methods directed towards both changing health behaviours in individuals and achieving environmental, organisational, and policy changes that support healthy choices.9 Among the resources developed and interventions undertaken by Heartbeat Wales were television series with BBC Wales and HTV such as Don’t Break your Heart, Fit for Life, and the BBC Diet Programme; “Quit and Win,” a smoking cessation project; food labelling and nutrition education with a major grocery retailer; “Heartbeat Awards,” a restaurant and canteen scheme to increase the availability of healthy food choices and smoke free areas; and Make Health Your Business, a worksite health promotion programme with CBI Wales.
Further details of the Heartbeat Wales intervention have been published elsewhere.9–13 To assess behavioural outcomes of the intervention a quasi-experimental evaluation design was adopted on the basis of comparison of change in modifiable behavioural risks for cardiovascular disease in Wales with that in a reference area in the United Kingdom closest in sociodemographic and health profile to Wales at the 1981 census. The reference area selected was north east England (Tyne and Wear, Cleveland, Durham, and North Yorkshire). The Health Education Council indicated that there would be no major additional resources in that area for heart health promotion between 1985 and 1990.
Two population surveys were conducted in 1985 and 1990 in Wales and the reference area. To assist with the interpretation of the findings from these surveys, a range of other studies described elsewhere14 was also planned for Wales but not the reference area. These studies have suggested that Heartbeat Wales achieved its basic aim of establishing a region-wide approach to the prevention of cardiovascular disease and that many of the key elements of the programme have been taken up and used elsewhere both in the United Kingdom and overseas.4,10–13,15–18 It has also been shown that there were significant reductions in prevalence of smoking and improvements in food choices between 1985 and 1990 in Wales.19 This current paper compares these and other changes in modifiable behavioural risks for cardiovascular disease in Wales with those that took place in the reference area over the same time to assess net intervention effects of the Heartbeat Wales programme.
Subjects and methods
Data were collected in random sample, cross sectional surveys during the summer and autumn of 1985 and 1990. In each survey households were selected with a multistage cluster sampling design, within 10 strata defined by the nine Welsh district health authorities and the reference area. Sample size in the 1985 survey was determined (α=0.05, β=0.2) to detect a 5% change in prevalence of smoking within each strata by using a two tailed significance test. In 1990 sample size in the reference area was increased to improve the power of analyses that compared Wales with the reference area. Brief interviews were undertaken at each household, and one self completion questionnaire was then left for each resident aged 18-64. Respondents to all three surveys were asked a set of identical questions covering key health related behaviours such as smoking, diet, and physical activity as well as health knowledge and beliefs. In Wales the response rate for the household interview was 88% in 1985 and 79% in 1990 and the self completion response was 67% and 61%, respectively. In the reference area the respective figures were 84% and 77% for the household interview and 64% and 61% for the self completion questionnaire. Altogether, 31 583 questionnaires (18 538 in 1985 and 13 045 in 1990) were returned over the two surveys in Wales, with 6017 (1483 and 4534, respectively) returned in the reference area. Data were weighted before analysis by sex, age group, social class, and population distribution within each strata to minimise bias due to differential response rates between groups. Further details of survey methodology and weighting are available elsewhere.19
Data analysis
Fifteen indicators were selected as key outcomes for analysis. They represented those health related behaviours that were most consistently targeted during the intervention period and for which measurements were available. These indicators are listed and defined in table 1. Of the 15 indicators, eight represent dietary choices, five are concerned with smoking, and one each with participation in regular exercise and being overweight. Two sets of analyses were undertaken: firstly, at the level of the individual respondent; secondly, at the community level.
Table 1.
Definition of outcome variables
| Indicator | Definition |
|---|---|
| Health enhancing behaviours | |
| Chicken | Consume chicken or other poultry ⩾2 days/week |
| Fish | Consume fish ⩾2 days/week |
| Fruit | Consume fresh fruit ⩾4 days/week |
| Green vegetables | Consume green vegetables or salad ⩾4 days/week |
| Low fat milk | Mainly use skimmed or semi-skimmed milk at home |
| Wholemeal bread | Mainly use wholemeal bread |
| Smoking harmful | Smokers who agree that their present level of smoking is harmful to their health |
| Tried to stop smoking | Smokers who have made a serious attempt to give up in 12 months before survey |
| Advice on smoking from general practitioner | Daily smokers visiting their general practitioner in 12 months before survey who were advised to cut down or give up |
| Exercise | Engage in moderate or strenuous activity ⩾2 times/week for >20 minutes each time |
| Health compromising behaviours | |
| Butter | Mainly use butter on bread |
| Fried food | Consume fried food cooked in lard or other solid fat ⩾2 days/week at home |
| Daily smoking | Smoke daily |
| Cigarettes/day | Mean No of cigarettes/day smoked by daily smokers |
| Overweight | Body mass index (kg/m2) ⩾24 for women; ⩾25 for men |
Individual level analysis
Standard errors of survey estimates and 95% confidence intervals were estimated on weighted data from the 37 600 completed questionnaires returned in the two surveys by using the sudaan (survey data analysis for multistage sample designs) statistical software package.20 This package uses the Taylor series linearisation method to compute appropriate standard errors for estimates obtained from complex survey designs and takes account of the effects of stratification and clustering on the precision of survey estimates. For each indicator, percentage point changes between 1985 and 1990 in Wales and the reference area (with 95% confidence intervals) were calculated.
Community level analysis
Analysis to compare change in Wales with change in the reference area was undertaken at community level by using the nine district health authorities in Wales and the four counties in the reference area as the units for analysis.
The intervention effect to be estimated was defined as the ratio of percentage change between Wales and the reference area (PC90W/PC85W)/(PC90R/PC85R), where PCY= proportion reporting characteristic C in year (Y=1985 or 1990) in Wales (W) or the reference area (R).
For each of the 13 analysis units (nine district health authorities in Wales and four counties in the reference area) the prevalence of each of the 14 binary categorical variables was calculated for the baseline and follow up (1990) surveys. Similarly, mean number of cigarettes per day for daily smokers was calculated for each unit in the two surveys. The logarithm of the intervention effect ratio was then estimated by fitting the following model:
 |
where PCY =proportion reporting characteristic C in year (Y=1985 or 1990) in each unit (or mean number of cigarettes per day for daily smokers); Int=exposure to intervention: 1 in Wales, 0 in the reference area; and Z=ln(RC90)−ln(R85), where R is the proportion of respondents in each unit living in a household where the head of household is in a non-manual occupation. The variable Z was included as a covariate to adjust the analysis for variations between the surveys in the composition of social group of the samples within each unit. The parameter d was included to control for the possibility that the degree of change may be dependent on the baseline value.
Weighted least squares linear regression models were fitted for each of the 15 variables weighted by the mean sample size in each unit over the two surveys. Analyses were undertaken for all respondents, and additional analyses were also run for seven subgroups: men, women, three age groups (18-34, 35-49, 50-64 years), and people living in households where the head of household was in a manual or non-manual occupation. Two tailed t tests of the null hypothesis that the parameter b (the logarithm of the intervention effect) was equal to 0 were undertaken and the adjusted intervention effect ratio (exp(b)) and its 95% confidence interval were calculated.
Results
Table 2 shows the prevalence of the 15 key outcome indicators in both Wales and the reference area in 1985 and 1990. These data indicate that in Wales there were positive (for health) changes in all 15 indicators except the proportion overweight, with all changes being significant except for the proportion of smokers who had tried to stop and mean daily cigarette consumption. Similarly, in the reference area there were positive changes in 13 of the 15 indicators, although only nine of these were significant.
Table 2.
Prevalence of key indicators in Wales and reference area, 1985 and 1990, and percentage point changes, 1985-90 (95% confidence intervals). All subjects aged 18-64
| Key indicator | Wales
|
Reference area
|
|||||
|---|---|---|---|---|---|---|---|
| 1985 | 1990 | Change | 1985 | 1990 | Change | ||
| Chicken | 16.5 (15.7 to 17.3) | 31.8 (30.5 to 33.1) | 15.3 (13.8 to 16.8) | 15.8 (13.4 to 18.2) | 30.0 (27.7 to 32.3) | 14.2 (10.9 to 17.5) | |
| Fish | 19.5 (18.7 to 20.2) | 27.4 (26.4 to 28.5) | 7.9 (6.6 to 9.2) | 21.7 (19.4 to 24.0) | 28.4 (26.3 to 30.5) | 6.7 (3.6 to 9.8) | |
| Fruit | 48.3 (47.2 to 49.4) | 56.7 (55.4 to 58.0) | 8.4 (6.7 to 10.1) | 46.2 (41.1 to 51.3) | 54.8 (52.5 to 57.0) | 8.6 (3.0 to 14.2) | |
| Green vegetables | 47.8 (46.5 to 49.1) | 55.0 (53.4 to 56.5) | 7.2 (5.1 to 9.3) | 46.2 (40.0 to 52.5) | 55.6 (53.0 to 58.2) | 9.4 (2.7 to 16.1) | |
| Low fat milk | 16.4 (15.5 to 17.3) | 44.1 (42.2 to 46.0) | 27.7 (25.7 to 29.7) | 20.0 (16.7 to 23.3)* | 47.8 (44.9 to 50.7)* | 27.8 (23.4 to 32.2) | |
| Wholemeal bread | 28.2 (27.0 to 29.4) | 35.9 (34.4 to 37.4) | 7.7 (5.7 to 9.7) | 30.2 (24.9 to 35.4) | 37.6 (34.6 to 40.7) | 7.4 (1.3 to 13.5) | |
| Smoking harmful | 56.4 (55.1 to 57.7) | 65.6 (63.9 to 67.3) | 9.2 (7.0 to 11.4) | 52.9 (48.0 to 57.8) | 65.1 (62.3 to 68.0) | 12.2 (6.5 to 17.9) | |
| Tried to stop smoking | 33.1 (31.7 to 34.5) | 34.8 (32.9 to 36.7) | 1.7 (−0.6 to 4.0) | 35.6 (31.2 to 40.1) | 34.5 (31.9 to 37.1) | −1.1 (−6.3 to 4.1) | |
| GP advice on smoking | 38.4 (36.5 to 40.2) | 49.6 (47.1 to 52.1) | 11.2 (8.1 to 14.3) | 41.8 (35.4 to 48.2) | 49.0 (45.3 to 52.6) | 7.2 (−0.2 to 14.6) | |
| Exercise | 31.7 (30.9 to 32.5) | 33.8 (32.8 to 34.8) | 2.1 (0.8 to 3.4) | 33.7 (30.3 to 37.2) | 36.9 (35.0 to 38.7)† | 3.2 (−0.7 to 7.1) | |
| Butter | 43.1 (42.0 to 44.2) | 26.1 (24.8 to 27.4) | −17.0 (−15.3 to −18.7) | 36.8 (34.4 to 39.2)† | 22.1 (20.2 to 24.0)† | −14.7 (−11.6 to −17.8) | |
| Fried food | 32.5 (31.1 to 34.0) | 13.8 (12.6 to 15.0) | −18.7 (−16.8 to −20.6) | 42.8 (36.5 to 49.1)† | 21.3 (18.6 to 24.0)† | −21.5 (−14.6 to −28.4) | |
| Daily smoking | 32.5 (31.4 to 33.6) | 27.6 (26.2 to 29.0) | −4.9 (−3.2 to −6.6) | 36.0 (30.5 to 41.4) | 30.6 (28.3 to 32.9)* | −5.4 (−11.3 to 0.5) | |
| Cigarettes/day‡ | 17.16 (16.9 to 17.5) | 17.14 (16.8 to 17.5) | −0.02 (−0.48 to 0.44) | 17.55 (16.8 to 18.3) | 17.39 (16.7 to 18.0) | −0.16 (−1.18 to 0.86) | |
| Overweight | 43.2 (42.3 to 44.1) | 45.7 (44.5 to 46.9) | 2.5 (1.0 to 4.0) | 39.8 (36.3 to 43.3) | 40.9 (39.0 to 42.8)† | 1.1 (−2.9 to 5.1) | |
GP = general practitioner.
P<0.05 in Z tests for difference in proportions, Wales v reference area.
P<0.01 in Z tests for difference in proportions, Wales v reference area.
t test for difference in means.
Further analysis showed that the baseline prevalence of two of the indicators was significantly lower in Wales than in the reference area—namely, the preference for low fat milk and the consumption of fried food. By 1990 four indicators were significantly lower in Wales than in the reference area; these were preference for low fat milk, consumption of fried food, daily smoking, and participation in exercise. The preference for butter was significantly higher in Wales than in the reference area in both 1985 and 1990, while the proportion overweight was significantly higher in Wales in 1990 only.
Table 3 presents the findings from the community level analyses and indicates that there were two outcome indicators (consumption of fried food and daily smoking) for which there was a consistent intervention effect across all seven subgroups in favour of Wales. There was also one consistent difference (consumption of green vegetables) in favour of the reference area. These effects were each significant in no more than one subgroup, however, and when the community level analysis was undertaken with all the respondents no significant differences were found.
Table 3.
Community level regression models: estimates of intervention effect ratio (95% confidence intervals) for seven subgroups and all subjects
| Key indicator | Age (years)
|
Occupation
|
Men | Women | All subjects | ||||
|---|---|---|---|---|---|---|---|---|---|
| 18-34 | 35-49 | 50-64 | Manual | Non-manual | |||||
| Chicken | 1.10 (0.90 to 1.35) | 0.92 (0.72 to 1.18) | 1.09 (0.83 to 1.42) | 1.08 (0.85 to 1.36) | 0.87 (0.70 to 1.09) | 1.04 (0.85 to 1.28) | 0.96 (0.79 to 1.17) | 1.01 (0.84 to 1.20) | |
| Fish | 0.96 (0.72 to 1.26) | 1.16 (0.66 to 2.05) | 1.01 (0.82 to 1.24) | 1.05 (0.90 to 1.22) | 0.94 (0.76 to 1.16) | 1.01 (0.80 to 1.27) | 1.02 (0.85 to 1.21) | 1.00 (0.84 to 1.19) | |
| Fruit | 1.04 (0.90 to 1.19) | 0.93 (0.80 to 1.07) | 0.88 (0.73 to 1.08) | 0.99 (0.84 to 1.17) | 0.94 (0.82 to 1.08) | 0.97 (0.86 to 1.09) | 0.98 (0.84 to 1.15) | 0.97 (0.86 to 1.09) | |
| Green vegetables | 0.96 (0.88 to 1.03) | 0.95 (0.82 to 1.10) | 0.99 (0.87 to 1.12) | 0.96 (0.89 to 1.04) | 0.98 (0.90 to 1.06) | 0.95 (0.88 to 1.02) | 0.97 (0.88 to 1.07) | 0.96 (0.89 to 1.03) | |
| Low fat milk | 0.98 (0.85 to 1.13) | 1.02 (0.91 to 1.16) | 1.05 (0.86 to 1.27) | 1.03 (0.87 to 1.21) | 0.99 (0.81 to 1.20) | 1.05 (0.92 to 1.20) | 1.03 (0.95 to 1.12) | 1.03 (0.94 to 1.14) | |
| Wholemeal bread | 0.91 (0.76 to 1.09) | 0.91 (0.78 to 1.06) | 1.03 (0.87 to 1.21) | 0.96 (0.79 to 1.16) | 0.93 (0.82 to 1.05) | 0.96 (0.76 to 1.20) | 0.99 (0.90 to 1.09) | 0.98 (0.85 to 1.13) | |
| Smoking harmful | 1.02 (0.89 to 1.17) | 0.90 (0.69 to 1.17) | 0.96 (0.80 to 1.16) | 0.97 (0.84 to 1.12) | 1.01 (0.78 to 1.30) | 0.99 (0.87 to 1.13) | 0.94 (0.81 to 1.08) | 0.95 (0.85 to 1.07) | |
| Tried to stop smoking | 1.06 (0.85 to 1.31) | 0.94 (0.76 to 1.16) | 1.15 (0.92 to 1.45) | 0.95 (0.83 to 1.09) | 1.24 (0.94 to 1.63) | 1.19 (1.05 to 1.35)* | 0.96 (0.81 to 1.13) | 1.09 (0.89 to 1.32) | |
| GP advice on smoking | 0.97 (0.70 to 1.36) | 0.93 (0.74 to 1.18) | 1.08 (0.83 to 1.40) | 1.03 (0.88 to 1.19) | 1.07 (0.78 to 1.46) | 1.05 (0.84 to 1.32) | 1.02 (0.83 to 1.25) | 1.02 (0.95 to 1.09) | |
| Exercise | 1.00 (0.90 to 1.11) | 1.41 (1.00 to 1.99) | 0.94 (0.70 to 1.25) | 0.97 (0.87 to 1.07) | 0.93 (0.85 to 1.01) | 1.00 (0.93 to 1.08) | 0.87 (0.74 to 1.01) | 0.96 (0.78 to 1.17) | |
| Butter | 0.82 (0.60 to 1.13) | 1.18 (0.92 to 1.51) | 0.96 (0.72 to 1.19) | 0.87 (0.73 to 1.03) | 1.12 (0.89 to 1.39) | 0.96 (0.78 to 1.18) | 0.95 (0.83 to 1.09) | 0.96 (0.83 to 1.11) | |
| Fried food | 0.99 (0.72 to 1.37) | 0.86 (0.65 to 1.13) | 0.55 (0.37 to 0.83)* | 0.78 (0.57 to 1.08) | 0.92 (0.63 to 1.34) | 0.86 (0.65 to 1.14) | 0.84 (0.61 to 1.16) | 0.86 (0.64 to 1.14) | |
| Daily smoking | 0.93 (0.67 to 1.28) | 0.90 (0.73 to 1.10) | 0.83 (0.60 to 1.15) | 0.91 (0.75 to 1.09) | 0.99 (0.82 to 1.19) | 0.92 (0.76 to 1.12) | 0.91 (0.70 to 1.17) | 0.90 (0.73 to 1.10) | |
| Cigarettes/ day | 1.01 (0.90 to 1.13) | 0.97 (0.85 to 1.10) | 0.98 (0.87 to 1.09) | 0.99 (0.91 to 1.07) | 1.02 (0.93 to 1.13) | 1.01 (0.92 to 1.10) | 0.96 (0.86 to 1.07) | 0.99 (0.93 to 1.07) | |
| Overweight | 1.02 (0.77 to 1.36) | 1.14 (0.60 to 2.18) | 1.04 (0.94 to 1.15) | 1.07 (0.89 to 1.28) | 1.02 (0.87 to 1.19) | 1.06 (0.92 to 1.23) | 1.00 (0.86 to 1.15) | 1.02 (0.98 to 1.06) | |
GP=general practitioner.
P<0.05 in t test of log (adjusted intervention effect ratio).
Discussion
The results indicate important changes in modifiable risks to health among the population in Wales and in the reference area in the north east of England for the period 1985-90. These changes should lead to subsequent reductions in premature death from cardiovascular disease in Wales and the north east of England over the coming decade. As welcome as improvements in smoking levels, dietary habits, and exercise patterns may be, the results do not show clear and consistent net intervention effects of the Heartbeat Wales programme after 5 years in comparison with activities in the reference area.
Sample size and contamination
Interpretation of the results reported here, however, requires a clear understanding of the strengths and weaknesses of the study design and of the context of the intervention. The critical assumption made in the study design was that the contrast between the intervention in Wales and existing activity in the reference area would be large enough and sustained over a 5 year period to show a clear net intervention effect. This was not the case for two reasons. Firstly, the sample size at the baseline measurement in the reference area was too small to give sufficient statistical power for the detection of a likely net intervention effect. Secondly, a previous paper by the authors has documented the diffusion of Heartbeat Wales projects and programmes to the reference area far faster and to a far greater extent than had initially been expected, along with the introduction of additional resources for heart health promotion through the development of the Look After Your Heart project, which was launched across the whole of England in 1987, and the Heartbeat Yorkshire programme, which was conducted in the reference area from 1988.4 This paper clearly shows the “contamination” of the reference area with initiatives that can be traced back to Heartbeat Wales in whole or part, including the uptake of policy changes in the health services, adoption of several unique projects, and receipt of a number of mass media interventions, especially those developed with BBC Wales. In addition, the paper provides evidence of increases in funding for heart health promotion in the reference area commensurate with increases in Wales during the Heartbeat Wales intervention period.
In retrospect, it was naive to believe that a high profile programme such as Heartbeat Wales could remain in quarantine for such a long period. A direct result of its success as a national demonstration programme was the attenuation of differential exposure to heart health promotion between Wales and the reference area and thus a dilution of any measurable intervention effect. In addition to these identifiable confounding factors are the favourable secular trends in smoking and dietary choices in the United Kingdom as a whole, which further confuse the interpretation of results.21,22
Conclusions
Because of these design problems, no definite conclusions can be drawn concerning the efficacy of the Heartbeat Wales programme in terms of behavioural outcomes. Indeed, two directly conflicting conclusions could be drawn, both of which would be compatible with but not proved by the results presented here: on the one hand, the improvements in risk behaviours for cardiovascular disease in Wales suggest that the Heartbeat Wales programme has been effective, with positive changes in the reference area also associated with increased community heart health promotion; on the other hand, the lack of any net intervention effect compared with the reference area suggests that the Heartbeat Wales programme has had no impact additional to secular trends.
These results from the United Kingdom can be set alongside results from other comparable programmes in the United States operating during the same period. Although encouraging results have been reported from the Stanford five city project, these were obtained from a cohort of subjects in the intervention communities. As in Wales, no differences were found in the comparison between independent samples in the intervention population and the reference communities.6 Similarly, analysis of results from independent samples in the Minnesota and Pawtucket heart health programmes showed that intervention effects were “modest in size and duration and generally within chance levels”7 and “very limited.”8 The strengths of positive secular trends relating to behavioural risks for cardiovascular disease were cited as a reason why net effects were difficult to identify despite positive results in the intervention communities.23 Unlike Heartbeat Wales, there is no indication that these studies included active monitoring of the diffusion of the programme in the reference area(s) or the extent of contamination through other unpredicted events.
The major conclusion to be drawn from this study is that the basic quasi-experimental design was inappropriate and insufficiently sensitive to answer the complex research question being asked. By their very nature, successful long term community based programmes can result in complex and wide ranging effects, many of which may be unexpected and not confined to any one predetermined intervention community, making the measurement of any impact and attribution of causality highly problematic.23–28 Solving these problems will remain a continuing dilemma for advocates of prevention and should be a cause for reflection among academics and researchers concerning appropriate methods for assessing the results from such programmes. New evaluation techniques need to be developed that combine the strengths of quantitative and qualitative research methods and make better use of more proximal outcomes.29
Acknowledgments
We thank Geoffrey Berry, Jim Dwyer, Les Irwig, Russell Luepker, Lucie Richard, Judy Simpson, Chris Roberts, and colleagues at Health Promotion Wales for their help and constructive comments.
Footnotes
Funding: Welsh Office.
Conflict of interest: None.
References
- 1.Boaz A, Rayner M. Coronary heart disease statistics. London: British Heart Foundation; 1995. [Google Scholar]
- 2.WHO Expert Committee. Prevention of coronary heart disease. World Health Organ Tech Rep Ser 1982;678. [PubMed]
- 3.Rose G, Ball K, Catford J, James P, Lambert D, Maryon-Davis A, et al. Coronary heart disease: plans for action. London: Pitman, 1984.
- 4.Nutbeam D, Smith C, Murphy S, Catford J. Maintaining evaluation designs in long term community-based health promotion programmes: Heartbeat Wales case study. J Epidemiol Community Health. 1993;47:127–133. doi: 10.1136/jech.47.2.127. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Puska P, Tuomilehto J, Nissinen A, Vartiainen E, editors. The North Karelia project. 20 years results and experiences. Helsinki: National Public Health Institute (KTL); 1995. [Google Scholar]
- 6.Farquhar JW, Fortmann SP, Flora JA, Taylor CB, Haskell WL, Williams PT, et al. Effects of community wide education of cardiovascular risk factors: the Stanford five-city project. JAMA. 1990;264:359–365. [PubMed] [Google Scholar]
- 7.Luepker RV, Murray DM, Jacobs DR, Mittelmark MB, Bracht N, Carlaw R, et al. Community education for cardiovascular disease prevention: risk factor changes in the Minnesota heart health program. JAMA. 1994;84:1383–1393. doi: 10.2105/ajph.84.9.1383. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Carleton RA, Lasater TM, Assaf AR, Feldman HA, McKinlay S the Pawtucket Heart Health Program Writing Group. The Pawtucket heart health program: community changes in cardiovascular risk factors and projected disease risk. Am J Public Health. 1995;85:777–785. doi: 10.2105/ajph.85.6.777. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Catford J, Nutbeam D. Heartbeat Wales. In: Williams K, ed. The community prevention of coronary heart disease. London: HMSO, 1992:164-75.
- 10.Clarkson J, Nutbeam D. Introducing healthy catering practice into hospitals: a case study from Wales. J Nutr Health. 1991;7:101–110. doi: 10.1177/026010609100700205. [DOI] [PubMed] [Google Scholar]
- 11.Phillips CJ, Prowle M. Economics of a reduction in smoking: case study from Heartbeat Wales. J Epidemiol Community Health. 1993;47:215–223. doi: 10.1136/jech.47.3.215. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Roberts C, Smith C, Catford J. Quit and win Wales: an evaluation of the 1990 pilot contest. Tobacco Control 1993;2:114-9.
- 13.Murphy S, Powell C, Smith C. A formative evaluation of the Welsh heartbeat award scheme. J Nutr Health. 1994;9:317–327. doi: 10.1177/026010609400900407. [DOI] [PubMed] [Google Scholar]
- 14.Nutbeam D, Catford J. The Welsh heart programme evaluation strategy: progress, plans and possibilities. Health Promotion. 1987;2:5–18. [Google Scholar]
- 15.Nutbeam D, Catford J. Modifiable risks for cardiovascular disease among general practitioners in Wales. Public Health. 1990;104:353–361. doi: 10.1016/s0033-3506(05)80528-1. [DOI] [PubMed] [Google Scholar]
- 16.Smith C, Nutbeam D. Assessing non-response bias: a case study from the 1985 Welsh heart health survey. Health Educ Res. 1990;5:381–386. [Google Scholar]
- 17.Pullen E, Nutbeam D, Moore L. Demographic characteristics and health behaviours of consenters to medical examination. Results from the Welsh heart health survey. J Epidemiol Community Health. 1992;46:455–459. doi: 10.1136/jech.46.4.455. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Roberts RJ. Can self-reported data accurately describe the prevalence of overweight? Public Health. 1995;109:275–284. doi: 10.1016/s0033-3506(95)80205-3. [DOI] [PubMed] [Google Scholar]
- 19.Smith C, Moore L, Roberts C, Catford J. Health-related behaviours in Wales, 1985-1990. Health Trends 1994;26:18-21. [PubMed]
- 20.Shah BV, Barnwell BG, Bieler GS. SUDAAN users manual: software for analysis of correlated data, release 6.40. North Carolina: Research Triangle Institute; 1995. [Google Scholar]
- 21.Bennett N, Jarvis L, Rowlands O, Singleton N, Haselden L. Living in Britain. Results from the 1994 General Household Survey. London: HMSO; 1996. [Google Scholar]
- 22.Ministry of Agriculture, Fisheries and Foods. Household food consumption and expenditure. London: HMSO, 1970-1994.
- 23.Susser M. The tribulations of trials—intervention in communities [editorial] Am J Public Health. 1995;85:156–158. doi: 10.2105/ajph.85.2.156. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Chapman S. Unravelling gossamer with boxing gloves: problems in explaining the decline in smoking. BMJ. 1993;307:429–432. doi: 10.1136/bmj.307.6901.429. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Green LW, Richard L. The need to combine health education and health promotion: the case of cardiovascular disease prevention. Promotion and Education. 1993;0:11–17. doi: 10.1177/175797599300100407. [DOI] [PubMed] [Google Scholar]
- 26.Winkleby MA. The future of community-based cardiovascular disease intervention studies. Am J Public Health. 1994;84:1369–1372. doi: 10.2105/ajph.84.9.1369. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Black N. Why we need observational studies to evaluate the effectiveness of health care. BMJ. 1996;312:1215–1218. doi: 10.1136/bmj.312.7040.1215. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Speller V, Learmonth A, Harrison D. The search for evidence of effective health promotion. BMJ. 1997;315:361–363. doi: 10.1136/bmj.315.7104.361. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Nutbeam D. Health outcomes and health promotion: defining success in health promotion.Health Promotion J Aust 1996;6:58-60.