Evidence into practice: combining the art and science of injury prevention

Abstract

Objectives

To bring together scientific evidence of what works in injury prevention with the knowledge and experience of practitioners, using a case study of smoke alarm installation from England.

Design

There is good evidence of strategies to reduce injuries but less is known about the art of translating those strategies to implementation in real‐world settings. England's Health Development Agency developed a structured process applicable to many public health fields, which integrates practitioner knowledge into the evidence base and reflects local contexts. The multistep process includes convening structured field meetings with local practitioners and policy makers, which focus on a mapping exercise of strategies, policies, targets, and funding streams related to childhood injury prevention, and barriers and facilitators relating to implementation of specific interventions.

Setting

Meetings were held in six venues across England with 98 participants from a range of professional backgrounds and sectors.

Results

The collective knowledge of participants provided many local insights unlikely to emerge in conventional research. Discussion topics covered key partners and sectors to include when planning a program; national policies and programs that could be used to drive the agenda; potential sources of funding; the importance of providing and installing appropriate smoke alarms; targeting of programs; and suggestions for gaining access to hard‐to‐reach populations.

Conclusion

This methodology represents an efficient way of gaining insight necessary for successful implementation of evidence based programs. It may be particularly useful in lower and middle income countries, serving to translate evidence into the local contexts and circumstances within which practitioners operate.

Increasingly available systematic reviews and meta‐analyses provide good evidence of promising strategies to reduce injuries. But little is known about the art of translating strategies from the realm of science to implementation in real‐world settings. The importance of implementation issues of any program cannot be overstressed, yet rarely are data available to help understand influences on the success of new programs.

A program showing promise in one context may not yield similar results in other settings. One smoke alarm distribution program in a high risk area of Oklahoma City, US showed an 80% drop in fire related morbidity and mortality.¹ A similar program targeting a high risk area of London, UK had no impact on fires, fire related injuries, or the presence of working smoke alarms.² Subsequent qualitative work examining the barriers to functioning smoke alarms identified the importance of initial qualitative work with the target population to improve the take‐up and maintenance of smoke alarms.³

Researchers are not solely to blame for neglecting implementation issues. Their priorities reflect the practice of scientific journals to concentrate on outcome rather than process. Furthermore, the voice of the practitioner implementing the program is rarely heard in such publications. Arai et al highlight the dearth of evidence regarding implementation issues in a review of smoke alarm programs.⁴,⁵ They sought to identify the most effective methods for these programs. They concluded that implementation data in the 37 articles retrieved were insufficient to be useful to practitioners and policymakers attempting to implement programs.

Not only do practitioners and policymakers lack sufficient information for implementation, but what is available generally does not reflect the context in which they work. Given this state of affairs, there can be little surprise at the well documented difficulties in translating evidence into practice.⁶,⁷ The climate is ripe for new and innovative ways to fill this gap.

Kelly et al describe a structured process for use in public health to translate evidence into practice in a way that reflects local contexts.⁸ A multistep process is followed which includes convening appraisal of practice field meetings. At the meetings, practitioners are presented with a list of recommendations regarding promising and proven practices. Their combined input aids the integration of scientific evidence with knowledge gleaned from practice. The meetings are based on the premise that practitioners and policymakers are ideally placed to offer insights into local contexts, and that collectively they provide a comprehensive understanding of practicalities of program implementation. Information is captured on macro and micro political, social, cultural, and economic contexts that affect daily practice, and practitioners' implicit and explicit understandings of the practice environment in which they work. The aim is to determine issues important to consider in implementing interventions and to make judgements regarding the likelihood of success of interventions in practice in a given setting. The output of the process combines evidence based recommendations with practitioner expertise and experience specific to the national and local context at that point in time.

This paper outlines our experiences in using this process designed to combine the art and science of injury prevention. The objective was to bring together scientific evidence of what works in injury prevention with the knowledge and experience of practitioners. To illustrate this process, a case study of smoke alarm distribution programmes provides a snapshot of the context in England.

Method

The process described by Kelly et al was adapted to reflect the injury prevention field.⁸ Table 1 summarizes the steps involved. Steps 1 to 3 occurred before the meetings (step 4) and steps 5 to 7 afterwards.

Table 1 Method to translate evidence into practice.

Steps	Process
1	Use existing reviews of reviews to identify best practice
2	Draw up a “longlist” of interventions based on reviews of reviews
3	Reduce to a shortlist of interventions through consensus process considering:
	• quality of evidence
	• potential transferability
	• applicability to real‐world conditions
	• burden of injury
	• potential to impact inequalities in health
4	Convene appraisal of practice field meetings until “sufficient saturation” is reached*
5	Verification of meeting summaries by participants
6	Interview key informants to supplement findings
7	Production of Effective Action Briefing

*“Sufficient saturation” occurs when the same points are reiterated in all groups and ideas become repetitive. As a rule of thumb, six meetings are recommended to reach this point.⁸

Step 1

Systematic reviews and evidence briefings regarding the effectiveness of interventions to prevent unintentional injuries in children and older people were consulted.⁹,¹⁰

Steps 2 and 3

Each member of the research team independently considered what were the most important interventions that had an impact on reductions in inequalities in health. A “longlist” of 20 interventions was compiled. This was reviewed by the research team to arrive at a shortlist, prioritizing feasible interventions for practitioners to implement in England. Through consensus, it was reduced to the five interventions shown in box 1. From these, three were selected for consideration by practitioners because they reflected a spread of injury topics, and avoided overspecialization.

Box 1 Shortlist of recommendations, with those selected in italics

• Smoke alarm distribution programs

• Practical pedestrian skills training

• Education through age paced materials to promote parental teaching of pedestrian safety skills

• Traffic engineering measures to prevent pedestrian injuries

• Cycle helmet promotion

Steps 4 and 5

Meetings were convened with practitioners throughout England to assess the potential for success of the three injury prevention strategies highlighted in box 1. For the sake of brevity, this paper focuses solely on discussion surrounding smoke alarm distribution programs.

Participants

Potential participants were identified through the Child Accident Prevention Trust's extensive database and contacts with local practitioners and policy makers familiar with injury prevention activities in their geographical area. Participants were invited if they had specialist knowledge of the subject areas (smoke alarms, cycle helmets, pedestrian training) and/or recent experience of running local or regional programs; or were in management positions that required involvement in program development, strategic development, or budgetary issues relating to unintentional injury or more generally; and were from the range of agencies that are actively involved in injury prevention at a local or regional level.

In total, 239 invitations were sent out for six meetings. Of these, 56 practitioners could not attend, 55 did not reply, 20 proposed substitutes, and 108 accepted. A total of 98 practitioners and policy makers attended the six meetings, with meetings including from 10 to 22 participants each as illustrated in table 2. Table 2 also indicates participants' range of professional backgrounds.

Table 2 Participants attending appraisal of practice field meetings.

Meeting	Sector
	Health	Road safety	Fire safety	Early child/education	Local authority	NGO*/other	HDA†	Total
1	5	4	2	1	0	2	0	14
2	6	1	2	0	2	3	1	15
3	3	0	1	2	2	0	2	10
4	6	3	1	0	4	1	1	16
5	6	2	1	4	2	5	1	21
6	4	1	3	4	1	6	3	22
Total	30	11	10	11	11	17	8	98

*NGO, non‐governmental organizations; †HDA, Health Development Agency.

Format of meetings

Venues for meetings were sought that were accessible and reflected a geographical spread across England. Initially, meetings were held in four venues. After examining preliminary data from these meetings, the research team determined that “sufficient saturation” had not been reached and two additional meetings were run. Table 3 summarizes the meeting agenda.

Table 3 Agenda for appraisal of practice field meetings.

Length of discussion	Item
20 minutes	Welcome and introductions
10 minutes	Project background and description of process
40 minutes	Macro‐environment discussion
105 minutes	Subgroup discussions
60 minutes	Report back subgroup discussions and consider in wider macro‐environment context
15 minutes	Summation of key points and next steps

The first portion of the day was important in setting the tone for the meeting. We attempted to find a balance between getting people actively involved and providing them with information. Participants were sent a summary of the process prior to the meeting, making a brief introduction sufficient for the day to run smoothly.

In the “macro‐environment” discussion, participants were asked to discuss the broad policy and economic frameworks in which they worked. They were invited to identify relevant strategies, policies, targets, funding streams, etc, that related to injury prevention generally and the three injury prevention strategies specifically. Barriers, facilitators, and linkages between policies were raised during this session. This allowed everyone to share knowledge on frameworks within which they worked and problems faced given the nature of these frameworks.

The subgroup discussions encouraged participants to consider how to implement one of the three specific interventions. Participants with relevant expertise were assigned to the appropriate group (for example, fire prevention officers were assigned to the smoke alarm subgroup). Other participants were then assigned to subgroups to provide a balanced mix of professional backgrounds. Each subgroup was facilitated by a member of the research team and supported by at least one note‐taker. Subgroup discussions followed a similar structure to ensure a systematic approach to data collection as far as was practical. Subgroups were invited to consider how to design and run local programs, how to get them on local policy agendas, and where to find funds, staff time, and other resources to facilitate programs. They were to consider their experience and knowledge, not just of injury prevention, but more generally.

The reporting back discussion session at the end of the day allowed the conclusions of the subgroup sessions to be considered in the wider context. This discussion drew out some of the crosscutting issues such as partnership working, targeting, and training.

Members of the research team took detailed notes throughout each meeting. These notes were consolidated and organised by topic. After participants had verified these summaries, key words and phrases were identified in each set of notes, providing the basis for the Effective Action Briefing.

Step 6

To provide examples of real‐world programmes and fill in any remaining gaps, face‐to‐face interviews were conducted with key informants regarding specific smoke alarm programmes that emerged during the meetings.

Step 7

Based on meeting output, a short Effective Action Briefing was created to give practitioners insights into smoke alarm programme implementation in England. A brief description of contents is below.

Results

Discussions around the promotion of smoke alarms were animated and extensive. A full description of the outputs from these meetings is available in the fieldwork report.¹¹ The nature of the discussion is illustrated below.

Policy drivers and funding opportunities

Participants unanimously agreed on the importance of national policies and drivers, influencing resources, and staffing. For example, the Fire and Rescue Services Act 2004 was viewed as a powerful driver, placing an obligation on Fire and Rescue Services (FRSs) to be active in community fire safety and providing some funding to install free smoke alarms in the most vulnerable households. Because participants from other sectors were missing drivers for injury prevention, creative interpretation of policies was required to make the smoke alarm agenda fit in. Potential sources of funding mentioned included partnerships with local government or organizations.

Multi‐agency partnership

Multi‐agency partnerships were perceived as crucial to the promotion of smoke alarms, offering opportunities for referrals between agencies and access to hard‐to‐reach groups.

Issues for consideration in program design

Providing smoke alarms to target populations was not seen as sufficient. Any program must include installing of smoke alarms, particularly for high risk groups such as older people who may not be capable of installing them. Training on correct alarm installation would ensure optimal placement and reduce false alarms. Tamper proof smoke alarms with 10 year batteries were recommended for distribution.

Targeting of interventions

High risk households

Accessing high risk households was a major concern and targeting interventions was seen as an important strategy to address this. Other methods were also suggested, including the media (for example, TV soaps, radio, and free newspapers). In certain vulnerable populations, alternative solutions such as sprinklers were suggested as more appropriate. It was felt important to continue to direct the fire prevention message at the general population as well.

Minority ethnic groups

Minority ethnic groups were seen as requiring specific strategies for targeting interventions. Community and religious leaders and community activists were mentioned as key collaborators. For example, religious leaders could include fire safety messages in sermons, or provide lists of households requiring help with smoke alarm installation.

Children

Education of school children on fire prevention was perceived as a key component of smoke alarm programmes to ensure that they grow up with this knowledge. Different skills and knowledge should be taught at different ages.

Program implementation barriers and facilitators

Barriers

Participants identified a number of barriers to the successful implementation of local programs. The absence of injury/fire prevention from agendas and targets outside FRSs was seen as a major concern making the dedication of resources to this issue difficult. When funding was available, it was often inflexible or short term, limiting the impact of the program. Work was often described as fragmented, due to the lack of coordination of home safety in one central organization. The need for interagency collaboration was seen as challenging, with difficulties such as lack of communication, data sharing obstacles, and cultural barriers which may result in insular working.

Both private and public landlords were seen as creating barriers. The lack of requirement to install and maintain smoke alarms meant landlords could ignore this issue, and some councils were even found to be removing alarms to limit liability if they malfunctioned.

The rapid turnover of tenants and the high likelihood of damaging or disabling smoke alarms in the most at‐risk households meant that repeat visits to the same households were required. An added difficulty mentioned in high risk communities was the suspicion of officialdom.

Facilitators

The FRS Act was seen as a major facilitator to smoke alarm programs, providing national leadership and targets. Participants from other sectors saw the potential for linkage with other health messages or initiatives. They felt there were persuasive cost‐benefit arguments to be made for these programs.

Participants mentioned sophisticated data and scoring systems within FRSs and through data sharing with partners meant that high risk properties could be identified and targeted. They also noted changes in building regulations that meant that all new properties required smoke alarms.

Meeting output

The Effective Action Briefing based on the meetings described above outlines recommendations and points of action for smoke alarms programmes.¹² They include:

• Recommendations on key partners and sectors to include when planning a program.

• Suggestions on national policies and programs to drive the agenda, as well as opportunity for action in activities without smoke alarm promotion as a central target.

• Potential sources of funding.

• Highlighting the importance of providing and installing appropriate smoke alarms.

• Comments and suggestions on targeting programs, particularly for hard‐to‐reach populations.

• Suggestions regarding potential settings.

• Case studies of real‐world programs.

Discussion

Combining high level scientific evidence with specialist knowledge and experience of practitioners is an important part of successful adoption of interventions.⁷ Kelly et al's methodology provides a promising way to accomplish this.⁸ The results from the field meetings are abundant with examples of insights unlikely to emerge in conventional research. For example, the report that some councils remove smoke alarms to limit liability if smoke alarms do not function. The practitioners we consulted were frustrated by lack of policy drivers for injury prevention but were full of suggestions for using existing policies to their advantage. Participants also drew on their experiences in many areas to offer strategies for accessing high risk populations—an issue of universal concern.

Comment on the process

Kelly et al's methodology was developed for use in all public health fields, but requires minor modifications to suit injury prevention. For example, injury prevention in the UK suffers from an absence of consistent high level policy drivers, especially in the health sector. This, combined with the lack of “ownership” for the subject, means there is little incentive for practitioners from sectors other than fire to participate. It was most difficult to recruit senior managers to the field meetings. When present, their input was extremely helpful and unlike that provided from even the most experienced practitioners. Three strategies were employed to address these gaps. First, key regional people were consulted to determine the most appropriate participants to include. Second, fieldwork meetings were supplemented by face‐to‐face key informant interviews to obtain greater detail and fill gaps. Third, additional information was collected on policy developments with an impact on injury prevention.¹³

The optimum number of participants was at least eight per subgroup with a minimum of two subgroups. With fewer there was a risk that not all relevant professions were adequately represented. For a topic subgroup to work, it needed two or three subject specialists, leaving a limited number of places for the range of other potential collaborators.

Implications for prevention

The methodology described in this paper has the potential to be of considerable value in improving the implementation of known effective interventions. Substantial evidence is available regarding what works, but missing are practical recommendations for program implementation that reflect real‐world practice. This paper aims to illustrate the two‐way flow between researchers and practitioners and improve the “fit” between research and practice.¹⁴ We tapped into practitioners' wealth of knowledge and experience regarding how to make things work. Simultaneously, meetings acted as valuable training tools, fostering learning from colleagues' experiences, and providing a mechanism for strengthening local partnerships.

This methodology has broad implications for prevention with the potential for transferability to many injury prevention topics as well as other areas of health promotion. It represents an efficient way of gaining insights necessary for successful implementation of evidence based programs, and may prove useful and cost effective in lower and middle income countries. The international literature on effective interventions comes mainly from higher income countries, but implementation of interventions requires translation into specific contexts and circumstances within which practitioners operate. Rather than repeating scientific research in multiple settings—an option that may be too prohibitively expensive for many lower and middle income countries—this process represents an accessible way of combining the art and science of injury prevention to reflect local contexts.

Key points

• This article presents methodology to combine scientific evidence with the specialist knowledge and experience of practitioners.

• The move from effectiveness literature to implementation considers local context and results in practical recommendations that reflect real‐world practice.

• The appraisal of practice field meetings have the potential to act as valuable training tools, allowing practitioners to learn from colleagues' experiences, and provide a mechanism for strengthening local partnerships.

• The methodology represents an efficient way of gaining insight necessary for successful implementation of evidence based programs.

• The methodology may be particularly useful in lower and middle income countries. Evidence based practice can reflect local contexts without repeated scientific research in multiple settings.

Abbreviations

FRS - Fire and Rescue Service