Use of the modified Delphi technique to identify and rate home injury hazard risks and prevention methods for young children

Abstract

Objective

For children aged 1–5 years, the authors used the Delphi method to determine (1) the most important injury hazards in each area of the home; (2) the most important injury prevention behaviors; and (3) feasible and efficacious safety devices and behaviors to reduce injury risks.

Design

The authors used a modified Delphi method to prioritize home injury hazards for children 1–5 years of age. The Delphi method is an indirect, anonymous, iterative process aimed at achieving consensus among experts; in this study, the authors queried key informants electronically. Thirty four key informants, primarily from the United States, participated in at least one of the three rounds of questionnaires. Responses were submitted by email or fax. Participants identified, rated, and ranked home injury hazards and prevention methods.

Results

The overall response rate for each survey ranged from 82% to 97%. Initially, 330 unique hazards and prevention behaviors/devices were identified in seven areas of the home. The 126 home injury hazards were rated based on frequency, severity, and preventability of injury; and the 204 behaviors and devices were rated by efficacy and feasibility. These experts rated firearms and pools as the most significant hazards, and smoke alarms and safe water temperature as the most important preventions.

Conclusions

The modified Delphi method of consensus was useful to prioritize home injury hazards and prevention methods for children under the age of 6 years.

Deaths and injuries resulting from unintentional causes occur in significant numbers in the United States. American homes represent the scene of 20% of these deaths and 40% of the unintentional injuries, with older adults and young children most at risk.¹ The societal costs associated with unintentional home injuries are staggering—estimated to be in excess of $200 billion annually.¹,² For children less than 6 years of age, mechanical suffocation and drowning claim more lives than any other cause of injury; whereas unintentional injuries resulting from falls remain the leading source of morbidity for children.¹,³ These statistics emphasize the relationship between housing and health.⁴,⁵ Most certainly the magnitude of the problem is understated. Information regarding location of unintentional injury events is not routinely and systematically collected, leaving substantial gaps in available data sets.¹ For example, the location of one third of non‐transportation related unintentional injury deaths is unknown.

Assessing potential home injury risks and designing appropriate and effective interventions pose considerable challenges for the public health community. It is generally agreed that most home injuries are preventable. However, determining which risks are most important for a particular developmental age group and the best means to mitigate them remain elusive. Currently, few data exist about which hazards are most important with regard to young children. And, whereas much is known about the effectiveness of certain injury prevention strategies, much is still undefined. For example, we still need to link the appropriate combination of active versus passive strategies; legislative versus educational or behavioral training; and targets of intervention (children, families, caregivers) to reductions in unintentional injuries.⁶

This lack of data led us to seek consensus among injury experts to determine injury prevention priorities for children 1–5 years of age. The Delphi method can be used to develop consensus among a group of experts without engaging them in direct discussions.⁷,⁸ The basic principle of the method is to provide experts with multiple rounds of questionnaires along with controlled feedback from the previous responses. With each successive round, only the least informed participants tend to alter their judgments, eventually leading to a consensus. Anonymity afforded by this method is thought to increase the likelihood that participants will express novel ideas, whereas the lack of direct interaction increases the accuracy of final judgments.⁷ The Delphi method is particularly useful in those situations where subjective opinions are likely to form the primary basis for decision making.⁷ Investigators have previously used the Delphi technique to identify pediatric counseling priorities and research topics for injury prevention.⁹,¹⁰

We performed a three‐round Delphi survey with a group of experts to identify priorities for an injury prevention program aimed at preschool aged children in the home environment. Based upon their input, we produced a list of the most important home hazards for this group, and a list of effective and feasible prevention behaviors and devices.

Methods

Identification of experts

One of the authors (MLK) and four injury prevention colleagues developed a list of 39 injury prevention experts with a wide variety of publications and professional, policy, and programmatic leadership in the field of childhood injury prevention. Each of the 39 key informants was invited to participate, provide demographic information, including their area(s) of injury expertise, and submit 3–5 names of other potential participants. Potential participants from the same organization, geographic setting, and area of injury interest were limited to maintain diversity in the pool of experts. Of 55 total invitations sent, 36 individuals (primarily from the United States) agreed to participate; 19 declined, could not be contacted, or did not reply.

Survey methods

The University of Wisconsin Health Sciences Human Subjects Committee approved the study protocol. The Delphi consisted of three rounds of surveys, with participants blinded to each other's identity.

The Delphi survey 1 requested a list of 5–7 injury hazards and 5–7 potential prevention behaviors and/or devices for children aged 1–5 years in each of the following areas of the home: bedroom/play area, kitchen/dining area, bathroom, living room, basement/garage (including other outdoor areas such as the driveway), pool, stairs/hallway, and multiple rooms/general safety. We asked participants to develop their lists of hazards by considering the frequency, severity, and preventability of the potential injury from each hazard, as well as the efficacy and feasibility of each prevention method. Efficacy was defined as the ability of the behavior or the device, if implemented, to eliminate the hazard and/or to prevent the injury. Feasibility was defined as the likelihood of implementation of the behavior or the device (depending on acquisition, installation, utilization, and maintenance).

Delphi survey 2 asked participants to rate each hazard and behavior/device listed in the responses submitted to survey 1 using a scale of 1 to 3 (with 3 being highest priority). Participants could also assign a score of zero (0) if they believed that an item should not remain on the list. In rating each item, the participants were instructed to consider the same factors used in the first round (for example, children aged 1–5 years; frequency, severity, and preventability for the hazards; and efficacy and feasibility for the behaviors/devices). We calculated a mean score for each item by summing all ratings reported for a single item. Items were subsequently listed in descending order of priority.

The 47 hazards and 52 prevention methods with the highest mean scores were selected for inclusion in Survey 3 based upon natural clusters, rather than just choosing the top 50 of each. We confirmed natural clusters of the items using an agglomerative hierarchical cluster procedure (SPSS v12.0 for Windows; SPSS, Inc, Chicago, IL, USA).¹¹ The similarity and distance levels at each step of the agglomerative schedule for combining clusters were displayed in an SPSS dendrogram plot (data not shown). The defined clusters in the final partition were subjected to a linear discriminant analysis to check the cluster groupings for logical and internal consistency.

For the 99 selected items, the third Delphi survey round asked participants to rate each hazard using a Likert scale of 1 to 5 (with 5 being the most important) considering overall importance in an injury prevention program for preschool aged children, 3–5 years of age. This age group request differed from previous rounds as we sought to use the panel's findings for a future injury prevention program targeted at children aged 3–5 years. By including younger children in earlier rounds, we have recommendations for all children under 6 years old. For each prevention behavior/device, we asked participants to rate separately both the efficacy and feasibility using a Likert scale of 1 to 5 (with 5 being the most efficacious or the most feasible). We calculated a mean score for each item based upon summation of the assigned ratings. Participants were also asked to place in rank order the 10 most important injury hazards and behaviors/devices from the lists. We determined the final rank order by calculating the percentage of participants ranking each hazard in the top 10. In addition, participants were to report factors that influenced their decisions.

Results

Of the 36 key informants who agreed to participate, 34 responded to at least one of the three rounds of the survey. The two who chose not to participate in round 1 were removed from subsequent rounds. Areas of expertise and demographic information are listed in tables 1 and 2. We listed all self‐identified academic credentials; some individuals possessed multiple degrees and areas of interest. As shown, the panel collectively possessed a broad scope of expertise in several areas of injury prevention. Furthermore, participants represented academic, government, advocacy, and policy interests.

Table 1 Self‐identified areas of expertise* (number of participants).

General injury (22)	Consumer products (3)	Firearms (2)	Home safety (1)
Outreach/training (9)	Counseling (3)	Law (2)	Interventions (1)
Pediatrics (9)	Drowning (3)	Playgrounds (2)	Minorities (1)
Fires/burns (7)	Economics (3)	Poisoning (2)	Nursing (1)
Epidemiology (5)	Policy (3)	Sports injury (2)	Research (1)
ER Medicine (4)	Traffic safety (3)	Advocacy (1)	Surveillance (1)
Violence (4)	Falls (2)	Disabilities (1)	Unintentional injury (1)

*Areas of expertise for 34 participants responding to at least one of three rounds of survey.

Table 2 Participant demographics*.

Country		Degree
Australia	1	MD	14
Canada	2	PhD	8
United States		ScD	1
East Coast	22	JD	1
West Coast	3	MPH	9
Midwest	6	MS	2

*Demographics for 34 participants responding to at least 1 of 3 rounds of survey.

In the first round, participants identified a total of 126 unique home hazards. Listed by the frequency with which hazards appeared in various room locations, these hazards were distributed as follows: kitchen, 16.7%; garage, 14.3%; living room, 12.7%; bathroom, 10.3%; bedroom, 10.3%; pool, 9.5%; stairs/hallway, 6.3%; and multiple rooms, 19.8%. They also listed 204 injury prevention behaviors and devices: kitchen, 18.1%; garage, 11.8%; bedroom, 15.2%; living room, 13.2%; pool, 12.7%; bathroom, 9.8%; stairs/hallway, 5.9%; and multiple rooms, 13.2%. The majority of the experts contributed between five and seven hazards or prevention devices/behaviors for each room as requested. Several items listed were not unique to an individual room (for example, access to uncovered electrical outlets), or were not specifically associated with any room of the home (for example, access to firearms; adult supervision).

Twenty eight key informants responded to survey 2. Results from this survey identified 99 high rated items (47 hazards and 52 behaviors/devices) for further prioritization in the final survey (fig 1). Many of the selected hazards and devices/behaviors applied to multiple rooms within the house, or to behaviors in general (for example, exposed electrical outlets and cords, supervision of children). The number of prevention strategies (fig 2) was generally comparable with the number of hazards listed for each type (for example, fire/burn). The only notable exception to this was for those hazards relating to suffocation and choking. Although this category contained the highest number of identified hazards, the participants either chose not to rate corresponding prevention strategies highly, or fewer appropriate devices or behaviors were available. The highest rated preventions in the second round were those involving singular actions or installation of devices to achieve passive protection; only 25% of selections for the final survey involved active, repetitive behavioral modifications on the part of the caregiver.

Figure 1 Frequency of injury hazards by type and room, Delphi round 2 expert ratings. Participants highly rated 47 hazards for prioritization in round 3. The number of depicted hazards equals 51 to accommodate the four hazards fitting into two simultaneous injury types (that is, playground surfaces and equipment  =  falls and suffocation).

Figure 2 Frequency of injury prevention strategies by type and room, Delphi round 2 expert ratings. Participants highly rated 52 devices or behaviors for prioritization in round 3.

Thirty three participants responded to survey 3. We generated composite ratings for all of the 99 items with the goal of quantifying the safety level of each item. In addition, we also asked the participants to rank the most important hazards and prevention devices to be included in an injury prevention program targeted to preschool aged children. The top 10 injury hazards identified from the participants are listed in table 3, and prevention strategies are listed in table 4 (ranking of “1” as the most important). Factors listed as affecting the ranking of potential hazards included: the age range given (3–5 year olds); potential severity of injury; and frequency of injury occurrence. In general, panel members ranked hazards associated with potentially severe outcomes more highly compared to more common hazards with potentially less severe outcomes. Participants gave “known effective strategies for prevention” and “assessment of likely parental knowledge” as factors for their ranking of prevention strategies.

Table 3 Expert identified top 10 injury hazards.

Hazard	Ranking*	Experts (%)†	Composite rating‡	SEM§
Access to firearms in the home	1	87.9	4.656	0.115
Direct access to a pool	2	72.7	4.688	0.130
Access to fire starting materials	3	69.7	4.219	0.160
Unsupervised around pool	4	66.7	4.688	0.114
Access to medications, poisons	5	66.7	4.000	0.180
Unsecured windows	6	60.6	4.094	0.187
Lack of adult supervision	7	57.5	4.032	0.239
Unsafe playground surfaces and equipment	8	54.5	3.968	0.183
Excessive hot water temperature	9	51.5	3.906	0.170
Playing in driveway	10	39.4	4.000	0.174

*Ranking of 1 represents most important.

†% of experts ranking item within the top 10.

‡Mean rating score using five‐point Likert scale (5 represents most important).

§Standard error of the mean (composite rating).

Next 10 most highly rated hazards: unattended child in bathroom; unguarded stairs; bike/scooter; fires related to cooking; space heaters; crib sleep position; bed placement; open fireplace; foods that can be choked on; flammable clothing.

Table 4 Expert identified top 10 injury devices/behaviors.

Prevention	Ranking*	Experts (%)†	Composite rating‡	SEM§
Install smoke alarms	1	86.7	4.563	0.100
Set water temperature at <48.9°C (120°F)	2	70	4.688	0.138
Use child resistant packaging	3	63.3	4.156	0.156
Install pool fencing	4	60	4.375	0.140
Install window guards	5	50	4.531	0.090
Remove firearms from the home	6	46.7	4.938	0.044
Store and lock firearms and ammunition separately	7	43.3	4.531	0.156
Use bicycle helmets	8	40	4.250	0.162
Use fire‐resistant sleepwear	9	30	4.281	0.156
Create safe play area	10	30	4.094	0.137

*Ranking of 1 represents most important.

†% of experts ranking item within the top 10.

‡Mean rating score using five‐point Likert scale (5 represents most important).

§Standard error of the mean (composite rating).

Next 10 most highly rated prevention strategies: install cabinet latches; place EMS numbers by phone; install ground fault circuit interrupt; use CO detectors; install stairway gates; never leave children unattended; fire escape plan; cover spas/hot tubs; remove non‐compliant cribs; remove baby walkers.

Devices/behaviors listed as most important for an injury prevention program were those that required the least repetitive effort by the child's caregiver and those that were technically simple. The most active strategy—constant adult supervision—was only considered the 16th most important safety prevention behavior. Moreover, the experts were somewhat divided as to the importance of this strategy: two thirds (23 individuals) felt adult supervision to be effective (rated “4” or “5”), but at the same time most (29 individuals) felt it not highly feasible (rated as a “2” or “1”).

As we analyzed the rankings presented in tables 3 and 4, we determined the responses clustered into three groups dependent upon the academic degree possessed by each participant. (Similar analysis based on geographic location, gender, or employer type (government, academic, private) failed to yield discernible clusters.) Cluster 1 was composed of approximately half MDs (6/13) and half PhDs (5/13). Those in cluster 2 held mainly MPH degrees (6/10), whereas those in cluster 3 held mainly MDs (7/9). The majority of each cluster agreed on access to pools and firearms as important hazards. Nevertheless, few other items elicited substantial accord either among or within clusters. Cluster 1 ranked access to poisons (11/13), unsupervised swimming (9/13), and lack of adult supervision (9/13) in the top 10. Those in cluster 2 chose installing smoke detectors (10/10), access to fire‐starting materials (8/10), and unsafe playground surfaces (8/10). The majority of cluster 3 named pool fencing (7/9) and unsupervised swimming (7/9) as key items. The participants also ranked items as important hazards, but then did not rank the corresponding prevention device or behavior, or vice versa. For example, the majority of each cluster listed access to fire‐starting materials as an important hazard (22/33), but only five participants listed storing such materials out of reach as a comparably significant behavior.

Discussion

With this study, our experienced injury specialists rated home injury hazards and preventions relevant for young children. The ranking of the most important hazards and remedies, and mean composite ratings of both, can be used to establish priorities for prevention programs aimed at young children, as well as a means to quantify the relative effectiveness of such interventions. Currently, many injury prevention programs, such as SAFE KIDS at Home, target leading causes of injury in the home. Home risk assessment occurs followed by education and perhaps installation or delivery of safety devices.¹² Generally, no attempt is made to quantify the level of danger presented by home hazards. Priorities for remediation are left to the family to decide, or are based upon the materials available at little or no cost to the prevention program (for example, smoke alarms).

The results we obtained are largely similar to those of a Delphi study to determine pediatric injury prevention counseling priorities for children under the age of 2 years.⁹ (Note that eight of the participants were in common between both studies.) The major exception is the greater importance placed upon firearm access by the panel in the current Delphi study; this may relate to the developmental age range stipulated for the third round of the current Delphi (3–5 years). Furthermore, the current study did not consider injuries related to motor vehicles, as the focus was injuries occurring in the home. Smoke alarms, pool fencing, and lower water temperature comprised the top three prevention strategies for both studies. Experts in both panels deemed hazards related to fire/burns, drowning, falls, and poisoning as the most crucial for children 5 years of age and younger. With the exception of suffocation, these priorities address the main causes of death and non‐fatal injuries in this age group.¹,³ The American Academy of Pediatrics (AAP) similarly developed counseling priorities for children based upon developmental age.¹³,¹⁴ Despite using a more evidence based method compared with the Delphi survey technique, the results of all three studies are consistent. This agreement suggests our results obtained using the Delphi method can serve as a useful guideline for assessing and mitigating home injury risks until additional research is available to verify the findings of our panel.

Our results, perhaps not unexpectedly, differed from those based on the frequency of product related injuries resulting in an emergency room visit compiled by the US Consumer Product Safety Commission and described in the National Electronic Injury Surveillance System (NEISS; http://www.cpsc.gov/LIBRARY/neiss.html). Our expert panel used not only frequency, but also severity and preventability of injury to prioritize potential hazards. In addition, the NEISS database focuses on the removal or modification of consumer products (for example, toys, appliances, clothing) as a means to prevent unintentional injury. Again, our panel used efficacy and feasibility as the main criteria for assessing prevention strategies.

We solicited expert advice from a variety of government, private, and academic injury prevention professionals. Because the Delphi method relies upon consensus opinion, the quality of the expert participants is always of potential concern. It is likely that not all of the individuals we enlisted possessed expert knowledge in all areas under consideration. Furthermore, the panel as a whole may not have represented all areas of injury prevention; for example, we did not include child care providers. We also acknowledge that the conclusions of our panel are connected to their identity as predominately professionals living in the United States; priorities for communities in other countries are likely to vary somewhat. Given these caveats, we were careful to select individuals with diverse academic training and areas of expertise. In addition, to avoid selection bias, we asked invited experts to nominate others. We limited the number of individuals from a single institution to minimize chance interactions and similarity of opinion. All of those who participated are actively working in the field of injury prevention and are recognized for their commitment and achievement in this area. Similarities among the different studies noted above suggest we were largely successful in achieving a competent panel. We did observe however, a clustering of responses from our expert panel based apparently upon academic degree. Because of the small number of individuals in each cluster, we were unable to determine whether there might be a systematic bias. Nevertheless, the partitioning of MDs into two different clusters indicates that academic degree cannot be the sole determinant. The inclusion of experts with a variety of academic degrees proved to be an important design component.

Key points

• The Delphi technique is a useful means of determining priorities for injury prevention programs.

• A diverse panel is essential to assure relevant expertise and to minimize professional bias.

• Additional research is needed to broaden the number of effective injury prevention strategies.

The results of the Delphi also highlighted areas for additional research. Whereas the panel noted numerous hazards related to suffocation, they identified few potential prevention behaviors and devices. Again, this cause of unintentional death is highest among children under 5 years of age, and is particularly fatal for infants.¹,³ Moreover, many of the prevention strategies the experts identified for all hazards required less parental action (that is, installing and maintaining safety devices) versus those requiring action such as active adult supervision. This underscores the call for more research into developing a variety of effective prevention strategies.¹,⁶ It may also be of interest to further characterize the observation of clustering of expert judgments.

Conclusions

Prioritizing unintentional injury hazards and corresponding prevention behaviors and devices is crucial to the success of programs aimed at lowering the rate of unintentional home injuries. Achieving this goal is possible by enlisting a panel of experts comprising a diverse array of injury prevention expertise and training. The experiences of each panel will impact the outcome of the prioritization, necessitating different panels for communities (or countries) with markedly different hazards (for example, firearm accessibility), or for different developmental groups (for example, young children and the elderly).