Limits of “Skills And Drills” Interventions to Improving Obstetric and Newborn Emergency Response: What More Do We Need to Learn?

A “skills and drills” intervention in 4 hospitals in Karnataka, India, produced modest improvement in provider knowledge and skills but not in actual response to obstetric and newborn emergencies. We explore possible explanations, which include (1) the need for a more intensive intervention; (2) other weaknesses in the health system; and (3) behavioral or organizational barriers related to hierarchical structures, roles, and team formation.

A “skills and drills” intervention in 4 hospitals in Karnataka, India, produced modest improvement in provider knowledge and skills but not in actual response to obstetric and newborn emergencies. We explore possible explanations, which include (1) the need for a more intensive intervention; (2) other weaknesses in the health system; and (3) behavioral or organizational barriers related to hierarchical structures, roles, and team formation.

See related article by Varghese.

In this issue of GHSP, Varghese and colleagues present the results of a set of "skills and drills" activities aimed at improving maternal and newborn emergency care in a high-mortality setting in Karnataka, India.¹ The Argentine Institute for Clinical Effectiveness and Health Policy with funding from the Maternal Health Task Force provided catalytic training and orientation activities over 2.5 days to a core group of coordinators. These coordinators formed a regional team composed of 8 nurses and 20 physicians from medical colleges in Karnataka. They instituted the skills and drills practice-improvement activities in 4 intervention facilities with mainly local human and material resources. The activities consisted of a 2-day skills course at the study facilities followed by visits to the facilities every 2 months over a 12-month period to carry out drills and supportive supervision activities.

Practitioners deemed the skills and drills intervention feasible and acceptable, and there was a modest increase in provider knowledge and skills. But the intervention appeared not to have achieved its desired results in improvements in clinical practice, either in the identification of cases in need of emergency intervention or in improved response to identified emergencies. This may be attributable to continued poor reporting, but since that also was a focus of the intervention, this lack of improvement in documentation is itself instructive.

PREVIOUS RELATED WORK

Teamwork and emergency preparedness have been shown to be key determinants of successful obstetric emergency response.² Therefore, in addition to coordinating the technical aspects of care, “skills and drills” interventions, including realistic simulation exercises for obstetric and neonatal emergencies, have been used for several decades in a variety of clinical settings in high-income countries to improve the quality of providers' responses as they work as a team in stressful and time-sensitive situations.^3–6 Such simulation approaches focus on procedures for preparation and coordinated clinical response. Facilitators set a scripted realistic emergency scenario, and providers respond as they would in a real situation. The simulation is often videotaped, and the clinical team debriefs afterward to review and reflect on what could have been done differently to improve the response.

A low-technology obstetric emergency simulation-based course named Advance Life Support in Obstetrics has been in use in North America since 1993.⁷ Over the last 10–15 years, use of obstetric simulation in high-technology simulation centers has become more common. There has been increasing interest in the use of simulation approaches in low- and middle-income countries as well, usually done on-site in a facility rather than in a special simulation center. For example, PRONTO International has introduced simulation-based training in 7 countries, much of it focusing on improving obstetric emergency care.⁸ In addition, the Expanding Maternal and Newborn Survival (EMAS) Program, a bilateral maternal newborn care project in Indonesia supported by the U.S. Agency for International Development, introduced emergency preparedness and simulation approaches at moderate scale.⁹

There is evidence that such simulation approaches can improve provider knowledge, skills, and confidence, but current evidence is thin that these approaches improve provider practices, especially when implemented at larger scale.^10,11 A current Cochrane protocol seeks to explore this gap in knowledge (or possibly enlighten us on the extent to which this knowledge gap has been addressed).¹² It is interesting that even in high-income settings, there is interest in implementing simulations as they are usually done in low- and middle-income countries—that is, as more realistic on-site simulations instead of being conducted in a simulation center—because there is some indication that this may give better results in terms of system-oriented improvements.¹³

Simulation approaches can improve provider knowledge, skills, and confidence, but current evidence is thin that they improve provider practices.

POSSIBLE REASONS WHY THE INTERVENTION IN INDIA HAD LIMITED EFFECT

There are several possible explanations for the negative results reported by Varghese et al. First, the authors suggest, as a possible explanation for the lack of practice improvement, that the intervention did not address other systems weaknesses (beyond provider skills and team functioning) such as commodity stock-outs or human resources shortages and turnover. The presence of a provider with knowledge and skills is a necessary but not a sufficient condition for the delivery of quality care. If a skilled provider does not possess a needed commodity, then clearly she will not be able to perform a lifesaving practice that depends on that commodity. Although the authors assert that this may account for the lack of improved care, they do not present data to show how much of a role systems weaknesses such as commodity supply might have played. However, the fact that the diagnosis of reported complications was not better than the comparison facilities suggests that commodity issues do not fully account for the negative results, as diagnosis and reporting requires nothing more than provider knowledge, skills, and motivation. So this underreporting of complications remains unexplained. Providers told the mentoring teams that they only report emergencies when something is done. This leaves open an important set of questions. Are providers still not recognizing problems and so not intervening? Or are they recognizing problems but still not thinking them serious enough to intervene? Or did providers intervene in some cases but did not report them because of poor outcomes?

Lack of improvement in provider practices in the India intervention may be due to other systems weaknesses, such as commodity stock-outs.

We should also consider other possible explanations for the negative results, related to provider motivation and behavior. This may have been a well-designed and suitable intervention, but there simply might not have been enough of it. As an intervention to change provider behavior and encourage teamwork, this was a relatively "light-touch" intervention. Supervisory and simulation visits happened only once every 2 months, were externally driven, and were not reinforced by more frequent on-site, peer-to-peer mentoring. So it is possible that this intervention was on the right track, but the dose was too low to have the desired effect. Emergencies happen infrequently, are high-risk situations, and require rapid responses. In order to build confidence, the desired behavior may simply need to be modeled and practiced more often to truly cement it.

The simulation intervention in India may have been on the right track, but the “dose” may have been too low to have the desired effect.

Another possible explanation for the negative results could be that the intervention did not effectively address important barriers to teamwork. How improvement goals are set has been shown to be critical.¹⁴ Teams also function best in situations where there is a flat, non-hierarchical organizational structure.² In a hierarchical health system, there may be hesitancy on the part of some providers to form, lead, or play an equal role on a team. Were there procedures and protocols for determining who would call for the formation of a team response and then lead it? Could a nurse lead a team in which a physician was a member, or play an equal role, for instance? Did lack of clarity about roles cause critical delays in emergency response? Providers may have been able to play non-traditional roles during simulations, but in actual practice they might well have reverted to usual behavior even when this may have blunted the effectiveness of the emergency response. In their debriefs, the teams noted the quality of communication and rapport, but it is not clear how much the training and mentoring directly addressed possible organizational/behavioral barriers to team formation and functioning nor to the differentiation of roles within the team. In addition to modeling the desired behavior of team members during drills, it may be necessary to discuss with providers how team members ought to act, how this might be different from usual practice, and their reactions to this.

IMPLICATIONS FOR FUTURE EFFORTS

The authors have given us an example of an intervention designed to address the need for clinical practice improvement for obstetric and newborn emergencies. The failure to show an effect on this primary outcome opens up a set of questions that future investigations ought to address. Future investigations should systematically take account of other systems issues in order to better establish what incremental value there may be to a skills and drills intervention in terms of clinical practice improvement. But other questions also need to be answered, such as the “dose” of the activities needed to improve team formation and functioning in emergencies. At a minimum, it would be instructive for future investigations to “open the black box” of what actually is happening on the ward after the skills and drills training to see how teams are functioning in emergency situations and how, if at all, this has been influenced by the skills and drills initiative. Further formative inquiry is also warranted on barriers and facilitators to team formation and functioning, centering on the issues of hierarchy, role differentiation, and team communication. The issues identified are likely to be quite context-sensitive, with the consequence that those interested in skills and drills interventions will need to engage in implementation research on proposed solutions in their own environment. Finally, of course, we need to remain open to the idea that there may be other significant behavioral barriers to effective provider response to emergencies not addressed directly by skills and drills (e.g., fear of negative consequences for unsuccessful intervention, exacerbated by a perceived non-supportive and punitive organizational environment). Future investigators may also be able to address identified problems through “restructuring the incentive environment,”¹⁵ perhaps rewarding desired behavior through one or more mechanisms (e.g., peer recognition, financial incentives). To be effective, such restructuring of the organizational environment would require the inclusion of facility managers and health system leaders in the design at a level above that of individual providers and facilities.

Future simulation interventions should focus on barriers and facilitators to team formation and functioning, including issues of hierarchy, role differentiation, and team communication.

Competing Interests: None declared.

First Published Online: 2016 Dec 19

Cite this article as: Ricca J. Limits of “skills and drills” interventions to improving obstetric and newborn emergency response: what more do we need to learn? Glob Health Sci Pract. 2016;4(4):518–521. http://dx.doi.org/10.9745/GHSP-D-16-00372