Development and application of a primer and reference assessment tool for neonatal abstinence syndrome: A phase I pilot study

Abstract

Background

Despite evidence for the efficacy of strict neonatal abstinence syndrome (NAS) treatment protocols, no national standardized education, diagnosis or treatment strategy is available.

Objectives

To describe the development and preliminary usability of an electronic bedside primer and decision support tool for medical providers, with embedded, interactive education and reference modules.

Methods

A panel of NAS experts established a standard operating procedure for the best practices of NAS management and developed an interactive mobile primer and reference and assessment tool to assess NAS with a curriculum and decision support system. We tested the feasibility and usability of this tool with n = 8 users, including registered nurses, last-year undergraduate nursing students and neonatal physicians.

Results

Participants rated the usability of the modules positively, with an average rating of 4.5 (scale of “1 = Strongly disagree” to “5 = Strongly agree”). Participants appreciated the ability to score the infant at the bedside using real time electronic entry. Seven users noted that the electronic device entry would be as accurate as paper or computer-based Electronic Medical Records entry and one user indicated it would potentially be more accurate during post-usability interviews. Users recommended improvements to the curriculum, including increasing detail of definitions and adding videos for additional NAS signs.

Conclusion

The assessment tool appears to be acceptable and usable by potential users. The strong ratings across users provides support for further testing whether its acceptability and usability remain high in a hospital setting, while assessing the impact on clinical outcomes such as newborn hospital length of stay.

1. Introduction

Neonatal Abstinence Syndrome (NAS) refers to a group of problems experienced by newborns exposed to substances in utero, typically to opioids. Relatedly, Neonatal Opioid Withdrawal (NOW) refers specifically to postnatal opioid withdrawal affecting 27%–94% of newborns born to women using illicit (e.g., heroin) or licit (e.g., opioid medication) drugs during pregnancy [[1], [2], [3], [4]]. In 2014, roughly 32,000 infants were diagnosed with NAS in the US, with an aggregate associated hospital charge of $1.5 billion [[5], [6], [7]]. NAS signs are highly variable [8], and often mimic other conditions such as infection, hypoglycemia, hypocalcemia, and hyperthyroidism [9].

Multiple assessment tools were developed to evaluate NAS signs, but no tool has emerged as ideal. The Finnegan Neonatal Abstinence Scoring Tool (FNAST) [10,11] is widely used since its original development in 1975, with later modified versions that incorporate supportive measures along with scoring in use more frequently in both research and clinical practice [[12], [13], [14]]. Inconsistency across raters is the primary measure of efficiency for the FNAST, stressing the importance of feedback in reaching competency [14]. While the FNAST has several noted limitations [15,16], it has demonstrated acceptable inter-observer reliability [17] and is recommended by the American Academy of Pediatrics (AAP) [9,15,18]. Other commonly used tools include the Neonatal Withdrawal Inventory [19,20], MOTHER NAS scale [21], and Eat, Sleep, Console (ESC) [22]. In addition to the scoring tools mentioned, clinical experiences of providers are also pivotal in screening and management of NAS and may override scoring tools in some instances. However, often, the danger in that is that experienced providers may change how they score an infant but fail to go back to the basic foundation of the scoring tool, and therefore fail to maintain competency.

Use of a strict protocol to treat NAS is associated with reductions in length of opioid withdrawal and hospital stay for newborns [18,23], yet there is no nationally standardized diagnosis or treatment strategy for NAS to date [24]. While the AAP recommends that hospitals “should develop and adhere to a standardized plan for the evaluation and comprehensive treatment of infants at risk for or showing signs of withdrawal” [18], only about half of surveyed Neonatal Intensive Care Units (NICUs) have a written policy regarding the management of NAS [25], although this number is expected to grow following the national implementation of the Vermont Oxford Network NAS Universal Training beginning in 2015 [26]. This universal NAS care training is intended for systematic dissemination of current evidence-based education and resources to the interdisciplinary workforce engaged in caring for substance-exposed infants and families [26,27].

Few comprehensive NAS-related educational and training opportunities exist for healthcare providers, and most of these are overviews providing only brief summaries of common NAS signs and treatment strategies. While some of the existing screening tools, including the FNAST and MOTHER NAS Scale do have training materials, such as videos or DVDs [20], only two training programs supporting inter-rater reliability for Finnegan scoring exist: NeoAdvances, a commercial training product developed by Loretta Finnegan and Karen D'Apolito specifically for the use of the FNAST, and a module developed by Gateway Health in Pennsylvania, both of which utilize a DVD and a manual [28].

As mobile technology becomes ubiquitous, a growing body of evidence has demonstrated that smartphone apps can be successfully used for bedside medical reference, [29] medical education and interactive training [[30], [31], [32]], and diagnostic assistance [33]. Despite the potential for a comprehensive, educational bedside reference tool for neonatal and pediatric providers, only one mobile app related to NAS exists. The Neonatal Drug Withdrawal Tools for Android by Kyle Gunter [34] provides five pharmacological protocols with dosage calculators; however, it does not provide any background reference or assistance with the interpretation of the FNAST.

The growing prevalence of NAS and the absence of an easily applied, standardized treatment protocol has formed an urgent need for education and decision support tools for healthcare professionals to successfully recognize and treat NAS. Below, we describe the initial development and implementation of a primer and reference assessment tool to streamline and unify education offered to NICU providers to more reliably assess NAS signs.

2. Methods

The tool described here was developed by Ringful Health, LLC for use in the first reported evaluation of this mobile-based, interactive tool for NAS providers in the NICU.

This first project had the following objectives: 1) Develop an educational and reference curriculum for NAS recognition and treatment; an interactive education course to deliver the NAS curriculum; and a bedside primer, reference, and decision support tool for clinicians; 2) Evaluate usability of the educational curriculum and technical tool. The study protocol was reviewed and approved by Washington State University's institutional review board.

2.1. First objective

To meet the objectives of developing an educational and reference curriculum for NAS recognition and treatment and a bedside primer, reference, and decision support tool for clinicians, we first assembled a panel of NAS experts to determine the availability of data references, educational and clinical guidelines, and presentation strategies. Panel members included Washington State University faculty, practicing clinicians from the regional health district, and experts from two regional hospitals who either have worked or currently work in the NICU with NAS patients. An in-house subject matter expert, a registered nurse with over 20 years of clinical experience working with pregnant women with substance use disorders and their newborns, served as Ringful Health's Clinical Director to facilitate discussions and translate the consultant team recommendations into the education program and bedside assessment tool.

We arranged a face-to-face session wherein we worked to identify NAS topics, assign sub-panels of experts to each topic, and deliver a written outline of key knowledge points and knowledge test questions under the assigned topic. We then defined and collected reference materials such as medical publications, scientific references, and best-practice guidelines and delivered the established standard operating procedure for the best practice of NAS management. Next, we provided feedback based on the review of the FNAST to identify content pieces that can be directly referenced from the scoring questionnaire as “online help”. We then developed an interactive education course to deliver the NAS curriculum. This interactive course allows the user to go through NAS training at her/his own pace. The education module was developed in a manner that allows integration into multiple delivery platforms such as mobile apps, websites and common learning management systems for CME courses.

To develop a bedside primer, reference, and decision support tool for clinicians, we identified an electronic platform for software implementation and developed a smartphone/tablet application that incorporates an interactive version of the NAS scoring system. We then incorporated educational courses into the bedside electronic tool as contextual help for the scoring system and merged the real time contextual links from the scoring questions. We demonstrated Electronic Medical Records (EMR) connectivity to prefill patient data into the scoring questionnaire, sent scores back into the EMR, and electronically escalated or referred patients to regional NICUs as needed. Finally, we demonstrated CME course feasibility and delivered a “web player” prototype of the primer, reference, and decision support tool.

2.2. Second objective

To meet the second objective of conducting feasibility and usability testing of the educational curriculum and technical assessment tool, we recruited n = 8 NAS professionals (e.g., registered nurses, last-year undergraduate nursing students, and neonatal advanced practice nurse; see Table 1 for demographic characteristics of the participants) from the participating university and Level 3 NICUs and tested them for knowledge and observational experience on the use of the tool in a simulated care environment. We revised and improved the tool context and software prototype in response to perceived needs and delivered feedback from this pilot study survey on the context and technology acceptance.

2.2.1. Descriptive evaluation plan

We calculated descriptive statistics for the seven basic questions about the tool's usability and summarized qualitative feedback from the participants.

3. Results and discussion

3.1. First objective

We developed an educational curriculum for NAS recognition and management and incorporated this curriculum into the interactive course. This curriculum is available in two formats. The first is a series of PowerPoint slides with instructor notes and bibliography. This is for use in the rare situation that the content cannot be delivered through the optimal training experience of an electronic device. The second is an interactive education course with video content deliverable via web, mobile devices and common learning management systems. The course was developed by the assembled panel of NAS experts identified above. Content experts also completed a review of the format and the content to ensure it would be manageable for a range of students. The key factor identified in this regard was the ability to format content in most cases to sub-segments of 5–8 min, allowing users to self-pace, with breaks at logical points. The course contains 7 lessons and an overview. The combined video length is 1 h and 45 min. The seven components are:

• Lesson 1 – Epidemiology of NAS and Neonatal Opioid Withdrawal and Pathophysiology of NAS/NOW

• Lesson 2 – Signs of NAS

• Lesson 3 – FNAST Scoring System

• Lesson 4 – Toxicology

• Lesson 5 – Pharmacological Treatment for NAS

• Lesson 6 – Non-Pharmacologic Treatment

• Lesson 7 – Transition to Follow Up Care

The bedside primer, reference and decision support tool for clinicians is formatted for the iPad (see Fig. 1). The education module is a series of content screens (videos, images and text) and contains written posttest as well as optional case studies. Details of the lessons are currently being refined and are changing in preparation for further testing, however, readers who are interested in further details regarding the lessons or who are interested in following up with the current status of the tool may contact Ringful Health at info@ringful.com or 7401 W. Slaughter Lane Suite 5076, Austin, TX 78739.

Fig. 1.

Reference Overview and Example.

Panel A. Schematic of iPad Symptom and Goal Entry and EMR Scoring Integration.

Panel B. Example Screen of Reference and Training Support Module.

These case studies provide the capability to select a patient exhibiting NAS/NOW signs, view the patient history, enter a FNAST score and track dosages related to pharmacologic therapy (see Supplemental Fig. 2 for an overview). The user can proceed through the learning path via a software “player”, allowing the user to view content in sequence or according to the course's branching logic, and to take quizzes. The reference tool provides contextual reference information from the education module in a portable format. This e-reference format allows for access in multiple locations supporting the education module. The content is organized by F-NAST screening element. Finally, the tool operates in a decision support mode which, subject to further development and FDA approval, could be used at the bedside for integrating NAS/NOW scoring with an EMR. The curriculum and training components meet the requirement of Continuing Nursing Education under California Board of Nursing and can potentially be offered for CME credit.

3.2. Second objective

The n = 8 users of this tool indicated that overall, the modules were 1) clear and concise, 2) kept their attention, 3) allowed self-pacing and 4) provided useful content. Feedback from an experienced NICU user indicated that “because the video does such a good job of teaching people, it might prevent some kids from coming to us that don't need to come.” The structured user feedback found that the content on the electronic scoring device was relevant to the simulation, M = 4.75 (on a 5-point scale: 1 = Strongly disagree, 5 = Strongly agree) and that the device was easy to use for all participants, M = 4.5 (5-point scale: 1 = Strongly disagree, 5 = Strongly agree). The users commented positively about the ability to score the infant at the bedside. Two experienced users noted that their scores are often memorized at the bedside for later entry into the computer-based EMR, allowing for multiple transcription errors in the process. Seven users believed that the electronic device entry would be as accurate as paper or EMR entry, and one user indicated it would potentially be more accurate. User characteristics are included in Appendix B under Table 1.

Users also recommended improvements to the curriculum, which are currently being integrated in preparation for the next phase of research for this tool. The primary improvement was more videos for additional NAS signs. Users also recommended reformatting the case scenario at the end of Lesson 3 from a narrative to a video format. Finally, users also recommended increasing detail of definitions used in the modules.

4. Conclusions

The bedside primer and reference assessment tool described here responds to the urgent need for online education, reference and decision support tools for frontline healthcare professionals to successfully recognize and treat NAS/NOW. This mobile tool assists clinicians in identifying, interpreting, scoring and responding to NAS/NOW signs and thus has the potential to improve neonatal outcomes. Further, this tool's potential benefits equally extend to facilities that experience few NAS births and have less experience/practice with scoring and operating a NAS protocol. The curriculum is available for use by instructors in a classroom setting and via distribution to mobile devices utilizing web-based video or a common learning management system. The curriculum content supports delivery as a stand-alone course and as a contextual reference for users of the tool. The educational module elements were structured to enable delivery via integration into Learning Management Systems or as a CME course.

A possible limitation of this bedside primer and reference assessment tool is its integration of FNAST as its standardized NAS assessment measure. While widely utilized, FNAST has several limitations such as a large number of items, with some that are difficult to rate and others that occur infrequently; establishing reliability requires extensive training and continuous inter-rater reliability checks; the validity of differential item weighting has not been examined, nor the validity of its cutoff scores for medication administration [15,22]. In addition, FNAST is associated with increased pharmacologic intervention and it requires the scorer to frequently disturb the newborn, which contradicts the AAP recommendation of providing first-line, nonpharmacologic interventions [22]. It is our hope that the development of an electronic training and decision support tool such as this will ease the way for future updates for guided care of NAS, e.g. ESC treatment and other, future modes of treatment and training of medical providers. In addition, it is a relatively easy and inexpensive mode of standardizing both training and treatment and hospitals could choose to require the completion of such training modules prior to working with babies in the NICU.

In the years since the phase 1 development of the bedside primer and reference assessment tool described here, the ESC treatment has rapidly gained traction as a better alternative to FNAST with preliminary data showing that compared to FNAST, newborns managed with ESC required significantly less pharmacotherapy and had shorter lengths of stay [22]. Unlike the FNAST, the ESC's main principal is that both the pharmacological and non-pharmacological treatment of the infant is based on infant functioning and comfort (as expressed in the areas of sleeping, eating, and consoling), rather than reducing signs and symptoms of withdrawal. Nonetheless, the bedside primer and reference assessment tool is a flexible prototype that can be amended to include educational modules and scoring assessment based on ESC treatment. Notably, additional investigation of this novel tool is currently underway in a clinical trial to further test its ability to impact clinical outcomes and assist with better, streamlined treatment for babies experiencing NAS, and it includes a newly integrated ESC treatment module.

Role of funding

This work was supported by funding from the National Institute on Drug Abuse (N44DA171210, PI; Johnson) and National Institute on Minority Health and Health Disparities (P20 MD006871; PIs McPherson and Buchwald). These funding sources had no other role other than financial support.

Declaration of competing interest

Dr. McPherson has received research funding from the Bristol-Myers Squibb Foundation. Dr. McPherson has received research funding from Ringful Health, LLC. Dr. McPherson has also received research funding from Orthopedic Specialty Institute and consulted for Consistent Care company. This funding is in no way related to the investigation reported here.

None of the other authors have any financial, personal, or other type of relationship that would cause a conflict of interest that would inappropriately impact or influence the research and interpretation of the findings.