Development of an App to Facilitate Communication and Shared Decision-Making with Parents of Febrile Infants ≤60 Days Old

Abstract

Objectives:

We aimed to develop and test a tool to engage parents of febrile infants ≤60 days of age evaluated in the emergency department (ED). The tool was designed to improve communication for all parents and to support shared decision-making (SDM) about whether to perform a lumbar puncture (LP) for infants 29 to 60 days of age.

Methods:

We conducted a multi-phase development and testing process: 1) individual, semi-structured interviews with parents and clinicians (pediatric and general emergency medicine (EM) physicians, and pediatric EM nurses) to learn their preferences for a communication and SDM tool; 2) design of a “storyboard” of the tool with design impression testing; 3) development of a software application (i.e., app) prototype, called e-Care; and 4) usability testing of e-Care, using qualitative assessment and the System Usability Scale (SUS).

Results:

We interviewed 27 parents and 23 clinicians. Interviews revealed several themes, including: a communication tool should augment but not replace verbal communication; a web-based format was preferred; information about infections and testing, including the rationales for specific tests, would be valuable. We then developed separate versions of e-Care for infants ≤28 days and 29 to 60 days of age, in both English and Spanish. The e-Care app includes 4 sections: 1) homepage; 2) why testing is done; 3) what tests are done; and 4) what happens after testing, including a table for parents of infants 29 to 60 days of age to compare the risks/benefits of LP in preparation for an SDM conversation. Parents and clinicians reported that e-Care was understandable and helpful. The mean SUS score was 90.3 (95% confidence interval: 84-96.6), representing “excellent” usability.

Conclusions:

The e-Care app is a useable and understandable tool to support communication and SDM with parents of febrile infants ≤60 days of age in the ED.

INTRODUCTION

Nearly 200,000 infants ≤60 days of age in the U.S. are evaluated for fever annually in either a pediatric or a general emergency department (ED).^1,2 Because approximately 2% of these infants have an invasive bacterial infection,³ including 1% with bacterial meningitis,⁴ febrile infants frequently undergo extensive diagnostic testing, including lumbar puncture (LP), and are often hospitalized.^5,6 Parents have expressed substantial information needs during this testing process that, if unaddressed, can affect their understanding and confidence in decision-making, particularly about LP.⁷

To reduce the frequency of both unnecessary LPs and unnecessary hospitalizations, several models have been developed to stratify febrile infants as “high-risk“ or “low-risk” for invasive bacterial infection without the use of routine LP.^3,8–10 Febrile infants 29 to 60 days of age classified as low-risk by these models, and infants with abnormal urinalyses but who are otherwise low-risk, have a low but non-negligible risk of bacterial meningitis.^3,8,11–14 When deciding whether to perform an LP on these low-risk infants, emergency medicine physicians must weigh the risks of LP, including stress and anxiety for parents, against the potential risks of neurologic injury or death in the rare instance in which diagnosis of bacterial meningitis is delayed.^15–17 Consequently, the management of febrile infants varies widely depending on individual physician’s risk tolerance and institutional norms.^5,6,18,19

Since the balance between potential risks and benefits of LP for low-risk infants is uncertain and likely to be sensitive to parents’ values, decisions should incorporate parents’ input through a shared decision-making (SDM) process.²⁰ SDM requires that parents: 1) are aware that there is a choice; 2) are informed of potential benefits and downsides of the options; and 3) work with a clinician to make a choice consistent with their values.²¹ However, there are potential challenges to SDM implementation for febrile infants in the ED, including: 1) parents’ difficulty in understanding information presented; 2) parents’ emotions; 3) the age of the infant – SDM might not be appropriate for infants ≤28 days of age; and 4) time constraints in the ED.¹⁹ Facilitators include repeating information to support parent’s understanding, and giving parents the opportunity to express their opinions to boost their confidence in decision-making.⁷

A tool that addresses these barriers and facilitates SDM can positively impact the quality of care delivered to low-risk febrile infants by aligning the use of LPs with the values and preferences of informed parents.²² Additionally, ensuring that all parents of febrile infants are informed about management decisions, including those whose infants are ≤28 days of age or otherwise high-risk, can improve their understanding and confidence.⁷ Our objective was to develop and test a tool to inform parents of febrile infants in the ED and to support their participation in SDM about whether to perform an LP.

METHODS

Guiding Theoretical Model

The tool development was informed by Fuzzy Trace Theory.²³ Fuzzy Trace Theory posits that people retrieve and use information using gist and verbatim representations. Gist refers to the general meaning that people attach to a specific characteristic, and is dependent on factors such as the person’s education, culture, and experiences.²³ For example, a parent’s gist representation of the 1% risk of meningitis for a febrile infant might be either “high-risk” or “low-risk.” In contrast, a parent’s verbatim representation would refer to the literal, quantitative risk, such as “1% risk.” Studies suggest that people preferentially rely on gist representations when making decisions.^23,24 Gist-based reasoning can also improve knowledge and decision-making.^25,26 Gist representations are linked to a person’s values, which influence decisions.²³ For example, if a parent believes that LP is a painful procedure and values avoiding pain, the parent might be reluctant to have the infant undergo an LP.

Study Design

We conducted a multi-phase study at an urban, quarternary-care academic medical center that includes a pediatric ED with approximately 40,000 vists per year. We followed the process recommended by Elwyn et al. for the development of web-based decision-support interventions (Figure 1).²⁷ The study was approved by the Yale University Institutional Review Board.

Figure 1.

Phases of app development

Phase 1

We conducted individual, semi-structured interviews with 2 groups of stakeholders: 1) parents with infants ≤60 days of age with at least one rectal temperature ≥38.0° C (100.4° F), who were evaluated in the pediatric ED or hospitalized after evaluation at an affiliated general ED; 2) clinicians, namely pediatric and general emergency medicine physicians and pediatric emergency medicine nurses. Parents were approached for enrollment in the ED or on the inpatient floor; physicians and nurses were invited to enroll during division meetings or over email.^7,19 We conducted interviews in either English or Spanish dependent on the parent’s preferred language. Most parents were interviewed within 24-48 hours of the visit; 2 parents were interviewed on day 5 and 6 of their infant’s hospitalization, and 2 parents were interviewed by phone 4 days after their infant was discharged. English interviews were conducted by a pediatric emergency medicine attending physician (PLA) trained in qualitative interviewing. Spanish language interviews were conducted by a bilingual medical student trained in qualitative interviewing (EF). Purposive sampling was used to ensure inclusion of clinicians with a spectrum of years of experience and training, and parents of diverse socio-demographic backgrounds and whose infants had varying management (e.g., LP vs. no LP).^28,29

Questions from the interview guides for the semi-structured interviews that specifically elicited the preferences of parents and clinicians for a tool to facilitate communication and SDM are listed in Supplemental Table 1. Additional details about the interviews have been published.^7,19 All interviews were audio-recorded. English-language interviews were transcribed verbatim by a professional transcription service. Spanish-language interviews were transcribed verbatim and subsequently translated by a bilingual medical student who is a native Spanish speaker (EF). Data collected from participants included parent’s health literacy using a validated screening question.^30,31

Based on principles of grounded theory, two investigators (PLA and PS) developed the initial codebooks and independently applied codes to two transcripts within each stakeholder group using the constant comparative method.^28,32 For data triangulation, a third investigator also applied codes to two transcripts within each stakeholder group (LMN for clinicians and LF for parents). The investigators compared codes and the codebook was iteratively revised. Using the revised codebook, two investigators (PLA and PS) each independently applied the codes to all the transcripts in an iterative process, met weekly to compare codes, made further revisions to the codebook, and resolved discrepancies through consensus. Consensus codes were used in final analyses, and the revised codebook was then applied to all the transcripts. The coded data were grouped into categories and themes were identified. Thematic saturation for the communication tool was achieved for each stakeholder group after interviews with 23 clinicians (15 physicians and 8 nurses) and 27 parents (of 24 infants), which is within the range of 20 to 30 interviews often needed to achieve saturation.^28,29,32 ATLAS.ti (version 8) was used for data management.

Phase 2

Using the themes identified from the qualitative interviews, including on barriers and facilitators to SDM,^7,19 as well as a synthesis of evidence on febrile infants, three investigators (PLA, MCP, and LF) created PowerPoint “storyboards” of the proposed app. Separate storyboards were created for infants ≤28 days of age, for whom LP is recommended without SDM,³³ and for infants 29 to 60 days of age, which included an SDM component. All of the content was written at an eighth grade reading level or below, assessed through a readability tool that provides the average grade level using 7 different formulas.³⁴ The storyboards were reviewed with 2 parents of febrile infants (who did not participate in Phase 1), 1 physician, and 1 nurse, with iterative revisions made to the app. From the final version of the storyboards, the designs of the app were created by the Yale University Information Technology Services (ITS) Web Technologies team. We then completed Design Impression Testing with a total of 10 parents, physicians, and nurses using an online survey. First, 2 designs were displayed (1 at a time) for 5 seconds, and participants were asked to focus on their “gut” reactions and list what message they thought the designs were intended to convey.^35,36 Second, participants were given a randomized list of “positive” and “negative” adjectives and were asked to choose the adjectives that they thought best described the designs.³⁷

Phase 3

The prototype of the app was then developed by the Yale University ITS Web Technologies team, for use on an iPad. We used the International Patient Decision Aid Standards conceptual framework for the design of the SDM component of the prototype. Specifically, the app included: 1) information about options in sufficient detail to facilitate SDM; 2) balanced outcome probabilities; 3) data from current evidence; 4) a value clarification exercise; 5) structured guidance in deliberation and communication; and 6) plain language.³⁸ The English language version of the app was created first, and the full content was translated into Spanish by a native Spanish speaker (EF) to create the Spanish language version.

Phase 4

Parents, physicians, and nurses enrolled during Phase 1 were re-contacted to participate in usability testing. Parents of febrile infants evaluated in the pediatric ED during Phase 4 were also approached for enrollment, using the same enrollment criteria as Phase 1. Each testing session was conducted jointly by a Yale ITS User Experience researcher and by 2 investigators (PLA and EF); each session included qualitative and quantitative analysis. Qualitative assessment occurred through cognitive interviews using a “think aloud” protocol.^39,40 Participants were given an iPad with the app and a scenario was provided. Participants were asked to perform tasks with the app while “thinking aloud” during navigation.⁴¹ Interviewers then asked each participant a set of questions about the content and format of the app. Quantitative assessment was conducted by asking participants to: 1) rate the app’s ease of use on a scale of 1 (very difficult) to 7 (very easy); and 2) complete the System Usability Scale, a 10-item validated measure, scored from 0-100,⁴² with high scores indicating excellent usability.^43,44 The app was refined if an issue was identified by multiple participants and/or significantly affected usability. Enrollment of 8 to 10 subjects can identify up to 80% of usability problems;⁴¹ thus 9 participants were enrolled. The refined app was field tested in the ED with 3 parents of febrile infants age 29 to 60 days. The investigator provided the parents with an iPad and instructed the parents to use the app in the ED. The investigator asked for feedback near the end of the ED visit.

Written informed consent was obtained from all participants for Phase 1 of the study, and verbal consent was obtained for Phases 2 and 4 (Phase 3 was exempt). A small monetary incentive was provided to participants in Phases 1 and 4.

RESULTS

Phase 1

Participant demographics are displayed in Table 1. Six themes emerged about stakeholders’ preferences for a communication tool for febrile infants (see Table 2 for representative quotations):

Table 1.

Demographics of participating parents, physicians, and nurses

Demographic	N (%)
Parents, n=241
Gender
Female	20 (83.3)
Male	4 (16.7)
Race/Ethnicity
White	10 (41.7)
Black	3 (12.5)
Hispanic	8 (33.3)
Asian	1 (4.2)
>1 Race/Ethnicity2	2 (8.3)
Highest Education Degree
GED	2 (8.3)
High School3	12 (50.0)
College	4 (16.7)
Graduate	5 (20.8)
None of the above	1 (4.2)
Limited Health Literacy4	8 (33.3)
Performance of Lumbar Puncture on the Infant in the ED
Yes, successful	12 (50.0)
Yes, attempted but not successful	3 (12.5)
No	9 (37.5)
Disposition of Infant on Initial ED visit
Hospitalized	19 (79.2)
Discharged from the ED	5 (20.8)
Infant with Bacterial Infection5	5 (20.8)
Physicians, n=15
Training
Pediatric Emergency Medicine Fellow	2 (13.3)
Pediatric Emergency Medicine Attending Physician	11 (73.3)
General Emergency Medicine Attending Physician	2 (13.3)
Years as an Attending Physician6
1-5	7 (46.7)
6-10	4 (26.7)
>10	4 (26.7)
Nurses, n=8
Years as a Nurse in Pediatric Emergency Department
1-5	2 (25.0)
6-10	3 (37.5)
>10	3 (37.5)
Previously Worked in a General Emergency Department	4 (50.0)

¹For interviews conducted with both parents (n=3), demographics provided for parent who predominantly participated in the interview

²1 parent identified as “Mixed Race” and 1 parent identified as Hispanic and White

³1 parent completed high school and was currently completing college

⁴Defined as answering “somewhat,” “a little bit,” or “not at all” to the question, “how confident are you in completing medical forms by yourself?”

⁵4 infants had a urinary tract infection; 1 infant had bacteremia and bacterial meningitis

⁶2 pediatric emergency medicine fellows classified as 1-5 years

Table 2.

Preferences of parents, physicians, and nurses for a communication tool

Theme	Representative Quotation
1.  Parents vary in how they want to receive information. Having different methods of communication available would be helpful.	Pediatric emergency medicine nurse: “You know, everybody learns differently, so I think that would give an option for people who are more visual learners to understand what it is that we’re actually doing. If they can see a video or see some diagrams or something. I think it might help us as a nurse and a provider to explain everything if we had some sort of sheet to make sure that we’re all going over the same points or hitting on the same pieces of that septic workup to make sure everything is discussed, or in a way that we want it discussed.” Mother of a 7-week old infant on whether an additional source of information would have been helpful: “I think they should have it in the event that they need it. I think that like, we were perfectly capable of sitting and having a conversation and understanding and asking questions and getting answers. We felt good about that and I think they felt good that we understood. But if you have somebody who maybe that type of communication is not their preferred way of communicating or they seem like they’re not understanding it, maybe if you have that kind of thing handy, that’s a good thing. If you need it. I don’t think we needed it, but if somebody were to need it, that would be a good thing to have.” Father of a 3-week old infant: “I mean, in some cases, like some people understand better by reading it than just hearing it. Some people understand better by hearing it than reading it, so I mean, it could work both ways. It depends on what type of person it is.”
2.  A communication tool should augment, but not replace, verbal communication from nurses and physicians.	Pediatric emergency medicine physician: “I actually do think that certainly in that initial – because the nursing staff, we all want to get everything done quickly, not only for like, flow in the ER, but just to make sure the patient is safe. I think having something, like a tool like that, that we can give the parents either after we talk to them initially, for them to use as a reference or to read after we leave the room and the nurses are doing all the procedures. The parents could reference that and maybe if they have any questions, use that. I think it would be very helpful. And to read again – because they’re probably going to forget a lot of the information we told them.” Father of a 6-week old infant who had bacterial meningitis: “For me it would be good to get it talking and through writing…. because when one is being talked to, one can easily go into shock and not pay attention to what they are saying. Instead if they give it to you written, the words are not going to leave the paper and they will be there. And one can review and understand well what is happening.”
3.  A web-based tool was preferred over a paper format.	Pediatric emergency medicine physician: “Let’s say the residents obtain the history and others are kind of like, gathering their resources and other things that a parent or others would take a moment to kind of get that primer. I mean, you know, for lack of a better understanding, I mean, Disney does this. What ride have you been on recently or whatever? They use a little multimedia prime.” Mother of a 6-week old infant, with limited health literacy: “Or you know, give me the names of stuff that I can look up on my phone. They even have a tablet here, you know, if you want more information, here’s a card and this is the website. Go up there and read it and then tell me if you’re comfortable with it.” Mother of a 6-week old infant: “Maybe if it was like, brief and short or if the option was given and then handed them an iPad and like, this is what an LP is or something like that. That might be more interesting than a piece of paper.” Father and Mother of a 3-week old infant: “The iPad. It’s a lot easier. You get to the main point a lot faster. As opposed to have to read everything…. everybody has a phone, so all they have to do is just swipe up and switch pages…. You can zoom the letters and you don’t have to read the little old letter…. That’s the generation now anyway. Technology is the era now.”
4.  Although clinicians suggested videos for the tool, parents felt that videos might be alarming or distracting.	Pediatric emergency medicine physician talking about using a video to communicate information to parents: “You know, something as mundane as buying a pair of shoes. If you go out onto Zappos now, there’s a video embedded in the website and you can click on it and you get a one-minute review of a pair of shoes. It’s a great idea. People have short attention spans, but videos are great ways to communicate ideas.” Mother of 1-week old infant talking about using a video to show parents what to expect for LP: “I don’t know so much of a video. I think a video might scare some parents because it is a scary procedure and especially when the baby is so young.” Father of a 6-week old infant: “I don’t know how we would’ve seen a video without it feeling like we were – I don’t know if you’ve ever been go-cart racing or anything like that, but they make you watch a video before you start and I feel like you’re just kind of on hold, waiting for things to start. I think that’s what it would’ve felt like for us. Basically on hold to see a doctor because we have to watch this video. I don’t think it would’ve been helpful for me.” Mother of a 5-week old infant: “I always love a good video, but I don’t know if I would’ve watched something. I mean, it’s a good question. The more information, the better, but how to get that information out, like what – that’s tough. I watch videos a lot on how to do things. Would I watch a video of an ER experience? No, probably not. It’s more – the videos were more for like a tutorial. However, articles, bullet points, a fact sheet, would 100 percent be helpful.”
5.  Information on infections and testing, including the rationale for specific tests, would be valuable.	Pediatric emergency medicine physician: “If there were a sheet that talked about the safety of a test like lumbar puncture, that you could leave with the family, because they will have time potentially in between tests or when the nurse is doing tests to look at something like that.” Father of a 3-week old infant with a urinary tract infection: “It would’ve been nice, you know, if they had like, a little booklet for why they get something like the urinary tract infection, like why they get it – just frequently asked questions or something like that…. yeah, I guess more like what to expect and explaining the process of all the procedures that they do. You know, just more knowledge of generally what they do.” Mother of a 5-week old infant, on the content of an additional communication tool: “I mean, it probably would be helpful….having a decision tree and having people see why you’re doing things to make more sense.”
6.  Parents could review the communication tool after speaking with the physician and while waiting for test results.	Pediatric emergency medicine physician: “Yeah, I mean, you certainly could let them review that information while they’re waiting for the lab results or something like that. I think that would be a good – that’s a lot of downtime, particularly at our institution where they could review that and see why we’re talking about what we’re talking about. Especially if you’re debating whether you’re going to do the LP based on bloodwork.” Mother of a 6-week old infant: “Um, actually, when we were downstairs, I noticed that they had like, a sheet on the wall that had said, “Waiting for results.” So it was kind of like, they didn’t use it, but things were kind of moving quick. If we were to be down there for a while, like, I think that sheet would be helpful for people just so that you know – because I kept saying like, okay, what else do we have to do? But that sheet would kind of like, keep the family, like okay, we have this next and then this and then we still have to do this.”

1). Parents vary in how they want to receive information. Having different methods of communication available would be helpful.

Clinicians (i.e., physicians and nurses) and parents thought that an additional method of communication would be helpful due to parents’ different information needs and ways of understanding information. Although some parents stated that talking with the physicians and nurses was sufficient, most felt that parents would benefit from the having the option to read additional written information.

2). A communication tool should augment, but not replace, verbal communication from nurses and physicians.

Nurses in particular felt that having a communication tool would benefit parents and clinicians by standardizing the information communicated to parents, especially by nurses or trainees. However, some physicians raised concerns that the information would need to be adapted to parents’ educational and health literacy levels, which would be achieved better with verbal communication rather than relying solely on a standardized tool.

3). A web-based tool was preferred over a paper format.

Although some suggested paper-based information sheets or pamphlets, most clinicians and parents noted that many people are used to getting information electronically through a computer, iPad, or smartphone, and thought a web-based communication tool delivered through one of those devices could be more effective. Clinicians felt that parents might not look at a paper-based tool, and that providing information through an electronic format would allow for information to be presented in a more visually-appealing manner and using different displays.

4). Although clinicians suggested videos for the tool, parents felt that videos might be alarming or distracting.

Clinicians felt that videos of procedures would be useful to help parents learn about the LP procedure. However, many parents expressed concerns that watching videos of procedures would be “scary” and would distract from the information. In addition, several parents were concerned that having to watch a video might delay their infant’s evaluation. Some parents felt they would have been comforted if they listened to narratives from other parents of febrile infants describing their experiences. However, other parents reported that narratives would be useful only if the parents had a good experience, since hearing about negative experiences could increase anxiety or cause alarm.

5). Information on infections and testing, including the rationale for specific tests, would be valuable.

Clinicians and parents wanted information about the procedures involved in testing and the low risk of serious complications from LP. Additionally, clinicians and parents felt that giving parents information on the timeline and what to expect after learning test results would be valuable. Parents also wanted more information on the possible infections, particularly bacterial meningitis.

6). Parents could review the communication tool after speaking with the physician and while waiting for test results.

Both clinicians and parents felt that the tool should be given to parents after the initial discussion between physicians and parents. Both groups reported that it would be helpful to have parents, while waiting for the test results, review the information that was discussed, including the next steps.

Phase 2

Based on the qualitative data, separate PowerPoint “storyboards” were created for infants ≤28 days and 29 to 60 days of age. The storyboards included the proposed text and format for each of the following 4 sections of an app named “e-Care: Emergency Room Care of a Baby with Fever”.:

• 1) Home “menu” page;

• 2) Why testing is done, including the types of bacterial infections;

• 3) What tests are done, including the procedures for urine, blood, and cerebrospinal fluid testing, and the expected timeline for results;

• 4) What happens after testing (for infants ≤28 days of age) or what happens after the test results are known (for infants 29 to 60 days of age).

For infants ≤28 days of age, the “what happens after testing” section describes that the infant would be hospitalized after LP and would receive antibiotic therapy for at least 1 or 2 days. For infants 29 to 60 days of age, the “what happens after the test results are known” section lists the 3 possible categories based on the results of urine and blood testing: 1) bacteremia or bacterial meningitis is possible (high-risk infants); 2) probable urinary tract infection (UTI) but unlikely to have bacteremia or bacterial meningitis (positive urinalysis but otherwise low-risk); and 3) unlikely to have any bacterial infection (low-risk infants). For the latter 2 categories, tables support values clarification, help parents’ deliberate about whether they want their infant to have an LP, and facilitate SDM between the physician and parents. The tables include a comparison of how bacterial meningitis would be found with an LP vs. no LP, possible results (including unsuccessful procedure with LP vs. return to the ED with no LP), and potential benefits and risks of LP vs. no LP.

In reviewing the storyboards, parents and clinicians thought that both the proposed structure of the app and the amount of content was appropriate. Two suggestions from parents were incorporated into the storyboard: 1) adding text that febrile infants were evaluated frequently, to provide comfort to parents; and 2) using language in the SDM component that would allow parents to weigh the pros and cons of LP without scaring them into making a decision. The physician thought it would be important to review the risk estimates in the app with physicians prior to implementation, to ensure consistent messaging. The nurse suggested adding descriptions of measures used to help comfort the infants during testing (e.g., sugar water on a pacifier, parents’ presence during procedures).

The Yale ITS Web Technologies team then created the initial designs of the e-Care app. The overall design aimed for neutral, comforting colors, with limited graphics to minimize distractions. In each section of the app, the format was a left-sided arrow with text that oriented parents to the content on the right side of the screen, including what to tap for more information (Figure 2 shows the design of the homepage).

Figure 2.

Design of the e-Care homepage

During Design Impression Testing, participants viewed the designs of the homepage and the “what tests are done” section. Participants identified that the designs were intended to inform parents what would happen in the ED and what tests might be performed. Participants then chose adjectives from a randomized list to describe the designs. Overall, 76% of adjectives chosen were “positive” adjectives, most commonly “accessible”, “understandable,” “useful,” “easy to use,” and “meaningful.” “Negative” adjectives were chosen less commonly, and included “impersonal” and “simplistic,” though it was noted that “simplistic” was also an intentional choice to minimize distractions. Based on the results of Design Impression Testing, the designs were determined to be appropriate and ready for prototype development.

Phase 3

The prototype of the e-Care app was created, with separate versions for infants ≤28 days of age and infants 29 to 60 days of age (each in English and Spanish). To ease navigation, parents could move between sections of e-Care through a top navigation bar or by tapping on boxes on the right side of the screen (Figure 2). The SDM component of the 29 to 60 day version of e-Care included the risk estimates for bacterial meningitis and a statement describing the importance of the physician knowing how parents feel about LP (Figure 3A). The parents were instructed to review the table comparing the risks and benefits of LP vs. no LP (Figure 3B). For infants with likely UTIs, the risk of bacterial meningitis was estimated as 0.4%, which is the upper limit of the 95% confidence interval (CI) for the prevalence of meningitis among infants with UTIs or positive urinalyses in a recent meta-analysis.⁴⁵ For low-risk infants, the risk of bacterial meningitis was estimated as 0.1%, which was the upper limit of the 95% CI for the prevalence of meningitis among low-risk infants, combined from studies of prediction models that do not include cerebrospinal fluid testing for infants 29 to 60 days of age.^3,8,11 The risk for serious complications from LP was estimated as 0.1% using available online risk estimates.^46,47 Risk estimates were provided using text and icon arrays, to facilitate gist-based reasoning (Figure 3C).^48,49

Figure 3.

A) Text to facilitate SDM between the parents and physician, B) Table for parents to compare the risk and benefits of LP, C) Risk estimate as icon array

Phase 4

Of the 9 participants enrolled for usability testing, 6 were parents (4 who did not participate in Phases 1, 2, or 3, including 1 with experience in app development and 1 whose preferred language was Spanish), 2 were physicians, and 1 was a nurse. For quantitative analysis, 7 of 9 participants (77.8%) gave a rating of 7 for “ease of use” while 1 (11.1%) gave a rating of 6.5 and 1 (11.1%) gave a rating of 6. The mean score on the 100-point System Usability Scale was 90.3 (95% CI: 84-96.6), which represents “excellent” usability.⁴³ For qualitative analysis, participants felt that the e-Care app was understandable without too much medical terminology, easy to navigate, and would be very helpful for parents. The primary feedback for changes, and the corresponding revisions, are listed in Table 3. For implementation, some, but not all, parents felt that they would need a brief introduction to e-Care by the nurse or physician. Parents of infants 29 to 60 days of age expressed that when the urine and blood testing results were available, they would want a clinician to show them which risk category applied to their infant. When testing the revised e-Care app in the ED, parents felt that the app was very helpful, including the information on potential benefits and downsides of an LP. The parents did not suggest any additional changes.

Table 3.

Qualitative feedback and corresponding revisions during usability testing

Primary Areas of Feedback	Revision
1) Content  -Inclusion of viral testing confusing as focus of app on bacterial infections and testing  -Clarity needed on which tests are always done vs. which tests are contingent on results of other tests (i.e., LP for infants age 29 to 60 days)	-Description of viral testing removed from the app and language added to the “why testing is done” section that the app will focus on testing for bacterial infections -Text modified for the 29 to 60 days version of the app to provide clarity that the decision about LP will be contingent on the results of urine and blood testing
2) Navigation  -In the “what happens after the test results are known” section for infants 29 to 60 days of age, participants did not know to scroll down to the table to review the risks and benefits of LP vs. no LP  -In the “what happens after the test results are known” section, participants had difficulty navigating back to the main screen	-Hyperlink added to direct parents to review the table: “Please tap here for information to help you think about whether or not you want your baby to have a spinal tap” -Hyperlink added to direct parents to return to the main screen: “Tap here to go back to the prior screen”
3) Icon Array  -Most participants did not know how access the icon array	-The hyperlink to the icon array was bolded

DISCUSSION

Using an iterative development process that engaged key stakeholder groups, we designed and conducted usability testing of an app, e-Care. E-Care supports communication with parents of febrile infants ≤60 days of age, and facilitates SDM with parents of infants age 29 to 60 days about whether to perform an LP. Because SDM might not be appropriate for infants ≤28 days of age,¹⁹ we developed a version of the e-Care app for parents of infants in this age group to address their information needs, but without an SDM component. We also developed a Spanish language version of e-Care, as non-English patients have been underrepresented in past communication studies.⁵⁰ Stakeholders found e-Care to be usable and understandable, though its efficacy needs evaluation in a multicenter study.

Although a 2016 Academic Emergency Medicine consensus conference identified decisions about the management of febrile infants as well-suited for SDM,⁵¹ we previously identified several barriers to SDM in this patient population. These barriers included parents’ need for information that may be unmet with verbal communication alone, parents’ understanding, parents’ emotions, and time constraints.¹⁹ We designed e-Care to overcome these barriers by: 1) creating content written at an eighth grade reading level or below; 2) using a web-based format that was rated as having excellent usability; 3) giving information on testing, including the rationale for testing and an explanation of possible results, to proactively address parents’ information needs and help reduce their anxiety;⁷ 4) allowing parents to compare the potential benefits and downsides of LP while they await the results of urine and blood testing; and 5) facilitating gist-based reasoning by using icon arrays to convey risks, and linking risks to parents’ values.^48,49,52 By informing and preparing parents for SDM, the time required of physicians to engage parents in SDM could be reduced, addressing one perceived barrier to SDM.⁵³ Time to engage in SDM needs to be tested in future investigation.

A challenge when developing e-Care was balancing the provision of information to parents while not exacerbating their fear or anxiety. The possibility of an infant having an invasive bacterial infection, and the testing process, are stressful and anxiety-provoking for parents.^7,15,54 Although clinicians suggested videos for e-Care to show the test procedures, many parents felt that watching a video would be distracting, alarming, or would delay the infant’s evaluation. Therefore, videos were not included in the app. We also included information in e-Care to help calm parents, including descriptions of measures taken during testing to provide comfort to the infant. Using e-Care to ensure that all parents, including those with infants ≤28 days of age, are informed and aware of what to expect can also help parents’ anxiety.⁷

Making a choice about LP is an affect-rich decision for parents which elicits a strong emotional response.⁵⁵ While many parents of febrile infants want the opportunity to express their opinions and concerns during decision-making, their preferences for participation in making the decision vary, and some parents expressed anticipatory regret about making a choice about LP.⁷ The SDM component of e-Care was therefore designed to allow parents to compare the benefits and downsides of LP and to consider their values, to help them think about whether they wanted their infant to have an LP. This process is intended to prepare them for an SDM discussion with their physician. This approach can empower parents to express their opinions to the physician, but does not leave decision-making solely to parents.

Using SDM interventions with adult patients is associated with increased knowledge, improved risk perceptions, higher congruency between patient’s values and choices, reduced decisional conflict, and, in some studies, a higher likelihood of patients choosing more conservative options over invasive procedures.⁵⁶ In pediatrics, SDM is similarly associated with improved knowledge and reduced decisional conflict for parents.^57,58 Yet there have been few studies conducted outside the primary care setting – and none with parents of young infants in the ED. Thus the impact of SDM on clinical outcomes in this setting is unclear.^57–59 Although e-Care showed excellent usability and understandability, a future randomized trial comparing e-Care to usual care can evaluate its impact on parents’ knowledge, decisional conflict, and informed, value-concordant choices about LP.²⁵ The future trial can also evaluate clinical outcomes, including LP rates and delayed diagnoses of bacterial meningitis. Finally, future studies should monitor e-Care implementation, including how treating clinicians interact with and use the app, and their fidelity to the SDM component.⁶⁰ For infants ≤28 days of age, the impact of e-Care on parents’ knowledge and anxiety should be evaluated. Additional testing of e-Care with Spanish-speaking parents is also needed.

LIMITATIONS

There are several limitations to our study. First, the study was conducted at a single quaternary-care academic medical center, and the perspectives of participants might not represent all clinicians or parents of febrile infants. However, we used purposive sampling to ensure that we obtained the perspectives of a diverse group of participants, including clinicians with a range of years of experience and parents from diverse socio-demographic backgrounds, education and health literacy. We also included parents who preferred language was Spanish and the perspective of one parent whose infant had bacterial meningitis. Second, parents’ perspectives on SDM during interviews may not reflect their perspectives during actual decision-making, though most parents were interviewed within 24-48 hours of the ED visit. Third, most interviews were conducted by a pediatric emergency medicine physician, which may have impacted responses. However, the physician was not involved in the care of any of the infants whose parents were interviewed. Fourth, the translation of the English language version of e-Care to Spanish was done by a bilingual, native Spanish speaker but not a certified interpreter. Fifth, the risk estimates for bacterial meningitis were calculated using the upper limit of the 95% CI for infants with UTIs or positive urinalysis based on a recent meta-analysis,⁴⁵ and for low-risk infants based on the upper limit of the 95% CI combined from studies of prediction models that do not include cerebrospinal fluid testing for infants 29 to 60 days of age.^3,8,11 While these risk estimates were based on the best available evidence, actual risks may be different. The risk estimate for serious complications from LP was based on online risk estimates with limited data.^46,47

CONCLUSIONS

The e-Care app is a useable and understandable communication and SDM tool for clinicians to use with parents of febrile infants ≤60 days of age in the ED. The iterative, stakeholder-engaged development process facilitated the design of e-Care to meet the preferences of parents, physicians, and nurses. A randomized trial is needed to evaluate the impact of e-Care on clinical and implementation outcomes.