Acceptability, Usability, and Effectiveness: A Qualitative Study Evaluating a Pediatric Telemedicine Program

Abstract

Background:

Pediatric emergency telemedicine consultations have been shown to provide support to community emergency departments treating critically ill pediatric patients. However, despite the recognized value of telemedicine, adoption has been slow. To determine why clinicians frequently do not use telemedicine when it is available for pediatric patients, as well as to learn how to improve telemedicine programs, we conducted a qualitative study using stakeholder interviews.

Methods:

We conducted a qualitative study using grounded theory methodology, with in-depth interviews of referring and accepting physicians and referring, transport, and transfer center nurses. We analyzed data iteratively and adapted the interview guide based on early interviews. We solicited feedback from the participants on the conceptual model.

Results:

Sixteen interviews were conducted; all respondents had been involved in a telemedicine consultation at least five times, with some having used telemedicine more than 30 times. Analysis resulted in three themes: (1) Recognizing and addressing telemedicine biases are central to gaining buy-in, (2) As technology advances, telemedicine processes need to adapt accordingly, and (3) Telemedicine increases collaboration among health care providers and patients/families in the patient care process.

Conclusions:

To improve patient care through increased use of telemedicine for pediatric emergency consultations, processes need to be modified to address provider biases and end-user concerns. Processes should be adapted to allow users to utilize a variety of technologies (including smartphones) and to enable more users, such as nurses, to participate. Finally, telemedicine can be used to improve the patient and family experience by including them in consultations.

BACKGROUND

Few emergency departments are fully equipped to manage pediatric emergency conditions.¹ Pediatric emergency telemedicine consultations can address this issue by providing on-demand, synchronous videoconferencing between community emergency department clinicians and specialty hospital pediatricians.^2,3 Such consultations can offer education, assistance, and reassurance to community physicians, thus improving care and disposition decisions and preventing some unnecessary transfers.^4–6 With virtual face-to-face communication and a means for specialty providers to remotely assess the patient, telemedicine has advantages over telephone communication and can result in care that is of higher quality, more family-centered, and more accessible.^4,7–11

Telemedicine has been available at our center for pediatric emergency consultations since 2003.¹¹ Our 121 bed children’s hospital, a quaternary care center located in Northern California, is the referral center for many children across a 33-county region covering 65,000 square miles and serving approximately 6 million people.¹² The children’s hospital receives transfers from over 130 emergency departments and hospitals in the region and accepted over 2500 patients as transfers in 2017. In order to begin the transfer process, or request a telemedicine consultation, referring emergency department providers contact the Transfer Center. The Transfer Center collects basic information about the patient, then determines the appropriate provider to complete the transfer/telemedicine consultation. While these consultations are primarily completed by pediatric critical care physicians or neonatal critical care physicians, consultations can be completed by any pediatric provider, including pediatric hospitalists and pediatric surgeons.

Despite its recognized value and availability, telemedicine is not universally employed during hospital to hospital transfers, suggesting that barriers exist that prevent its adoption and uptake.¹³ Prior studies have explored the acceptability of telemedicine; however, these studies have included mostly participants with little to no pediatric telemedicine experience, have explored telemedicine use for non-emergency settings, or have used survey data collection methods rather than in-depth interviews.^13–18 Identified barriers to telemedicine use from these prior studies include technology challenges, workflow integration, perceived usefulness, regulatory issues, and costs.^13–18 Participatory design methods ensure that systems are usable, acceptable, and effective. Unfortunately, such approaches are often not integrated into telemedicine interventions.¹⁹ The objective of this qualitative study was to understand facilitators and barriers to using telemedicine for pediatric emergency department transfer consultations from the perspectives of healthcare providers.

METHODS

Study Design

We conducted a qualitative study using in-depth interviews and grounded theory methodology.²⁰ Prior to our qualitative study, there was a period of three years where the use of telemedicine (as opposed to telephone) was strongly encouraged for transfer consultations from community emergency departments to our pediatric critical care unit. The physician who conducted the consultation was usually the physician that accepted the patient; occasionally, due to shift change, severity of illness, or bed availability, the patient would be accepted by another provider or another hospital. To inform development of the interview guide, we reviewed 139 free-text responses that were collected during the three-year period regarding patients who had a telephone consultation as opposed to a telemedicine consultation; telemedicine coordinators asked the pediatric critical care physicians why telemedicine was not used for these patients. Four researchers (J.R., M.G., H.S.F., G.W.) independently performed open-coding of the free-text responses, discussed the results, and formulated initial categories from the open-coding process. These categories were used to develop an initial interview guide with the following main topics: (1) experiences with using telemedicine for transfer consultations and (2) decision-making process regarding using telemedicine.

The initial interview guide was revised as data were analyzed and new categories of findings developed. Exact wording of the interview guide questions was adapted based on the provider type being interviewed; the version for the referring physicians is provided as Appendix 1. Specifically, based on preliminary analyses, interviewers (J.R., M.G., H.S.F., G.W.) probed more into two topics: (1) suggested changes to the telemedicine system or process and (2) specific patient types/diagnoses who should receive telemedicine. Interviews were conducted in-person or by phone, and were audio recorded and transcribed. Interviewers maintained field notes with contextual observations. Each participant provided verbal informed consent and received a $50 gift card. The University of California Davis Institutional Review Board approved the study.

Study Population

To identify the scope of the phenomenon and overall trajectory of the study, we initially used convenience sampling²¹ to identify referring physicians (physicians working in community emergency departments) and accepting physicians (pediatric critical care physicians) who had used telemedicine during the prior three year period in which telemedicine use was strongly encouraged. We subsequently used purposive sampling²² to identify referring emergency department nurses, transfer center nurses, and transport nurses in order to further explore topics that arose in the initial interviews. Sampling continued until thematic saturation was reached. Interviews were conducted in August and September 2018. Participants were identified through suggestions from accepting providers and community hospital site leads and were recruited via e-mail. Eligible participants were aged 18 years and older and English-speaking. Eligible participants had experience using or coordinating telemedicine for transfer consultations for pediatric patients. Demographic information, including gender, age, occupation, years’ experience, and frequency of telemedicine use, was collected during interviews.

Analysis

Data were analyzed in an iterative process; analysis used a constant comparative approach.^20,23 The process included the following steps: (1) Each investigator open-coded the first 4 interviews; (2) Full group met to discuss findings, distill open coding results into categories, and generate a codebook; (3) Adapted the interview guide based on the initial codes; (4) Individual memo-writing and focused coding of next 4 interviews using categories; (5) Full group met to compare codes, discuss discrepancies to ensure consensus on application of codes, refine dimensions of existing codes, add new codes, develop tentative categories, and identify theoretical direction. The process was repeated twice for each following group of four transcripts and ended when the full group agreed thematic saturation was reached.

Interviews were conducted until theoretical saturation was reached; at this point the categories were fully developed and demonstrated conceptual coherence. The team then reviewed the final coded data to identify major themes and reviewed relationships between themes to develop a conceptual model. We performed member checking by soliciting feedback from the interviewed participants on the preliminary conceptual model and themes.²⁴ Additional data validation occurred through investigator triangulation.²⁵ The research team consisted of two inpatient pediatricians with some telemedicine experience (J.R., M.H.), a sociologist (M.G.), a qualitative research analyst (G.W.), and a clinical research associate (H.S.F.). Two investigators (J.R., M.G.) had extensive qualitative research experience. An additional inpatient provider with telemedicine and qualitative research experience (L.K.) and the director of the pediatric telemedicine program (J.M.) were brought in to review the data and assist with the interpretation. We used ATLAS.ti to organize and store coding and data analysis.²⁶

RESULTS

We conducted sixteen approximately 60-minute interviews with referring physicians working in community hospital emergency departments (n=5), accepting pediatric critical care physicians (n=5), referring emergency department nurses (n=2), transfer center nurses (n=2), and transport nurses (n=2). One referring provider declined to participate but suggested their colleague, who participated; no other providers declined. Characteristics of participants are provided in Table 1. We identified three overarching analytic themes and six main categories across the transcripts that pertained to use of telemedicine for transfer consultations. Major themes included: 1) Acceptability-Recognizing and addressing telemedicine biases are central to gaining buy-in, 2) Usability-As technology advances, telemedicine processes need to adapt accordingly, and 3) Telemedicine increases collaboration among health care providers and patients/families in the patient care process. Main categories included: overlooking the benefits of telemedicine, conditionally buying-in to telemedicine, technology barriers extending beyond the equipment, expanding roles and clarifying processes, communicating with more collaboration, and strengthening the involvement of the consultant. These are explored in more detail below with representative quotes in Tables 2–4. Respondent validation demonstrated that participants agreed with the conceptual model (Figure 1) and description of the themes.

Table 1:

Participant Demographics

	N (%)
Gender
Male	5 (31)
Female	11 (69)
Age, in years
35-44	10 (63)
45-55	6 (38)
Role
Referring emergency department physicians	5 (31)
Accepting pediatric critical care physicians	5 (31)
Referring nurse	2 (13)
Transport nurse	2 (13)
Transfer center nurse	2 (13)
Years in profession
<5	3 (19)
6-10	6 (38)
11-20	5 (31)
21+	2 (13)
Number of times telemedicine used
5-10	3 (19)
11-20	5 (31)
21-30	5 (31)
31+	3 (18)

Table 2:

Theme 1: Acceptability-Recognizing and addressing telemedicine biases are central to gaining buy-in

Category	Subcategory	Representative Quotations	Participant
Overlooking the benefits of telemedicine	Resistance to using telemedicine	Our physicians, the PICU docs would offer it… ‘Do you mind if I log onto the system and look at the kid?’ And sometimes they would be willing and sometimes they would not. But a lot of those times the physicians just wanted to get our PICU doc on the phone.	Transfer center nurse
		I’d ask, ‘Would you like me to telemedicine?’ And – because I wanted to, actually, and the other side was like, ‘No, we’re running the code, and, we’re you know doing things.’ Or, ‘I can’t do that right now.’	Accepting physician
		People get frustrated and they don’t, like, it’s a sigh, we go, ‘Oh, God, I’ve got to get the telemed.’ Because it’s always like, is it going to work this time or is it not? So, if it worked better, it’d be one of those things we wouldn’t be so dreading when it comes up… I think if it was a little more streamlined, in that sense, it would be amazing all the time.	Referring nurse
	Negative telemedicine perceptions that might not be based on reality	I think the level of care we provide is not different [with telemedicine]…It helps us give more precise recommendations; it’s always good to have more information. So, I think, in that regard, it’s better [than telephone].	Accepting physician
		I think that it actually doesn’t take as much time as what we think it does and that even though we think it’s a barrier the time and busyness even though we think it’s a barrier to using it, I’m not so sure that it actually takes that much more time.	Accepting physician
	Benefits of telemedicine	I’ve been in those situations and said, ‘Gosh, I’d feel better about all this if I could take a peek at this kid before he came.’ I do think that I’ve come to appreciate some of the benefits of it… I think it makes for safer transport and for less anxiety on my end because I’ve got to see the kid.	Accepting physician
		Before we had telemedicine, you were kind of making guesses about how sick the patient is and you learn a lot by seeing them. So, not having the opportunity to see them, I felt like that would put me at a disadvantage sometimes, which meant the child was at a disadvantage.	Accepting physician
		I actually think it speeds up our disposition and the patient getting to the higher level of care, faster. So, while it might sometimes seem to take a few minutes more, I actually think the overall decision about disposition, definitive treatment, occurs faster by taking the possible few extra minutes to initiate the telemedicine.	Referring physician
		For parental benefit, and that feeling of communication and understanding what’s going on for parent’s sake…to encourage its use, even though maybe physician to physician you don’t feel like you need it. But to encourage its use for families.	Referring physician
Conditionally buying-in to telemedicine	Telemedicine felt to be useful for certain circumstances	You may get a call about a patient who maybe you’re really worried about and you really want to see what they look like so you can offer more directed intervention. Telemedicine is great. On the other hand, sometimes, you have a patient that you’re hearing about, but they don’t actually sound too sick for an ICU. And then you see them. It’s like, hmm, I really don’t think they are that sick and therefore, I can divert that patient to another level of care. For either patient because it helps you treat that patient more efficiently and effectively.	Accepting physician
		More a complex, complicated code, like an extremely premature newborn, I would use telemedicine… The more severe the more likely I am to use telemedicine.	Referring physician
	Competing demands and time constraints / sense of urgency limited telemedicine acceptability	In a situation where they feel like they have to intubate, there’s been a drowning, there’s something kind of urgent, they don’t want to take the time to do it. They want to speak with the physician, they want to get the kid here. They want to get the kid out of ER.	Transfer center nurse
		Telemedicine, it actually doesn’t take that long, but it feels like a time constraint, and it’s actually something that you can’t multitask during. You have to be, you know, in one place, and you can’t be talking to other people and stuff. And it’s a period of time where people also feel like they can’t interrupt you. So, it definitely feels like you kind of have to set everything aside and do this thing for a few minutes. And especially overnight or when things are very busy in the afternoon in the ICU, it definitely feels like a huge thing – distraction if it needs to happen. And it makes it less likely to wanna do it.	Accepting physician
		When we’re busy here doing-you know, telemedicine consults, they’re not particularly quick. They don’t feel like they’re quick… You have to find the computer, and you’ve gotta log in and get all set up… I think balancing those with what we felt were our immediate responsibilities here was a little bit kind of annoying for lack of better words.	Accepting physician

Table 4:

Theme 3: Effectiveness-Telemedicine increases collaboration among health care providers and patients/families in the patient care process

Category	Subcategory	Representative Quote	Participant
Communicating with more collaboration	Integrating the family into the conversation	I do think that, no matter what, it benefits – I presume it benefits the families… It’s helpful for them to be like, ‘Hi, I’m Dr. X. I’m helping with the transport. I’m gonna be here when you get here. If you any questions,’ etcetera, etcetera. Because they get that sort of bridge of contact.	Accepting physician
		I would want to fly with my kid. That would be hard, to let my kid go. So, it would be nice to see a face and go, ‘Okay, this is who’s going to be there.’ Or somebody telling you what’s to be expected. It gives them an opportunity to ask questions of that hospital, too.	Referring nurse
	Bringing the bedside nurse into the conversation	It’s nice to get everybody’s view of what’s going on with that patient including very much the nursing view… when we have these interfacility transports, it’s often a doctor-to-doctor conversation to get the consult going… Telemedicine is nice because you can often get the nurse’s perspective of what’s going on with the patient as well as the doctor’s. And it can give you a fuller picture and then the nurses can talk to the receiving nurses. So, I think being able to talk to the referring, you see the referring doc and the referring nurse and sometimes, even the referring respiratory therapist is very helpful.	Accepting physician
Strengthening the involvement of the consultant	Involvement in the clinical management	It helped us with the resuscitation and it helped us with the ventilator settings and management. They were able to see how the baby was doing and really provided good guidance.	Referring physician
		Our physician was on telemedicine prior to us departing, then went off telemedicine and came back on when we got there. Our physician and the physician there both worked together in making the right decision to inform mom what potential outcomes could be with the baby and also survival of transport. We withdrew support there with the physician on telemedicine here, and then with the physician up there both working together, and it ended up being very nice to have our physician there to help facilitate that.	Transport nurse
		[The tele-intensivist was] able to, like, pull a neurologist into the room, and we were having a conversation all together. And, like, you know, we’re all flipping through books and our resources to try and come up with, like, the best game plan for this patient and get as rapid a transfer as possible and get them stabilized, and it just, it ended up working out really, really well. And that’s the way that I think that should work, you know?	Referring physician
	Emotional support to the bedside providers	Having the telemedicine person be there, physically there, the family could see that the specialist was involved and there was lots of discussion and anticipatory guidance about brain death and what might be coming down the line. And they were there, they kind of took control and said, ‘If this child stops having a beating heart again, then we’re not going to keep going because there’s, you know, evidence he’s got brain death, but we’ll still take care of him.’… Feeling like they’re saying, ‘You’re doing what you can. You haven’t missed a step. This is happening.’ That was the main thing.	Referring physician
		I think with telemedicine, the difference is feeling not that Lone Ranger. Yes, physically, you’re the only one there, and oftentimes, the deficit is not physically being able to do something. It’s having the knowledge or expertise to know what to do when something isn’t going as expected or there’s a deviation from, there’s a unique situation and you’re not quite sure how to handle it for a pediatric patient. And so, having the support, the knowledge support, makes it feel less intimidating.	Referring physician
	Communicating and building relationships with the patient’s family	The mom was right there at the bedside and saw Dr. X on telemedicine up close and had a lengthy conversation of what our options were. We could take the baby back, but may not survive transport. The baby’s very sick. Or we can withdraw. So giving her all that information with the physician up front and personal was extremely beneficial.	Transport nurse
		It’s always good – is that the family, and I was able to introduce myself. And tell them what I was worried about, get some history, and kind of give them a heads up about what was to come. So, that they weren’t so surprised when they got here. And then I was able to call them on the phone while they were driving down ‘cause the kid flew, but they couldn’t fly. And they could attach my voice to my face.	Accepting physician
		All of a sudden, the mother started demanding a transfer, so that she could speak with a specialist. Yeah, it was ugly. It was really sad. So, I had the idea, well, why don’t we put her in touch with the neonatologist via telemedicine? We did that, and it was great. And she heard the very, very difficult and painful things she needed to hear which was ‘No, your baby is not going to survive,’ a difficult conversation for her, but we were able to get that done via telemedicine.	Referring physician
		And I think that’s been one of the biggest benefits is they have a connection to a physician and potentially to a nurse who will be caring for their child when they come to this facility.	Accepting physician

Figure 1:

Conceptual Model Describing Themes and Categories Acceptability influences usability, which influences effectiveness (represented by the downward arrows). Additionally, usability and effectiveness also feed back to influence acceptability (represented by the upward arrows).

Theme 1: Acceptability--Recognizing and addressing telemedicine biases are central to gaining buy-in (Table 2)

Category 1: Overlooking the benefits of telemedicine

Almost every participant reported biases against telemedicine despite also acknowledging numerous benefits of using this technology for consultations; those with the strongest or most extreme negative perceptions were all accepting physicians. Multiple accepting and referring physicians began the interview discussing their resistance to using telemedicine, stating that it took too long, that it did not change their care plans, or that the technology was frustrating. However, as the interviews proceeded, many providers acknowledged that their negative perceptions may not be based in reality. One accepting provider specifically acknowledged that they perceived telemedicine to take a long time, but that it did not take much more time than a phone call. When asked about specific cases or instances where they had used telemedicine, interviewees acknowledged the benefits of telemedicine they had experienced. Benefits included allowing accepting providers to provide more tailored recommendations after visually assessing the patient, expediting patient disposition, improving family experience, increasing knowledge and confidence for referring physicians, and increasing comfort for accepting physicians.

Category 2: Conditionally buying-in to telemedicine

Almost every participant stated telemedicine was only or mostly useful for certain clinical circumstances. Participants shared diagnoses or illness severities that did or did not benefit from telemedicine consultations. However, there was no consensus among participants for when telemedicine should be used. For some, including all referring physicians, the most beneficial circumstance for using telemedicine was for severely ill patients. However, many accepting physicians and one referring physician believed telemedicine was also useful for patients who were stable or whose presentation was unclear; for these patients, telemedicine was useful in preventing some transfers or avoiding unnecessary intensive care utilization. Some participants stated telemedicine was most beneficial for patients with respiratory illnesses. Multiple participants used the example of diabetic ketoacidosis as a diagnosis for which telemedicine was particularly not useful. However, one accepting physician shared a story that involved benefits to family satisfaction when telemedicine was used in the care of a child with diabetic ketoacidosis.

Time limitations and competing demands were other components of providers’ conditional telemedicine buy-in. Many participants stated that providers wanted to use telemedicine, and appreciated this technology, when there was adequate time to use it. However, when there was limited time and competing demands, telemedicine was perceived to be less acceptable. Also, for very sick patients at referring hospitals, telemedicine can be seen as a barrier to efficient transport.

Theme 2: Usability--As technology advances, telemedicine processes need to adapt accordingly (Table 3)

Table 3:

Theme 2: Usability-As technology advances, telemedicine processes need to adapt accordingly

Category	Subcategory	Representative Quote	Participant
Technology barriers extending beyond the equipment	Technology interface limiting mobility or availability	In order to do telemedicine, you have to be near a telemedicine machine. Because we don’t have it on our laptops or desktops here in our offices. So, in the mornings, there’s a backup person that takes the transport calls so that the people on service can round and do everything they need to get done.	Accepting physician
		I would try to improve the hardware that is utilized so it is handheld or goes into the room on a tripod… whether it’s on an iPhone or a tablet or an iPad so that I can, even if I’m not in that patient’s room, I can coordinate with the telemedicine physician back at my workstation or have an ongoing discussion. Because as of now, you have to be in the room with the patient, with the telemedicine equipment available.	Referring physician
		[T]o be able to see X-ray images on the telehealth side. So, finding a way to actually be able to access those without, like, pulling the robot next to my PACS, for instance. And again, that’s where that communication tool [secure messaging tool via smartphone app] really helps, because I can take a quick photo and send it over.	Referring physician
	Technology issues occurring and not being adequately solved	I just think the technology is difficult and it doesn’t always work. I mean, it seems like 50 percent of the time, they can’t get the machines on. That’s the frustrating part for me… We don’t know whether they’ve got the machine on. Is it them, the referring hospital, or is it our own docs that are not doing it right. Maybe they’re doing everything right and it’s just not working.	Transfer center nurse
		I still, to this day, feel like it’s very confusing… it’s always like this waiting game and you sit in the room and you’re waiting, you’re waiting… we’ll run and get the telemed and then we wait 45 minutes to get them on there… I think it’s kind of on both ends, we need to have a better process on our end to know how to get it in the room quickly and be ready.	Referring nurse
	Wanting or expecting the technology to advance	I think it’s about the ease of the process and how much equipment can make a big difference… telemedicine just seems so cumbersome from a physical process standpoint of connecting and all that. In today’s age, it just seems silly with cell phones and iPads and all that to then have this big cart and it looks like it’s from the 1990s sort of a thing… the cumbersomeness of the actually connecting and getting it to work. That’s kind of the down side of it.	Referring physician
		The integration of [telemedicine and interpreter services], I think, will, at some point in time, hopefully happen. And where there will be a translator that can pop onto the robot itself and help provide those services. The way it works now is not as seamless, and at times, depending on what’s going on and how busy it is in the room, it can end up being fairly difficult, I think, and can impact the overall efficiency of the process.	Referring physician
	End-users not knowing how to use the telemedicine equipment	The experienced nurses [who know how to use the telemedicine equipment] also tend to be our best nurses in pediatric situations… So, if there’s competing demands for skills, we’re obviously going to commit them to the physical care… we have a clerk that can always make phone calls, they’re just not well versed on how to get the machine set up.	Referring physician
		The ERs often have their cart laid out in a hallway or something, and then you’re supposed to be able to [remotely] drag the cart to the room. But, I don’t actually – I’ve never been able to do that well. So, then you have to wait for somebody to walk by and say, ‘Do you mind taking me to the patient’s room?’… I wish that there was a way to just, like, say, ‘Let’s do telemedicine,’ and then you turn it on, and you’re there. But, it’s never like that.	Accepting physician
	Workaround using smartphones	They would take their phone and scroll through the CT and then send a video of that to Dr. X so they can get an idea of what they’re looking at. They’ve done it with X-rays and things like that… It’s just regular, the texting a message app on their phone, and they will actually go to the computer and scroll through it. Now, with videotape as they’re scrolling through, the doctor can get a picture of it.	Transfer center nurse
		In the helicopter, you can text. I text very often coming back in the helicopter with the physicians.	Transport nurse
	Vulnerability and impression management	When you don’t do one for a little while, and you’re sort of fumbling. And you feel a little bit silly fumbling when someone is watching you do it… Sometimes when I’m on the phone talking to outside docs, I will look things up on the computer system because sometimes they call us with things that are more rare. Or, I need to look up a dose or something like that. So, I’ll be multitasking. I’ll be talking to them, and I’ll quickly look something up. You can’t really do that when they’re watching you on a camera.	Accepting physician
Expanding roles and clarifying processes	Physicians as gatekeepers to using telemedicine	Sometimes, I feel like there have been cases where we felt, as nurses, that the transfer was unnecessary. And we may have voiced that to the intern or one of the residents to a certain extent, but it kind of seems to be a pretty hierarchical, top down thing to where once they’ve kind of decided that that’s what they’re going to do we just comply with the plan… perhaps we might be a little bit more assertive in suggesting telemedicine be utilized by the team of physicians.	Referring nurse
		It’s not usually prompted by the nurses, generally. It’s prompted by the doctors. ‘I want, you know, telemed in this room.’… Most of the time, it is the doctor that preempts it. But I’ve never been the one to initiate it, that I could think of off the top of my head. But I’m sure a nurse could.	Referring nurse
	Deciding when to use telemedicine	We’ll get a call like, oh, the baby’s seizing, hurry up and get there. And sometimes it takes us at least an hour to get there – and we get there and the baby’s been seizing the whole time. Whereas, if they’d done telemedicine and actually looked at the baby, I think we could have intervened, you know, an hour ahead of time instead of waiting for us to get there.	Transport nurse
		There’s not a direct relationship between how sick the kid and their likelihood to initiate the telemedicine call. I think it’s so much provider preference, and probably the culture at their institution.	Accepting physician
		And each attending seems to kind of have their own way of doing things and also their own institutional knowledge, or lack thereof, about how things are done at Hospital X and the resources that are available… [telemedicine use is] generated by the attending and the residents and whatever that team decides that they want to do on that particular case, on that particular day.	Referring nurse
		I just feel like there’s a lot of resistance… I think if there was some sort of algorithm that made, or really encouraged the doctors to implement telemedicine, I think that would be helpful. Because they just don’t think about it on that side.	Transport nurse
	Who is present on the telemedicine consultation
		Sometimes, I’ve seen patients where they move the telemedicine unit into the patient’s room and the patient is kind of there by themselves. That’s a little less helpful. From a process point of view, it’s really nice to have somebody else in that room.	Accepting physician
		To be totally honest – I would say half the time the doctor is not there [on the telemedicine consultation].	Accepting physician
	Telemedicine process duplicating work	The other issue with the telemedicine consult that’s different than our regular phone call or phone calls is a bridge call, which allows the transport team, [charge nurse, accepting physician, referring provider]… all those groups, to hear the same story… When we use telemedicine, we don’t have that… And that’s very frustrating… adds not only inefficiency, but potential errors in transmitting the story.	Accepting physician
		We’re calling and we’re saying, ‘Hey, we’ve got a critically ill kiddo. My doc needs to talk to the tele-intensivist, pediatric intensivist.’ Then I shouldn’t also have to have a long conversation with [the transfer center nurse]… I think any time that we have to have additional communication back and forth by phone in the emergency department, it draws us away from patient care and it’s extremely difficult for us.	Referring physician

Category 3: Technology barriers extending beyond the equipment

Every participant, even those who had strong buy-in, mentioned equipment issues. Referring emergency departments often had only a few telemedicine “experts” who knew how to find the cart and turn it on, but these were often nurses who had more pediatric experience, so they were wanted at the bedside instead of troubleshooting telemedicine equipment. Equipment issues were not solely related to the hardware, but also related to the inability to integrate interpreter services and the limitations with where the telemedicine equipment was located. Some participants stated that smartphones were a ubiquitous, more user-friendly technology that providers often used to communicate during transfer consultations. Providers used smartphones as a workaround to transmit images and videos instead of using the formal telemedicine equipment.

Technology barriers also included issues related to impression management between accepting and referring physicians. Many accepting physicians thought that they had to respond immediately to telemedicine consultation requests. Some accepting physicians thought telemedicine did not permit the flexibility to multi-task and simultaneously address competing demands. Accepting physicians shared they could not reference online resources during telemedicine consultations; whereas, telephone communication permitted that without compromising the impression of being the knowledgeable expert. Although telemedicine left some accepting physicians navigating feelings of vulnerability and anxiety, telemedicine gave some referring physicians and nurses a way to demonstrate their skillset. With telemedicine, accepting physicians could watch referring providers deliver care at the bedside, which fostered trust and built relationships.

Category 4: Expanding roles and clarifying processes

Most physicians and nurses described a lack of clarity about who could use telemedicine and when it would be most appropriate to use. While nurses explained that they sometimes wanted to activate telemedicine, most believed that physicians were the only ones who could activate telemedicine. There was also a disconnect with physician expectations; accepting physicians thought that the role of the referring provider during telemedicine consultations was to be physically at the patient’s bedside, but that frequently did not happen—instead, the nurse and family participated in the consult.

Telemedicine also duplicated processes. Transfer center nurses, transport nurses, and physicians expressed frustrations that information was first communicated by telephone and then repeated via telemedicine. Because transport nurses and transfer center nurses were not on telemedicine consultations, but otherwise would be on a bridged telephone line, telemedicine consultations were followed with another phone call to loop the nurses into the conversation. This process duplicated work and left the nurses receiving secondhand information. All transport and transfer center nurses wanted to expand their roles to be included on telemedicine consultations.

Theme 3: Effectiveness--Telemedicine increases collaboration among health care providers and patients/families in the patient care process (Table 4)

Category 5: Communicating with more collaboration

Telemedicine brought more individuals into the transfer communication process. It allowed the family to virtually meet the accepting physician before the transport, which brought comfort and reassurance to the family. Nurses and physicians also described how telemedicine brought the referring hospital’s bedside nurse into the conversation with the accepting physician, something that did not occur with telephone communication. Telemedicine allowed the nurse to provide and receive additional information.

Category 6: Strengthening the involvement of the consultant

Telemedicine expanded the role of the accepting physician during the transfer consultation, resulting in increased involvement in clinical management and perceived better recommendations. Referring physicians and nurses explained how telemedicine enabled accepting physicians to provide them with deeper levels of support, including reassurance and emotional support. Telemedicine increased involvement of the accepting physician in communicating with the patient’s family. First, accepting physicians virtually saw and talked directly with the family; they were then more likely to engage further with the family, such as calling with updates until the family arrived at the receiving hospital. This improved family communication was reported by almost every participant to be one of the most beneficial aspects of telemedicine. Participants shared that even if telemedicine did not always alter the clinical management of a patient, participants perceived that telemedicine almost always enhanced the family’s experience.

DISCUSSION

This study gathered perspectives from telemedicine users to understand facilitators and barriers to using telemedicine for pediatric emergency transfer consultations. All participants had experienced pediatric emergency telemedicine consultations at least five times, and 18% had experienced more than 30 telemedicine consultations. Informed by our respondents, we constructed a conceptual model that illustrates influencers of the acceptability, usability, and effectiveness of telemedicine for transfer consultations (Figure 1) pertaining to the three study themes. Acceptability influences usability, which influences effectiveness. Additionally, usability and effectiveness also feed back to influence acceptability. We theorize that (1) interventions are needed to address telemedicine biases, (2) telemedicine processes need to adapt to address technology-related needs, and (3) the end-users need to be informed of the broad effectiveness of telemedicine – particularly the increased collaboration between providers and patients/families. Our data support the belief that these strategies will collectively assist with gaining telemedicine buy-in for pediatric emergency transfer consultations, which will ultimately increase the effectiveness of telemedicine.

Many of the barriers to telemedicine acceptability identified in this study are consistent with previously reported findings. These barriers include technology challenges, poor workflow integration, and uncertainty of the benefits.^13–18 To our knowledge, classifying certain barriers as negative biases towards telemedicine is unique to our study. We use the term “bias” because our stakeholders, especially physicians, had inclinations or prejudices against telemedicine that contradicted their own or others’ statements. We must address these biases in order to gain end-user buy-in. Although a prior mixed-methods pediatric telemedicine study highlighted the need to gain buy-in among community clinicians,¹⁴ another study emphasized the need to gain buy-in from both referring and accepting physicians.¹³ Our study identified the greatest resistance among accepting physicians; we thus propose efforts to address buy-in should target this group. As depicted by the arrows in the conceptual model (Figure 1), improving usability and effectiveness perceptions are strategies to improve acceptability.

Telemedicine technology challenges are commonly reported in the literature, but prior studies largely report technology learning barriers and technically-challenged individuals.^13,27,28 As such, existing strategies to address technology issues focus on educational programs, trainings, and frequent test calls.^13,28 Our study identified that technology barriers extended beyond the equipment. To address these broader technology issues and improve telemedicine usability perceptions, we need to adapt our local telemedicine processes to better address end user concerns.

First, many telemedicine platforms permit multiple users to videoconference simultaneously. Adapting our processes to include other individuals (e.g., professional interpreters, transport nurses, transfer center nurses) on the consultation is one way to address the reported usability problems related to integrating nurses and interpreter services in the conversation. Second, telemedicine consultations can be performed on various types of equipment; our current processes use designated telemedicine carts and desktops. Adapting our processes to encourage providers to use the equipment type of their choice (e.g. smartphone, laptop) would likely improve usability, as well as efficiency. Leveraging new technology would allow for busy emergency department providers to participate in the consult “on the fly”, instead of waiting for a telemedicine cart to power on and connect. Lastly, future research is needed to understand and address the negative impressions identified. Possibly, processes can be implemented to minimize the perceived pressures on accepting physicians to respond immediately to telemedicine consultation requests and to mitigate their concerns during telemedicine consultations. Tackling the broad technology challenges might improve workflow integration and time constraint issues reported in our study and in prior telemedicine literature.^13,16

Regarding the effectiveness of telemedicine, recognized benefits include improving care quality, enhancing family and physician experience, reducing transfers, decreasing costs, and providing educational value to clinicians.^4,7,10,11,29 Despite this body of literature, our study and prior studies suggest that uncertainty regarding telemedicine’s benefits continues to impede its adoption.^13,30 Ray et al¹⁴ proposed developing guidelines for which specific clinical conditions are appropriate for telemedicine use. However, our respondents did not recognize any clear or consistent clinical conditions for which telemedicine should be used. We did find, though, that telemedicine consistently improved collaborative communication and providers believed that telemedicine provided benefits for the family by bringing families into the conversation.

Current pediatric emergency transfer consultations involve telephone communication primarily between the physicians. Telemedicine transforms the communication framework by bringing the patient, family, and bedside nurse into the conversation; this may address the issue that families are often uninformed and uninvolved during transfers.³¹ We propose encouraging the use of telemedicine for all pediatric emergency transfer consultations by promoting its effectiveness in increasing collaboration and integrating more individuals in the process. Prior telemedicine studies suggest that telemedicine use for transfer consultations can improve family satisfaction. Our respondents indicated that “meeting” patients and their families before transfer helped to provide a more family-centered handoff.^11,32 Family-centered rounds at the bedside with multidisciplinary teams have become standard of care at many institutions given the perceived benefits of increased family involvement and understanding and effective team communication.³³ We believe that telemedicine has potential to transform the communication paradigm for transfers, similar to how family-centered rounds has transformed the paradigm for rounds; further research is needed in this area.

Our study had several limitations. The findings represent only the perceptions from this group of participants. However, we interviewed a variety of physicians and nurses who had used pediatric telemedicine frequently and reached thematic saturation after the 16 participants. Although participants share their perceptions of families’ experience and family-centeredness of care, the participants in our study did not include family members. Participants could have recall bias. Participants who agreed to participate could also have polarized or extreme perceptions. The findings were specific to our institution and may reflect specific telemedicine workflows at our hospital. However, many of the findings are likely transferable to other pediatric emergency telemedicine programs.

Physicians and nurses in our study provided valuable insights on their experiences with and perceptions of telemedicine. Addressing negative biases to gain buy-in, adapting processes to meet technology-related needs, and informing end-users of the broad effectiveness of telemedicine are potential strategies that may be incorporated into future interventions to enhance telemedicine use for pediatric emergency transfer consultations.