Using mobile devices for inpatient rounding and handoffs: an innovative application developed and rapidly adopted by clinicians in a pediatric hospital

Abstract

Objective: To describe the usage of a novel application (The FLOW) that allows mobile devices to be used for rounding and handoffs.

Materials and Methods: The FLOW provides a view of patient data and the capacity to enter short notes via personal mobile devices. It was deployed using a “bring-your-own-device” model in 4 pilot units. Social network analysis (SNA) was applied to audit trails in order to visualize usage patterns. A questionnaire was used to describe user experience.

Results: Overall, 253 health professionals used The FLOW with their personal mobile devices from October 2013 to March 2015. In pediatric and neonatal intensive care units (ICUs), a median of 26–26.5 notes were entered per user per day. Visual network representation of app entries showed that usage patterns were different between the ICUs. In 127 questionnaires (50%), respondents reported using The FLOW most often to enter notes and for handoffs. The FLOW was perceived as having improved patient care by 57% of respondents, compared to usual care. Most respondents (86%) wished to continue using The FLOW.

Discussion: This study shows how a handoff and rounding tool was quickly adopted in pediatric and neonatal ICUs in a hospital setting where patient charts were still paper-based. Originally developed as a tool to support informal documentation using smartphones, it was adapted to local practices and expanded to print sign-out documents and import notes within the medicolegal record with desktop computers. Interestingly, even if not supported by the nursing administrative authorities, the level of use for data entry among nurses and doctors was similar in all units, indicating close collaboration in documentation practices in these ICUs.

INTRODUCTION

Handheld mobile devices have been found in the pockets of clinicians for over 2 decades.¹ From personal digital assistants, used mainly for referencing and calculation purposes, to smartphones, handheld devices are now used by many doctors for a variety of professional tasks.² Recent surveys conducted in France,³ Ireland,⁴ the United States,⁵ the United Kingdom,⁶ and Canada⁷ have documented that approximately 75–95% of medical students and residents are using their personal smartphones for clinical work, usually to utilize applications (apps) or communicate with the care team via email, text messages, or phone calls. The quality of these apps is seldom verified,⁸ and the use of less secure networks to communicate confidential information with the care team is a concern.³ In fact, various problems have surfaced with the use of personal smartphones by clinicians, including confidentiality and security issues, as well as distractions and interruptions while performing clinical tasks.^5,6,8–11 Moreover, the diversity of activities that are possible with smartphones, including personal communication with emails and text messages, online shopping, and browsing social media websites such as Facebook, may impede their use as a clinical device.^12,13

Despite these potential shortcomings, the multifunctionality and mobility of smartphones makes them highly attractive for team-based work in hospitals, especially given the high level of coordination needed among clinicians to ensure continuity of care.^13,14 Efforts are being made worldwide to identify the best ways to support clinicians with appropriate tools during rounds and at care transitions (handoffs).^15–18 Development of handoff and rounding tools is also stimulated by a study conducted by Starmer and colleagues in 9 pediatric teaching hospitals in the United States and Canada, demonstrating a reduction in medical errors by 23% and preventable adverse drug events by 30% after implementation of a handoff program that included a standardized document.¹⁹ Information technologies can be used to extract, structure, and synthesize clinical data.^16,20,21 There is a great diversity of handoff tools developed within commercial electronic medical records (EMRs),^22,23 as stand-alone applications,²⁴ or simply as electronic templates (in Word, Excel, or Access). To date, mobile technologies have been used mainly to support coordination of care through text messaging and email among care team members in an informal way,²⁵ but their use in structured rounding and handoff processes is limited.^{15–17,20,21,23,24,26}

OBJECTIVE

The objective of this study was to describe the usage patterns and user experiences of a novel application that allows mobile devices to be used for rounding and handoffs in an academic tertiary health care center.

MATERIALS AND METHODS

Context: Application design and implementation

This study took place at the McGill University Health Centre (MUHC) in Montreal, Canada, in 2015. During the study period, patient charts were paper-based, combined with an EMR allowing access to the hospital’s clinical databases and scanned paper documents. In 2010, the hospital built a web-based app (V-Sign) to allow all types of mobile devices (tablets, smartphones) to connect to the hospital’s information system in a “bring-your-own-device” (BYOD) model. This app could be installed on personal devices from any app store by hospital-based clinicians. No training was provided; diffusion happened mostly through word of mouth among users.

A new module, The FLOW, was added to the V-Sign app in 2013. It was developed by a team of 2 engineers (J.P.C. and J.P.), 1 programmer, and 1 clinician, in short iterative development cycles with input from physicians in 2 pilot units. It allows care team members to enter and share short free-text notes, hereafter referred to as flows, for admitted patients. Each flow can be up to 200 characters in length and is tagged by the user as relevant to 1 category (Figure 1). Categories were developed based on the identify, situation, background, assessment, and recommendation (ISBAR) mnemonic for structured sign-outs,^27,28 and adapted by unit according to clinicians’ needs. Flows themselves are not part of the EMR, but are informal documentation.

Figure 1.

Categories tagged by clinicians and associated visual item in application.

Implementation of The FLOW targeted 2 intensive care units (ICUs) (1 pediatric unit, PICU, and 1 neonatal unit, NICU) in October 2013 and May 2014, respectively, at 1 of the 5 hospital sites. In October 2014, it was also implemented at 3 medical units at the same site. In January 2015, the NICU at another hospital site spontaneously started to use the application. During the development phase, users asked for the app to be available on desktop computers to extract data into templates in order to print out for the medical record (paper chart). It was thus made available on all desktop computers in the pilot units. Downloads of the last entry per category in The FLOW were possible in 2 different templates: the list of patients as the handoff document and the daily progress note, which could be printed and then incorporated into the medical record. Figure 2 presents the app’s interface on a smartphone and the template of the daily note.

Figure 2.

Two interfaces of The Flow application. (A) The left menu includes access to the data in the EMR (labs, meds, vitals). The central screen is the list of patients in the unit. Clicking on The Flow module for a given patient will open the right screen, with flows listed in chronological order by category. (B) The last flow entered per category can be extracted and edited to print the daily progress note or the handoff document (not shown).

Data collection

A sample of usage patterns of The FLOW was obtained by retrieving audit trails of all entries in March 2015. User experience was assessed using an adaptation of the technology acceptance questionnaire that was administered online in April 2015.²⁹ All clinicians (i.e., physicians, nurses, other health care professionals) who had used the application were sent an invitation to complete the questionnaire.

Questionnaire development and administration

A bilingual (English and French) questionnaire was developed using tailored design survey methods.³⁰ Survey items were adapted from previously published surveys on mobile devices used in the ICU.^25,31 A paper version of the questionnaire was submitted to 5 FLOW users to assess clarity, comprehensiveness, and content validity. A researcher (A.M.) met individually with each user, and his or her comments were integrated into a second version that was then translated into French. This version was translated back into English using the back-translation method to ensure that the English and French versions were identical. The second version of the questionnaire, including both languages, was submitted electronically to 5 users and 5 health informatics researchers, and their comments were integrated into a final version. The final questionnaire contained 32 items (see Appendix). An online survey tool (EFS Survey from Unipark®) was used to distribute the survey, because it allowed the interface to adapt to mobile devices. The survey link was sent by personal email in April 2015 to all 253 user profiles created in The FLOW since its implementation, and follow-up reminders were sent 2 weeks and 4 weeks after the first invitation. A $25 gift certificate was provided as compensation for questionnaire completion.

User characteristics

User characteristics included hospital unit, role (i.e., attending physician/fellow, resident, medical student, or nurse/other), gender, and experience with The FLOW (described by the user’s duration and frequency of use). Users were categorized into 2 groups based on their reported duration of use: continued users who had used the application for 8 weeks or longer, and episodic users who had stopped using the application within 8 weeks. The experience of users was also described by summing the response to questionnaire items on the frequency of use of the various features of The FLOW to create a comprehensiveness score (9 items using a 5-point scale, maximum possible score of 45). Users were classified as comprehensive users if their total score was greater than 35 (meaning they used most features “often” or “very often”).

Data analysis

For each unit, usage patterns of The FLOW were analyzed by calculating the number of entries per user and per patient, as well as the number of distinct user roles entering flows for each patient. Differences in usage patterns by unit and user role were assessed using the Kruskal Wallis test (KWT). Social network analysis (SNA) was applied to audit trails from The FLOW, providing a visual representation of the frequency of data entry within the care team. Descriptive statistics were used to summarize responses to each questionnaire item. Differences in questionnaire responses by user characteristics were assessed using the Wilcoxon Rank Sum test for user characteristics with 2 categories and the KWT for user characteristics with more than 2 categories. To account for multiple testing, a more conservative p < .01 was considered statistically significant. Logistic regression was used to assess whether gender, unit, and role were associated with more comprehensive use of The FLOW. All statistical analyses were conducted using SAS software (version 9.4), and Gephi (version 0.8.2-beta) was used for visual representations. This study was approved by the MUHC ethics board.

RESULTS

Usage patterns

In March 2015, The FLOW was being actively used in 3 different ICUs, but all general medical units had stopped using the app. This decision was made in January 2015 because the design of the app was not optimized for creating patient lists in medical units. At the time, the patient list for a given unit in The FLOW was based on the physical location of the patient. For general medical units, this was problematic because patients assigned to 1 team were often located in different physical locations. Clinicians on these teams therefore had to manually add and delete each patient on their patient list, which was cumbersome. Since all general medical units were no longer using the app in March 2015, usage patterns of The FLOW were described for the 3 ICUs only.

The level of overall use of the app for data entry was similar among all the ICUs, with a median of 26–26.5 daily flows entered per user (Table 1). There was no significant difference in the number of daily flows entered by nurses, residents, medical students, and attending physicians in the PICU and NICU 1 units. However, in NICU 2, attending physicians entered significantly lower numbers (p < .01) of daily flows than other users (except other professionals who were excluded from the KWT) (Table 1).

Table 1.

Number of flows entered per day, by user role and unit

Roles	PICU		NICU 1		NICU 2
	N	Median [IQR]	N	Median [IQR]	N	Median [IQR]
Attending physicians and fellows	3	27 [13–37]	9	18 [10–43]	5	5 [2–13]a
Residents	14	25 [14–32]	13	26 [14–39]	5	40 [22–56]
Medical students	1	24 [16–27]	1	27 [21–32]	11	24 [17–34]
Nurses	2	29 [26–33]	9	31 [22–38]	7	37 [11–48]
Other professionalsb	0	–	3	1 [1–2]	0	–
All roles	20	26 [14–33]	35	26 [14–35]	28	26.5 [15–42]

^aLower than all other roles on NICU Site 2 (p < .01).

^bOther professionals were excluded from the Kruskal Wallis Test for each unit.

Visual representation of flows entered per unit revealed different network patterns of the app’s usage by unit (Figure 3). At NICU Site 1, the network was dense and all users were connected closely, indicating that users were using the app collaboratively for all patients. Four attending physicians (blue circles) at NICU Site 1 were central to the network, while nurses (yellow circles) and residents (pink circles) in this unit were also very active in entering notes in The FLOW. At NICU Site 2, where the app had been adopted only 2 months earlier, usage patterns were different. The 5 attending physicians in this unit were all outside of the network, while 1 resident (central purple circle) was driving usage of the app, surrounded by 2 groups of users: a group of 11 medical students (teal circles) on 1 side of the network and a group of 7 nurses on the other side, indicating that these groups were entering data on different groups of patients. In the PICU, 1 physician (central blue circle) was driving the app’s usage. While residents were also using the app to enter data, only 2 nurses and 1 medical student were using the app, resulting in a significantly lower number of different roles entering flows per patient in the PICU compared to the other 2 units (Table 1).

Figure 3.

Visual representation of data entry in March 2015 by unit.

Questionnaire

Respondents

Among 253 users who were sent the questionnaire, a total of 127 (50%) provided responses to 96% or more of the items and were included in the analysis. Most respondents were <30 years old (71%), female (60%), and physicians or physicians in training (35% residents, 33% medical students, and 17% attending physicians/fellows). Only 15% of respondents were nurses or other professionals (e.g., social workers). Respondents had used The FLOW in 4 clinical units: 45% in a pediatric medical unit, 24% in a PICU, 14% in the NICU at Hospital Site 1, and 17% in the NICU at Hospital Site 2.

Level of use of different features of The FLOW

Nearly three-quarters of respondents used The FLOW frequently (“often” or “very often”) to enter notes (72%), to print the electronically annotated list of patients for handoffs (73%), and to communicate during handoffs (72%) (Figure 4). A majority of users (70%) also reported using The FLOW frequently to read or review notes. By role, nurses and other professionals reported a statistically significantly lower use of The FLOW compared to other users, with a comprehensiveness score of 24.84 compared to the physicians’ score of 34.90 (Table 2). For all features except reading or reviewing notes, nurses and other professionals reported using The FLOW less frequently than other users, but this difference was statistically significant for only 2 features (entering notes during handoffs and printing the list of patients for handoffs). On the other hand, attending physicians and fellows reported using The FLOW more frequently than other users, with a mean response significantly higher for 2 features: reading or reviewing notes when on call (4.24) and reading or reviewing notes from home (3.76), reflecting their roles and responsibilities on the care team. By unit, The FLOW was used significantly more in NICU Site 2 than NICU Site 1 and medical units, with a comprehensiveness score of 35.71 vs 28.88 and 29.47, respectively. Regarding specific features, the mean response in NICU Site 2 was significantly higher than all other units for 2 items: entering notes (4.48) and exporting daily progress notes (4.43). In terms of user characteristics associated with more comprehensive use of The FLOW, male users were less likely to be comprehensive than female users (OR = 0.37, 95% CI, 0.16-0.87) (Table 3). When compared to residents, nurses and other professionals (OR = 0.22, 95% CI, 0.05-0.87) were less likely to be comprehensive users (Table 3).

Figure 4.

Response to selected items from the questionnaire on the frequency of use of different features of The FLOW (N = 127). Numbers are in percentage of respondents.

Table 2.

Users’ responses about how often they used The FLOW

How often do you use The FLOW to:	By role				By unit
	Attending physicians and fellows	Residents	Medical students	Nurses and other professionals	PICU	NICU Site 1	NICU Site 2	Medical Units
	(n = 21)	(n = 43)	(n = 41)	(n = 19)	(n = 29)	(n = 17)	(n = 21)	(n = 55)
Scale: 1 = never, 2 = rarely, 3 = sometimes, 4 = often, 5 = very often
	Mean (SD)				Mean (SD)
Enter notes	4.14	4.07	4.12	3.32	4.31	3.88	4.48a	3.73
	(1.35)	(1.24)	(1.05)	(1.49)	(1.28)	(0.78)	(0.98)	(1.35)
Enter notes when rounding	3.29	3.14	2.71	2.58	3.21	2.53	3.48	2.75
	(1.55)	(1.23)	(1.57)	(1.71)	(1.37)	(1.28)	(1.6)	(1.52)
Enter notes during handoffs	3.52	3.37	3.27	2.11b	3.38	2.65	3.38	3.16
	(1.4)	(1.22)	(1.57)	(1.24)	(1.52)	(1.41)	(1.47)	(1.37)
Print the list of patients for handoffs	4.14	4.35	4.10	2.11c	4.31	3.41	4.19	3.71
	(1.2)	(0.92)	(1.28)	(1.37)	(1.17)	(1.5)	(1.08)	(1.52)
Communicate about patients during handoffs	4.10	4.16	4.12	3.26	4.24	3.59	4.42	3.89
	(1.34)	(1.08)	(1.19)	(1.69)	(1.30)	(1.37)	(1.07)	(1.27)
Read/review notes	4.24	3.88	3.76	3.95	4.07	3.88	4.29	3.76
	(1.18)	(1.37)	(1.04)	(1.35)	(1.25)	(1.22)	(1.10)	(1.22)
Read/review notes when on call	4.24d	3.77	3.61	2.89	4.00	3.24	4.05	3.51
	(1.18)	(1.00)	(1.07)	(1.56)	(0.96)	(1.30)	(1.07)	(1.25)
Read/review from home	3.76e	2.74	2.49	2.37	3.00	2.53	3.00	2.71
	(1.30)	(1.11)	(1.33)	(1.12)	(1.51)	(1.23)	(1.26)	(1.18)
Export daily progress notes from The FLOW	3.48	3.09	3.17	2.26	3.66	3.18	4.43f	2.25
	(1.69)	(1.69)	(1.80)	(1.79)	(1.70)	(1.55)	(1.12)	(1.62)
Comprehensiveness score (sum of all items)	34.90	32.58	31.34	24.84g	34.17	28.88	35.71h	29.47
	(8.33)	(7.01)	(7.43)	(10.26)	(8.44)	(8.22)	(5.27)	(8.28)

^aClinicians at NICU Site 2 used The FLOW more frequently to enter notes than at Medical Units (p = .006) (difference not significant [NS] vs other units).

^bNurses and other professionals used The FLOW to enter notes during handoffs less frequently than attending physicians and fellows (p = .005), residents (p = .001), and medical students (p = .009).

^cNurses and other professionals used The FLOW less frequently to print the list of patients for handoffs than attending physicians and fellows (p < .0001), residents (p < .0001), and medical students (p < .001).

^dAttending physicians and fellows used The FLOW to read/review notes when on call more frequently than nurses and others (p = .007) (NS vs other groups).

^eAttending physicians and fellows used The FLOW to read/review notes from home more frequently than residents (p = .004), medical students (p = .002), and nurses and others (p = .003).

^fClinicians at NICU Site 2 used The FLOW more frequently to export daily progress notes than clinicians in the PICU (p = .003), NICU Site 1 (p = .002), and Medical Units (p < .0001).

^gUse of The FLOW was less comprehensive by nurses and other professionals than by attending physicians and fellows (p = .004) (NS vs other groups).

^hUse was more comprehensive at NICU Site 2 than NICU Site 1 (p = .008) and Medical Units (p = .004).

Table 3.

User characteristics associated with comprehensive^a use of all features of The FLOW

Variables	N	Odds ratio (95% CI)
		Unadjusted	Adjustedb
Gender
Female	74	1.00 (reference)	1.00 (reference)
Male	49	0.51 (0.24-1.09)	0.37 (0.16-0.87)
Role
Residents	43	1.00 (reference)	1.00 (reference)
Medical students	41	0.54 (0.23-1.31)	0.45 (0.17-1.17)
Physicians	21	1.70 (0.59-4.94)	1.20 (0.67-3.90)
Nurses and others	19	0.28 (0.08-0.98)	0.22 (0.05-0.87)
Unit
Medical Unit	55	1.00 (reference)	1.00 (reference)
PICU	29	2.33 (0.93-5.85)	1.80 (0.66-4.96)
NICU Site 1	17	0.58 (0.17-2.04)	0.36 (0.09-1.39)
NICU Site 2	21	3.08 (1.09-8.72)	2.60 (0.82-8.23)

^aA user was identified as comprehensive if he or she used most of the features often or very often, with a comprehensiveness score of 35 or more.

^b122 users had complete data on all variables and were included in the adjusted multivariable logistic regression model.

Medium of use

To read or review notes, most users preferred accessing The FLOW via computer (42% of respondents), followed by smartphone (34%) and tablet (24%). To enter notes, more than three-quarters (77%) of respondents preferred using a computer, 17% preferred using a smartphone, and only 6% preferred using a tablet.

Perceived usability and impact

On average, respondents were positive in their perceptions about the usability and impact of The FLOW (Table 4). Seventy-nine percent of respondents agreed (40%) or strongly agreed (39%) that The FLOW improved the legibility of progress notes (Figure 5). For nearly half of respondents (49%), progress notes were perceived as being more comprehensive using The FLOW. The accuracy of information transmitted during handoffs was considered improved by 51% of respondents, and 58% believed that handoffs were quicker when using The FLOW. Compared to usual care, The FLOW was perceived as having improved patient care and patient safety by 57% and 54% of respondents, respectively, and 79% believed that patient care would be improved if all care team members used the application. User responses about the impact of The FLOW did not differ significantly by role, unit (Table 4), or duration of user experience.

Table 4.

Users’ perceptions about the impacts of The FLOW

Survey item	Overall	By role				By unit
		Attending physicians and fellows	Residents	Medical students	Nurses and other professionals	PICU	NICU Site 1	NICU Site 2	Medical Units
	(N = 127)	(N = 21)	(N = 43)	(N = 41)	(N = 19)	(N = 29)	(N = 17)	(N = 21)	(N = 55)
Likert scale: 1 = strongly disagree; 5 = strongly agree
	Mean (SD)	Mean (SD)				Mean (SD)
Quality of information
Using The FLOW has improved the legibility of daily progress notes	4.16	4.14	4.12	4.22	4.11	4.28	4.18	4.52	3.98
	(0.79)	(0.79)	(0.82)	(0.72)	(0.88)	(0.80)	(0.73)	(0.68)	(0.78)
Using The FLOW has improved the comprehensiveness of daily progress notes	3.46	3.24	3.40	3.68	3.32	3.55	2.88	3.62	3.55
	(1.04)	(1.04)	(1.16)	(1.01)	(0.82)	(1.15)	(0.99)	(1.12)	(0.94)
Using The FLOW has improved the accuracy of information transmitted during handoffs	3.76	3.62	3.77	3.93	3.58	3.90	3.35	3.52	3.95
	(0.90)	(0.92)	(0.95)	(0.93)	(0.69)	(0.94)	(0.86)	(0.87)	(0.87)
Quality of patient care
Using The FLOW on the unit has improved patient care	3.65	3.57	3.56	3.78	3.58	3.83	3.35	3.48	3.69
	(0.69)	(0.75)	(0.59)	(0.69)	(0.84)	(0.66)	(0.61)	(0.75)	(0.69)
Using The FLOW on the unit has improved patient safety	3.61	3.52	3.53	3.71	3.53	3.83	3.12	3.38	3.69
	(0.77)	(0.81)	(0.67)	(0.84)	(0.77)	(0.85)	(0.7)	(0.59)	(0.74)
I feel patient care would be improved if all physicians and members of the care team used The FLOW	4.04	4.14	3.91	4.02	4.26	4.21	3.65	4.00	4.11
	(0.82)	(0.96)	(0.72)	(0.94)	(0.65)	(0.86)	(0.93)	(0.89)	(0.74)
Usability of the tool
The FLOW is easy to use	3.89	3.90	3.70	4.05	3.95	4.07	3.65	3.95	3.91
	(0.89)	(0.94)	(0.96)	(0.8)	(0.85)	(1.00)	(0.61)	(1.02)	(0.91)
The more I use The FLOW the more useful I find it	3.80	3.86	3.72	3.88	3.84	3.90	3.59	3.76	3.87
	(0.76)	(0.73)	(0.67)	(0.9)	(0.69)	(0.62)	(0.87)	(0.89)	(0.75)
Overall satisfaction
I would like to keep using The FLOW	4.20	4.38	4.16	4.20	4.11	4.38	3.82	4.38	4.18
	(0.79)	(0.74)	(0.78)	(0.9)	(0.66)	(0.82)	(1.07)	(0.67)	(0.70)
Scale: 1 = much slower; 5 = much faster
Productivity of work processes
Using The FLOW has made the documentation of daily progress notes	3.62	3.43	3.74	3.76	3.26	3.72	3.12	3.67	3.73
	(0.96)	(0.98)	(0.93)	(0.99)	(0.93)	(1.03)	(0.99)	(1.15)	(0.83)
Using The FLOW has made patient handoff	3.75	4.00	3.74	3.83	3.42	4.00	3.76	3.86	3.64
	(0.77)	(0.71)	(0.73)	(0.83)	(0.69)	(0.8)	(0.56)	(0.79)	(0.78)

Figure 5.

Responses to selected items from the questionnaire on the impacts of The Flow (N = 127). Numbers are in percentage of respondents.

Twelve respondents (8%) reported that using The FLOW may have prevented an adverse event. An incident where The FLOW may have had a negative impact on patient care was reported by only 6 respondents (5%). Reasons for negative impact were either there was outdated information or a patient was missing from the list. The FLOW was considered easy to use by 77% of respondents and easy to learn by 82% of respondents. It took users a few minutes (25%), a few hours (32%), or a few days (25%) to become comfortable using The FLOW, and only 4 users (3%) were still not comfortable using it when they answered the survey. Most respondents (86%) wished to continue using The FLOW, even if the majority of them (59%) were no longer using it at the time of the questionnaire.

DISCUSSION

This study describes a rounding and handoff tool that was developed and quickly adopted by clinicians at 3 ICUs in a pediatric teaching hospital where patient charts were still paper-based. Overall, user feedback about the app was very positive. The vast majority of respondents (86%) wanted to keep using the app, even though a high proportion (59%) were no longer using it at the time of the questionnaire. Moreover, the fact that the app was spontaneously adopted in an ICU at another of the hospital’s sites suggests that users were able to find immediate and tangible benefits to using it in the ICU. The successful uptake and diffusion of the app in a BYOD model supports the well-recognized need for handoffs and rounding tools in hospital settings, especially in the ICU, where care is time-sensitive and complex.^22,32

The FLOW has 2 innovative features: first, it allows nurses and other professionals to enter notes in the same string as physicians; second, users are allowed to incorporate notes entered in The FLOW within the medicolegal record, by exporting notes in a template and printing (without copying and pasting). Although the app was designed primarily for physicians, The FLOW was used by nurses not only to view notes, but also to enter notes, indicating close collaborative documentation practices with this informal documentation tool. This was especially true at NICU Site 1, as shown by SNA, with nurses being close to the center of the network. This is interesting, given the fact that the nursing administrative authorities of the hospital were not supportive of the BYOD model for nurses. Other studies have observed this phenomenon, where physician-centered handoff tools were found useful by nurses, because the handoff document gives an overview of the patient history, status, and plans.^23,32,33 Moreover, the literature supports shared documentation practices between nurses and physicians to better reflect shared decision-making processes occurring around patients.^34–37 While challenging from a liability perspective, this was not seen as problematic by users in this study. This might be related to the fact that all notes entered in The FLOW were informal documentation. Only when requested by the user would a note become part of the medicolegal record. In most settings, this decision is related to the design of the tool or is made at the organizational level. For example, when the handoff tool is fully integrated within the EMR, such as the one developed at Lucile Packard Children’s Hospital in Palo Alto, California, this option of informal documentation is usually not available and all notes entered in the handoff tool are part of the legal record.²² On the other hand, when the tool is stand-alone or can be installed as a “plug-and-play” within commercial EMRs, the decision as to whether or not data entered will become accessible to the legal discovery process is usually made at the organizational level.^18,19,24,38 In the latter case, it may not be possible to extract notes from the stand-alone tool to the EMR due to technical constraints, leading to a lot of copy-and-pasting. The FLOW was able to allow this integration, and to let clinicians make the decision as to which notes to incorporate into the medicolegal record.

Interestingly, The FLOW was initially designed as an informal documentation and team communication tool for use on mobile devices, but a desktop version was soon required by physicians to print notes entered in it. In 2013, it was installed on all desktop computers in the pilot units, and templates to allow for retrieval and printing of notes, as either a daily note or handoff document, were elaborated. Overall, it is a typical example of a technology that developed into something different from what it was originally intended to be.³⁹ This also suggests that clinicians still require paper forms to be in their pockets along with their smartphones.^40,41

Given the initial success of the app, going forward the MUHC decided to incorporate The FLOW into its plans. Funding was obtained to implement the app in the adult hospital units and extend the features to support documentation practices in the ambulatory clinics. However, the future of The FLOW seems to rely on the provincial government, which decided in November 2015 that all hospital centers in the province would change their EMR systems to the same one the Ministry of Health had already chosen. Whether or not integration of The FLOW as a plug-and-play within this new EMR will be possible is still unknown. This decision reinforced the reality of politics around IT development and implementation in public organizations.

Limitations

In this study, only those individuals who actually used the app were surveyed. As a result, the app’s benefits may have been overestimated, because clinicians who had chosen not to use it could not express their concerns and barriers to use. Thus, to get a deeper understanding of perception of the app, it would be useful to investigate the impressions of nonusers in terms of their attitudes and intention to use, from the same unit and other units. This study used visual network representation to describe usage patterns. However, it is not known if differences in patterns observed through SNA are related to different teamwork practices. To complement this analysis, it could be combined with direct observation as well as interviews or group discussions to get a deeper understanding of how the technology is used and perceived in different hospital units.

CONCLUSION

Analysis of logs and questionnaires allowed us to describe user experiences and patterns of use of The FLOW in different units. While it was abandoned by users in medical units, it was spontaneously adopted in the NICU at another site in the hospital. Experience of users, mostly physicians, was positive, with the vast majority of them (even past users) wanting to continue using it. Some nurses were also active users of the app for data entry, indicating collaborative documentation practices. This study describes an interesting example of an innovative technology that allows informal notes to be extracted within the medicolegal record, with the clinician able to make this decision. While the technology was originally intended primarily for smartphone use, it was eventually adapted to fit data entry and printing needs by using desktop computers. Future research should investigate how the different usage patterns of The FLOW in different units reflect different team-based work practices, and how it can be integrated within a new EMR system that will be implemented in the health care center.

FUNDING

Development and implementation of The FLOW module was financially supported by the Canadian Foundation for Innovation. A.M., a postdoctoral fellow, is financially supported by the Canadian Institute of Health Research.

COMPETING INTERESTS

The authors have no competing interests to declare.

CONTRIBUTORS

A.M., J.W., J.P.C., J.P., R.T., and J.B. were involved in the design of the study. A.M., J.W., R.T., J.P.C., and J.P. were involved in the data collection and/or analysis and interpretation. The manuscript was written by A.M. and J.W. and critically revised by all authors. All authors approved the final version.

SUPPLEMENTARY MATERIAL

Supplementary material is available at Journal of the American Medical Informatics Association online.