Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2015 Jul 2.
Published in final edited form as: Comput Inform Nurs. 2013 Sep;31(9):439–449. doi: 10.1097/01.NCN.0000432126.99644.6c

Mixed Methods Approach for Measuring the Impact of Video Telehealth on Outpatient Clinic Triage Nurse Workflow

Rhonda G Cady 1, Stanley M Finkelstein 1
PMCID: PMC4489569  NIHMSID: NIHMS701804  PMID: 24080753

Abstract

Nurse-delivered telephone triage is a common component of outpatient clinic settings. Adding new communication technology to clinic triage has the potential to not only transform the triage process, but also alter triage workflow. Evaluating the impact of new technology on an existing workflow is paramount to maximizing efficiency of the delivery system. This study investigated triage nurse workflow before and after the implementation of video telehealth using a sequential mixed methods protocol that combined ethnography and time-motion study to provide a robust analysis of the implementation environment. Outpatient clinic triage using video telehealth required significantly more time than telephone triage, indicating a reduction in nurse efficiency. Despite the increased time needed to conduct video telehealth, nurses consistently rated it useful in providing triage. Interpretive analysis of the qualitative and quantitative data suggests the increased depth and breadth of data available during video triage alters the assessment triage nurses provide physicians. This in turn could impact the time physicians spend formulating a diagnosis and treatment plan. While the immediate impact of video telehealth is a reduction in triage nurse efficiency, what is unknown is the impact of video telehealth on physician and overall clinic efficiency. Future studies should address this area.

Keywords: Telehealth, Workflow, Nursing Informatics, Qualitative Research, Time and Motion Studies

Telephone triage is a regular component of outpatient clinic settings1 and a telephone triage office is often established to handle patient-clinic communication between visits.2 Commonly staffed by registered nurses (RNs), telephone triage evolved during the 1980's in response to the ‘gatekeeping’ focus of managed care,3 using the lowest cost provider in the lowest cost setting to provide safe and effective care.4 In 1997, the American Academy of Ambulatory Care Nurses recognized the importance of telephone triage and published the first standards and practice guidelines for telephone nursing.5

A steady increase in broadband availability and corresponding home adoption6 has generated new types of patient-clinic communication. Ranging from remote monitoring to video telehealth, these technologies have the potential to transform the telephone triage nurse toolset. Determining the safety and efficacy of new technology on patient outcomes is paramount to adoption. Equivalently, understanding clinician experience with new technology and factors that increase usability are needed to create evidence-based implementation guidelines7 that maximize workflow efficiency.

Despite increased investment in technology, there is mixed evidence supporting a corresponding increase in clinician productivity and overall system efficiency.8 Qualitative research has documented both positive and negative effects of new technology on clinician workflow9,10 while quantitative research, primarily in the form of time-motion study, has shown little or no effect of new technology on clinician workflow.11,12 A single research paradigm may not capture subtle organizational or workflow issues resulting from new technology,13 and may explain the discrepancy between quantitative and qualitative workflow research.

The purpose of this study was to describe and evaluate the impact of adding video telehealth (VTH) to the workflow of telephone triage nurses located in a large hospital-based general pediatrics clinic. In the context of this research, VTH is the experimental use of PC-based, HIPAA-compliant video-conferencing software between home and outpatient clinic triage office, to communicate health questions and concerns that would otherwise be communicated by telephone. Whether different workflows emerge after the introduction of VTH, and the impact on triage nurse efficiency as measured by time, has not been studied and represents a significant gap in the current knowledge of nursing informatics. A mixed methods workflow analysis14 was chosen over a single research paradigm because the workflows of telephone and video triage have not been documented and the variables that require measurement are unknown.15 Equally important, the mixed methods workflow analysis addresses key issues needed to advance telehealth use: contextual analysis of the implementation environment to identify the positive and negative consequences of telehealth, and understanding how integration of telehealth improves the efficiency of the delivery system.16

Methods

Setting

The study was conducted in the telephone triage office of a large urban general pediatrics clinic affiliated with a non-profit children's hospital. Eight physicians and four nurse practitioners provide primary care to approximately 10,000 children, which include a large population (~ 600) of children with complex conditions. RN-delivered telephone triage is standard service for all children enrolled in the clinic, including those with complex conditions. Two part-time RNs staff the triage office weekdays from 8:00 a.m. - 4:30 p.m. The triage office is an integral component of the general pediatrics clinic and telephone is the standard modality for parents to communicate with providers between appointments. Incoming clinic calls are answered by the scheduling or front desk clerks. Calls from parents regarding a concern about their child's health are routed to the triage office and have two possible outcomes; they are answered ‘live’ by the triage nurse or sent to voice mail. Unanswered calls are prioritized by the triage nurse and returned in order of priority.

The RN's primary role during a triage call is data collection and assessment. Assessments for non-emergent conditions use standardized protocols when applicable; all other assessments are sent to the child's pediatrician for a prescribed plan of care which may include having the parent schedule a clinic visit. Assessments for emergency conditions are directed to the nearest emergency facility. When the clinic is closed, triage calls are routed to an off-site service. In addition to telephone calls, the triage nurses are responsible for completing and coordinating forms (e.g. prescription refill requests, school forms for medication administration and special accommodations, insurance prior authorization requests) sent to the clinic.

Participants

Appropriate Institutional Review Boards approved the study protocol. No incentives were provided to informants. All RNs (n=2) working in the triage office were recruited; both provided written informed consent. These nurses, the primary informants of the workflow analysis, had 35 and 21 years nursing experience and 21 and 5 years of triage experience. The workflow analysis of video triage required observation of health-related interactions with parents and children. A secondary group of informants, parent/child or family informants, were recruited from children enrolled in the clinic. Triage nurses identified ‘frequent triage callers’ from the subset of children with complex conditions, and recruitment materials were given to the families during scheduled well-child clinic visits. Interested families were screened for inclusion in the video telehealth protocol, which required access to a Windows-based computer and a broadband connection. Over a five month period, a convenience sample of eleven family informants provided written informed consent that allowed the researcher to observe nurse-parent-child interactions during VTH triage and conduct documentary analysis of the child's electronic medical record (EMR).

Video Telehealth Intervention

The health related nature of transmissions during experimental video triage with the convenience sample of eleven informant families mandated HIPAA-compliant software. Each informant family (n=11) received a telehealth packet containing HIPAA-compliant video-conferencing software (PVX, Polycom Inc., Pleasanton, CA), a web camera, and instructions for installation. Three families required in-home installation assistance; the remaining families successfully installed the video software. To assess the impact of VTH on triage nurse workflow, informant families agreed to use video for all communication with the general pediatrics clinic triage nurses. To minimize clinic disruption during the VTH intervention, informant families initiated triage communication using the existing telephone protocol. Upon receipt of an informant family telephone call, the triage nurse assessed the family's readiness to conduct a VTH session. If ready, the triage nurse opened the video application and initiated a VTH session with the informant family; otherwise the triage session was completed by telephone.

Workflow Analysis Protocol

This research used a mixed methods approach that explored the workflow of telephone and VTH triage and used this information to measure the efficiency of the two triage modes. This approach was chosen over a singular qualitative or quantitative approach because telephone and VTH triage workflows have not been documented and the activities and interactions requiring measurement are unknown.15 The combined qualitative and quantitative protocol resulted in a robust design that offset the weaknesses of one method with the strengths of the other,15 and allowed corroboration of findings across methods.17

In this study, the focus of analysis was the workflow of the general pediatrics clinic triage nurses and their interactions with persons and artifacts (things). Artifacts are man-made devices that store, display or transform information, such as paper and electronic records, e-mail, voice mail, telephone, fax and video.18 Triage workflow was investigated using a mixed methods protocol consisting of four distinct components that were conducted sequentially (Figure 1). Results from the qualitative component informed development of the data collection tool used during the quantitative component. Results from the qualitative and quantitative components were merged and integrated to identify the positive and negative consequences of adding VTH to outpatient clinic triage nurse workflow.14 Data collection and data analysis methods for each of the four components are described below.

Figure 1.

Figure 1

Open in a new tab

Mixed Methods Workflow Analysis Protocol

Data Collection

The workflow analysis protocol utilized both qualitative and quantitative data collection methods. All data was collected by a single observer (RC) whose background included nursing, information systems and health informatics research. Data collection focused on the workflow of the triage nurses during telephone and VTH triage. No data was collected from family informants. Observation and semi-structured interviews with the triage nurses took place in the telephone triage office; documentary EMR data collection was conducted at a clinic workstation. Collection of telephone triage data took place during periods of high (Monday and Friday) and normal (Tuesday – Thursday) triage activity and was deliberately selected to allow equal observation of both triage nurses. Collection of VTH triage data was limited to eleven family informants and occurred sporadically when a change in the child's health precipitated a call to the triage office. This resulted in “event sampling”19 of VTH triage data, a common qualitative data collection approach.

The qualitative component of the workflow analysis protocol utilized cognitive ethnography, where the “researcher learns from informants the meanings that they attach to their activities, events, behaviors, knowledge and rituals.”19 Cognitive ethnography was chosen over other qualitative methods since it is best suited to describe the distributed and collaborative nature of nurse workflow. Ethnographic data collection was purposeful, with deliberate focus on a pre-planned intervention (VTH) that impacts work practice.20 Data collection by a single observer is typical of ethnographic studies21 and observation of triage workflow used an “observer as participant”19 role, where the researcher's purpose is openly known to informants. Qualitative methodology requires researchers to be aware of their presence on the culture being studied, and the impact of personal biases and perceptions on data analysis and interpretations.22

Ethnographic data were collected via direct observation, semi-structured interview and documentary analysis of the EMR. Direct observation of nurse informants provided purposive and specific information about the work activities performed and the things (artifacts) and people they interacted with. Observations were captured in handwritten notes for later transcription. Semi-structured interviews clarified questions noted during observation sessions and facilitated “member checking,”23 comparing the researcher's description of workflow to the nurse's description to ensure accurate documentation of workflow activities. Documentary analysis of the child informant EMR provided retrospective data on telephone and video care delivery episodes that occurred outside of the data collection sessions.

Ethnographic data collection continues until data saturation; it does not follow a pre-defined time frame. The workflow of telephone triage was observed during 13 sessions over a six month period beginning August 2009. The prolonged time frame was due to a three month project suspension during the swine flu pandemic. Data saturation occurred during the 11th session and was evident when “redundancies and duplication of similar ideas, meanings, experiences, and descriptions occur from informants so that no more information is forthcoming.”24 Two additional sessions were conducted to validate data saturation. The workflow of VTH triage was observed during 6 sessions over a four week period beginning June 2010. Data saturation occurred during the 4th session and was validated during two additional sessions.

The quantitative component of the workflow analysis protocol utilized time-motion study to measure the frequency and time of triage workflow activities and interactions, and determine whether a difference in efficiency, as measured by time, existed between telephone and VTH triage. Time-motion data were collected by observation from both nurses during 27 sessions over a three month period beginning June 2010 using the repetitive or snap-back timing method.25 At the start of each data collection session, the researcher coded the nurse informant and snapped-back a digital stop-clock to zero. As the nurse moved from one activity to another, four pieces of data were coded: the time from the stop-clock, the code for the activity observed, the person(s) interacted with, and the artifact(s) interacted with. The activity, person and artifact codes were taken from the validated triage workflow list generated from the qualitative data (Table 1). Data from the coded workflow observations comprised the sample used in the time-motion study.

Table 1.

Validated Triage Time-Motion Workflow List

TRIAGE ACTIVITY LIST
1. Coordination of Activities
    a. Coordinate Triage Episode of Care
    b. Coordinate Prescription / Refill Request
    c. Coordinate New/Rejected Prior Authorization
    d. Coordinate Appointment / Test Scheduling
    e. Coordinate / Process Forms and Service Request
    f. Handoff open episodes to other triage nurse
        g. Coordinate preparation for Video Triage
2. Information Flow
    a. Collect faxes / Sort through forms
    b. Document new episode
    c. Check e-mail
    d. Transfer pending lab rpts to CPN (Friday only)
3. Problem Solving
    a. Search for information (non-EMR)
    b. Review call log for open episodes
    c. Information request from caller
    d. Ways to increase documentation/triage efficiency
4. Workarounds, Interruptions, Inefficiencies
    a. Go to person/artifact when no electronic communication
    b. Search for pmt options when no insurance
    c. Coordination / provision of direct care for clinic patient
    d. Search for missing EMR documentation
e. Troubleshoot Video Triage
TRIAGE INTERACTION ARTIFACT LIST
1. EMR Search / Review Clinical Document
2. EMR Message (review/update)
3. EMR e-prescribe
4. Telephone
5. Fax
6. Printer
7. Call log
8. Form
9. Paper prescription form (controlled substances)
10. Internet
11. Reference Manuals
12. Paging System
13. Children's ‘Tube System’
14. Voice Mail
15. Video Software / Audio Headset
TRIAGE INTERACTION PERSON LIST
1. Parent / Caregiver
2. General Peds MD / PNP
3. General Peds MA
4. Triage Nurse / SNC Coordinator
5. Other Gen Peds staff
6. Other Children's personnel
7. Community providers (MD, RN, ER, HHA, dentist)
8. Community non-provider (school, insurer)
9. Child
10. Investigator
Open in a new tab

Italic indicates activity or interaction added by VTH

Nurse perception of the usefulness of VTH in providing triage was captured after each VTH session. Using a four category Likert scale (not at all, a little, somewhat, very clear/very much) nurse informants rated the audio/video quality and usefulness of the VTH session. Usefulness data was not collected for telephone triage sessions.

Data Analysis

The workflow analysis protocol utilized qualitative and quantitative data analysis methods during each of the four components shown in Figure 1. Qualitative analysis of ethnographic data searched for patterns and their explanation26 and relied on the strategies “asking questions and making comparisons”27 to identify the activities and interactions conducted by outpatient clinic triage nurses during telephone and VTH triage. Asking questions was accomplished with a process analysis focusing on the “who, what, when, where and how”27 of triage workflow with and without VTH. Comparisons using a directed content analysis grounded in the framework of distributed cognition provided a mechanism to determine the interactions relevant to activity performance and identify interdependencies between persons and artifacts.28 Transcripts from ethnographic data collection were color-coded using the distributed cognition categories: coordination of activity, interaction with artifacts, interaction with persons, information flow, changes in representational state and problem solving.14 Inefficiencies, interruptions and workarounds were identified and coded within each of these categories. The small number of informants observed was not a limitation to the consistency of qualitative analysis since data collection focused on purposeful observations of telephone and VTH triage from knowledgeable key nurse informants.21

Connecting data sources is an integral component of mixed methods design and uses results from the qualitative component to inform development of the quantitative measurement tool. In this study, the categories and subcategories emerging from the cognitive ethnography formed the basis for the triage workflow list, which specified the activities, persons and artifacts measured during the quantitative time-motion study. Validity of the measurement tool was established across content, concurrent and convergent domains.

Quantitative analysis of time-motion data used Intercooled Stata version 9.0 (StataCorp, College Station, TX). Frequency charts were generated with Excel 2007 (Microsoft, Redmond, WA). Time-motion data exhibited a left-skewed, non-normal distribution that warranted non-parametric tests for significance testing. Mutually exclusive telephone and VTH sub-samples were extracted from the time-motion observation sample. These sub-samples included only activities that involved interaction with a parent/child by telephone or VTH, respectively. Between-group comparison using the Wilcoxon rank sum test determined whether a difference existed between the times required to conduct telephone and VTH triage activities. All observations measured during the time-motion study were performed by two nurses and differences between telephone and VTH activity times could be attributed to a spurious dependence on a third variable.29 Spuriousness was tested by between-group comparison using the Wilcoxon rank sum test to determine whether a difference existed between the times each nurse spent on all activities in the time-motion observation sample. Logistic regression of the ordinal perception of VTH usefulness survey data using a continuous latent variable model tested the predictive relationship between audio/video quality and the usefulness during VTH triage only; telephone triage was not tested. Chi-square analysis tested whether an association existed between the activities and interactions (people and artifacts) of triage nurse workflow.

The final component of the mixed methods protocol merged results from the quantitative and qualitative components. Time-motion results were compared to ethnographic results to corroborate findings across methods, enhancing the overall strength of the workflow analysis. Interpretive analysis used the five dimensions of the Interactive SocioTechnical Analysis (ITSA) Framework30 to describe the positive and negative consequences of VTH on triage nurse workflow. The ITSA framework examines the effects of new technology on an organization's workflow and social interactions, how an organization's culture and technical and physical infrastructure effect new technology use and how these interactions lead to technology redesign.30

Results

Qualitative Ethnographic Findings

The workflows of telephone and video triage were observed for 28 hours during 19 sessions and documented in 45 pages of typed notes. The final themes emerging from the ethnographic data fit into four categories: coordination of activities, information flow, problem solving and inefficiencies, interruptions and workarounds. In the context of this study, coordination of activities between clinical and non-clinical persons using artifacts such as telephone, VTH, EMR, e-mail, voice mail and fax was the central theme that facilitated the information flow and problem solving activities of the triage nurse.

Analyzing triage workflow uncovered the collaborative and interdependent nature of triage activities and interactions. All activities of triage workflow are initiated by interaction with an artifact (telephone, fax, mail, EMR messaging) or person (parent, clinic staff, non-clinic staff). This finding is significant since changes to an artifact or person could change the interactions and activities of triage workflow. A key component of triage workflow was communication with clinical providers during assessment, planning and implementation activities. Although clinic providers were located in the adjoining outpatient clinic, electronic messaging via the EMR artifact was the primary mode of communication between triage nurse and provider. This allowed immediate two-way communication that was not disruptive to either person's workflow.

VTH changed the workflow of the triage nurse. These changes are illustrated in the triage workflow process map (Fig 1), where activities and interactions added by VTH appear in varying shades of gray. The first new activity, prepare for VTH, reflects the additional activities needed to initiate a VTH triage session and the new interactions with video software and audio headset artifacts. The second new activity, troubleshoot VTH, reflects interruptions to triage nurse workflow when video transmission issues occurred. The third new activity, conduct VTH assessment, reflects the additional information obtained by visual interaction with the child and parent. During the VTH session, triage nurses interacted with the child, who is the focus of but not a participant in, telephone triage. Triage nurse comments during initial parent-child interactions included:

  • I have never seen X before. Just talked to his mom on the phone. This is so nice. Hi X!

  • I have not seen Y for 2 ½ years. She has grown a lot. Hi Y (waving)

  • We have talked a lot by phone. It is so nice to finally see you. Oh, there's the new baby.

She can hold her head up already.

Depending on the visual information received, the triage nurse invited the child's pediatrician to participate in the VTH session. This differs from telephone triage, where physician consultation occurs separately from the parent interaction and most commonly via EMR messaging. An episode of VTH involving a toddler illustrates the changed nurse-physician interaction. The child was seen in clinic a week earlier and diagnosed with an infected gastrostomy tube (g-tube). The mother was concerned about a recurring infection. After visual assessment, the triage nurse told the parent she wanted the physician to view the site. Using the triage workstation and audio headset, the physician assessed the g-tube and child while interacting with the parent. The triage nurse assessment of granulamatous was confirmed and a home treatment plan was prescribed. After the session, the physician stated that a verbal-data only (telephone) assessment would have resulted in a clinic visit recommendation, not home treatment. After the episode, the triage nurse provided the following analogy:

“What we do is like a good car mechanic. They know what questions to ask to figure out your car problem and are usually correct. However, they still hook it up to the diagnostic machine to verify the problem. We do the same with telephone triage. The video allowed the MD and me to verify the diagnosis. It was very cool.”

Generation and Validation of the Quantitative Data Collection Tool

The categories and subcategories that emerged from the cognitive ethnography of triage workflow formed an initial 64-item triage workflow list. Duplicate activities were collapsed. Interaction persons and artifacts were separated from activities to capture the collaborative nature of triage workflow. The final triage time-motion workflow list (Table 1) contained 20 activities, 15 interaction artifacts and 10 interaction persons.

Quantitative Time-Motion Results

The triage time-motion observation sample consisted of 642 observations recorded during 24 ½ hours of data collection, using the activities and interactions from the validated triage workflow list. The primary measurement of each observation was time. The average time to conduct a single activity in the observation sample is 2:17 minutes, with a median time of 1:30 minutes. Frequency and time of observations grouped by workflow category appears in Table 2. Coordination of activities comprised over 2/3 of the observation sample, corroborating the primary theme emerging from the qualitative strand.

Table 2.

Frequency and Time of Time-Motion Observation Sample by Activity and Interaction

TRIAGE WORKFLOW LIST Triage Time-Motion Observation Sample
Frequency Time Mean Median (Min/Max)
No. % Total %
WORKFLOW CATEGORIES
Coordination of Activities 435 67.8% 14:54:30 60.9% 02:03 1:23 (0:01, 17:39)
Information Flow 88 13.7% 4:21:29 17.8% 02:58 2:00 (0:14, 16:11)
Problem Solving 57 8.9% 1:45:56 7.2% 01:51 1:10 (0:09, 10:35)
Workarounds, Interruptions, Inefficiencies 62 10% 3:27:18 14% 03:20 2:43 (0:20, 13:35)
Totals 642 100% 24:29:13 100% 02:17 1:30 (0:01, 17:39)
WORKFLOW ARTIFACTS
Telephone 151 24% 7:02:43 29% 02:48 2:07 (0:01, 17:39)
Voice Mail 31 6% 1:17:33 5% 02:30 1:30 (0:43, 11:19)
Video Software / Audio Headset 40 12% 2:42:03 11% 04:03 3:10 (0:31, 13:40)
EMR Search / Review Clinical Document 76 12% 2:05:28 9% 01:39 1:08 (0:02, 8:46)
EMR Message (review/update) 105 16% 2:45:06 11% 01:34 1:19 (0:02, 7:14)
EMR e-prescribe 49 8% 1:39:59 7% 02:02 1:28 (0:15, 17:15)
Fax 15 2% 0:11:17 1% 00:45
Internet 5 1% 0:13:34 1% 02:43
Paging System 1 0% 0:02:20 0% 02:20
Printer 9 1% 0:10:23 1% 01:09
Call log 34 5% 0:39:54 3% 01:10 0:49 (0:09, 7:30)
Form 64 10% 2:25:53 10% 02:17 1:32 (0:13, 8:35)
Paper prescription (controlled substances) 8 1% 0:08:52 1% 01:07
Reference Manuals 3 0% 0:06:37 0% 02:12
Children's ‘Tube System’ 0 0% 0:00:00 0% 00:00
No Artifact Used 51 8% 2:57:31 12% 03:29 1:45 (0:05, 16:11)
Totals 642 100% 24:29:13 100% 02:17 1:30 (0:01, 17:39)
WORKFLOW PERSONS
Parent / Caregiver 144 22% 6:43:38 27% 02:48 2:06 (0:16, 13:33)
Child 10 2% 1:01:35 4% 06:10 6:00 (1:22, 13:40)
Clinic Provider (MD / PNP) 110 17% 4:13:03 17% 02:18 1:37 (0:05, 16:11)
Clinic Medical Assistant 1 0% 0:01:15 0% 01:15
Triage Nurse 29 5% 1:08:54 5% 02:23 1:27 (0:16, 13:00)
Other Clinic Staff 3 0% 0:14:46 1% 04:55
Other Hospital Personnel 16 2% 1:04:15 4% 04:01
Community Providers 14 2% 0:36:31 2% 02:36
Community Non-Provider (school, insurer) 7 1% 0:19:48 1% 02:50
Investigator 3 0% 0:07:58 1% 02:39
No Person Involved 305 48% 8:57:30 37% 01:46 1:11 (0:01, 17:15)
Totals 642 100% 24:29:13 100% 02:17 1:30 (0:01, 17:39)
Open in a new tab

Efficiency of Video Triage

This study determined whether a difference in efficiency, as measured by time, existed between telephone and VTH triage. The VTH observation sub-sample included 15 activities conducted over 1:26 hours, with a median activity time of 5:04 minutes. The telephone observation sub-sample included 92 activities conducted over 3:50 hours, with a median activity time of 1:53 minutes. The activities in both sub-samples were from the coordination of activities and information flow categories only. Between-group comparison of the time spent on telephone and VTH triage, as defined by the telephone and VTH observation sub-samples, was performed at the category level because of inadequate sample size at the activity level. These comparisons indicated that video triage required significantly more time than telephone triage (p <= 0.05). “Spuriousness”29 between nurse, activity and triage type was eliminated, strengthening validity of the result. This finding raises the possibility that video triage is less efficient, as measured by time, than telephone triage.

Usefulness of Video Telehealth Triage

Both triage nurses rated the quality and usefulness of VTH after each VTH triage episode (n=13) using the four category Likert scale. All episodes with good quality two-way audio and video were rated as very useful (77%). Several family informants had reoccurring video transmission problems, where the nurse received audio and video from the parent/child but the parent/child received only audio from the nurse. All episodes with this problem were rated a little or somewhat useful (23%). Logistic regression analysis of the ordinal survey data supported audio/video quality as a potential predictor of video usefulness (p <= 0.001) and indicates the importance of reliable VTH technology.

Importance of Person and Artifact Interactions

A unique aspect of the time-motion study was the collection of interaction data. The frequency and time of observations grouped by workflow artifact and person appears in Table 2. Triage nurses interacted with another person during 52% of activities but interacted with an artifact during 92% of activities. The EMR was the most frequent artifact interaction and clinic providers and parents were the most frequent person interactions. Artifacts were grouped by communication type (verbal/visual, electronic, paper) and persons were grouped by relationship to the triage nurse (parent/child, clinic personnel, non-clinic personnel); both were charted by workflow category. Chi-square analysis indicated a strong association (p < 0.01) between workflow category and both artifact category and person category. The frequency of interaction type by workflow category (Fig 3) illustrates the importance of person and artifact interactions during triage workflow and is consistent with qualitative findings.

Figure 3.

Figure 3

Figure 3

Open in a new tab

Frequency of Person and Artifact Interactions by Workflow Category

Frequency of Person and Artifact Interactions by Workflow Category (black & white)

Merging and Integrating Workflow Analysis Data

Merging and integrating the qualitative and quantitative data within the ITSA framework30 illustrated the trade-off of adding VTH to triage workflow. Positive workflow consequences resulted from VTH changing the triage social system. This included visual and non-verbal communication between parent, child and triage nurse, and increased confidence in the quality and usefulness of parental communication and when requested by the triage nurse, provider use of VTH to facilitate diagnosis and treatment plan. These changes subsequently generated a negative consequence, with VTH triage requiring significantly more time than telephone triage. Additional negative workflow consequences resulted from technical infrastructure security that was not compatible with VTH routing, creating unpredictable connectivity and transmission problems that disrupted nurse workflow.

Discussion

A mixed methods workflow analysis combined cognitive ethnography with time-motion study to analyze the workflow of triage nurses in a general pediatrics clinic before and after the implementation of video telehealth (VTH). The workflow analysis found that VTH added activities and interactions to triage workflow and disrupted triage workflow with unpredictable connectivity issues. VTH triage required significantly more time than telephone triage, yet nurses consistently rated it as useful. This important but contradictory finding could be explained by the increased depth, breadth and quality of data available during VTH assessment, which included visual, verbal and non-verbal data from the parent and child.

The workflow analysis illustrated the importance of merging qualitative and quantitative results to describe the consequences of adding new technology to an existing workflow. The time spent on video triage activities was significantly longer than the time spent on equivalent telephone activities, suggesting a decrease in triage nurse efficiency. Exploration of the qualitative data provided context for this finding. Triage nurses interacted with parents primarily by telephone, and VTH was often the nurse's first ‘face-to-face’ interaction with a parent. The face-to-face, non-verbal communication supported by VTH resulted in a positive consequence: increased confidence in the quality of the communication. A study of videophone communication where nurses felt more engaged during patient communication supports this finding.31

An additional consequence of VTH was the ability to communicate with the child, often for the first time, and obtain verbal, non-verbal and visual data that is not available during telephone triage. This finding is supported by a study of children with severe congenital heart disease that found video was “a more effective form of follow-up in terms of clinical observation” than telephone.32 In addition, the triage nurses are pediatric nurses and spent a portion of each video call interacting with the child. While not measured during this study, video interactions with children could increase triage nurse job satisfaction.

Despite the positive consequences of VTH, transmission issues influenced triage nurse willingness to use VTH. An organization-wide video gateway and firewall caused unpredictable connectivity issues during VTH call initiation. Families with successful VTH connections one day were unable to connect with the triage nurse another day. When VTH connectivity issues occurred, the triage nurse stopped her workflow and waited for issue resolution. Triage nurses are very busy and often work alone. After several lengthy troubleshooting episodes, triage nurses requested VTH use only when both nurses were working. This negative consequence resulted from replacing a working, efficient triage delivery modality (telephone) with an experimental and at times unpredictable delivery modality (VTH). This finding is supported by several studies. An assessment of nurse readiness to implement telehealth found that unlike the EMR, telehealth is not mandatory, and nurses must be motivated to use it.33 Furthermore, an unreliable system provides little motivation for use.34 A reliable VTH platform could reduce workflow disruption, increase nurse confidence in VTH triage and yield different results.

Even with the disruptive impact of VTH on triage workflow, the nurses consistently rated VTH as useful. This rating was dependent on good two-way audio/video quality and matches the results of previous studies.35 Although VTH was consistently rated as useful by the nurses, the practicality of VTH determined when and how it was used. This dilemma is illustrated in the following scenario.

A parent informant called regarding a child's ileostomy stoma. The triage nurse felt VTH would help with assessment and asked the parent to initiate a VTH call. The child was sleeping and parent did not want to awaken the child. More importantly, removal of the ileostomy bag, which can be very messy, was needed to view the stoma. The parent and nurse agree that video would be useful, but was not practical, in this situation.

The workflow analysis protocol illustrated the benefit of identifying and measuring person and artifact interactions within a workflow. Triage nurses use the EMR more than any other artifact and the EMR messaging function is the primary communication mode with clinic physicians. Changes to the EMR that impact the messaging function could have a highly disruptive effect on clinic workflow. The workflow analysis revealed that triage nurses interact with other clinic personnel during an episode of triage. Physicians are key components of a triage episode, with responsibility for assessing and formulating a treatment plan. The direct impact of VTH on triage nurse workflow was a reduction in nurse efficiency, as measured by time. What is not known is the impact of VTH on the efficiency of physicians.

Limitations

While a purposeful sample of triage nurse workflow was selected to yield information-rich cases, sampling limitations occurred. The small size of the triage office limited observation sessions to a maximum of three hours/day. The VTH observation sample was limited by timing and participant selection bias. All observations occurred during summer and early fall. Data collected during winter months could yield different results. The VTH inclusion criteria limited the sample to eleven families with a home computer and broadband connection. Data collected from families excluded by these criteria could yield different results. The use of VTH as an ‘experimental’ and not standard triage delivery mode presents a limitation since triage nurse and family informant inexperience with the technology could increase activity times for VTH as compared to telephone triage. This was minimized by excluding the time spent preparing for and troubleshooting VTH in the analysis.

The time-motion component utilized a quasi-experimental equivalent time-samples design where nurse and family informants had knowledge of the experimentation and repeated exposure to the experimental variable - triage delivered by VTH.36 The possibility of multiple-X interference36 was present because nurse informants provided care to family informants in the presence and absence of the experimental variable. These threats, along with a small number of nurse (n=2) and family (n=11) informants, limits generalization of quantitative results.

Conclusion

The primary responsibility of the triage nurse is coordination of activities between clinical and non-clinical persons using artifacts such as telephone, VTH, EMR, e-mail, voice mail and fax to facilitate information flow and problem solving. These activities involved assessing a child's condition, transmitting information to the physician for diagnosis and treatment, and implementing the physician's plan of care. VTH assessment provided visual and non-verbal data unavailable during telephone assessment. The increased quantity and quality of data increased the time triage nurses spent during VTH triage. One question that remains unanswered is the impact of VTH on physician workflow efficiency. The increased depth and breadth of data provided during VTH could change the time spent by physicians assessing and formulating a treatment plan. VTH could also change the implementation or delivery of a treatment plan. Measuring the time of activities conducted by physicians during an episode of triage provides a more robust measure of triage efficiency and a more accurate assessment of the impact of VTH on overall clinic workflow. Future studies should address this question.

Figure 2.

Figure 2

Open in a new tab

Process Map: The Impact of Video Telehealth on Triage Workflow

Acknowledgments

This project was supported in part by grant 1R01NR010883 from the National Institute of Nursing Research, National Institutes of Health

References

  • 1.Huber DL, Blanchfield K. Telephone nursing interventions in ambulatory care. J Nurs Adm. 1999;29(3):38–44. doi: 10.1097/00005110-199903000-00007. [DOI] [PubMed] [Google Scholar]
  • 2.Katz HP. Telephone medicine : Triage and training : A handbook for primary care health professionals. Slack; Thorofare, NJ: Thorofare, NJ: 1990. [Google Scholar]
  • 3.Salk ED, Schriger DL, Hubbell KA, Schwartz BL. Effect of visual cues, vital signs, and protocols on triage: A prospective randomized crossover trial. Ann Emerg Med. 1998;32(6):655–664. doi: 10.1016/s0196-0644(98)70063-0. [DOI] [PubMed] [Google Scholar]
  • 4.Poole SR. Are you interested in telephone triage research? Pediatrics. 1995;95(5):802–802. [PubMed] [Google Scholar]
  • 5. [7/23/2012];Telehealth | AAACN. http://www.aaacn.org/cgibin/WebObjects/AAACNMain.woa/wa/viewSection?s_id=1073743920.
  • 6. [6/5/2012];Pew internet: Broadband | pew research center's internet & american life project. http://pewinternet.org/Commentary/2012/May/Pew-Internet-Broadband.aspx.
  • 7.Demiris G, Charness N, Krupinski E, et al. The role of human factors in telehealth. Telemedicine Journal & E-Health. 2010;16(4):446–453. doi: 10.1089/tmj.2009.0114. [DOI] [PubMed] [Google Scholar]
  • 8.Lapointe L, Mignerat M, Vedel I. The IT productivity paradox in health: A stakeholder's perspective. Int J Med Inf. 2011;80(2):102–115. doi: 10.1016/j.ijmedinf.2010.11.004. [DOI] [PubMed] [Google Scholar]
  • 9.Campbell EM, Sittig DF, Ash JS, Guappone KP, Dykstra RH. Types of unintended consequences related to computerized provider order entry. Journal of the American Medical Informatics Association. 2006;13(5):547–556. doi: 10.1197/jamia.M2042. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Ash JS, Sittig DF, Poon EG, Guappone K, Campbell E, Dykstra RH. The extent and importance of unintended consequences related to computerized provider order entry. Journal of the American Medical Informatics Association. 2007;14(4):415–423. doi: 10.1197/jamia.M2373. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Hollingworth W, Devine EB, Hansen RN, et al. The impact of e-prescribing on prescriber and staff time in ambulatory care clinics: A time motion study. J Am Med Inform Assoc. 2007;14(6):722–730. doi: 10.1197/jamia.M2377. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Cornell P, Herrin-Griffith D, Keim C, et al. Transforming nursing workflow, part 1: The chaotic nature of nurse activities. J Nurs Adm. 2010;40(9):366–373. doi: 10.1097/NNA.0b013e3181ee4261. [DOI] [PubMed] [Google Scholar]
  • 13.Ash JS, Berg M, Coiera E. Some unintended consequences of information technology in health care: The nature of patient care information system-related errors. Journal of the American Medical Informatics Association. 2004;11(2):104–112. doi: 10.1197/jamia.M1471. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Cady RG, Finkelstein SM. A mixed methods approach for measuring the impact of delivery-centric interventions on clinician workflow. AMIA Annu Symp Proc. 2012;2012:1168–1175. [PMC free article] [PubMed] [Google Scholar]
  • 15.Creswell JW. Designing and conducting mixed methods research. SAGE Publications; Los Angeles, Calif.: 2011. [Google Scholar]
  • 16.Bashshur RL. Telemedicine effects: Cost, quality, and access. J Med Syst. 1995;19(2):81–91. doi: 10.1007/BF02257059. [DOI] [PubMed] [Google Scholar]
  • 17.Bryman A. Integrating quantitative and qualitative research: How is it done? Qualitative Research. 2006;6(1):97–113. [Google Scholar]
  • 18.Carroll JM, editor. Designing interaction : Psychology at the human-computer interface. Cambridge University Press; Cambridge; New York: 1991. Cambridge series on human-computer interaction ; 3. [Google Scholar]
  • 19.Germain CP. Ethnography: The method. In: Munhall PL, editor. National League for Nursing, eds. Nursing research : A qualitative perspective pub. (national league for nursing) ; no. 14-1357. 3rd ed. Jones and Bartlett Publishers; Sudbury, Mass.: 2001. pp. 277–306. [Google Scholar]
  • 20.Ball LJ, Ormerod TC. Putting ethnography to work: The case for a cognitive ethnography of design. International Journal of Human-Computer Studies. 2000;53(1):147–168. [Google Scholar]
  • 21.Muecke MA. On the evaluation of ethnographies. In: Morse JM, editor. Critical issues in qualitative research methods. Sage Publications, Inc.; Thousand Oaks, Calif.: 1994. pp. 187–209. [Google Scholar]
  • 22.Patton MQ. Enhancing the quality and credibility of qualitative analysis. Health Serv Res. 1999;34(5 Pt 2):1189–1208. [PMC free article] [PubMed] [Google Scholar]
  • 23.Mays N, Pope C. Qualitative research in health care: Assessing quality in qualitative research. 2000;320(7226):50. doi: 10.1136/bmj.320.7226.50. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Leininger M. Evaluation criteria and critique of qualitative research studies. In: Morse JM, editor. Critical issues in qualitative research methods. Sage Publications, Inc.; Thousand Oaks, Calif.: 1994. pp. 95–115. [Google Scholar]
  • 25.Barnes RM. Motion and time study. Wiley; New York, New York: 1958. [Google Scholar]
  • 26.Boyle JS. Styles of ethnography. In: Morse JM, editor. Critical issues in qualitative research methods. Sage Publications, Inc.; Thousand Oaks, Calif.: 1994. pp. 159–185. [Google Scholar]
  • 27.Corbin JM. Basics of qualitative research : Techniques and procedures for developing grounded theory. 3rd ed. Sage Publications, Inc; Los Angeles, Calif.: 2008. [Google Scholar]
  • 28.Rinkus S, Walji M, Johnson-Throop KA, et al. Human-centered design of a distributed knowledge management system. J Biomed Inform. 2005;38(1):4–17. doi: 10.1016/j.jbi.2004.11.014. [DOI] [PubMed] [Google Scholar]
  • 29.Aneshensel CS. Theory-based data analysis for the social sciences. Pine Forge Press; Thousand Oaks, Calif.: 2002. [Google Scholar]
  • 30.Harrison MI, Koppel R, Bar-Lev S. Unintended consequences of information technologies in health care an interactive sociotechnical analysis. J Am Med Inform Assoc. 2007;14(5):542–549. doi: 10.1197/jamia.M2384. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Jonsson AM, Willman A. Implementation of telenursing within home healthcare. Telemedicine Journal & E-Health. 2008;14(10):1057–1062. doi: 10.1089/tmj.2008.0022. [DOI] [PubMed] [Google Scholar]
  • 32.Morgan GJ, Craig B, Grant B, Sands A, Doherty N, Casey F. Home videoconferencing for patients with severe congential heart disease following discharge. Congenital Heart Disease. 2008;3(5):317–324. doi: 10.1111/j.1747-0803.2008.00205.x. [DOI] [PubMed] [Google Scholar]
  • 33.Browning SV, Tullai-McGuinness S, Madigan E, Struk C. Telehealth: Is your staff ready to implement? A descriptive exploratory study of readiness for this technology in home health care. Home Healthc Nurse. 2009;27(4):242–248. doi: 10.1097/01.NHH.0000349911.12860.f2. [DOI] [PubMed] [Google Scholar]
  • 34.Travers DA, Downs SM. Comparing user acceptance of a computer system in two pediatric offices: A qualitative study. Proc AMIA Symp. 2000:853–857. [PMC free article] [PubMed] [Google Scholar]
  • 35.Wakefield BJ, Buresh KA, Flanagan JR, Kienzle MG. Interactive video specialty consultations in long-term care. J Am Geriatr Soc. 2004;52(5):789–793. doi: 10.1111/j.1532-5415.2004.52220.x. [DOI] [PubMed] [Google Scholar]
  • 36.Campbell DT. Experimental and quasi-experimental designs for research. Houghton Mifflin; Boston, Mass.: 1963. 1916-1996. [Google Scholar]

RESOURCES