Traversing the many paths of workflow research: developing a conceptual framework of workflow terminology through a systematic literature review

Abstract

The objective of this review was to describe methods used to study and model workflow. The authors included studies set in a variety of industries using qualitative, quantitative and mixed methods. Of the 6221 matching abstracts, 127 articles were included in the final corpus. The authors collected data from each article on researcher perspective, study type, methods type, specific methods, approaches to evaluating quality of results, definition of workflow and dependent variables. Ethnographic observation and interviews were the most frequently used methods. Long study durations revealed the large time commitment required for descriptive workflow research. The most frequently discussed technique for evaluating quality of study results was triangulation. The definition of the term “workflow” and choice of methods for studying workflow varied widely across research areas and researcher perspectives. The authors developed a conceptual framework of workflow-related terminology for use in future research and present this model for use by other researchers.

Introduction

Public policy¹ and private groups² increasingly advocate use of health information technology (HIT) as an important element in efforts to transform the healthcare system, with potential contributions to patient safety, healthcare effectiveness and cost savings. For example, the Institute of Medicine (IOM) in the USA identified HIT as a key component of transitioning to a healthcare system that is (1) safe, (2) effective, (3) patient-centered, (4) efficient, (5) timely and (6) equitable.³ Despite the potential contributions of HIT, concerns about the impact of this technology on clinical workflow abound. Healthcare providers in particular often cite the impact of HIT on productivity and workflow as a potential barrier to implementation.^4–6 Researchers have also raised serious questions about HIT design and implementation strategies that may risk patient safety.^7–10 As healthcare organizations increase information technology investments,^{1 11} constructive analyses of workflow are needed to inform effective design and implementation of HIT and avoid costly implementation failures.^{12 13}

The concept of studying workflow and especially the interaction between workflow and technology has longstanding roots in industries outside of healthcare. Researchers with engineering and management perspectives such as Frederick Taylor¹⁴ and Lillian Gilbreth¹⁵ began considering workflow and efficiency in manufacturing settings in the early 1900s. Researchers including W Edwards Deming¹⁶ expanded on their early research, with a continued emphasis on industrial applications. The structured manufacturing work environment studied in much of this early research has limited similarity to complex and dynamic healthcare environments. However, researchers in fields such as sociology, psychology, engineering and computer supported cooperative work (CSCW) have continued to refine and develop approaches to study the interaction between workflow and technology. Incorporating these cross-disciplinary research concepts into healthcare workflow studies could save time and effort, with the added benefit of providing healthcare researchers with new conceptual and methodological tools for understanding work.

Our previous research demonstrated the value of evaluating the impact of HIT on workflow.^{17 18} The methods we applied included direct observation, semistructured interviews and documentation analysis. These methods proved effective but were labor- and time-intensive. As we prepared for additional research on the interaction between workflow and HIT, we sought a central resource to understand methods applied by other researchers across disciplines in studying workflow. Although there are multiple articles on workflow in healthcare and in other industries, no systematic review of the literature had been conducted to categorize and discuss different approaches to evaluate workflow. A preliminary assessment of workflow research literature revealed a wide range of workflow-related research questions and varying approaches to workflow study. We determined that a systematic literature review was an appropriate and necessary technique to understand the depth and breadth of workflow research.

We defined two primary study questions prior to beginning the study. First, what methods have been used to study workflow? Second, how have researchers ensured and evaluated the quality of the results of workflow studies? Two additional secondary research questions emerged during the study. First, how is workflow defined across research domains? Second, what components are included in definitions of workflow?

Methods

The study began with an extensive search of the literature. Eligible studies included articles published between 1 January 1995 and 1 January 2008, and were restricted to peer-reviewed sources published in English. Peer-reviewed conference proceedings were included in addition to peer-reviewed journals due to the emergent nature of workflow research.

After a thorough examination of available databases, we selected databases covering a broad range of fields incorporating engineering,^{19 20} basic sciences,²¹ healthcare²² and social sciences.^23–25 Through an iterative testing process, we developed a common set of terms for use across all of the databases, limiting the search to title and abstract fields to focus on articles with a major focus on workflow or workflow-related topics.

Information for all articles matching the search terms was retrieved, including title, abstract, date of publication, journal, database source, database unique identifier (when available) and authors. We then transferred the article information into a database to facilitate collection of article review data.

After establishing the corpus of review literature, two reviewers (KMU, LLN) pilot-tested the abstract review process, and after reviewing exclusion criteria and other elements of the review process, both reviewers independently evaluated abstracts for the full literature corpus. Final exclusion criteria categories included: focus on bioinformatics or basic science, focus on computer science or technology, focus on a medical condition, workflow was a minor part of study and not peer reviewed. The reviewers also excluded cognitive work analysis studies,²⁶ concluding that these studies engaged a well-articulated toolset based in cognitive engineering that is more appropriate to evaluate separately. We assessed inter-rater agreement for the title and abstract review using Yule's Q, as previously described by Dexheimer et al.²⁷ Any article that either or both reviewers selected for inclusion was included in the next phase of review.

The full text of all included articles was retrieved. Both reviewers independently evaluated the full text articles for inclusion, using the criteria established during the abstract review. All articles included by either or both reviewers were included in the final phase of review. Disagreements on inclusion status were resolved by consensus.

We developed and pilot-tested a form to standardize data collection for the included articles. The data collection form (see Appendix A Data Collection Form, available as an online data supplement at http://www.jamia.org) was integrated into the FileMaker database and included fields related to researcher perspective, article type, study design information, methods details and dependent variables. During the pilot phase of the abstract review, we identified widely varying definitions of workflow across studies and included a free-text field on the data collection form to capture these differing definitions.

Initial data analysis focused on descriptive statistics of key variables for the included article corpus and examining key variables for interactions, such as methodology selection trends over time. The wide-ranging review results prompted inductive analyses of text-based data fields including definitions of workflow, scope of study and dependent variable selection. NVivo qualitative analysis software²⁸ and Microsoft Excel were used to facilitate the inductive analysis.

Applying techniques developed in our previous qualitative research,^{17 18} we pursued two distinct but complementary strategies for identifying patterns in the workflow definition data. The first strategy focused on grouping workflow definitions based on researcher perspectives toward workflow, including methodological and motivational orientations. In the second strategy, we extracted key phrases based on content and context from each workflow definition and analyzed the data to identify common components that played roles in defining workflow across research fields. The two analysis strategies focused on identifying cross-disciplinary commonalities in the study of workflow, while still maintaining awareness of discipline-specific concepts.

For an extended discussion of the study methods and data analyses, see Appendix D Extended Methods, available as an online data supplement at http://www.jamia.org.

Results

Search results

The database search retrieved 6221 matching articles (figure 1). The ISI Web of Science contributed 1787 articles, IEEE Xplore contributed 1497 articles, the ACM Digital Library contributed 1459 articles, PsycINFO contributed 696 articles, PubMed/Medline contributed 473 articles, Sociological Abstracts contributed 184 articles, and IBSS contributed 125 articles. We excluded 941 duplicates. The two reviewers (KMU, LLN) independently evaluated 5280 abstracts, excluding 4477 articles and including 803 articles. The inter-rater agreement for the abstract phase of the review as determined by Yule's Q was 0.91. We extracted 23 additional articles from references and included them in the next phase of the review, resulting in a total of 826 articles for full-text review. The two reviewers independently evaluated 826 full-text articles, with an inter-rater agreement as determined by Yule's Q of 0.77. All articles included by either reviewer were then evaluated jointly, with disagreements resolved by consensus. The final corpus of papers included 127 articles (table 1).

Figure 1.

Flow of systematic literature review.

Table 1.

Included manuscripts and methods categories

Type of methods	No of included manuscripts	References
Qualitative only	65	29–93
Quantitative only	13	94–106
Mixed methods	35	107–141
Unclear	9	142–150
Not applicable	5	151–155

Analysis of descriptive statistics

Of the 127 included articles, 82 were published in peer-reviewed journals, and 45 were published in peer-reviewed conference proceedings. Year of publication ranged from 1995 to 2008 (figure 2). The researcher perspectives represented in the selected articles included engineering, social sciences, management and other perspectives (table 2). Dependent variables, or the phenomena being affected by workflow (ie, efficiency, clinical outcomes, resource allocation), were categorized along the six IOM aims for improving the healthcare system (table 3). Few studies clearly defined dependent variables, but variables were extrapolated based on article contents.

Figure 2.

Publication year.

Table 2.

Researcher perspectives

Researcher perspective	No of included articles
Computer supported cooperative work	27
Human factors engineering	24
Process and quality improvement	21
Sociotechnical	21
Industrial engineering	13
Management	13
Cognitive science	12
Other engineering	12
Computer science	9
Unknown/unclear	8
Design	6
Anthropology and sociology	3
Health services research	2
Organization studies	2

Several articles incorporated multiple researcher perspectives.

Table 3.

Dependent variables, categorized by the six Institute of Medicine aims

Institute of Medicine aim	No of articles with related goals
Efficient	64
Effective	60
Safe	38
Timely	24
Patient-centered	13
Equitable	7

Selection of multiple categories of aims was allowed.

Table 4 summarizes the design of included studies, incorporating study type, setting, subjects and length. The majority of the studies were descriptive, and a larger number were set in healthcare than in other industries. Subject selection within healthcare was divided evenly among nurses and physicians, with smaller numbers of studies including other healthcare staff members and patients. The majority of the studies were conducted over weeks or months, but several multiyear studies involved repeated data collection in the same environment to produce a longitudinal evaluation of workflow changes. The majority of the studies utilized qualitative or mixed methods. Studies frequently applied multiple methods to gather data. Table 5 summarizes methods categories and applied methods, ranging from ethnographic observation to usability techniques. For additional details of specific methods for each article, see Appendix C Analysis of Methods in Included Articles, available as an online data supplement at http://www.jamia.org. We evaluated interactions among key variables to determine if there were links between any of these variables. For example, we assessed methodology selection against date of study publication. No significant interactions among key variables were found.

Table 4.

Overview of study design for included articles

Study type	Descriptive	102
	Intervention	33
	Theory	23
	Viewpoint	22
	Literature review	9
Study setting	Healthcare	78
	Outside healthcare	49
Study setting (outside healthcare)	Manufacturing & industry	15
	Military & public infrastructure	14
	Technology design & development	8
	Offices	6
	Virtual environments	2
	Home	1
	Did not apply	10
Study subjects (healthcare)	Nurses	51
	Physicians	45
	Other healthcare staff (administrative staff, pharmacists, laboratory and radiology technicians, community-based healthcare workers)	25
	Patients	12
Study subjects (outside healthcare)	General office workers	22
	Technical staff	14
	Military & public service workers	13
	Creative workers	5
	Manufacturing workers	3
	Home	2
	Virtual	2
	Unclear	7
	Did not apply	12
Study length	Hours	8
	Weeks	27
	Months	31
	Years	8
	Unclear	40
	Did not apply	11

Several articles spanned several types, settings and subjects.

Table 5.

Method selection for included studies

Overall method type	Qualitative	65
	Quantitative	13
	Mixed	35
	Unclear	9
	Did not apply	5
Specific methods applied	Ethnographic observation	65
	Interviews	58
	Artifact collection*	29
	Structured observation†	26
	Surveys	19
	Recording‡	17
	Focus groups	15
	Software extraction§	12
	Simulation	11
	Modeling¶	7
	Usability methods**	7
	Diary††	6
	Expert panel	3
	Participant observation‡‡	3
	Discourse analysis	1

^*Artifact collection: analysis of documents, software tools, physical objects.

^†Structured observation: work sampling, task analysis, timing studies.

^‡Recording: photographs, audiotaping, videotaping.

^§Software extraction: tracking usage of specific software features, tracing flow of information through a software system, analyzing overall patterns of software use.

^¶Modeling: various approaches to creating flow charts of work processes.

^**Usability techniques: Collaborative Analysis of Requirements and Design (CARD) methodology, technology profile analysis, root cause analysis, use of a “think aloud” protocol.

^††Diary: subjects self-recorded work activity or behavior data.

^‡‡Participant observation: researcher actively participated in work activities.

For 87 of the 127 articles, strategies to ensure or evaluate the quality of study results were not explicitly addressed. For an additional eight Theory or Viewpoint articles, the concepts did not apply. For the 32 articles where we identified clear strategies for evaluating the quality of study results, multiple different approaches often were used together. Different forms of triangulation, or cross-verifying results from multiple sources, were most frequently used: methods triangulation (17 articles),^{30 34 44 45 62 64 77 80 88 89 92 95 121 126 133 137 139} researcher triangulation (seven),^{30 52 80 89 91 92 121} and subject triangulation (four).^{53 71 75 89} In methods triangulation, researchers applied multiple different methods, such as ethnographic observation supplemented with interviews, to gather data. In researcher triangulation, multiple researchers conducted the study. In subject triangulation, multiple subjects often in differing roles (ie, physician, nurse) or with other differing characteristics were studied. Reviewing and verifying findings with the study subjects, also known as member checking, was applied in eight articles.^{45 46 52 53 112 114 126 137} Researchers applied a standardized data-collection process such as work sampling, extensive training of data collectors and structured data collection approaches in seven articles,^{46 52 71 89 91 112 117} tested inter-rater reliability in three articles,^{46 71 138} and used a validated data-collection instrument in two articles.^{71 98} Researchers identified achieving data saturation, a point where collecting additional data did not change the findings, as an approach to ensure the quality of study results in four articles.^{52 80 92 133} In three articles, researchers compared computer-generated data such as simulations or data extracted from a computer system to other sources of data such as observation or self-reported actions.^{94 110 141} Sensitivity analysis was used in one study as part of verifying a workflow simulation model of podiatry services, varying multiple parameters such as staffing levels and medical condition severity.¹¹¹ Finally, one article identified their overall cross-referenced study design as a strategy to ensure the quality of study results.⁹⁸

Inductive analysis of workflow definitions: two approaches

When articles provided precise and unambiguous definitions of how the researchers viewed the term “workflow,” we recorded the definition. In cases where a clear workflow definition was not provided, we synthesized article-specific definitions based on overall article contents and article contextual factors. An example of the definitions we developed is “Process steps that are available to measure through the extant information system.”¹⁰⁰ For a list of workflow definitions for each included manuscript, see Appendix B Workflow Definitions, available as an online data supplement at http://www.jamia.org. Our first approach to analyzing workflow definitions examined data from Researcher Perspective, Scope of Study and Definition of Workflow fields, and resulted in 18 categories related to motivational and methodological orientations toward workflow (table 6).

Table 6.

Motivational and methodological orientations towards workflow

Category	Motivational and/or methodological orientation
Cognition and information processing	Information needs and cognitive processes of workers are essential elements in workflow analysis
Communication and collaboration	Workers accomplish work activities through interaction with others
Construction of meaning	People accomplish work through the creation of shared meaning
Design	Analysis of work produces insights useful for technology and work system design
Ergonomics	Contextual factors (eg, environment, task demands) impact workers on physical and mental levels
Idealized process for simulation	Developers study workflow to create idealized models of work for use in computer simulations
Interruptions	Studying the nature and impact of interruptions produces insights about workflow
Invisible work	Analysis of non-categorizable and contingent work adds to the overall understanding of workflow
Management and business process redesign	Management controls workflow, which links directly to organizational objectives
Safety and resilience	Analysis focuses on controlling elements of work impacting process safety and resilience
Systems view	Analysis of workflow covers multiple levels (eg, individual, group, environment, and technology)
Tasks and processes in the abstract	Descriptions of routine and marked tasks produce generalizable process information
Taxonomy	Elements of workflow require further definition
Temporality	Dimension of time impacts tasks, the relationships among routine tasks, and interactions among workers
Time study	Analysis of how much time specific tasks consume contributes to understanding workflow
Use of artifacts	Actors' use of technology, documents and other items provides insight into understanding overall workflow and informs the design of specific technologies
Work activities in context	Examining routine and non-routine work in the real-world context reveals the complex nature of work
Work sampling	Data on actual work activities collected at set intervals serves as an empirical basis for work analysis

In a second approach to analyzing workflow definitions, key phrases were extracted from each definition. For the previously described example, the extracted terms were: “process steps,” “measure,” and “information system.” The eight categories that emerged from thematic analysis of the data included: context, temporal factors, aggregate factors, actors, artifacts, characteristics, actions and outcomes. The context category included terms that described the work setting such as environment, culture, social context and space. The temporal factors category included terms related to timing of events including: sequence, rhythms, stages and time. Aggregate factors described terms relating to combinations of actors or events such as categories of tasks, networks, patterns, relationships, systems and work system. Artifacts included items such as documents, technology or tools used in work. Characteristics were terms used to describe work such as: articulation, behavioral, cognitive, formal, informal, personal, shared, routine, strategies and visible. The actions category incorporated specific and general activities related to work such as: allocate, balance, collaborate, communicate, evaluate, manage, mediate, plan and redesign. Finally, the outcomes category incorporated terms related to the output of work, whether physical products or virtual constructs.

Discussion

Our results demonstrated the wide range of current approaches to workflow research. The majority of the included studies were descriptive and used qualitative methods to gather data, but with many different motivations, methods and perspectives on workflow. The wide range of perspectives and motivations was expected, as we deliberately selected databases and search terms to retrieve a broad literature base. The lack of a coherent definition for workflow and other workflow-related terms presented challenges in transferring methods and findings to different contexts. We developed a model of elements defining workflow grounded in the literature review.

Purpose of workflow research

We assigned study dependent variables to the six IOM categories for health system quality improvement, which seek to develop a healthcare system that is (1) safe, (2) effective, (3) patient-centered, (4) efficient, (5) timely and (6) equitable.³ The importance of workflow research in healthcare and other fields is not always immediately apparent. Applying the IOM categories highlights the significant purposes and potential impacts of workflow research. We found efficiency and timeliness to be common dependent variables, as workflow research originates in the operations research and industrial engineering legacy of Taylor's Scientific Management approach.¹⁴ We also found an emphasis on effectiveness and safety in many studies, highlighting the important role workflow plays in quality improvement research. The small number of studies related to patient-centered and equitable-dependent variables suggests that researchers have not found value in examining questions in these areas yet. Workflow research can potentially inform all six IOM categories for health-system improvement; focusing on patient-centered and equitable variables may present an opportunity for novel research.

Many of the studies informed other processes, such as software design or business redesign. In these cases, the workflow assessment was one element of a larger project. For example, in several papers, the workflow study was part of a needs assessment during design of a software application. In other studies, looking at changes in workflow was one piece of an evaluation of a software application. While workflow studies deliver valuable information on their own, understanding the role workflow plays in the larger project is important.

The study length results demonstrate that the amount of time needed for descriptive studies is often substantial, often stretching into months. Several studies that sought to understand the evolution of a work system over time even lasted for years. While shorter studies in our literature corpus yielded helpful descriptive information, generalizing from these brief studies to other research contexts is challenging. Depending on the research goals, researchers need to be aware of the time demands of workflow-focused studies and allocate adequate time for data collection and analysis.

Selection of methods for studying workflow

A standardized approach for studying workflow did not emerge from the included literature; different methods were applied in multiple ways across multiple research fields. This is not surprising considering the lack of a coherent definition of workflow across the studies and within researcher perspective categories. The variety of motivational and methodological orientations toward workflow research (table 6) highlighted the complex and intertwined nature of method selection and purpose of workflow research. For example, a methodological focus on cognitive and information processing served the purposes of business process re-engineering in some studies and the design of informatics tools in others. This complexity results in difficulty establishing clear patterns relating to rationale for methodology selection. We observed no clear patterns linking methods to research motivation.

Qualitative methods were used in most of the included studies either alone or combined with quantitative methods. These approaches aligned well with the largely descriptive nature of the studies. Methods applied to study workflow represent a continuum of research, with open-ended ethnographic-based approaches on one end and highly structured approaches on the other. Even approaches appearing qualitative on the surface can be quantitative, depending on the design of data-collection instruments and data-analysis processes.

The variety of methods for workflow analysis and the paucity of discussions of strategies for ensuring and evaluating the quality of study results in the included articles raise the question: are conclusions about workflow in one context applicable to other settings? The included articles represent a wide variety of approaches to workflow research applied in a variety of contexts. Workflow research is intrinsically tied to context due to the interaction between contextual elements and work activities. The highly descriptive nature of workflow research may lead to perceptions that study findings are not applicable outside their specific contexts. However, well-written in-depth reports of workflow studies can provide useful insight regarding the applicability of the same methods across multiple environments or the formulation of general theories about workflow in a variety of contexts.

Determining how context-dependent a specific workflow study is represents a shared responsibility between researchers and scholarly readers. Researchers need to provide rich descriptions of methods and results, while readers need to consider whether the findings can be generalized to their own research circumstances. A rich description of contextual elements, beyond a typical brief research setting characterization, can provide readers with insight about how to apply the findings in local environments. For example, if a workflow study is conducted in an organization experiencing organizational difficulties after implementing a new electronic medical records system, findings could be relevant to other organizations considering implementing similar technology. Determining relevance to other contexts requires an in-depth description of the contextual factors such as organizational structure, technology features and work practices prior to introducing the new technology as well as a thorough description of data-collection and analysis methods. Without access to thorough and in-depth descriptions of context and methods, applying findings of workflow research across contexts involves making risky assumptions about the relevance of research findings to the target setting.

Furthermore, only a small percentage of the included articles unambiguously discussed steps to ensure and evaluate the quality of study findings, which raises concerns about conclusions based on the research. Addressing study quality, even in a purely ethnographic approach or in an exploratory study, is part of a rigorous approach to ensuring that findings are representative of the real situation and that conclusions are faithful to the data. An open discussion of techniques to address quality, such as triangulation of methods, is crucial to include when describing workflow research findings.

Developing a conceptual framework of workflow-related terms

There are many different perspectives on the term “workflow.” Definitions of workflow often focus on static processes that can be fully captured by a flow chart. Terms such as “workflow management systems” and “workflow solutions” are used in business to describe approaches to automate repetitive processes, again promoting a static and linear view of workflow. In computer-supported cooperative work, workflow is viewed as an evolving and continuously changing set of processes. While some elements of workflow may be static, the overall workflow of an individual, work group or organization is dynamic. Exceptions, such as interruptions, surprises and unintended consequences, play a significant role in this dynamic view of workflow.

Because of the myriad definitions of the term workflow, lack of precision in language when discussing workflow presents challenges in understanding the purpose and findings of workflow research. Identifying a precise definition of workflow during design of studies and dissemination of research results would assist others in understanding the purpose and impact of the research. Considering context is also critical, as context is an intrinsic part of workflow. A standardized picture of “normal” workflow is difficult to ascertain in exceptions-driven fields like healthcare. A flow chart can capture expected behavior, rules and routines but fails to present a full picture of the complex adaptive and dynamic nature of healthcare. As a result, definitions of workflow appropriate to the context being studied should be developed and applied.

Several other workflow-related terms have similar degrees of ambiguity in definition and use, including “work system,” “modeling,” “work practices,” and “work processes.” The term “model” in particular had two divergent definitions. From one perspective, a model was considered a representation where measurements against the model could be tested for statistical significance. In the second design-oriented perspective, a model was an abstract representation of relationships among real-world actors, activities and artifacts. Each perspective can present valuable insights into workflow, but models should be evaluated against the appropriate expectations.

We analyzed the definitions of workflow in the included studies and developed a conceptual framework of elements to consider including when studying workflow regardless of field, the Workflow Elements Model (figure 3). The model has two levels: pervasive and specific. The pervasive level includes three components that apply throughout specific elements of workflow: context, temporal factors and aggregate factors. Context constrains and enables workflow. Considering context is critical in workflow studies including the physical workspace, the virtual workspace and organizational factors. The concept of temporality involves scheduling, temporal rhythms and coordination of events, and is important on individual, work group and organizational levels. Aggregate factors are the relationship and interaction among different tasks and actors, including elements of coordination, cooperation and conflict. The specific level is composed of: the people performing actions (actors), the physical and virtual tools the actors are using (artifacts), specific details of the actions being performed (actions), characteristics that describe the actions (characteristics) and the end products of the actions (outcomes). Other factors outside of our model and not directly related to workflow also potentially contribute to the outcomes.

Figure 3.

Workflow elements model (based on analysis of workflow definitions).

The relationship among these elements and the importance of the various elements in the analysis of workflow depends on researcher perspective, dependent variables, research questions and contextual factors. We developed the Workflow Elements Model to provide a flexible structure for consideration by researchers designing and reporting on workflow studies. The model captures attributes of workflow repeatedly discussed in workflow literature across contexts, research fields and industries. Determining which workflow elements to focus on when applying the model to future research studies is a dynamic process intrinsically linked to each individual research study. Our goal in presenting this model is to highlight commonalities across research domains and to stress the importance of terminology usage when designing and reporting on workflow research.

Methodological opportunities on the horizon

The current state of workflow research in healthcare presents a clear opportunity for cross-disciplinary research. Utilizing concepts and methods from different research perspectives and contexts can deepen and strengthen the conclusions of workflow research in healthcare. Considering design thinking as being complementary to science thinking¹⁵⁶ rather than being in opposition can also aid in this pursuit. For example, combining the multilevel ethnographic approach toward workflow with the linear task-oriented approach of business process redesign can yield information on both the static routine elements of workflow and dynamic exceptions from the routine. Acknowledging the contributions of differing perspectives will paint a deeper and more accurate portrait of workflow.

Our current work involves applying the results of this literature review to our ongoing workflow research projects. The lack of clarity in existing literature regarding definitions of workflow stressed the importance of clearly defining workflow-related terminology as we design and report on research studies. The analysis of methodological and motivational orientation has assisted us with method selection and research design. The literature review also identified a clear need to rigorously define, implement and report on strategies to ensure and evaluate workflow research quality, which we have carried forward in our own research studies. We also applied the Workflow Elements Model to a variety of organizations and clinical contexts participating in the MidSouth eHealth Alliance, a Regional Health Information Organization in Memphis, TN. The model assisted researchers with categorizing patterns of technology-related workflow across contexts, and we continue to consider revisions to the conceptual model as we apply it in research practice.

Study limitations

The open-ended questions that motivated this review resulted in enormous logistical challenges due to the high number of matching abstracts. The two reviewers coordinated the study through a customized database that blinded the reviewers during early stages of the review and then allowed collaboration to finalize conclusions in the end stages of the review. The electronic tools enabled us to easily adhere to our analytical objectives and to follow-up on interesting topics that arose during data analysis. While the inter-rater reliability was high for the abstract review phase, the inter-rater reliability was lower for the full paper review phase. Adding a third reviewer may have strengthened the review process. Significant variability in how different fields view the concept of workflow presented challenges in determining common elements of workflow. Reviewers considered conceptual and theoretical differences across fields when examining article-specific definitions of workflow and incorporated contextual factors into the content analysis process. There are many additional workflow-related terms such as “routines” and “coordination” that were not included in our search criteria due to logistical constraints. Future studies could focus on these additional terms to cover additional research areas.

Conclusion

Cross-disciplinary workflow research presents enormous opportunities for improving the fit between technology and work. The first step toward cross-disciplinary research in this area is understanding the many different perspectives toward and definitions of workflow. Most existing workflow research focuses on descriptive studies and applies qualitative or mixed methods. Workflow is often studied as one element of a multistage research or design project. Although different fields view the concept of workflow differently, there are many common elements of importance to evaluate when studying workflow. Based on these common elements, we developed a conceptual framework of workflow components and have applied this conceptual framework to our ongoing workflow research across multiple contexts. We also plan on continuing to expand our understanding of the various methodological and motivational orientations toward workflow as we design future healthcare workflow studies.

The current state of workflow research can be compared to cartography. Like maps that differ in what they highlight (eg, political divisions, topography, population density, etc) and in scale, current methods for studying workflow highlight different attributes of work and are applied at different scales. Some methods are better suited to specific types of work depictions, but all of the methods have potential contributions. Just as one would not use a population density map to determine the height of a mountain, using a time-and-motion study to examine communication practices makes little sense. Selecting appropriate methods to fit research goals shapes the outcome of workflow research. Communicating these research goals and describing the appropriateness of the methods to the goals creates a useful key to the workflow research map.