Killer Apps: Developing Novel Applications That Enhance Team Coordination, Communication, and Effectiveness

Abstract

As part of the Lorentz workshop, “Interdisciplinary Insights into Group and Team Dynamics,” held in Leiden, Netherlands, this article describes how Geeks and Groupies (computer and social scientists) may benefit from interdisciplinary collaboration toward the development of killer apps in team contexts that are meaningful and challenging for both. First, we discuss interaction processes during team meetings as a research topic for both Groupies and Geeks. Second, we highlight teamwork in health care settings as an interdisciplinary research challenge. Third, we discuss how an automated solution for optimal team design could benefit team effectiveness and feed into team-based interventions. Fourth, we discuss team collaboration in massive open online courses as a challenge for both Geeks and Groupies. We argue for the necessary integration of social and computational research insights and approaches. In the hope of inspiring future interdisciplinary collaborations, we develop criteria for evaluating killer apps—including the four proposed here—and discuss future research challenges and opportunities that potentially derive from these developments.

Imagine two towns, one populated by Geeks and the other populated by Groupies. In Geektown, there is a particular tribe who works on social signal processing and affective computing. Every day, the people from this tribe go to their labs where they develop sophisticated algorithms that can extract and structure information from huge data sets of social interactions. They discover interesting patterns and relations in their data sets and have high hopes that their algorithms could be helpful for someone outside their little town.

Not too far away in Groupietown, there is another tribe of scholars, Groupies, trying to understand how observable social interaction processes in groups and teams influence performance and other group outcomes. They have diverse backgrounds including anthropology, communication, organizational behavior, sociology, and social, industrial, and organizational psychology. To answer their main question, the Groupies travel all over the globe to record groups and their interactions through audio and video. Back in Groupietown, they spend hours and hours of diligent work to code each observed behavior in these interactions. They have excellent theoretical explanations of how specific behaviors could contribute to effective team dynamics, as well as performance, but they are growing tired of spending days and weeks analyzing the minute details of the observed group interactions.

From an outside perspective, both groups seem to work on very similar problems, albeit from different perspectives—one favors what can be done with technology and the other favors behaviors that drive groups and teams. For example, both the Geeks and the Groupies are trying to understand the behavioral patterns underlying smooth group coordination and performance. The methodological approaches taken to address such questions are different, however. Whereas in Geektown the scholars rely on machine-learning principles, the scholars in Groupietown employ human raters who annotate human interaction processes. Both approaches have potentials and pitfalls, as discussed in detail in this special issue. Here, we focus on the practical implications of the collaboration between Geeks and Groupies toward developing feasible technologies that can be used in everyday practice and at the same time inform research. Such technologies have been termed killer apps.

Killer app (noun). An extremely valuable or useful computer program; a computer application of such great value or popularity that it assures the success of the technology with which it is associated; a feature or component that in itself makes something worth having or using. (Merriam-Webster, 2016).

Killer apps in the context of understanding and improving group processes can push both computer and group science toward a better cross-disciplinary collaboration and integration of research. In addition, they can offer new possibilities for interventions that affect our daily lives. In particular, killer apps (a) offer novel opportunities for real-time diagnostics, visualizations, and feedback on group and team processes in field settings (e.g., for improving coordination and performance in surgical teams); (b) provide advice for team managers and team members themselves for optimizing interactions (e.g., conducting more effective meetings); (c) enable organizations by providing decision guidelines for how to best compose, develop, and manage teams at work (e.g., improving decisions related to team design); and (d) trigger a radical shift in the accuracy and prognostic value of human resource planning and development efforts in organizations and society more broadly.

We propose that Geeks and Groupies need to collaborate to develop such killer apps. They need to look beyond the boundaries of their respective disciplines and visit each other to identify ways to communicate in each other’s language, pool their knowledge, create shared research questions and intervention interests, and create the building blocks necessary for developing killer apps. Based on our experiences from the Lorentz workshop that preceded this white paper, we expect that Geeks and Groupies will quickly recognize that they share more common ground than previously assumed. Nevertheless, the willingness and effort to “leave their home town and suspend native beliefs” to better communicate with scholars of the other discipline are important antecedents for enabling the development of killer apps.

The present article elaborates on this idea by outlining the basic requirements and criteria for a killer app and highlights four possible contexts in which such technologies could be developed to the benefits of Geeks and Groupies, firms and institutions and, eventually, society at large. We begin by pointing out how social and computer scientists are fundamentally working on the same problems and highlight several striking overlaps between the two disciplines. Next, we provide concrete examples of potential killer apps in four diverse group settings. Specifically, we discuss how organizational team meetings, surgical team interactions, team design, and massive open online courses (MOOCs) might benefit from killer apps. These different group contexts allow and require both disciplines to collaborate to advance the development of killer apps. Finally, we outline how this collaboration could result in specific technological applications and provide evaluation criteria that are considered essential for true killer apps, from an integrated Geek and Groupie perspective.

Building Blocks: Interdisciplinary Insights for Killer Apps

Over the past decades, social scientists have contributed to a better understanding of the relevance of behavioral group interaction dynamics. For example, we now know that observable verbal behaviors in group discussions can be contagious (e.g., Kauffeld & Meyers, 2009), that micro-behavioral patterns in group discussions relate to emergent group mood (e.g., Lehmann-Willenbrock, Meyers, Kauffeld, Neininger, & Henschel, 2011) and that affectively laden behavioral interactions influence cooperation and performance in groups (e.g., Barsade, 2002; Jung, Chong, & Leifer, 2012; Lehmann-Willenbrock & Allen, 2014). Social scientists have also shown that behavioral patterns in health care teams predict clinical team performance and patient safety (e.g., Kolbe et al., 2014), that micro-behaviors and behavioral mimicry can be used to distinguish leadership styles (e.g., Lehmann-Willenbrock, Meinecke, Rowold, & Kauffeld, 2015), and that early interaction patterns set the tone for team performance (Zijlstra, Waller, & Phillips, 2012). Researchers have also established that communication among team members reveals individual and group processes, often based on roles (Ervin, Bonito, & Keyton, 2017).

At the same time, computer scientists have developed the computational foundations to enable machines to analyze subtle human communicative behaviors during social interactions. Machines can be used to process continuous behavioral data that humans create through speech signals and natural language in social interactions (Narayanan & Georgiou, 2013). Operationalized through software, machines can be used to automatically analyze interactive behaviors in groups of children (Moreno & Poppe, 2016), to predict behavioral responses of participants during negotiations (Park, Scherer, Gratch, Carnevale, & Morency, 2013), to improve speaking in front of an audience (Wörtwein, Morency, & Scherer, 2015), to recognize and classify human gestures and interaction (Chiu, Morency, & Marsella, 2015; Song, Morency, & Davis, 2013), to support decision making in health care (Stratou & Morency, 2016), to detect psychological stress in interactive conversations (Venek, Scherer, Morency, Rizzo, & Pestian, 2017), and to predict leadership roles and behaviors in group discussions (Scherer, Weibel, Oviatt, & Morency, 2012).

Social signal processing researchers have addressed the challenges in the computational analysis of verbal and nonverbal behavior from sensor data (see Vinciarelli, Pantic, & Bourlard, 2009, for an overview). With video cameras, the missing depth information introduces challenges in dealing with variation in camera viewpoint, lighting condition, and (partial) occlusions. Currently, the limited video resolution and frame rate further complicate the data capture and analysis of facial and body motion. For the analysis of verbal behavior from audio recordings, the common occurrence of recording noise demands more complex processing algorithms.

Overall, social scientists have made important theoretical and empirical contributions to our understanding of how groups can work effectively, and how specific behaviors and conditions contribute to better interactions. To reach these conclusions, social scientists use different methods (questionnaires or surveys; audio and/or video recordings) to document observations of group interactions both in the field and in lab experiments. In particular, observational methods or video-recorded observations provide rich information on the micro-dynamics that are an integral part of interactions in groups (e.g., Waller & Kaplan, 2016). Social scientists examining micro-interactive behaviors are following the affirmative action for action suggestion by Baumeister, Vohs, and Funder (2007) who suggested that scholars should “try to put a bit more behavior back into the science of behavior” (p. 401). Likewise, Moreland, Fetterman, Flagg, and Swanenburg (2010) lamented that behavioral assessment was “gradually disappearing from [social] psychological research on groups” (p. 47).

One reason why behavioral studies of groups and teams are diminishing is the significant time and resources needed for the systematic study of micro-behaviors. The coding of behavioral interactions typically demands a substantial amount of resources due to the hours of human coding effort that go into each observational study. These disadvantages were also acknowledged in the opinion piece by Baumeister et al. (2007) who noted that “a failed behavioral study is an expensive failure and could even be a major career setback” and that “journals do not seem to give extra points or consideration to studies that observe behavior instead of just getting ratings” (p. 399). This is not just an issue for behavioral research but also for advancing diagnostic and intervention tools and methods that can promote productive interactions in groups and teams.

At the same time, research in social signal processing has led to impressive advances in the robustness of automated algorithms for the detailed measurement of behavior. While initial efforts focused mainly on data collection and analysis at the individual level, there is a shift toward interactions in groups. Moreover, researchers are increasingly focusing on the added value of interpreting, rather than merely observing, behavior. This is a more subjective task that often requires contextual knowledge. For example, a measured smile can be a sign of happiness, frustration, or relief. To distinguish between these semantic annotations, knowledge of the team members, the interaction context, and the mechanisms of how behavior manifests itself is required. To this end, social signal processing researchers are increasingly using computational models to interpret the automated measurements that can be captured to arrive at a higher-level, semantic interpretation of the behavior of an individual group member or the group as a whole. These computational models are typically derived from the social science theories that describe how interpretable concepts relate. When made quantitative, such models can lead to computational algorithms for automated analysis of higher-level behaviors (e.g., in terms of affect, relations between people and interaction outcomes). However, the adoption is not straightforward, as behavior theories often contain subjective, qualitative concepts that are difficult to operationalize quantitatively through objective measurements. A tighter collaboration between social signal processing and social science researchers could bridge this gap, and benefit both communities.

In terms of identifying specific topic areas for killer apps, we focus on four different group settings, ranging from small groups in organizations to large online collaboration groups. These group interaction contexts allow us to point out both the benefits and the challenges of interdisciplinary collaboration on the topic of team interactions. Moreover, the four identified topic areas will require novel insights into interactional dynamics, as well as conceptual and methodological integration efforts among Geeks and Groupies. Specifically, in a best-case scenario, the envisioned killer apps described below would allow us to live in a world in which (a) workplace team meetings are intensely productive and even enjoyable (i.e., not stressful or a waste of time); (b) surgical teams perform without error; (c) optimal teams can be composed, based on comprehensive information about member characteristics and their interdependencies; and (d) communication among learners of MOOCs flows easily and collaboration is highly effective (see Table 1).

Table 1.

Application Capabilities and Intervention Opportunities.

Application context	Functions and capabilities	Example intervention opportunities
Team meetings	Automatic assessment of motivational and affective states	Improved time management and meeting chairing
Surgical teams	Analyze current deviations from protocols and provide real-time interventions; detect shared awareness and shared mental models	Using automated alarms for avoiding human errors; increasing patient safety
Team design	Guide decisions for optimal team design using complementary and supplementary fit principles	Improved team functioning and effectiveness
MOOCs	Automatic assessment of individual retention and engagement	Prevention of student dropout, based on early indicators of withdrawal behaviors

Note. MOOCs = massive open online courses.

First Wish: A Killer App for Optimizing Team Meetings

Meetings are an integral part of daily work activities in organizations—with over 25 million meetings per day in the U.S. alone. On average, employees spend 6 hr per week whereas managers spend 23 hr per week in meetings (e.g., Allen, Lehmann-Willenbrock, & Rogelberg, 2015a). Unfortunately, there is ample evidence that about 42% to 50% of these meetings are considered ineffective and thus a waste of time (e.g., Schell, 2010). Researchers have found that meeting design factors and specific behaviors in meetings are negatively related to meeting quality (Cohen, Rogelberg, Allen, & Luong, 2011), meeting effectiveness (Leach, Rogelberg, Warr, & Burnfield, 2009), meeting satisfaction, team productivity, and organizational success (Kauffeld & Lehmann-Willenbrock, 2012). Overall, meetings in organizations are not only relevant from a practical point of view but they are also an important topic of academic study (for an overview, see Allen, Lehmann-Willenbrock, & Rogelberg, 2015b). In the following, we will outline why meetings are interesting from a Groupie as well as from a Geek perspective, and discuss the interdisciplinary collaboration requirements for creating a killer app for use in workplace meetings.

Why Meetings Are a Groupie Problem

Meetings are a management instrument that helps to promote knowledge distribution and decision-making processes in organizations (e.g., Allen et al., 2015a; Van Vree, 1999). From a Groupie perspective, a meeting constitutes a specific organizational event that brings together several individuals (i.e., a group or a team) to discuss work-related issues, engage in group problem solving and sense making, generate new ideas, build consensus for making decisions, and ultimately promote the implementation of individual or organizational actions (e.g., Rogelberg, Shanock, & Scott, 2012). As such, meetings provide a unique lens into a wealth of sociopsychological phenomena, such as information distribution (Tepper, 2004), strategy discussions (Beck & Keyton, 2009), and the generation of new ideas and innovation (Hodgkinson, Whittington, Johnson, & Schwartz, 2006). Participants can also engage in negotiations and bargaining (Boden, 1995), establish socioemotional relations (Nielsen, 2009), and exert social influence during meetings (e.g., Lehmann-Willenbrock et al., 2015). Given these multiple purposes, meetings are highly suitable for studying group interactions.

Meetings can provide unique insights into organizational culture, hierarchy, and sociointeractional dynamics. In other words, meetings are a petri dish for studying small group phenomena and teams “in the wild” (Salas, Cooke, & Rosen, 2008, p. 544). For example, meetings can provide a research context for investigating how team interaction patterns shape team performance (e.g., Kauffeld & Lehmann-Willenbrock, 2012; Lehmann-Willenbrock & Allen, 2014; Lehmann-Willenbrock, Chiu, Lei, & Kauffeld, 2017), how different team roles emerge through behavioral patterns (Lehmann-Willenbrock, Beck, & Kauffeld, 2016), or how affective states of team members are behaviorally expressed and converge over time (e.g., Jung et al., 2012; Paulsen, Klonek, Schneider, & Kauffeld, 2016). From a Groupie perspective, a killer app would be able to automatically detect and summarize participants’ verbal and nonverbal behaviors during meetings such that the multitude of interdependent individual- and group-level behaviors of team members become more easily accessible. Among others, these include team member negotiations, mutual social influences, emergent leadership, contributions of novel ideas, the ebb and flow of emergent group mood, conflict management practices, and behavioral signals that indicate necessary team interventions.

Why Meetings Are a Geek Problem

Meetings provide a relatively controlled setting for the automated study of group behavior. In large-scale research projects such as AMI (Augmented Multiparty Interaction), AMIDA (Augmented Multiparty Interaction With Distant Access), and CHIL (Computers in the Human Interaction Loop), researchers have focused on the automated analysis of vocal and visual behavior in meetings, including speech recognition, estimation of gaze direction, and the recognition of conversational gestures (e.g., Carletta et al., 2006). In the AMI project, a large multimodal data set of 100 meetings has been recorded and annotated for a large variety of verbal and nonverbal behaviors. These recordings have been used to address research questions related to turn-taking (e.g., Rienks, Poppe, & Heylen, 2010), dominance (e.g., Hung, Huang, Friedland, & Gatica-Perez, 2011), attention and social signals (e.g., Poel, Poppe, & Nijholt, 2008), and the convergence of intrapersonal and interpersonal processes (e.g., Ervin et al., 2017).

For the development of automated algorithms, the analysis of recordings of meetings has several marked advantages over using “in the wild” recordings from arbitrary sources. The analysis of audio is more easily facilitated, as noise from outside is controlled or limited. Regarding the visual analysis of behavior, controlled lighting conditions reduce the variation in the visual appearance of the face and body. Also, during most meetings, participants are seated and therefore occupy a fixed location in the observational space. Typically, participants have an unobstructed view of all others.

From an initial focus on the verbal and nonverbal behaviors of individuals, researchers have moved on to the analysis of behaviors in interaction with others. Often, interactions with others help us to interpret the behavior of the individual. In this sense, the direction of processing is the other way around as it is common in social science research. Computer scientists take a bottom-up approach by first analyzing individual, isolated behaviors and then infer how the sum of these behaviors constitute group behavior. To analyze groups based on individual behaviors, computer scientists either infer group behaviors directly using pattern recognition algorithms or use a model of group behavior.

When behavior is classified using pattern recognition algorithms, there is no explicit underlying model of how behaviors relate to automated measurements. Rather, this relation is determined automatically by analyzing a collection of data, also known as training data. It is assumed that the patterns in the occurrence of behaviors in the training set are representative of those found in the more general application domain.

Instead of learning a computational model from data, a model can be employed that prescribes how single and simple behaviors give rise to the observation of more complex behaviors. Such a computational model is typically derived from theory and simplifies how behaviors in groups are related to one another.

With either approach, analyses on different aspects of group behavior can be performed automatically. How can these computational advancements contribute to our understanding of human interactions in workplace meetings? On one hand, these analyses provide insights in group behavioral processes after the meeting has finished. For example, a killer app using speaker diarization could automatically produce transcripts of a meeting to facilitate documentation and detailed protocols. Other analyses from the audio and video recordings allow for the study of turn-taking patterns, number and type of speech acts, and the role of gestures. This could help to better understand how participants’ behaviors influence meeting outcomes. Such analyses provide insight into how communication works and can be used to validate (computational) interaction models (Reidsma et al., 2007). On the other hand, if the automated analysis of behavior is performed in real time, or with a minimal delay, these analytic techniques would allow meeting coaches or consultants to provide direct feedback during the meeting. For example, by analyzing who speaks when, a killer app could signal (e.g., by producing a buzz or other sound) if a participant is occupying too much speaking time. By taking gaze direction into account, a killer app could detect when participants are not actively involved or lose attention.

Second Wish: A Killer App for Improving Coordination in Surgical Teams

The outcome of teamwork in emergency rooms (ERs) is of vital importance. It is therefore not surprising that there is an increasing demand for black boxes, equipment that records actions in an ER with the aim to review, postsurgery, what has happened (Guerlain et al., 2005). Currently, these black boxes are merely recording tools and the focus is on the recording of equipment that has been used during the surgical procedure. An important aspect that is currently not being observed and analyzed systematically is the group behavior of the surgical team (Lingard, Reznick, Espin, Regehr, & DeVito, 2002). There is an increasing focus on standardizing the communication behavior in operating rooms, for example, using Crisis Resource Management (CRM, Carne, Kennedy, & Gray, 2012). Communication within an ER is of vital importance, and misunderstandings due to miscommunications or a lack of shared awareness can have serious effects on the outcome of the surgical operation. By analyzing the verbal and nonverbal behavior of a surgical team, we can study group behavior patterns in relation to surgical procedure quality, but also in relation to team satisfaction.

Why Surgical Teams Are a Groupie Problem

Surgical teams can be described as ad hoc action teams that operate in a high stakes environment and have particular responsibility for patient safety. Because problems in verbal communication constitute a major source of surgical errors (e.g., Lingard et al., 2004), medical action teams are an inherently interesting field of study for Groupies. Groupies are particularly interested in how team members implicitly and explicitly coordinate their behaviors, decisions, and performance (e.g., Burtscher, Kolbe, Wacker, & Manser, 2011; Tschan et al., 2006, 2009) and under which conditions team members point out errors in the procedure that might be crucial for safety reasons and ultimately have adverse or even mortal consequences for patients (Kolbe et al., 2014). For example, group researchers have meticulously analyzed video-recorded anesthesia and surgical teams to understand how noise affects communication processes in the operating room (Keller et al., 2016), how behavioral interaction patterns differ between high- and low-performance groups (Kolbe et al., 2014), how teams’ shared understanding of the operational task moderates the relationship between monitoring behaviors and team performance (Burtscher et al., 2011), and how they have classified the sheer amount of communication failures that might jeopardize patient safety (Lingard et al., 2004). Therefore, from a Groupie perspective, a killer app could serve to point out problems in implicit and explicit behavioral coordination between team members, track frequencies of verbal exchange between team members, highlight if team members’ attention decreases, and even reduce overall team workload by monitoring inaccurate behavioral actions of team members.

Why Surgical Teams Are a Geek Problem

Similar to meetings, ERs are relatively controlled in ambient settings. Moreover, technology in ERs is common, which facilitates the unobtrusive placement of cameras and microphones to study the behavior of surgical teams. When observing the team members, the faces are typically largely covered by face masks. Moreover, surgeons and other ER personnel have a tendency to look down at the patient, or at equipment. This hinders the view cameras have on their faces and consequently prevents a robust analysis of facial expressions. This puts the focus on the analysis of the head and the body.

Surgical operations are typically structured, both in task execution and in the communication among members of the surgical team. Still, this leaves for variation between teams. One particularly interesting avenue for Geek research is to analyze whether procedures are strictly adhered to. For example, when safety procedures or communication protocols are violated, this might introduce risks to the patient, with potentially dramatic effects. Using automated technology, we can analyze these behavior and communication patterns in the moment they occur. Offline, we can analyze whether procedures have been followed, and where improvements are possible. This can also lead to a desire to further train personnel to adopt communication guidelines. Eventually, such analyses could also lead to improved protocols.

A second avenue for research into automated behavior analysis is to examine shared awareness among surgical team members. Communication might be missed or misunderstood by some team member. By analyzing turn-taking patterns, we could identify when the addressee did not hear or understand what was said, or when someone is waiting for an opportunity to say something. By analyzing the smoothness of conversation and by analyzing the performance of each team member—in isolation and in relation to team progress—we could identify team members that have reduced awareness of the current situation.

Third Wish: A Killer App for Optimizing Team Design

A considerable body of research on strategic human resource management (SHRM) has examined the relation between HRM and firm performance (for a meta-analytical overview, see Combs, Liu, Hall, & Ketchen, 2006). In the past decades, the focus of attention has shifted from the individual to the team, as teams promise to tackle many of the challenges modern organizations are faced with (e.g., Hollenbeck, DeRue, & Guzzo, 2004; Kozlowski & Bell, 2003; Richter, Dawson, & West, 2011). For example, strong competitive pressures and fast-paced change within volatile markets force firms to innovate rapidly and make decisions under uncertainty conditions. However, the overall effect of teamwork on organizational effectiveness is weak at best (see meta-analytical evidence by Combs et al., 2006; Richter et al., 2011), as teams are not always effective. This variability of effect sizes for the relation between teamwork and performance suggests the presence of contingency factors determining whether teamwork yields high performance (Richter et al., 2011).

One such contingency factor is a team’s composition, in terms of characteristics such as demographics, educational background, personality, attitudes, and values (e.g., Buengeler & Den Hartog, 2015; Van Knippenberg & Schippers, 2007). Optimizing team composition through team design is thus an important way to increase team effectiveness (Humphrey, Hollenbeck, Meyer, & Ilgen, 2007). In the following, we outline why team design is interesting from a Groupie as well as from a Geek perspective, discuss the interdisciplinary collaboration requirements for creating a killer app enabling effective team design, and suggest interventions based on a team’s composition.

Why Team Design Is a Groupie Problem

Team design is an important (HR) management tool because a favorable team composition relates to improved decision making and problem solving (Van Knippenberg & Schippers, 2007), creativity and innovation (e.g., Van Dijk, Van Engen, & Van Knippenberg, 2012), and performance (e.g., Bell, 2007; Humphrey et al., 2007; Humphrey, Hollenbeck, Meyer, & Ilgen, 2011). Moreover, the right team composition can help reduce or avoid some of the disadvantages often associated with teamwork, including conflict, free riding, and conformity (e.g., Harrison & Klein, 2007; Humphrey et al., 2007).

When thinking about ways to optimize team composition, Groupies have focused either on increasing the average level of a valuable characteristic (e.g., mental ability or agreeableness) or on optimizing the dispersion of a characteristic, such as team members’ conscientiousness or extraversion (e.g., Barrick, Stewart, Neubert, & Mount, 1998; Harrison & Humphrey, 2010; Humphrey et al., 2007; Prewett, Brown, Goswami, & Christiansen, 2016). Last, for certain characteristics, the minimum or maximum value of a characteristic in a team may be relevant (e.g., physical ability in a manufacturing team; see also Steiner, 1972, for a taxonomy of team-task requirements). To decide which of the three options to pursue (i.e., average levels, dispersion, or minimum/maximum of a characteristic), Groupies draw from a wealth of theoretical knowledge and empirical insights.

For instance, team composition research suggests that extraversion—a personality trait describing individuals’ tendency to be sociable and outgoing, dominant, and assertive (Costa & McCrae, 1992; John & Srivastava, 1999)—is conducive to team cohesion and job performance when there is complementarity among the members (e.g., Barrick et al., 1998; Prewett et al., 2016). When some members score high on extraversion and others score low on extraversion, this allows for optimal role differentiation and reduces conflict. Similarly, team diversity with respect to the functional backgrounds present in a team has been linked to more innovation when certain conditions are present (Van Dijk et al., 2012). Organizations oftentimes deliberately design for diversity to stimulate creativity and innovation by bringing together individuals from different functional backgrounds (e.g., sales, marketing, engineering, IT in a cross-functional new product development team).

However, previous Groupie research has also shown that variability in traits such as conscientiousness—individuals’ tendency to be purposeful, organized, achievement-oriented, and self-disciplined (Costa & McCrae, 1992)—is negatively related to team performance (e.g., Barrick et al., 1998; see also Humphrey et al., 2007). Rather, team design should seek to achieve high average levels of conscientiousness given their positive relation with team performance (Bell, 2007).

However, team design may not always be feasible and does not automatically guarantee team effectiveness (e.g., Bell, 2007). Therefore, teams need to be actively enabled to make use of their respective design. Diversity research has shown that although differences among team members broaden the pool of informational resources, this potential advantage oftentimes remains unused given unfavorable categorization of dissimilar others, subgroup formation, and conflict (Van Knippenberg & Schippers, 2007). To avoid these issues and reap the benefits of diversity, teams may require a diversity training to learn how to constructively deal with the differences inherent in their design (Homan, Buengeler, Eckhoff, van Ginkel, & Voelpel, 2015).

Concluding, viewing team composition from a Groupie perspective can inform HR decisions on team design and may prove useful for selecting appropriate interventions for a given team composition. What is missing, however, are technological solutions for integrating and applying this knowledge toward usable recommendations for team design and related interventions.

Why Team Design Is a Geek Problem

From a technical perspective, team design brings new modeling challenges for interpersonal and group dynamics. The technical approach needs to be able to handle longitudinal data efficiently. How can we summarize computationally the behaviors of multiple team members over a long period of time? Currently, computational models typically keep only short-term memory. For an effective application in team design, these models need to be extended to be able to identify long-range dependencies (e.g., a person’s behavior today can impact the team a week later).

A further challenge comes from the use of knowledge about a team member’s personality and the relations between members. Especially in the interpretation of (group) behavior, both factors play an important role. For example, when team members know each other well, it is not uncommon to observe much more heated brainstorm sessions. When only the behaviors of the team members are considered, distinguishing a heated from an aggressive brainstorm session becomes much harder.

A third challenge comes with the privacy. We want to be able to recognize behavioral indicators for successful teams while respecting the privacy of team members. This will require the creation of new acoustic and visual features that are user generalizable and keep as little information as possible about the participant’s identity. For example, we want to recognize the facial expressions of the team members without necessarily keeping the information about face identity (i.e., face recognition). When considering interpersonal behaviors, the amount of congruency in the facial expressions of team members is an example of a privacy-aware representation of visual behaviors.

Fourth Wish: A Killer App for Analyzing Team Collaboration in Distributed MOOCs

The landscape of learning and education is changing rapidly with the introduction of online courses and MOOCs (Pappano, 2012). These new venues for learning comprise large catalogs of courses and topics available for free or at a much lower cost than traditional venues. They allow students from remote areas to learn from top professors about topics that may not be taught at their own university or college. The list of providers of online courses has increased dramatically over the past years, with new entities such as Coursera and Udacity dedicated only to online learning.

While these developments represent great opportunities for students and the academic community in general, they also come with new challenges. Students are attending online courses remotely and do not have face-to-face interactions with the professor or peers. One area that is particularly challenging from a team perspective concerns group homework assignments performed remotely by students. This setup implies a substantial challenge regarding how new technologies might support collaborative processes within MOOC student teams. As a first goal, we want to be able to build productive teams that can work and collaborate remotely. As a second goal, we also want these teams to be able to adjust their collaboration strategies during online meetings in real time. And finally, we need technologies to be able to assess the progress of online teams, both from an academic point of view and from the perspective of practical collaboration.

Why MOOCs Are a Groupie Problem

As the body of knowledge on computer-mediated and virtual teams demonstrates, teams comprised of students in MOOCs may experience particular benefits, but also particular pitfalls compared with more traditional face-to-face student groups. MOOCs create a challenging environment, as the interaction is happening remotely and sometimes asynchronously (Gibson & Cohen, 2003). The lack of proximity can lower familiarity and friendship, and induce misunderstandings, which can contribute to easily escalating conflict (e.g., Hertel, Geister, & Konradt, 2005; Hinds & Bailey, 2003). However, virtual collaboration bears considerable advantages including lowered traveling time, coordination, and cost, allowing MOOC participants to access learning opportunities that otherwise would not have been available. In addition, factors hindering effective learning (e.g., social inhibition in creative tasks; Connolly, Jessup, & Valacich, 1990) may be suppressed in MOOCs.

Setting up MOOCs in ways that keep potential pitfalls in check while utilizing the unique benefits of virtual group work requires a Groupie perspective. For instance, virtual groups require more clarity concerning goals and member roles than face-to-face groups (e.g., Hertel et al., 2005; Montoya-Weiss, Massey, & Song, 2001). Given the strongly reduced availability of communication cues, MOOCs need effective communication and coordination to avoid interpersonal conflict. Virtual groups such as MOOCs also require performance-related and socioemotional feedback (e.g., using computerized tools), as well as opportunities for informal communication (e.g., Hertel et al., 2005; Weisband, 2002). One way to achieve this, according to previous Groupie research, would be to promote connectedness among MOOC members, for instance via team-based rewards, interdependent goals, or task design that promotes collaboration (e.g., Hertel, Konradt, & Orlikowski, 2004). Ideally, there should be introductory workshops that enable virtual collaboration in MOOCs (e.g., Warkentin & Beranek, 1999). In sum, solid Groupie knowledge exists regarding strategies that can help ensure effective virtual collaboration. Currently, however, integrative, optimally real-time technological solutions built on these strategies are missing.

Why MOOCs Are a Geek Problem

MOOCs represent an excellent environment for developing new technologies to recognize and record human behaviors during social interactions. Students collaborate remotely using computers that can easily be instrumented to sense their actions and behaviors. For example, in remote Skype interactions, each student is facing a camera and uses a microphone to be heard. This allows the automatic digitalization of both audio and video and can be used as data for computer vision, speech analysis, and natural language processing algorithms. In other words, all the input modalities of a MOOC are already digitalized during these interactions.

Notwithstanding the excellent opportunities of this environment for automatic recognition of collaborative behaviors and affective phenomena, there are numerous technical challenges that should be addressed to promote students’ learning experiences. First, to overcome the challenges of the remote interaction setting, nonverbal behaviors such as facial expressions, gaze, and speech features of each student need to be recognized in real time. We face challenges in how recognition algorithms are designed so they can perform with minimal processing delays. Research is needed to understand the trade-offs between recognition accuracy and real-time processing. Second, nonverbal behaviors need to be analyzed in the context of the verbal content expressed by each student. For example, a student with furrowed eye brows may simply be showing signs of strong attention or, if a new concept was recently explained verbally, the student may be expressing confusion. The interpretation of nonverbal behaviors is typically more accurate when put in the context of the associated verbal content.

Finally, the verbal and nonverbal behaviors of all students need to be modeled jointly to better infer collaborative behaviors and detect possible problems. Some social states such as rapport (i.e., a close and harmonious relationship; Huang, Morency, & Gratch, 2011) are to a large extent characterized by the dyad or small group interacting together. Recognizing these social behaviors requires that we model the verbal and nonverbal behaviors of all students at the same time. This is an understudied research question in social signal processing and artificial intelligence in general.

The Importance of Integrating Needs and Fields of Expertise

To address the needs of Geeks and Groupies trying to understand and influence group meeting effectiveness, improve surgical team coordination, optimize team design, and increase the learning success of student groups in MOOCs, killer apps need to be based on interdisciplinary insights from both fields. Groupies alone typically lack the technological expertise and knowledge to automatize the detection of behaviors and behavioral patterns in real time. Geeks alone often lack the underlying theories of social behavior in groups that are necessary for making sense of the extracted data and also for creating practically feasible group interventions. When the skills and expertise of both fields are successfully integrated, this sets the stage for creating killer apps that can move both fields forward as well as advance practice.

To illustrate, we focus on group interactions during workplace meetings here. A killer app for workplace meetings might offer ways to automatically detect fluctuations in group affective states during the meeting (i.e., when is the mood more or less pleasant in the meeting room) and visualize feedback to the participants. These fluctuations can predict broader constructs such as group cohesion and group performance. To develop such a killer app, the application would require input from Groupies in terms of (a) the behavioral indicators of group affective states; (b) behavioral indicators of group cohesion; (c) methods for establishing theory-based, reliable coding schemes, and procedures, including the establishment of interrater reliability among human coders; and (d) objective measures of team performance. Geeks would need to supply expertise on (a) extracting and quantifying relevant audiovisual signals, (b) specifying which ground truths (i.e., information about behaviors provided by direct observation as opposed to inference) are necessary for developing algorithms that can reliably automatize behavioral measurement, (c) adapting and improving algorithms, and (d) providing the necessary technological expertise for visualizing information on meeting interaction patterns to develop feedback tools.

Toward an Evaluative Framework for Killer Apps

While the possible ideas for killer apps may be abundant, the decision whether the development of a particular killer app is worthwhile should be based on a set of evaluation criteria that speak to the academic and practical needs of both Geeks and Groupies, organizations and institutions, and society at large. To derive these evaluation criteria for killer apps, the authors of this article, who share a research interest in behavioral group interaction processes and have relevant expertise working with audiovisual behavioral data, collectively brainstormed and discussed many killer app opportunities.

This team of authors (and the focus group members for the discussions reported here) interacted face to face, with one team member serving as moderator, and discussed the following question: What are the (most) important evaluation criteria when developing a killer app that is meaningful and valuable for both social and computer scientists? The group engaged in 5 hr of discussion total. To ensure that contributions of all participants were documented and to incorporate feedback by the entire group, the focus group compiled a list of the most important evaluation criteria and presented it during a plenary session, followed by additional focus group work. In addition, all contributions were documented via a live online protocol. The focus group discussion resulted in the following four criteria for evaluating killer apps: (a) impact (breadth and strength), (b) time to market, (c) return on investment (ROI), and (d) feasibility and access to funding opportunities.

Impact: Breadth and strength

The first evaluation criterion is the impact of a killer app, consisting of two components. First, the breadth of a killer app captures the extent to which the application can impact different groups of individuals in the general public or society more broadly, as well as different academic fields. In terms of the impact on the general public, we can ask two types of questions to establish the impact breadth of a killer app: (a) What is the target audience for the killer app? and (b) How large is each audience, and what are the individual, group, organizational, and societal implications, respectively? Moreover, we can ask about the breadth of a killer app in terms of its scholarly impact. For example, will there be a meaningful impact within a scholar’s own specific line of research, within a research community (i.e., subgroup within a discipline), within an entire discipline, or across a range of disciplines?

The second component of the impact criterion concerns strength, which quantifies the extent of the impact on each of the target groups as identified by the questions above. For example, in terms of organizational impact, how much does the use of a killer app simplify work procedures and thereby enhance employee safety and well-being? In terms of scholarly impact, how many citations does a manuscript using a particular killer app generate? In terms of broader societal impact, to what extent does the implementation of a killer app affect quality of life in a population?

Time to market

The second evaluation criterion is time to market, or the length of time that is needed from the conception of a killer app until its application in the field. For instance, whereas minor technological advancements may only take a few weeks, major advancements could have times to market of 1 year, or even 5 or 10 years. Killer apps codeveloped by Geeks and Groupies require intense collaboration, demanding considerable time investments. Ethical and legal considerations are important as well. Research projects typically depend on the assessment by ethics committees, and need to adhere to legal demands. Because the proposed killer apps combine assessments of different types of sensitive information (e.g., physiological, attitudinal, assessment of traits), and interventions based on these assessments, we can expect extensive ethical assessments by the scholars’ universities to avoid potential harm for participants. Depending on the respective killer app, time to market may thus vary.

Anticipated ROI

A third criterion to evaluate a killer app concerns the ROI, or the quantified monetary benefit from an investment of a resource. To determine the ROI of a killer app, researchers must account for (a) costs of invested time, development costs, investment in the actual technology, and marketing activities on the one hand and (b) generated value on the other hand (e.g., time savings, generating objective data, making better decisions and saving money as a result). For the four potential killer apps proposed in this article, the ROI can only by anticipated. Hence, we assigned values of 1 (low anticipated ROI) to 4 (high anticipated ROI; see Table 2).

Table 2.

Positioning Four Potential Killer Apps in the Evaluation Framework.

Killer app	Societal impact	Scholarly impact	Time to market	Anticipated ROI	Funding opportunities
Team meetings	2	2	5 years	1	2
Surgical teams	4	2	10 years	3	3
Team design	1	3	5 years	2	1
MOOCs	3	3	3 years	3	3

Note. Values denote the extent to which each killer app fulfills each criterion, 1 = weak impact to 4 = profound impact. ROI = return on investment; MOOCs = massive open online courses.

Feasibility and access to funding opportunities

The fourth and last evaluation criterion for killer apps concerns feasibility and access to funding opportunities. Assessing feasibility involves estimating whether the respective killer app can be made (assuming reasonable amounts of effort). This is closely related to access to funding opportunities, denoting the likelihood with which funds can be attracted to support the development of the killer app. Without the latter, feasibility will be lowered even though the killer app could be developed from a technological point of view. When a killer app’s feasibility and access to funding opportunities is estimated to be low, it is assigned a 1 (difficult/rather unlikely). In contrast, a score of 4 denotes that a killer app is feasible, and highly likely to receive funding. Table 2 summarizes the evaluations of the four killer apps according to the four criteria (impact, time to market, anticipated ROI, and funding opportunities).

Discussion

In this article, we have outlined how mutual research endeavors of computer and social scientists can be used to promote technological advancement in four practical settings that fundamentally require effective interactions of small groups and teams. First, we outlined how automated behavioral signal processing could be applied to improve group interactions in organizational meetings. Second, we elaborated how the computation of behavioral signals can improve teamwork and context awareness in health care settings. Third, we highlighted how an automated solution for optimal team design could add to the effectiveness of work teams, contributing to more SHRM. Fourth, we provided examples of how social and computational researchers can advance our theoretical and practical understanding of team effectiveness in the area of MOOCs.

Outlook and Directions for Future Research

Importantly, we want to emphasize that this article only provides a preliminary glimpse of potential killer apps, and related opportunities, that can result from interdisciplinary work among Geeks and Groupies. We hope to have shared our enthusiasm for the rich potential inherent in such collaborations. Yet we would also like to point out that the four examples discussed here are important but not exhaustive. Our selection of the four application contexts reflects the expertise and research interests of the authors. However, other scholars may very well think of many other potential killer apps and application contexts. For instance, others may think about Geek-Groupie collaborations toward killer apps for diagnosing and preventing cyberbullying in small online groups, for promoting member well-being in groups or for predicting and preventing aggression among soccer fan clubs.

An important question for future research concerns the ways in which new killer apps may feed into new research opportunities, in addition to their practical relevance. We have focused our discussion of the four application contexts—from a Groupie and a Geek perspective, respectively—on existing research challenges. However, the development of a killer app may also generate novel, unforeseen research questions and challenges.

For example, consider the killer app for team meetings. When a killer app provides real-time feedback on team behaviors to a team leader (e.g., through a screen, wristbands or sound), a follow-up research question concerns the ways in which such feedback is utilized for reflection and intervention purposes. For instance, feedback information from the meeting killer app will likely create substantial cognitive load for team leaders who wish to improve their team’s interaction processes. In addition to being involved in the meeting himself or herself, the meeting leader would simultaneously need to monitor and process the information from the killer app. Again, both Geek and Groupie expertise will be needed to address such challenges, in terms of different ways of modeling and visualizing behavioral traces, anticipating and preventing cognitive load issues, and creating conditions that are conducive to team development more broadly.

Another important challenge for future research that likely applies to all of the potential killer apps concerns the ethical issues involved in the extensive data gathering. Ethical issues are particularly important with respect to recorded individuals and their right of confidentiality. In addition to this, the transparency of a meeting killer app that provides real-time information on the (individual) behavior of group members may promote intensive impression management among team members (for an overview, see Leary & Kowalski, 1990). Alternatively, team members may become anxious and/or withdraw from the team interaction to avoid being constantly monitored. As a second example, students participating in MOOCs may worry that their behaviors are used as performance criteria, which may demotivate some students, severely disrupt the online discussion process, and ultimately undermine the killer app’s potential to promote group functioning and student learning in these environments. One way to address such concerns resulting from ethical issues would be to involve the intended end users in the development of killer apps to ensure application is practical and not overly burdensome. Pilot projects would be a necessity. Such measures would hopefully eliminate the ethical backlash.

A third challenge surrounding the development and implementation of killer apps relates to change management more broadly. As any change initiative, the implementation of a killer app as a general practice requires careful communication before, during, and after the implementation to prevent resistance to change (for an overview, see Ford & Ford, 1995; Todnem By, 2005). The anxiety related to extensive data gathering by means of killer apps can be avoided when conditions are in place that promote transparent information about the aims and functions of a killer app, enhance the active involvement of users (e.g., by asking for user feedback to improve a killer app), and communicate the individual and team benefits of a killer app that cannot otherwise be attained. To address these challenges, Geeks and Groupies need to work together to identify critical technical as well as psychological and social factors that promote willingness to actively engage with a killer app. Pilot projects can be helpful in this regard to anticipate and solve problems before a killer app is implemented more broadly.

Particularly exciting new research opportunities result from the copious amounts of behavioral data that can be generated by killer apps in field settings. While the need to consider behavior and interaction dynamics continues to be a challenge for Groupies (e.g., Waller & Kaplan, 2016), the development of reliable algorithms in the field can be challenging for Geeks. Close interdisciplinary collaboration and pilot projects can help address these challenges and fine-tune a killer app such that the obtained data are both practically useful and scientifically sound.

A final point for future research concerns the selection of an evaluation paradigm for the effectiveness of a killer app. The evaluation criteria discussed in this article (see Table 2) can guide decisions to invest in the development of a killer app. However, after such a decision has been made, researchers and practitioners should still strive to establish the effectiveness of an implemented killer app (e.g., in terms of improved team productivity during meetings, reduced patient mortality in the ER, increased student learning in MOOCs, improved team effectiveness). To test whether such outcomes are indeed promoted by a killer app, an ideal evaluation design would involve an experimental group that uses the killer app in the intended application context, as well as a control group in the same context that does not use the killer app. Ideally, there should also be an additional condition in which more traditional feedback tools are used, such as using a team trainer who provides brief feedback at the end of a team meeting. Randomized control trials are preferable. If the killer app is effective, the control group should receive access to the technology. Once an effect of a killer app has been established, long-term effects should also be considered where applicable. For example, when the team interaction dynamics during a meeting have significantly benefitted from the feedback provided by the killer app, the question remains to what extent these positive changes can carry over into the next meeting, or into a team meeting 6 months down the road (i.e., short or near transfer; e.g., Blume, Ford, Baldwin, & Huang, 2010). Insights into these different short- and long-term effects can inform the further refinement and continuous improvement of a killer app.

Author Biographies

Claudia Buengeler is an assistant professor of human resource management and organizational behavior at the University of Amsterdam, Netherlands. She examines determinants of team effectiveness, including leadership, training, and team composition.

Florian Klonek is postdoc at the University of Western Australia, Centre for Transformative Work Design. He uses video recordings to study sociointeractional and motivational dynamics in teams and dyads to understand the linkages between social behavior, work design, and the organization.

Nale Lehmann-Willenbrock is an associate professor of work and organizational psychology at the University of Amsterdam, Netherlands. She investigates dynamic linkages and emergent behavioral patterns in organizational teams as well as leader–follower communications, using interaction analytical methods.

Louis-Philippe Morency is assistant professor in the Language Technology Institute at Carnegie Mellon University, United States, where he leads the Multimodal Communication and Machine Learning Laboratory. His research focuses on building the computational foundations to enable computers to analyze, recognize, and predict subtle human communicative behaviors during social interactions.

Ronald Poppe (PhD, University of Twente) is an assistant professor at the Information and Computing Sciences department of Utrecht University, Netherlands. His research interests include the analysis of human behavior from videos and other sensors, the understanding and modeling of human (communicative) behavior, and the applications of both in real-life settings.