Interactive Workshops in a Pandemic: The Real Benefits of Virtual Spaces

With the global pandemic preventing travel, and with the expectation that many workers will continue to work from home after the pandemic, the scientific community faces the challenge of how best to organize joint workshops in the future. In this article, we share our experiences of organizing online workshops. We discuss possible approaches for connecting participants in virtual settings, and creating environments that support productive collaboration online.

Due to the COVID-19 pandemic, many scientific conferences in 2020 were conducted in the form of virtual events, which pose a great challenge for conference-related activities. Workshops specifically might suffer from this transition, as they typically provide a forum for networking, information exchange, and intensive group-based collaboration. Conducting workshops virtually presents inevitable barriers to experiencing fully fledged, in-person interactions and exchanges with colleagues. However, virtual formats also have unique advantages. For example, there is no travel budget necessary to participate in a virtual workshop, making them more accessible to a broader range of potential attendees. Furthermore, online workshops might be better positioned to employ digital services for work and collaboration than physical events. The reason is that the use of technology is sometimes unpopular at in-person meetings, which often focus on physical or pen and paper prototyping. Thus, despite all hope that conducting physical workshops will be possible again soon, future events could try to integrate positive aspects from both worlds and, thereby, benefit from hybrid formats.

In this article, we reflect on two virtual workshops that we held during the ACM AutomotiveUI 2020 conference, where we aimed to provide participants an experience similar to in-person meetings while making use of digital services for group collaboration. In particular, we elaborate on relevant differences, advantages, and disadvantages of virtual and physical workshops, discuss which important aspects of in-person meetings can be substituted with virtual activities (and which ones cannot), and how they might change collaboration in the future. Furthermore, we examine the impact of virtual workshops on organizing committees who have to deal with issues of different time zones and durations of activities. The lessons learnt through organizing these events provide an outlook that could enrich workshop experiences for both scientific and educational purposes in the future.

VIRTUAL WORKSHOPS—A RECAP

Organizing a scientific workshop involves various tasks before, during, and after the main session (see Figure 1 for an overview of workshop-related activities). Broadly, those activities can be categorized into: “networking” (participant introductions, coffee breaks, breakout groups, postworkshop dinner), “presentation” (presentation of position papers and breakout group results), “collaboration” (brainstorming, prototyping), and “organization” (preparing material to be consumed before the workshop, reviewing, schedule preparations, wrap-up, postworkshop activities).

FIGURE 1.

Typical process of interactive workshops. We categorize each activity as organization (yellow), networking (green), collaboration (red), or presentation (blue).

We organized two workshops at the 2020 AutomotiveUI conference: The “Workshop on Virtual Reality (VR) in Automated Vehicles: Developing and Evaluating Metrics to Assess VR in the Car,”¹ and “AutoWork 2020: Second Workshop on the Future of Work and Well-Being in Automated Vehicles” ²—both of which were delivered virtually using Zoom.us as the video-conferencing platform, where we tried our best to convey the in-person experience in a virtual setting. In the following sections, we will discuss our attempts in line with the activity categories earlier.

ORGANIZATION

Virtual and physical workshops are no different when it comes to receiving and reviewing position papers. However, novel challenges in organizing a workshop began with the preparation of a schedule for the main session. First, we had to deal with much shorter main sessions compared to the typical half- or full-day length of physical workshops. Remote conferences are usually extended through multiple time zones; hence, the allocated time for virtual workshops are more limited than in-person ones.

The 2-h session format was the default format of the AutomotiveUI 2020 conference, and this is what the VR workshop followed. In contrast, we split AutoWork into a 1-h session during a conference day, and another 2-h session, which we held about a month later. We hoped that participants would develop some ideas during the initial 1-h session, and then spend time between the initial and second sessions to finalize their contributions from group work activities.

We put an emphasis on group activities at both workshops, at the cost of detailed topic introductions and longer position paper presentations. This decision can increase the organizational workload for both the organizers and the participants; in the VR workshop organizers prepared videos for participants to watch beforehand and we asked participants to introduce themselves (more on this in the "Networking" section) and to choose their breakout group asynchronously, before the workshop. However, we believe that only a small group of participants watched the prepared videos, and we had to allocate time in the beginning of the session for tasks that were intended to be conducted asynchronously. Hence, our experience is that expecting participants to perform outside-session activities may not be realistic, especially if the participants think they have enough background knowledge about the workshop topics.

NETWORKING

Considering the short timeframes for the main workshop sessions and the absence of coffee breaks or postworkshop dinners, a big challenge was finding ways for participants to introduce themselves and get to know each other. Thus, we invited participants to provide introductions for themselves before the session—either by submitting a short video in the AutoWork workshop, or by leaving content on a Miro board (a collaborative digital whiteboard platform) in the VR workshop. However, this attempt was not successful as most participants did not provide this content in advance. Thus, we had to spend some initial time at both workshops for introductions—for AutoWork, we utilized multiple series of one-to-one breakout sessions (speed-networking), whereas for VR, we had small breakout groups. Still, this may have felt problematic for some participants as we could not know which ones were already familiar with each other. Consequently, participants sometimes made introductions during the (already limited) time in the breakout sessions that we designed for collaborative work (discussed next).

COLLABORATION

For brainstorming and prototyping in both workshops, we split the sessions into breakout groups of 4–5 people including an organizer in each group for guidance. We used a single Miro board for all groups for brainstorming (prepared with specific board regions for different groups), and the groups that were communicating solely within their breakout rooms could see the progress of the other groups. For this part, we were quite satisfied with the process as it conveyed an experience similar to individual groups working on different desks in the same room: Each group could see as other groups posted new notes, or added images, and several attendees commented that they actively looked at progress in other groups.

However, we experienced a distinct challenge in encouraging participants, many of whom are experts in their respective fields, to actively contribute to the discussions. Technical constraints enable participants to justify partial engagement; that is, some participants would contribute with video and audio functions disabled, on occasion responding solely via Zoom's chat facility. It appears that more so than in a physical setting, the role of the organizer pivoted more heavily toward actively fostering engagement from and between participants. The unintended consequence of a more involved role of the organizers was that each organizer's unique background played a more prominent role in the breakout groups’ course of discussions, activities, and final result. Still, we believe such guidance was important to keep the breakout groups’ work on track. However, this necessitates an unusually high ratio of organizers to participants, given the reduced capacity for a single organizer to visit multiple groups rapidly, as one might do in a physical workshop setting.

PRESENTATION

One might think that the presentation and discussion of position papers and the general topic can easily be substituted with a video platform like Zoom. However, the short time allocated to each session forces a shorter presentation time, and long sessions and presentations may result in increased cognitive load and “Zoom fatigue.” This may be problematic for participants new to the topic, since they do not have the basic theoretical foundation to participate in the discussions. That is why we decided to prepare the basic background knowledge as an asynchronous activity for the VR workshop.

The other set of presentations were placed after group work, where the sessions were concluded with presentations for all workshop groups. The AutoWork workshop did not end after the first session, and participants were directed to Slack, a group chat application, to continue their discussions until the second session. At this second session, we had to re-mix groups, since a number of participants from the first session did not join this second session. This also forced us to build up new groups, after we as organizers summarized the results of the previous session. Strictly speaking, our expectation of participants reflecting on the results of the first session and collaborating before the second session was not realized.

LESSONS LEARNED

The prospect of engaging with the international community without leaving your place of residence, let alone your home or workplace, seems appealing. However, there are unintended consequences. Still, we have learned a lot from our experience organizing these workshops.

Do not put extra workload on the participants. Our initial belief that participants will provide introductions for themselves and read/watch topic-related materials was not fulfilled. Rather than requiring participants to engage with material that is delivered externally to the workshop session, we should expect participants to flexibly complete material depending on their existing familiarity with the topic. An important factor here is how participants prioritize nonworkshop-related activities. At physical conferences, attendees usually free their schedule of many tasks that are not related to the conference; however, these tasks persist during remote conferences and participants have to fit the conference in their daily lives.

Provide asynchronous participation options. Although this suggestion may sound contradictory given earlier, we still see advantages in providing material in advance. While this places a greater demand on organizers to offer a range of material that could end up not being used by participants, the outcome is a small library of resources that participants may benefit from after the session has ended. Overall, we are observing a trend toward the workshop experience stretching out before and after the designated session time. We have utilized such methods already in prior physical workshops, and the option of asynchronous engagement may simply be a feature that participants will take time adjusting to.

Think about the workshop goals and invite participants accordingly. If the goal of the workshop is to accomplish specific outcomes, the organizers may want to limit the attendees to those with relevant background knowledge, given that the organizers should not expect them to get up to speed asynchronously. However, there would be no such limit if the goals are more in line with networking and socializing.

Networking is hard. Workshop participation presents an opportunity for attendees to develop meaningful connections with other members of their field. Indeed, building these connections can be considered part of the value prospect of attending such workshops in the first place. Bringing participants together at a single moment in order to foster real-time discussions is already difficult, but providing a proper format for networking experiences similar to in-person meetings is even harder. We tried our best in providing networking opportunities like “virtual speed networking” or prerecorded introduction videos, but must acknowledge that this part may be the biggest challenge for virtual workshops in the future.

Virtual collaboration can be enriching. Although the online Miro/Zoom environment did not allow attendees to develop spontaneous, unstructured connections with their peers, we consider it a good substitution for physical breakout group activities. Digital collaboration tools like Miro, Overleaf, or Google Docs are fairly widespread nowadays, which lowers the entry-barrier of usage. We utilized Miro as a digital whiteboard as it offers all essential tools for collaborative brainstorming, like placing sticky notes, drawing, or voting. Feedback from workshop participants was that one big board, with all participants’ cursors floating around was particularly fun and helped them activate, instead of keeping groups separate within their own documents. Furthermore, providing a persistent virtual space allows participants to continue collaborating, exchange contact details, and communicate about the outcomes of their experience. The use of such digital tools also provides a unique advantage for organizers: The workshop content and results are already digital, and ready to be reused in another virtual workshop. In contrast, it takes longer for in-person workshops to wrap up and store the results of physical, hands-on-prototyping, or they may end up lost and thrown away.

What about hybrid formats? We expect that in-person workshop will return, and we hope this happens soon. However, we also tried to imagine future hybrid workshop formats connecting researchers all over the globe, as there are enormous potential gains to be had from fostering collaborative relationships with individuals who may not otherwise be able to attend an in-person workshop. Based on our experience with the virtual workshops, there are challenges associated with virtual workshops that need to be addressed as we work toward organizing hybrid workshops. “Going hybrid” will pose additional challenges, such as an emerging barrier between participants on- and off-site. Moving forward with these issues in mind, it might be necessary to rethink our understanding of the role of workshops in conferences—for example, it is possible that workshops can develop into focal points of community-building that start with an in-person meeting at a conference, and continue as a sequence of virtual meetings over a longer period of time after the conference.

CONCLUSION

The COVID-19 pandemic has forced the scientific community to change and adapt how colleagues interact and share research. Many conferences have adopted virtual formats to allow research to continue moving forward and help keep colleagues connected during trying times. Although workarounds have been developed for oral and poster presentations, there are still lessons to learn about hosting virtual workshops. Some of the common challenges that were encountered in the workshops described in this article included: Considerations of timing constraints for both the organizers and participants, determining expectations from participants to engage in asynchronous activities, and identifying tools to keep participants engaged. Additionally, with virtual workshops, we lose opportunities to socialize and network during and afterward. The interactions participants have during a virtual workshop are not organic—they are planned, timed, and infrequent compared to in-person workshops. Future workshop organizers should carefully consider ways to promote more socialization during and after the workshop to avoid losing important networking opportunities.

Even with these challenges, virtual workshops present valuable opportunities for researchers, and this will be the case even after the pandemic. The virtual format allows participants who cannot travel to the conference for different reasons (i.e., funding, visa, location, etc.) to participate and contribute to the work being done. It gives organizers an opportunity to recruit from a wider pool of participants and connect people from all over the world and different scientific communities. Virtual workshops also help reduce the environmental impact of traveling, since it does not require all participants to travel. Although there are many benefits to bringing researchers physically together for a workshop, we are just beginning to uncover the benefits and potential of hosting events virtually. As a community, we will have to continue to adapt and create better ways to collaborate, share ideas, and educate both in virtual and in hybrid physical/virtual settings. The pandemic has compelled everyone to innovate in light of a challenging situation, forcing us to rethink what the future of conferences and workshops entails.

Biographies

Zoe M. Becerra is currently a Design Researcher for the Experience Strategy and Design Team with Anthem, Inc., Indianapolis, IN, USA. Contact her at zoebece@gmail.com.

Nadia Fereydooni is currently working toward the Ph.D. degree in human-centered computing with the Georgia Institute of Technology, Atlanta, GA, USA. Contact her at nadia.fereydooni@gatech.edu.

Andrew L. Kun is currently a Professor of electrical and computer engineering with the University of New Hampshire, Durham, NH, USA. Contact him at andrew.kun@unh.edu.

Angus McKerral is currently working toward the Ph.D. degree in psychology with the University of Newcastle, Callaghan, NSW, Australia. Contact him at angus.mckerral@uon.edu.au.

Andreas Riener is currently a Professor of human–computer interaction and virtual reality with Technische Hochschule Ingolstadt, Ingolstadt, Germany. Contact him at andreas.riener@thi.de.

Clemens Schartmüller is currently a Human–Computer Interaction Researcher with Technische Hochschule Ingolstadt, Ingolstadt, Germany. He is currently working toward the Ph.D. degree in computer science with Johannes Kepler University Linz, Linz, Austria. Contact him at Clemens.Schartmueller@thi.de.

Bruce N. Walker is currently a Professor of psychology and of interactive computing with the Georgia Institute of Technology, Atlanta, GA, USA. Contact him at bruce.walker@gatech.edu.

Philipp Wintersberger is currently a Researcher with the CARISSMA Institute of Automated Driving, Technische Hochschule Ingolstadt, Ingolstadt, Germany. Contact him at philipp.wintersberger@carissma.eu.

Funding Statement

The work of Andrew L. Kun was supported in part by NSF under Grant CMMI-1840085. The work of Clemens Schartmüller and Andreas Riener was supported by the "Innovative Hochschule'' program of the German Federal Ministry of Education and Research (BMBF) under Grant 03IHS109A (MenschINBewegung). The work of Philipp Wintersberger was supported under the FH-Impuls program of the BMBF under Grant 13FH7I01IA (SAFIR).