Skip to main content
NCBI home page
Springer Nature - PMC COVID-19 Collection logoLink to Springer Nature - PMC COVID-19 Collection
. 2022 Dec 7;28(6):7631–7649. doi: 10.1007/s10639-022-11495-6

Promote collaborations in online problem-based learning in a user experience design course: Educational design research

Meina Zhu 1,✉, Ke Zhang 1
PMCID: PMC9735013  PMID: 36532793

Abstract

Since the Covid-19 pandemic, there has been an increasing growing need for learning experience or instructional designers. As a result, online courses on user experience (UX) design for learning are in demand to prepare those much-needed professionals. This paper reports the first circle of educational design research (EDR) on such a completely online problem-based course using group contracts and peer-evaluations to promote collaborations. Multiple datasets were collected from a range of data sources from a graduate-level course and were analyzed by two researchers. The study yielded several meaningful results: (1) group contracts helped students work smoothly and keep goals focused on collaborative learning in PBL; (2) the instructor could facilitate the group formation process; (3) peer evaluations helped group collaboration and monitoring; and (4) collaborative technologies are critical for collaboration. The findings also revealed important design principles for collaborative learning in PBL that can be used in similar courses. In addition, this study provides a successful example of how to teach UX design in an online PBL environment, and sheds light on how to further improve similar practices while contributing to the limited body of research on UX design for learning.

Supplementary Information

The online version contains supplementary material available at 10.1007/s10639-022-11495-6.

Keywords: Problem-based learning, Collaborative learning, Online education, User experience design for learning, Human–computer interaction, User experience design

Purpose

Online learning as an instructional delivery format has been around for approximately three decades. However, due to the COVID-19 pandemic, the need for online learning strategies exploded when over 1.5 billion students and youth, 87% of all students worldwide, were suddenly faced with university and school closings as of March 2020 (UNESCO, 2022). As a result, the course modality switched to online education at many schools, colleges, and universities (Gardner, 2020). Many instructors suddenly realized that they needed to change their approach to engage students who were often physically separated from each other. One of the strategies to engage online learners and develop higher-order thinking skills (Delialioğlu, 2012) and learning achievement (Schmidt et al., 2011) is problem-based learning (PBL). PBL, in which learners collaborate in a group to solve problems, has great potential to develop learners’ critical thinking, creative thinking, and problem-solving skills, and to support knowledge acquisition (Hmelo-Silver, 2004; Lajoie et al., 2014). A typical PBL assignment involves identifying problems, collecting and analyzing data, negotiating with group members, reflection, and presentation of the results (Glazewski & Ertmer, 2020; Kim et al., 2016). Since learners take the primary responsibility for their learning in PBL, they need to be motivated and engaged, and effectively and collaboratively manage and monitor learning progress and seek information and support. However, research has shown that learners often do not have these skills (English & Kitsantas, 2013). Group members typically encounter challenges, such as social loafing (Lee & Lim, 2012), lack of motivation, negative attitudes, lack of responsibility for learning, and management, all of which hinder effective PBL (English & Kitsantas, 2013). Therefore, learners need instructors to facilitate and support group collaboration and problem-solving skills through scaffolding (Savery, 2006).

Moreover, PBL could be one of the great approaches for courses on learning experience (LX) design. LX is an interdisciplinary area that involves both user experience (UX) design and instructional design (ID) (Schmidt et al., 2020). As a new, emerging area, it is critical to prepare future LX designers with the necessary skills to develop courses, particularly online. However, very little research and few practical examples exist for online courses focusing on LX/ UX design. This educational design research (EDR) study aimed to fill this gap by addressing two essential aspects: (1) provide an authentic example of such a course, and (2) conduct the first circle of EDR and explore selected strategies in the course. This study also explores students’ perceptions of their collaborative PBL. The primary aspects of this research include: (1) designing and developing a course focusing on UX design for learning, (2) using various methods to evaluate the effectiveness of the instructional approaches, (3) grounding findings in theory, and (4) establishing general design principles.

Theoretical framework

User experience design for learning

LX design consists of two primary interactions: (1) with the learning environment and (2) with learning spaces (Tawfik et al., 2022). Interactions with learning environments include customization for learning, expectation of content placement, functionality of the components, interface terms aligned with learners’ existing mental models, and navigation. And the interaction with the learning space consists of engagement the content, dynamic interaction, perceived value of technology affordance for learning, and scaffolding (Tawfik et al., 2022). LX design typically leverages UX design approaches to improve learning. UX refers to “a person’s perceptions and responses that result from the use or anticipated use of a product, system, or service” (International Organization for Standardization [ISO], 2010, Terms and Definitions section, para 2.15). UX originated and has been used in multiple disciplines, including human–computer interaction (HCI), interaction design, and cognitive psychology (Gary, 2020). However, in education, UX design for learning has been relatively limited compared to the more common user-centered design (UCD) approaches for learning design (Baek et al., 2008; Ebner & Holzinger, 2007; Fernandez-Lopez et al., 2013).

UX and traditional ID have some similarities, including a focus on UCD, as well as differences such as the processes and concepts. Gary (2020) compared the terms and conceptual vocabulary used to describe design activities in ID and UX and emphasized that the two fields have used different terms to describe design processes. For example, ID has traditionally included analysis, design, development, implementation, and evaluation, and UX has included user research (personas, scenarios, contextual inquiry, ethnographic engagement, interviews), prototyping (sketches, wireframing), and user testing (Gary, 2020).

To create an effective learning experience, one of the approaches is to leverage UX processes and approaches for LX design. However, teaching strategies in UX courses vary. Consequently, it is important to create effective courses on UX design for learning. Given that both theory and practice are essential in the UX field (Churchill et al., 2013), UX courses typically include both lectures and laboratory sessions (Koppelman & Dijk, 2006; Moroz-Lapin, 2009) so students can apply the theories in practical projects. For practice, students typically work on practical projects in groups of 3–6 students (Culen et al., 2014; Urquiza-Fuentes & Paredes-Velasco, 2017). More specifically, some UX courses have leveraged PBL to help students solve authentic problems in education (Koutsabasis & Vosinakis, 2012; Nordahl & Serafin, 2008).

Problem-based learning

PBL, based on constructivism theory, is an instructional approach that emphasizes learners’ autonomy, integrates theory and practice, and implements knowledge and skills to solve ill-structured authentic problems (Barrows & Tamblyn, 1980; Savery, 2006). In PBL, students work in small groups to learn and solve problems, and instructors are facilitators (Hmelo-Silver, 2004). PBL has several unique characteristics: (1) students have ownership of their own learning, (2) the problems are ill-structured, (3) it involves multiple disciplines, (4) students collaborate in a group, (5) PBL enables students to apply individual knowledge to solve problems, (6) the learning process is monitored, (7) PBL involves self-and peer- assessment, (7) the problems are authentic, and (8) both the knowledge and process are important (Savery, 2006). In online learning, PBL has been used to enhance students' higher-order thinking, knowledge acquisition, and learning outcomes (Delialioğlu, 2012; Hmelo-Silver, 2004; Lajoie et al., 2014; Schmidt et al., 2011). In addition, group collaboration plays an important role in enabling students to take responsibility for their learning and collaborate successfully with peers in PBL (Sun et al., 2022).

Collaborative learning

Collaborative learning as an educational approach provides an opportunity for students to work in a group to solve problems or create a product (Johnson et al., 1998). It includes five essential components: (1) positive dependence with common goals; (2) individual accountability; (3) encouragement for interaction and enhanced group communication; (4) education on soft social skills; and (5) reflection on the group process and learning from group members (Johnson & Johnson, 1989). Self-regulated learning (SRL; Wong et al., 2021) and socially shared regulated learning (SSRL) are critical for successful collaborative PBL (Bannert et al., 2014; Hadwin & Oshige, 2011; Rogat & Linnenbrink-Garcia, 2011). SRL refers to learners adjusting their learning process to achieve individual learning goals (Zimmerman, 2000), while SSRL highlights the social perspectives, such as interactions among team members to realize shared learning goals (Hadwin & Oshige, 2011; Järvelä & Hadwin, 2013; Malmberg et al., 2017).

Challenges exist in collaborative learning. The challenges include: (1) teamwork, (2) communication, (3) personal priorities, and (4) external constraints (Järvenoja & Järvelä, 2009). Researchers indicated that collaborative learning does not naturally happens and needs well-designed instruction to facilitate it (Pang et al., 2018).

To support successful collaborative PBL and facilitate students' SRL and SSRL, a new framework, namely 4S PBL, has been proposed. The framework provides detailed strategies for facilitating learning in four domains (i.e., motivation, affect, cognition, and behavior) in the three primary stages of PBL (i.e., launching the project, creating solutions, and concluding the project). Two of the strategies that support collaborative PBL are a group contract and peer evaluations.

Group contract and peer evaluations

The approach to forming a group and maintaining active participation in the group is critical in online collaborative learning since the success of group learning depends on the quality of student interaction in a group (Scager et al., 2016). This process may require instructor facilitation (Kirschner, 2001). Another strategy is to use a contract to facilitate learners’ group learning behavior (Volet & Mansfield, 2006; Zhang & Ge, 2006). Knowles (1986) proposed a learning contract for learner-centered instruction to foster personalized learning. A learning contract outlines the actions that the student expects and promises to take to achieve learning goals and is signed by the individual student and the instructor. The advantages of the learning contract are that it provides structure and support, engages students in learning, regulates their behavior, and creates an open communication platform among students and the instructor on learning goals and the approaches to achieve them (Frank & Scharff, 2013). A group contract, as a learning contract, helps groups create a shared understanding of group work and collaboration behaviors. It can be created by the instructor or generated by group members or formulated cooperatively by the instructor and the students (Zhang & Ge, 2006).

Besides a group contract, requiring peer evaluations is another effective approach to help collaborative work by motivating students, reducing free-rider issues, and engaging group members (Lin et al., 2015). Anonymous peer evaluations and recognizing individual contributions could motivate learners to actively participate in group projects.

This study aims to (1) document and critically analyze the first circle of EDR, (2) examine the impact of a group contract and peer evaluations in supporting collaborative learning in online PBL, and (3) explore students’ perceptions of their learning experiences. The following research questions guided our inquiry:

  1. How did the group contract and peer evaluations support student online collaborative learning in PBL?

  2. What challenges did students encounter in online collaborative learning in PBL?

Method

This study adopted the EDR approach using a qualitative approach to improve teaching practices through different iterations and to develop design principles and theories (McKenney & Reeves, 2018). EDR aims to make an impact on real education interventions and form or validate theoretical concepts. EDR typically has several basic characteristics indicating that the study findings are related to the design process in that it (1) refines theory and practice, (2) is conducted in authentic, real-world settings, (3) is iterative, and (4) is contextual (Wang & Hannafin, 2005).

We aimed to (1) investigate the effectiveness of the interventions, (2) track the process of iterative design, development, implementation, and evaluation, and (3) develop design principles and theories for UX design for learning technology courses in this project. EDR allowed us to continually design, evaluate, and re-design instructional interventions. This approach also provided an opportunity to conduct iterative revisions. Thus, EDR principles guided our efforts. The different data sources can increase the trustworthiness of the study through data triangulation.

McKenney and Reeves (2018) created a generic model for conducting design research in education. This model has three primary features: (1) the model has three core iterative phases: investigation/analysis, design/prototyping, evaluation/retrospection; (2) it has a dual focus on theory and practice (i.e., integrated research and design processes, theoretical and practical outcomes); and (3) it is planned for implementation and spread and is contextually responsive.

Guided by the model from McKenney and Reeves (2018), this project, designing, assessing, and revising the course, was conducted in two cycles over two years. Theoretical and pedagogical tasks and activities were used to improve students’ collaborative learning and satisfaction. This paper focuses on the first circle of EDR.

Context

The context of this study was an online graduate-level course at a public university located in the Midwest of the U.S. in winter semester, from January to May 2021. This course aimed to introduce UX design principles and processes for learning. The outcomes of this course were that upon successful completion, students would be able to apply current best practices in UX design and employ the fundamental principles of UX design to create an effective learning experience. Participants were graduate students recruited from the class who were majoring in learning design and technology (see Table 1). The nine participants had full- or part-time jobs while taking the course.

Table 1.

Participant information

Pseudo Name Gender Enrolled Program Professional background
Rebecca F Ph.D Health professional
Kate F Master Lecturer in higher education
Kacy F Master eLearning Specialist
Henry M Master School staff member (K-12)
Selina F Master Corporate instructional designer
Jane F Ph.D Program coordinator in higher education
Laura F Master Technology integration specialist in K-12
Amy F Master Graphic designer in higher education
Hellen F Master Medical educator in a hospital
Open in a new tab

The online PBL UX course environment

The design of this course was guided by constructivism (Vygotsky, 1978), which emphasized students’ active construction of knowledge by integrating new knowledge into existing knowledge in social interaction. Extending the PBL literature, the authors created a new framework, 4S PBL, to promote and facilitate self- and socially-regulated learning in online PBL, especially in technology-intense courses. This course was created based on general PBL principles and 4S PBL strategies. The course was a 15-week asynchronous course with several synchronous meetings with guest speakers and weekly synchronous office hours (see Table 2). The instructor recorded weekly 5–10 min instructional videos. The course was hosted in Canvas, a popular learning management system in higher education, and was organized in weekly modules. The typical module included weekly learning objectives and a to-do list, instructional videos, a reading list, online discussion, hallway conversations, and assignments. The weekly learning objectives and to-do list set overarching goals for the week and actions/tasks students should complete during the week. The instructional videos included lectures on the content knowledge of the week. Reading lists described all the readings that students needed to read. The online discussion forum provided a platform for students to discuss, reflect, and share their thoughts about the topic of the week. Hallway conversations were used as a channel for students to ask the instructor and teaching assistant questions.

Table 2.

Primary instructional design component in the UX course

Design component Detailed descriptions
Course participation Online engagement (self-introduction, discussion board responses, readings, peer evaluation & interactions)
Assignments Learning product analysis (I)
Learning design problem description (G)
Design project plan (G)
Sketches of design ideas (paper) (I)
Create your personas and scenarios (G)
Storyboard (G)
Mapping out inputs and outputs (G)
Wireframes of your design (digital) (G)
Interaction with the instructor Weekly online office hours
Hallway conversations in discussion forum
Emails
Scheduling individual meetings
Open in a new tab

Note: I: individual; G: group

The course focused on using the PBL approach for learning. Thus, the assignments of the course were based on PBL and a design thinking approach. Specifically, the assignment design was guided by the design thinking process (i.e., empathize, define, ideate, prototype, test), which is commonly practiced in the UX design field. The first stage was to obtain an empathic understanding of the real problems that students wanted to solve. The next stage was to define the core problems based on observations and synthesis. The third stage was ideation, when the designers were ready to start generating ideas and brainstorming solutions. Then, the design team started to create some inexpensive products using the ideas generated in the prior step for experimentation purposes. The fifth step was the user test, where the design team tested and evaluated the product with target users. However, the fifth step was not the end of the process. It was an iterative process in which the user test results could be used to refine and revise the product.

The first individual assignment, related to empathy, was learning product analysis, in which students critically analyzed the existing learning management system (LMS) and identified the pros and cons and areas that needed improvement. Then, students formed a team based on their background and expertise. As a group, they identified the design problems that they had identified for their project at the beginning of the class. As a team, they then developed a project plan that could guide their project throughout the semester. Once they had a plan, each person sketched possible solutions for the problems and brainstormed with other team members. To make the design ideas more concrete and easier to communicate, they created personas and scenarios, storyboards, mapped out inputs and outputs, and developed wireframes and prototypes as a group. Guided by the user-centered approach for the design, students conducted user tests to gather feedback from users and then finalized their design solution. In the end, students wrote a project report and presented their project in a synchronous online meeting. At the end of the course, individual students reflected on what they achieved and areas to improve in their project and collaborative process. Reflection is critical for designers so they can engage in constant improvement (Schön, 1983).

Group collaboration is critical for online PBL. To facilitate online collaboration, a group contract was adopted to support teamwork. Students formed three groups of three members based on their interests and background. Once they formed the group, students in one group met synchronously to develop their group contract, including the communication channels, frequency of meetings, and approaches to solving conflict. As a team, they were also encouraged to constantly reflect on their collaborative process and the project progress.

In addition, to better support group work, a peer evaluation was used to encourage students to actively contribute to the group project. The instructor graded the group project based on the quality of the group assignment (70%) and contributions to the group project (30%). The contribution to the group project was measured by peer evaluations. To do so, the instructor created a peer evaluation online survey via Qualtrics, an online survey tool. The peer evaluation questions included six 5-point Likert scale questions and three open-ended questions collecting data. For example, the Likert scale questions included attendance at group meetings, contributions to the group discussion, communication etiquette, and collaborative attitude. The open-ended questions related to specific comments about individual group members, and major issues. Once students submitted their group project, they submitted the peer evaluation through Qualtrics.

Last, the researcher-instructors consistently integrated various scaffolding strategies to foster collaborative PBL in a timely and dynamic manner. For example, the instructors held synchronous online office hours every week to address students’ questions and cleared up any confusion regarding the project and group collaboration. In addition, students were encouraged to schedule individual meetings with the instructor for help. Thus, the instructors played a key role in facilitating students’ PBL.

Data sources

The data sources of this study included: (1) students’ course evaluation, (2) students’ evaluations of peers’ participation and contributions throughout the semester, (3) students’ final reflection survey, (4) a group contract, and (5) the instructor’s reflection. The final course evaluation was conducted at the end of the semester and administrated by the university. The peer evaluations were conducted after each group project, where individuals evaluated their group members. The final reflection survey related to students’ achievement, challenges, experience with group collaboration, and suggestions for the course. Sample questions in the reflection survey included, “What went well in your group collaboration?” and “What needs to be improved in your group collaboration?” For the group contract, each team submitted their group contract at the beginning of the course. All the assignments throughout the semester were also used as data sources. Last, the two instructors wrote reflection notes regarding the challenges encountered and lessons learned.

Data analysis

For the qualitative data from the course evaluation, peer evaluations, group contracts, and reflection survey, the researchers conducted an inductive content analysis (Elo & Kyngäs, 2008). The researcher went through all the data to get a general sense of these components. For data from group contracts and reflection survey, the open coding, grouping, categorization, and abstraction were conducted. The data from the course evaluation and peer evaluations were analyzed using descriptive statistics. All data were then triangulated to ensure the trustworthiness of the study.

Findings

Overall, students were satisfied with using collaborative PBL in the online course. Students rated the course and the instructor, on average, 4.3 out of 5 in the final course evaluation. Although this course was an elective for all of the students, they rated whether they wanted to take this course 4.5 out of 5, on average. Students’ final artefacts also demonstrated a great success in the semester-long learning experience in this course. For examples, one group designed a learning management system integration feature to address the problem that Canvas does not integrate well with specific school district information systems, such as MISTAR (see Fig. 1).

Fig. 1.

Fig. 1

Open in a new tab

Canvas MISTAR integration core step interfaces. Images were intentionally blurred to protect participant’s personal or identifiable information

Research question 1 (RQ#1): How did the group contract and peer evaluations support student online collaborative learning in PBL?

All three groups finished a group contract at the beginning of the semester. The group contract helped students discuss and consider diverse aspects in their group collaboration for SSRL, such as their communication approach and frequency of meetings, individual group members’ contributions and roles, time management, collaborative working space, and strategies to solve possible conflicts. For communication, the three groups identified diverse approaches for communication, including both synchronous and asynchronous meetings. Two groups adopted Microsoft Teams, while one group used Zoom for synchronous communication. These two tools were available to students and faculty for free. All of the groups met synchronously at least once each week. For asynchronous communication, emails, phone text, and Google Docs were used. For regular communication, they used emails or Microsoft Teams and indicated that they checked these tools daily. In addition, phone texts or Teams chat were used as time-sensitive communication tools.

For individual group members’ contributions, all three groups mentioned that they used the weekly meeting to specify tasks for each group member. For example, the contract in Group A stated, “Each task will be evaluated prior to the due date. We will meet to discuss each project and assign individual tasks and due dates.” Similarly, Group B stated in the contract, “As we get into our project, we'll use our weekly meeting to specify additional tasks.” Some groups defined the role of individual members based on the project and rotated roles. For example, Group C indicated that “The roles will shift based on project need. We plan to rotate the roles, so each person has an opportunity to perform each role.”

Regarding the timeline, all three groups stated in the group contracts that they would finish the group projects ahead of time. For example, Group A’s contract indicated, “All final draft communications for submission will be made one day prior to assignment due date.”

Regarding collaborative working spaces, all the three groups identified some digital tools for collaboration such as Teams, Google Docs, or OneDrive. For example, Group C stated in the contract, “Our group will use [Microsoft] Teams to share documents and communicate updates on tasks throughout the week.” The final reflection survey results discussed their group contract statements. For example, Kate expressed how Microsoft Teams helped her group collaborate in the reflection survey:

Our group met every Sunday on MS Teams and made a plan for the week. We housed all documents on there and created a folder for each assignment. This system allowed all three of us to contribute. A few times, group members were unable to work on specific segments. However, since all content was housed on Teams, a different group member took on a more prominent role. The Teams’ technology allowed us to chat and communicate very quickly and keep each other updated.

When one member failed to deliver their part on time, the group contract stated that the other two members would try to step up and help out if the communication was made in advance. If one group member just did not work without any reason, they would notify the instructor. For example, Group B stated,

First, we'll make several attempts to reach out via multiple channels (Teams, text, email). If we lose track of someone, the remaining two people will divide and conquer. The remaining two people will have transparent conversations with each other and the professor about the situation and notify the professor in a timely manner.

If they had disagreements or conflicts, three groups suggested open communication and democracy in the group contracts. For example, Group B’s contract indicated, “If we have a disagreement about how to approach part of the project, we could each show facts or reasons why we feel a certain way to help others understand. Additionally, in situations of disagreement, we can move forward by voting (best 2 out of 3).” In addition, some groups emphasized the importance of respect in group collaboration. For example, Group A’s contract stated, “We will address all conflicts and disagreements directly and respectfully.” These indicated that learners socially shared regulated their behavior, motivation, and cognitions using group contract.

The last stage of self-regulated learning is evaluation. Consequently, in the peer evaluations and final reflection survey, most students reported that their group worked very well throughout the semester, and students received high peer evaluation results. The average score of each student's peer evaluation ranged from 4.9–5 out of 5. For example, Ellen commented, “Our group worked well and collaborated effectively to execute the assignment.” Amy also expressed her satisfaction with group work, “Each week I am amazed at the team effort my peers put in.” Similarly, Henry said, “We worked well together on this assignment.” In the individual final reflection survey, Henry expressed the achievement that he was proud of: “Completing a project in a design team. Initially, I was hesitant about working in a group, but I was able to learn how to manage that. I felt as though I learned some of my strengths in a design team.”

Students reported that the group contract played an important role in group collaboration in PBL. Group contracts helped them with effective communication and created shared learning goals. Laura shared her thoughts on how the group contract helped their communication in the reflection survey:

I think the best part of our group collaboration was the communication. This was the first group where we exchanged phone numbers at the beginning of the semester, and all agreed to communicate via text or email as necessary. Being able to text the group made a huge difference in communication and effectiveness, in my opinion. We all agreed to use those forms of communication in our team contract, so no one ever felt like boundaries were being crossed.

Jane also expressed that the group contract helped them communicate and helped them create shared goals in the reflection survey:

There are a few things that I believe went well in my group’s collaboration: 1) we were effective communicators. At the beginning of our newly formed group, we amended our group contract to outline our expectations and set the intention for the group. We then followed through with the contract by establishing ways to communicate through a Teams channel... 2) we had a shared goal. Each member of our team wanted to be successful in this course. Since this was a desire of each person, it made for commonality and bond amongst the team. It also helped us to maintain focus on each task and how to work together to achieve the tasks.

It is worth noting that graduate students as those in this course are typically working professionals with full time jobs and family obligations. In addition, online students may be from different time zones, including those outside of the USA. Thus, personal cell phone is not necessarily the preferred way of communication.

RQ#2: What challenges did students encounter in online collaborative learning in PBL?

Students had several challenges in group collaboration. First, they struggled with group formation after some students withdrew from the course. The group was formed by the students, with three or four students in a group in Week 1. However, one student from a group of three members withdrew from the class in Week 3. By that time, the groups had already completed their group contract and started working on the design problem. As a result, three groups had an uneven number of students: two, three, and four students, respectively. Considering the group dynamics for PBL, the instructor suggested that one group member from a group with four members move to the group with two members. It was complicated to negotiate and coordinate adjusting the groups as students were afraid of changing to a new group, which brought uncertainty to the group morale and atmosphere. One student who had a change in group members shared the issues he encountered in the peer evaluation: “I just wish we could have started together rather than shuffling groups after the start.”

Another critical issue also related to group formation: a lack of information about other students’ skills and strengths before forming the groups. Henry suggested, “I think what would have been helpful is a list of skills and times available prior to even selecting team members. We all work and find a regular meeting schedule is difficult.” Kate suggested using the survey to gain knowledge about group members’ skills and backgrounds as “most of the class did not know each other and finding group members in the first few weeks was anxiety-inducing for most.”

Third, most students in the class had full-time jobs or families, so some life events may have influenced their group collaboration even though they had a group contract. For example, Hellen said, “This week we all struggled with other life events which led to some confusion in the meeting; however, we as a team were able to communicate and get things done by the due date.” Moreover, Laura said,

I am most proud of how well our group worked together. Despite our full-time work schedules, multiple class course loads, childcare, surgeries, quarantines, and pregnancy, we were all able to commit to working together as a team. Both Amy and Hellen were fantastic communicators, and I found that collaborating on assignments with them was easy and beneficial. I'm proud of all of the work that we accomplished.

The course involved the use of Adobe XD and Figma software for the wireframe and prototype. Adobe XD is a design tool, developed and published by Adobe Inc. It can be used to create wireframe and click-through prototypes in both macOS and Windows system. Figma is a tool for interface design with both collaborative web application and desktop applications for macOS and Windows. Figma is used for vector graphics editing and prototyping. Although the students could access these tools for free, they said that they encountered challenges regarding collaborative use. It is critical to have software that enables students to collaborate together. For example, Kate said,

One significant challenge was access to the software…The limited access meant the collaboration was very challenging since not everyone could work on it simultaneously or at the same level. If the course could provide access to software for all the students, that would be amazing.

Most students reported that they liked the peer evaluations. However, it was challenging to complete a peer evaluation for each group project, and students felt that it was difficult to keep track of the evaluations. For example, Kacy expressed her struggle with keeping track of the peer-evaluation surveys,

I think the surveys were a bit challenging to keep track of. I wonder if a survey early in the semester, one mid-semester, and one at the end of the semester would suffice. I've never missed assignments like this before in my life, and I am excessively stressed out about the impact that had on my grade with how hard I worked this semester.

Laura suggested that the course should “make weekly peer-reviews optional and give students a choice to fill out one at the end of the semester instead.” For other students, completing a peer evaluation for each group project was also considered trivial when the group collaboration went well. Helen said,

I did not care for the week-by-week peer evaluations, but that's probably because everything went smoothly with my group. I never had anything specific to report on, but I can see how it would help in groups where someone was missing meetings or not contributing fully. I wish the weekly peer evaluations were optional and could be exchanged for an end-of-the-semester review if there were no issues arising week by week.

Discussion

This paper reports the first circle of EDR and the use of two key strategies, a group contract and peer evaluations, to support collaborative learning in online PBL courses. The research explored students’ perceptions of the learning experience, particularly strategies they used for group collaboration, and the challenges they encountered. We found that group contracts helped students work smoothly and keep goals focused on collaborative learning in PBL. This finding supports the conclusions by Frank and Scharff (2013).

Another finding is that students’ life events may influence the group collaboration process, even though students in this study handled it well and demonstrated mutual respect. This success can be attributed to creating a group contract that helped them think through different situations in the group collaboration process. As Zhang & Ge (2006) found in earlier work, group contracts help students create a shared understanding of the group work. The group contract could be created by the instructor or student group members or generated cooperatively by both the instructor and group members (Zhang & Ge, 2006).

Regarding group formation, as suggested by our previous work, Zhang & Ge (2006), this study found that the instructor could facilitate the group formation process. One strategy is that the instructor could survey all of the students regarding their skills, strengths, education, and working background, and then assign students to different groups. Alternatively, the instructor could encourage students to share their information in the class discussion forum so they could get to know each other better in terms of skill sets and be able to form groups on their own. Occasionally, group members may need to change groups, so the instructor could facilitate changing the groups and help them adjust to different groups, and revisit the group contract. Regarding the size of the groups, this study found that a group of three works well, which aligns with prior studies that 3–6 students for group projects are appropriate (Culen et al., 2014; Leslie, 2020; Urquiza-Fuentes & Paredes-Velasco, 2017). It may be even more important in online courses to keep the group size relatively small, given that students need to navigate out-of-class tasks and time.

For peer evaluations, most students liked their group members and evaluated their peers highly, which supports Lin’s et al. (2015) statement that peer evaluations could reduce free-rider issues and keep learners engaged. In this study, given that group collaboration went well for the majority of the groups, some students stated that a peer evaluation for each assignment was tedious and preferred to have them be optional or only at the end of the semester. Thus, the instructor may constantly monitor the peer-evaluation results and consider adjusting the peer-evaluations and making them optional or less frequent. More detailed strategies on how to effectively leverage peer evaluations need to be explored, including the frequency and the timing.

Given that both theory and practice are essential in the UX field (Churchill et al., 2013), this UX design for learning course involved hands-on activities using digital technologies to create the final project. Moreover, based on activity theories, “instruments” is one of the important elements along with “object,” “subject,” “community,” “division of labor,” and “rules” (Engeström, 2001). Therefore, in group collaboration, selecting the appropriate tool or instrument is critical. This study found that students encountered challenges regarding working collaboratively on a project using UX technologies. The Figma tool used in this class does have a free group collaboration function. Adobe XD also provides group collaboration functions; however, we provided a virtual desktop for students to access, which does not support collaborative group work. Due to the high cost, we did not purchase collaborative accounts for the students. It is critical to think through the collaborative technology used and the cost-benefits in the course design.

The research findings of this study support a few imperative design principles for collaborative PBL in similar online courses, including (1) balancing students’ autonomy and instructors’ facilitation or interventions in group formation, (2) encouraging group goal setting and constant reflections, (3) providing opportunities for peer-feedback in collaborative PBL (Scager et al., 2016; Volet & Mansfield, 2006; Zhang & Ge, 2006), and (4) considering accessibility, affordability, and cost-benefits of technology use for group collaboration.

Limitations and future research

As a first cycle of EDR, this study is limited in its scope by nature. First, the context of this study was a small graduate class with. Thus, applications of the findings of this study in large classes should be approached with caution. Future research is recommended to verify similar strategies in large online classes and with different body of students. Second, most participants in this class were non-traditional adult learners with full-time jobs and family obligations. It should also be noted that this was not a required course, so the students who chose to attend were a self-selected group with likely a higher level of motivation. Thus, the findings of this study may not apply to other students with different backgrounds. Future research could experiment with the strategies at different educational levels and in different settings. Last, despite that this study does not include detailed description about the course design and development process, sufficient information is provided regarding the context of the study, which avoids distracting readers from the purpose of this research study. Future research could share a practical design case study focusing on the design and development process and final course design to help instructional designers or educators who are interested in designing similar courses in the future.

Implications for practice

This study provides practical implications for courses utilizing online PBL, collaborative learning, or UX design, respectively. First, using group contract to set expectations for group collaboration and communication is important. It avoids potential conflicts due to lack of mutual understanding about group collaboration (Zhang & Ge, 2006). Second, peer evaluations incentivize active participation and meaningful contributions while minimizing social loafing in group assignments and promoting collaborative learning. Last, for online courses, like user experience design, that involves using learning technologies or tools to work on hands-on activities, it is critical to consider whether the tools support smooth group collaboration or not.

Conclusion

Group collaboration plays a critical role in problem-based learning (PBL), but it also has challenges in user experience design for learning courses. This study indicated that (1) group contracts helped students work smoothly and keep goals focused on collaborative learning in PBL; (2) the instructor could facilitate the group formation process; (3) peer-evaluation helped group collaboration and monitoring; and (4) the instructor could streamline technology used for collaborative projects. The findings confirm that strategies like group contract and peer evaluations support online PBL, through self- and socially-shared regulated learning as in the 4S PBL framework. With the increasing demand for UX design professionals and the lack of online training opportunities to prepare qualified UX designers, this timely study provides a successful example of how to teach UX design in an online PBL environment. It also sheds light on how to further improve similar practices, while contributing to the limited body of research on UX design for learning. As a first-cycle of the EDR, this study also sets up a foundation for future studies to strengthen and extend the research on online PBL and collaborative learning in authentic contexts.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 24 KB) (23.6KB, docx)

Availability of data and materials

The datasets used and/or analyzed during the current study are not publicly available due to their personal and private nature but are available from the corresponding author on reasonable request.

Declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Footnotes

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

References

  1. Baek E-O, Cagiltay K, Boling E, Frick T. User-centered design and development. In: Spector JM, Merrill MD, van Merriënboer JG, Driscoll MP, editors. Handbook of research on educational communications and technology. 3. Routledge; 2008. pp. 659–670. [Google Scholar]
  2. Bannert M, Reimann P, Sonnenberg C. Process mining techniques for analysing patterns and strategies in students’ self-regulated learning. Metacognition and Learning. 2014;9(2):161–185. doi: 10.1007/s11409-013-9107-6. [DOI] [Google Scholar]
  3. Barrows HS, Tamblyn RM. Problem-based learning: An approach to medical education. Springer; 1980. [Google Scholar]
  4. Churchill EF, Bowser A, Preece J. Teaching and learning human-computer interaction: Past, present and future. Interactions. 2013;2(20):44–53. doi: 10.1145/2427076.2427086. [DOI] [Google Scholar]
  5. Culen, A. L., Mainsah, H. N., & Finken, S. (2014). Design practice in human computer interaction design education. In L. Miller, & A. L. Culen (Eds.), Seventh international conference on advances in computer-human interactions (pp. 300–306).
  6. Delialioğlu, Ö. (2012). Student engagement in blended learning environments with lecture-based and problem-based instructional approaches. Journal of Educational Technology & Society, 15(3), 310–322. https://www.jstor.org/stable/10.2307/jeductechsoci.15.3.310
  7. Ebner M, Holzinger A. Successful implementation of user-centered game based learning in higher education: An example from civil engineering. Computers & Education. 2007;49(3):873–890. doi: 10.1016/j.compedu.2005.11.026. [DOI] [Google Scholar]
  8. Elo S, Kyngäs H. The qualitative content analysis process. Journal of Advanced Nursing. 2008;62(1):107–115. doi: 10.1111/j.1365-2648.2007.04569.x. [DOI] [PubMed] [Google Scholar]
  9. Engeström Y. Expansive learning at work: Toward an activity theoretical reconceptualization. Journal of Education and Work. 2001;14(1):133–156. doi: 10.1080/13639080020028747. [DOI] [Google Scholar]
  10. English MC, Kitsantas A. Supporting student self-regulated learning in problem-and project-based learning. Interdisciplinary Journal of Problem-Based Learning. 2013;7(2):128–150. doi: 10.7771/1541-5015.1339. [DOI] [Google Scholar]
  11. Fernandez-Lopez A, Rodriguez-Fortiz MJ, Rodriguez-Almendros ML, Martinez-Segura MJ. Mobile learning technology based on iOS devices to support students with special education needs. Computers & Education. 2013;61:77–90. doi: 10.1016/j.compedu.2012.09.014. [DOI] [Google Scholar]
  12. Frank, T., & Scharff, L. L. (2013). Learning contracts in undergraduate courses: Impacts on student behaviors and academic performance. Journal of the Scholarship of Teaching and Learning, 13(4), 36–53. https://scholarworks.iu.edu/journals/index.php/josotl/article/view/3453
  13. Gardner, L. (2020). Covid-19 Has Forced Higher Ed to Pivot to Online Learning. Here Are 7 Takeaways So Far. The Chronicle of Higher Education. https://www.chronicle.com/article/Covid-19-Has-Forced-Higher-Ed/248297
  14. Gary, M.C. (2020). Paradigms of Knowledge Production in Human-Computer Interaction: Towards a Framing for Learner Experience (LX) Design. In M. Schmidt, A. A. Tawfik, I. Jahnke, & Y. Earnshaw, (2020). Learner and User Experience Research: An Introduction for the Field of Learning Design & Technology. EdTech Books. https://edtechbooks.org/ux
  15. Glazewski KD, Ertmer PA. Fostering complex problem solving for diverse learners: Engaging an ethos of intentionality toward equitable access. Educational Technology Research and Development. 2020;68(2):679–702. doi: 10.1007/s11423-020-09762-9. [DOI] [Google Scholar]
  16. Hadwin AF, Oshige M. Self-regulation, co-regulation, and socially- shared regulation: Exploring perspectives of social in self-regulated learning theory. Teachers College Records. 2011;113(2):240–264. doi: 10.1177/016146811111300204. [DOI] [Google Scholar]
  17. Hmelo-Silver CE. Problem based learning: What and how do students learn? Educational Psychology Review. 2004;16(3):235–266. doi: 10.1023/B:EDPR.0000034022.16470.f3. [DOI] [Google Scholar]
  18. International Organization for Standardization. (2010). Ergonomics of human-system interaction–Part 210: Human-centred design for interactive systems (ISO Standard No. 9241). https://www.iso.org/standard/52075.html
  19. Järvelä S, Hadwin AF. New frontiers: Regulating learning in CSCL. Educational Psychologist. 2013;48(1):25–39. doi: 10.1080/00461520.2012.748006. [DOI] [Google Scholar]
  20. Järvenoja H, Järvelä S. Emotion control in collaborative learning situations: Do students regulate emotions evoked by social challenges. British Journal of Educational Psychology. 2009;79(3):463–481. doi: 10.1348/000709909X402811. [DOI] [PubMed] [Google Scholar]
  21. Johnson DW, Johnson RT. Cooperation and competition: Theory and research. Interaction Book Company; 1989. [Google Scholar]
  22. Johnson DW, Johnson RT, Holubec EJ. Cooperation in the classroom. Interaction Book Company; 1998. [Google Scholar]
  23. Kim HR, Song Y, Lindquist R, Kang HY. Effects of team-based learning on problem-solving, knowledge and clinical performance of Korean nursing students. Nurse Education Today. 2016;38:115–118. doi: 10.1016/j.nedt.2015.12.003. [DOI] [PubMed] [Google Scholar]
  24. Kirschner PA. Using integrated electronic environments for collaborative teaching/learning. Learning and Instruction. 2001;10:1–9. doi: 10.1016/S0959-4752(00)00021-9. [DOI] [Google Scholar]
  25. Knowles MS. Using learning contracts: Practical approaches to individualizing and structuring learning. Jossey-Bass; 1986. [Google Scholar]
  26. Koppelman, H., & Dijk, B. V. (2006). Creating a realistic context for team projects in HCI. In 11th annual SIGCSE conference on innovation and technology in computer science education, ITiCSE 2006 (pp. 58–62). ACM Press.
  27. Koutsabasis P, Vosinakis S. Rethinking HCI education for design: Problem-based learning and virtual worlds at an HCI design studio. International Journal of Human-Computer Interaction. 2012;28(8):485–499. doi: 10.1080/10447318.2012.687664. [DOI] [Google Scholar]
  28. Lajoie, S. P., Hmelo-Silver, C. E., Wiseman, J. G., Chan, L. K., Lu, J., Khurana, C., … & Kazemitabar, M. (2014). Using online digital tools and video to support international problem-based learning. Interdisciplinary Journal of Problem-Based Learning, 8(2), 60–75. 10.7771/1541-5015.1412
  29. Lee, H.-J., & Lim, C. (2012). Peer evaluation in blended team project-based learning: What do students find important? Educational Technology & Society, 15 (4), 214–224. https://drive.google.com/file/d/1Lyrgd0HRBKLM7XnDF2jSUBhp0CR9NRIy/view
  30. Leslie HJ. Facilitation fundamentals: Redesigning an online course using adult learning principles and trifecta of student engagement framework. Journal of Research in Innovative Teaching & Learning. 2020;14(2):271–287. doi: 10.1108/JRIT-09-2019-0068. [DOI] [Google Scholar]
  31. Lin JW, Mai LJ, Lai YC. Peer interaction and social network analysis of online communities with the support of awareness of different contexts. International Journal of Computer-Supported Collaborative Learning. 2015;10(2):139–159. doi: 10.1007/s11412-015-9212-4. [DOI] [Google Scholar]
  32. Malmberg J, Järvelä S, Järvenoja H. Capturing temporal and sequential patterns of self-, co- and socially shared regulation in the context of collaborative learning. Contemporary Journal of Educational Psychology. 2017;49:160–174. doi: 10.1016/j.cedpsych.2017.01.009. [DOI] [Google Scholar]
  33. McKenney S, Reeves TC. Conducting educational design research. Routledge; 2018. [Google Scholar]
  34. Moroz-Lapin, K. (2009). Role play in HCI studies. In International conference on HCI Educators: Playing with our education (pp. 64-67). British Computer Society Swinton.
  35. Nordahl, R., & Serafin, S. (2008). Using problem based learning to support transdisciplinarity in an HCI education. In Proceedings of HCIed (HCI in education) conference. Association for Computing Machinery. https://vbn.aau.dk/ws/files/16104806/HCIed08final.pdf
  36. Pang C, Lau J, Seah CP, Cheong L, Low A. Socially challenged collaborative learning of secondary school students in Singapore. Education Sciences. 2018;8(1):24. doi: 10.3390/educsci8010024. [DOI] [Google Scholar]
  37. Rogat TK, Linnenbrink-Garcia L. Socially shared regulation in collaborative groups: An analysis of the interplay between quality of social regulation and group processes. Cognition & Instruction. 2011;29(4):375–415. doi: 10.1080/07370008.2011.607930. [DOI] [Google Scholar]
  38. Savery JR. Overview of problem-based learning: Definitions and distinctions. Interdisciplinary Journal of Problem-Based Learning. 2006;1(1):9–20. doi: 10.7771/1541-5015.1002. [DOI] [Google Scholar]
  39. Scager, K., Boonstra, J., Peeters, T., Vulperhorst, J., & Wiegant, F. (2016). Collaborative learning in higher education: Evoking positive interdependence. CBE—Life Sciences Education, 15(4), ar69. 10.1187/cbe.16-07-0219 [DOI] [PMC free article] [PubMed]
  40. Schmidt HG, Rotgans JI, Yew EH. The process of problem-based learning: What works and why. Medical Education. 2011;45(8):792–806. doi: 10.1111/j.1365-2923.2011.04035.x. [DOI] [PubMed] [Google Scholar]
  41. Schmidt, M., Tawfik, A. A., Jahnke, I., & Earnshaw, Y. (2020). Learner and User Experience Research: An Introduction for the Field of Learning Design & Technology. EdTech Books. https://edtechbooks.org/ux
  42. Schön DA. The reflective practitioner: How professionals think in action. Basic Books; 1983. [Google Scholar]
  43. Sun, C., Shute, V. J., Stewart, A. E., Beck-White, Q., Reinhardt, C. R., Zhou, G., … & D'Mello, S. K. (2022). The relationship between collaborative problem solving behaviors and solution outcomes in a game-based learning environment. Computers in Human Behavior, 128, 107120. 10.1016/j.chb.2021.107120
  44. Tawfik AA, Gatewood J, Gish-Lieberman JJ, Hampton AJ. Toward a definition of learning experience design. Technology, Knowledge and Learning. 2022;27(1):309–334. doi: 10.1007/s10758-020-09482-2. [DOI] [Google Scholar]
  45. UNESCO. (2022). Alternative solutions to school closure in Arab countries to ensuring that learning never stops.https://www.unesco.org/en/articles/alternative-solutions-school-closure-arab-countries-ensuring-learning-never-stops?hub=800
  46. Urquiza-Fuentes J, Paredes-Velasco M. Investigating the effect of realistic projects on students' motivation, the case of Human-Computer interaction course. Computers in Human Behavior. 2017;72:692–700. doi: 10.1016/j.chb.2016.07.020. [DOI] [Google Scholar]
  47. Volet S, Mansfield C. Group work at university: Significance of personal goals in the regulation strategies of students with positive and negative appraisals. Higher Education Research & Development. 2006;25(4):341–356. doi: 10.1080/07294360600947301. [DOI] [Google Scholar]
  48. Vygotsky LS. Socio-cultural theory. Mind in Society. 1978;6:52–58. [Google Scholar]
  49. Wang, F., & Hannafin, M. J. (2005). Design-based research and technology-enhanced learning environments. Educational Technology Research & Development, 53(4), 5–23. https://link.springer.com/content/pdf/10.1007/BF02504682.pdf
  50. Wong J, Baars M, He M, de Koning BB, Paas F. Facilitating goal setting and planning to enhance online self-regulation of learning. Computers in Human Behavior. 2021;124:106913. doi: 10.1016/j.chb.2021.106913. [DOI] [Google Scholar]
  51. Zhang, K., & Ge, X. (2006). The dynamics of online collaboration: Team task, team development, peer relationship, and communication media. In Managing learning in virtual settings: The role of context (pp. 98–116). IGI Global.
  52. Zimmerman BJ. Attaining self-regulation: A social cognitive perspective. In: Boekaerts M, Pintrich P, Zeidner M, editors. Handbook of self-regulation. Academic Press; 2000. pp. 13–39. [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplementary file1 (DOCX 24 KB) (23.6KB, docx)

Data Availability Statement

The datasets used and/or analyzed during the current study are not publicly available due to their personal and private nature but are available from the corresponding author on reasonable request.

Articles from Education and Information Technologies are provided here courtesy of Nature Publishing Group

RESOURCES