Understanding the Development of a Design Thinking Mindset During a Biomedical Engineering Third-Year Course

Abstract

As students gain more experience with design concepts, they should progress from novice to expert design thinkers. The purpose of this research was to identify the constructs of growth in design thinking (DT) over short- (one weekend) and long-term (10 weeks) design challenges. A DT mindset questionnaire was completed by students in a third-year undergraduate biomedical design course at the beginning of the course, after a one-weekend design challenge, and on completion of the course. After the short design challenge, an improvement in 15 of the 19 constructs was observed relative to baseline. Six of these constructs: mindfulness and awareness of the process, embracing risk, abductive thinking, envisioning new things, creative confidence, and optimism to make an impact, were sustained over the course of the semester indicating that a prolonged period of experiential learning can maintain short-term gains in DT. Three of the constructs: holistic views (considering the problem as a whole), diversity, and curiosity showed improvement following the short-term design challenge, then deterioration suggesting that situational circumstances are significant contributors to these constructs of DT. DT generally improves with the opportunity to collaborate, communicate, and design for a specific outcome. However, situational factors including team diversity, instructor expertise, dedicated time for team collaboration, and prior experiences can affect changes in the DT skillset.

Introduction

The fundamental difference between an engineering education and that of the sciences is engineering’s central distinguishing focus on design, which is loosely defined as an ability to apply analysis, creativity, and experimentation to solve problems. Like other skills, aspects of design thinking (DT) can be taught, and as students gain more experience with the concepts, they should progress from novice to expert design thinker, and “the expert designer uses explicit problem decomposing strategies, which the novice designer does not possess”.¹³ Thus, DT is now frequently used as a term to describe methods for teaching design strategies, but the attributes a design thinker should possess and how to develop them in engineering students are still debated.

A recent scoping review⁹ showed that there is poor consensus on the definition of DT. For example, Brenner and Uebernickel¹ define DT as a future-oriented innovation method based on examples, while Brown² argues that it is a personality profile that includes empathy, integrative thinking, optimism, experimentalism, and collaboration. Of the 79 studies reviewed, only 20 studies actually assessed the impact on students as a result of an intervention.⁹ They found the most dominant impacts of DT training related to skills, attitudes, and educational outcomes. Skills attained included creative thinking, problem solving, teamwork and critical thinking.⁹

The goal of developing a DT mindset is to improve creativity and innovation to better meet the needs of the world by addressing the biases and behaviors that inhibit creativity.¹¹ DT, in a biomedical engineering context, is a process for solving problems where the needs, health, and safety of the user are prioritized over all other considerations, and for design thinkers to develop empathy for the target users.¹⁰ In an educational setting, outcomes could further include motivation, higher levels of achievement, higher order thinking, improved communication, conflict management, problem solving, creating confidence, and self-efficacy in innovation.

In an attempt to quantify the characteristics of a DT mindset, Dosi et al.⁶ developed a questionnaire to measure DT mindset awareness, which consisted of 71 questions to assess 19 constructs (Table 1). These constructs were attributes that an expert design thinker should attain over time, such as ‘embracing risk’, ‘working well in teams’, and ‘envisioning new things’. They defined ‘DT mindset’ as the set of attitudes, opinions, beliefs and behaviors that characterize an individual’s comfort levels with design and are rated using a five-point Likert Scale. Using this self-report measure to reflect on knowledge prior to undertaking a design project can provide an assessment of professional competence, but can also be used over time to assess enhanced aspects of one’s DT mindset.⁶

Table 1.

The constructs evaluated⁶ with some of the representative questions within the constructs of empathy, mindfulness, and abductive thinking (bolded constructs show differences relative to Chen and Chou³—see discussion).

A. Tolerance for - Being comfortable with Ambiguity - Uncertainty

B. Embracing Risk

C. Human centeredness

D. Empathy/empathic

I can tune into how users feel rapidly and intuitively  I am comfortable to see problems from the users point of view  I am comfortable to put myself into the shoes of user  I easily empathize with the concerns of other people

E. Mindfulness and awareness of process

I am capable to recognize when there is the necessity to iterate one phase of the process  I trust in the process to find new discoveries, rather than focusing on where the outcomes may fall I am able to recognize when we are in a divergent or convergent phase of the process

F. Holistic view/consider the problem as a whole

G. Problem reframing

H. Team Working

I. Multi-/ inter-/ cross-disciplinary collaborative teams

J. Open to different perspectives /diversity

K. Learning oriented

L. Experimentation or learn from mistake or from failure

M. Experiential intelligence/bias toward action

N. Critical Questioning (“beginners mind”, curiosity)

O. Abductive Thinking

I am comfortable to invent or simulate alternative contexts of use of the solution  I am comfortable to invent new conditions for future possibility of the project  I am comfortable to build conclusions from incomplete information I am comfortable to take decisions from a plausible hypothesis

P. Envisioning new things

Q. Creative confidence

R. Desire to make a difference

S. Optimism to have an impact

The purpose of the present research was to evaluate the growth of the DT mindset in a third-year biomedical engineering design course, with a goal of improving the students’ awareness of key design-thinking concepts, such as embracing risk, human-centeredness, empathy, mindfulness, and collaboration. To assess this growth, the DT mindset survey was administered to the students at three time points: at the beginning of the course, after a short, intense, design-focused session (two-day design challenge), and after delivery of all the design content over a semester-long course (twelve-week course), which culminated in a design competition, where students presented their team’s solution to a panel of judges. It was hypothesized that the measured attributes of DT would improve after the initial one-weekend health challenge and continue to improve over the duration of the study.

Methods

This research assessed the short-term and long-term impacts on DT mindset after participating in team design collaborations. Ethics approval was received from the Human Research Ethics Committee at the University of Canterbury. A letter of information and consent was provided as the first page of the survey, which was undertaken through Qualtrics, and implied consent was achieved when the student moved onto and completed the survey.

The students involved in this evaluation were enrolled in a third-year, biomedical design and production management course at the University of Canterbury. This was a core design course requirement for students pursuing a minor in biomedical engineering (a sub-discipline within mechanical engineering). The course was held three times per week for 12 weeks, with two standard, one-hour lectures, and a two-hour tutorial. The tutorial included an introductory lecture and then hands-on, experiential active learning exercises that students completed in small groups. Subject matter related to enhancing a DT mindset included understanding stakeholders, universal design, parallel design, bioethics, failure modes and risk analysis, liability, quality control, and errors in design (see Table 2 for course schedule and links to DT mindset). The pandemic required online learning for weeks 5-6, but students had the option to choose online learning or in-class learning for the remaining duration of the semester. The majority of students chose to attend lectures online (the lecturer presented within the classroom environment which was recorded and posted), but tutorials (in which they worked as teams) were attended in person. For the visit from the manufacturer (week 9), all students attended and conducted another (confidential) mini-design session of 2-h but they could choose to be in different groups.

Table 2.

Calendar of topics for the duration of the evaluation period (links to constructs of DT mindset).

All students were invited to voluntarily participate in the DT mindset study; they were not awarded any incentive or compensation to participate. A baseline evaluation of DT mindset was taken at the beginning of the course.

During the first weekend of the course, a ‘future of health’ design competition was offered by the University of Canterbury Center for Entrepreneurship. This was an intense, two-day design challenge (with cash prizes) and was open to all tertiary students in the Canterbury region of New Zealand. All students enrolled in the course were required to attend, but additional participants from the community were also involved. The mixed, collaborative design teams formed for this interaction included nursing, industrial design, engineering, and commerce students. The students worked in multidisciplinary teams with students beyond those involved in their course (thus the teams were more diverse than the teams developed for the course).

The student teams could choose to design for one of three challenges: develop a new, innovative model for integrated primary health care in 2025; design a high-level training program for nurses to be more responsive to the changing needs of patients and the healthcare system as a whole; or design a method for the health ecosystem to better support patients and communities to take ownership and engage actively with their wellness to improve health outcomes and prevent illness. While any of the three challenges could be chosen, all three were treated as “equal” by the judges. During the two-day timeframe, students worked together to define the question their team sought to answer and designed a method to answer it. Students were given access to community members who visited during a two-hour window to provide reflective insight on their designs.

The longer-term design project on heart health involved the design of a different medical device over the remaining 10 weeks of the semester. This design project was supported by the Christchurch Heart Institute and included the design of a point-of-care (POC) device that was able to detect biomarkers (troponin and NT-proBNP) of a heart attack with a specified turnaround test time of less than ten minutes. The design criteria for this device were better defined and were more explicit than for the projects in the two-day health challenge.

As stated earlier, students completed the Design Thinking Mindset Questionnaire at the beginning of the semester (baseline), immediately following the two-day health challenge design competition, and again at the completion of the course. Although the only respondents of the questionnaire in all three sessions were course students, the students worked in multidisciplinary teams during the health challenge design competition, which differed from their semester-long design teams.

Results

For each construct, a mean value for each of the three sessions was calculated by summing the scores for all questions under each construct and dividing by the total number of responses in the given session. At the beginning of the semester (baseline, session 1) there were 37 respondents, after the health challenge there were 20 respondents (Health Challenge, session 2) and following the POC heart health project there were 12 respondents (POC HHP, session 3). Statistical analyses were performed on each of the constructs using a two-sample t-test assuming unequal variances, since the number of respondents was different among sessions.¹⁷

Three main themes were identified among the different constructs with statistical significance being p < 0.05. These were (i) continued learning in which there were significant differences between sessions 1 and 2 and between sessions 1 and 3 (Fig. 1), (ii) short duration but sustained engagement of DT with statistical significance between sessions 1 and 2, but not in the other sessions (Fig. 2), (iii) situational gains from which there were significant gains between sessions 1 and 2 and losses of those situational gains between sessions 2 and 3, with no significant gains between sessions 1 and 3 (Fig. 3). There were no significant differences among the three sessions with the constructs of understanding uncertainty, collaborative teams, learning oriented, and experiential intelligence, and as a result, these constructs are not included within the figures.

Figure 1.

Evidence of continued learning (significant differences of p <0.05 between sessions 1 and 2, and sessions 1 and 3).

Figure 2.

Short-term but sustained engagement (significant between sessions 1 and 2 of p < 0.05, but not among any other sessions).

Figure 3.

Constructs that exhibited situational gains after the Health Challenge, but these improvements in DT were not sustained (significant difference of p < 0.05 between sessions 1 and 2 and between sessions 2 and 3, but not significant between sessions 1 and 3).

Discussion

Attributes of the DT mindset appeared to improve over the duration of the course. However, the number of respondents decreased over the period of the study making statistical analyses of direct comparisons among sessions difficult. This study was conducted during the COVID-19 pandemic. At the beginning of the course, all learning was undertaken in person. The students were able to interact with each other on-site at the design studio during the health challenge. However, shortly after the health challenge, New Zealand was placed in lockdown and many of the initial interactions and design processes for the POC heart health design project were undertaken online. It is difficult to determine what effect this change to the protocol had on the results of this study.

Short-Term Impact on Design Thinking

As hypothesized, in all constructs there was evidence of improvement in DT mindset from the baseline (session 1) following the health challenge (session 2), though in some of the constructs this was not significant. Fink⁷ argues that situational factors such as the context of the teaching and learning situation can positively affect the learning experience. This intense weekend offered an opportunity for the students to interact outside the typical teaching environment. Students were invited to the Christchurch District Health Board Design Lab (off- campus location), and mentored by nurses, engineers, and members of the community. Attributes of the instructor in this situation differed from those in the long-term POC heart health design challenge. The instructors involved in the two-day health challenge were intimately familiar with the problem, had skills that were different from the engineering students, and were enthusiastic to gain insight into immediate implementation of a solution. Meals were provided throughout the weekend and the students were also offered the opportunity to win monetary prizes. Course students were given the opportunity to meet students from other institutions and other disciplines. Interdisciplinary collaboration has been found to be beneficial in developing DT⁵ with improvement in aspects that include gaining confidence in addressing clinical and technical problems, enhancing communication, teamwork, and conflict resolution skills. Although students were required to participate in the weekend challenge for course purposes, it is evident that this event provided an opportunity for growth and improvement of DT. Tacit experiences allowed the students to experience all aspects of the variety of DT constructs within a short timeframe.

While many different approaches to cyclic design processes exist, Brown² argues, in its simplest form, that the stages of iterative design include inspiration, ideation, and implementation. The timeline of the health challenge required the students to iterate quickly and often. This resulted in quick failure and the opportunity to reframe the problem and increase experimentation. Following this health challenge, students were required to reflect on the interdisciplinary makeup of their teams, the aspects of the challenge that were surprising and frustrating, and how they could implement DT strategies in other aspects of their lives.¹⁵ These reflections may have better informed the answers to the Design Thinking Mindset questionnaire.

Using the same DT mindset questionnaire, Chen and Chou³ found similar results after an intense three-day design sprint. The differences between our current study and the Chen and Chou are bolded in Table 1. On the Chen and Chou teams, which consisted of clinician innovators, participants became more familiar with understanding uncertainty, yet did not exhibit gains when reflecting on DT constructs of empathy, reframing the problem, diversity, critical questioning (curiosity), desire to make a difference, and optimism to make a difference. These constructs that were enhanced in the engineering students but were not observed in medical students, are those humanistic attributes that are also essential for the selection of medical students and further cultivated within the medical program. Within the engineering education curriculum, students undertake design projects from their first year and appear more comfortable with ambiguity than medical students in dealing with unsolved problems.

Both studies showed similar results in the remaining twelve constructs evaluated using the DT mindset survey following an intense, short design innovation session. Neither study showed increased perception of skills in DT of collaborative teams, learning oriented, and experiential intelligence, while they did show improvements in the other nine.

Long Term Engagement in Design Thinking

While improvements in DT mindset were evident after the health challenge, only some of the DT constructs were sustained as the semester progressed following the POC heart health project. Evidence of continued learning was evident in six constructs that included: mindfulness and awareness of the process, embracing risk, abductive thinking, envisioning new things, creative confidence, and the optimism to make an impact. Improvement in these constructs of the DT mindset relative to baseline was observed both after the health challenge and after the POC heart health projects.

Mindfulness and awareness in DT is important such that participants become more cognizant of where they are in the process of becoming expert design thinkers and are able to identify when it is important to brainstorm and generate new ideas and when it is important to converge on a single solution.¹⁴ During the health challenge, students likely increased their awareness through an unconscious assimilation of ideas, while the design course provided explicit occasions for identifying and understanding stakeholders, idea generation, universal design, parallel design, and times for convergence. Compared to baseline, both events showed a significant improvement (p < 0.05); however, the difference between sessions 2 and 3 neared significance (p = 0.07), suggesting that the students appreciated and benefited from the structured process of engagement when learning in the longer-term setting. In both cases, students became more aware of the importance of when to use divergent and convergent thinking than at the beginning of the course and improved as the course progressed.

DT allows for students to become comfortable with failure and embracing risk. Students can explore different options and solutions throughout the design process, regardless of how predictable the outcomes (brainstorming crazy ideas). Fraser⁸ argues that iteration and constant change are necessary to keep the costs of failure low and rewards of breakthrough high. Within the classroom environment, students explored the concept of risk through the evaluation of failure modes and effects analysis, and analyses of failures that resulted from errors in design. While the initial improvement in comfort with risk-taking could be attributed to the situational opportunities within the health challenge, the sustained understanding of the importance and ability to take risks was better achieved through classroom experiences.

Abductive thinking, envisioning new things, and creative confidence are all cognitive styles or skills that lead to creative solutions. They include the ability to make hypotheses given incomplete sets of information, look to the future, see the complete picture, and be confident in creative problem-solving abilities. Lockwood¹² encourages the ability to challenge traditional processes and styles to evaluate ‘the logic of what might be’. As students progressed through the course, they met with the POC heart health device stakeholder, a cardiologist, on four different occasions providing them with additional insights and emphasizing the importance to the stakeholder of identifying a solution to the problem. Optimism to have an impact is regarded as a state of mind that allows teams to push forward through challenges, resistance, and major setbacks, which was also observed to improve throughout the duration of the course.¹⁴ This optimism was evident with both the health challenge and the POC heart health project.

Student attitudes toward the DT process that include human-centeredness, empathy, and a desire to make a difference showed significant improvement in the short-term, following the health challenge, but that improvement was not sustained (Fig. 2). Human-centeredness includes seeing people as a source of inspiration for design direction and devoting considerable time to understanding a user’s needs and empathy to see the user’s point of view. During the semester, the students were given an opportunity to explore these concepts through lectures in universal design, bioethics, and cultural differences, but these concepts are difficult to grasp and retain unless immersed in a setting in which they are valuable. The health challenge exposed the students to others from the community (team members, mentors, judges) with differing needs and values, requiring empathy to be experienced through the views of other disciplines. Students interacted with multiple mentors with a variety of differing views as to the definition of the problem and the most effective potential solutions, and students knew the healthcare professionals who sponsored the event sought to implement the design solutions in the short term. This may have increased the desire to make a difference, and the students sought a readily-implementable solution. The need to integrate and synthesize multiple perspectives was an attitudinal shift with which they were unfamiliar. When returning to the classroom environment, with mentorship from only course instructors and one client, empathy, human-centeredness, and the desire to make a difference in the design process were curtailed as compared to the weekend design challenge.

Skills associated with reframing the problem and team working did not persevere throughout the semester (Fig. 2). Given that students had previously worked on engineering design teams in the two years prior to this course, they were likely comfortable with this skillset as well as the people with whom they were cooperating on their design teams. The homogeneity of team members (all engineering students) for the semester-long project suggests that prior experiences may have informed these aspects of the DT mindset prior to the baseline testing.

In addition to having a longer timeline on which to work on the POC heart health project, students also had three other courses on which they needed to focus at the same time. While there were opportunities during “class time” to engage in the project, many students were not in attendance (virtually or in-person). Since the students did not use the dedicated time that was set aside for collaboration and interaction, it is possible that the students did not take the opportunity to engage with each other directly, preferring to use other means of producing course outcomes, such as making one person on their team responsible for each section of the design project, as was evident in the final reports for the course.

Experimentation or learning from mistakes is a design construct that one would expect to see improve over the duration of the course. Within the lecturing environment, students worked together on a variety of different tutorial experiences within different teams, including a two-hour hands-on brainstorming session with a local company. However, these experiences did not appear to provide the students with an increased comfort level in learning from their mistakes.

Situational Influences in Design Thinking

There were three constructs that improved only as a result of the situational factors of the health challenge. A significantly higher score was observed for embracing holistic views (considering the problem as a whole), diversity, and curiosity (critical questioning). However, in all three of these constructs, there was a significant decrease back to baseline after the semester-long POC heart health project. People who are open to diversity and different perspectives allow effective collaboration in which they tap into the experiences of others to learn how others approach the same problem. The constructs of critical questioning and curiosity refer to the same exercise that a toddler conducts —the act of questioning everything. Exposure to various perspectives through collaboration with multi-disciplinary teams in the health challenge allowed for evaluation of several concepts, which required a willingness to explore beyond personal experience. However, the experience of the engineers-only design teams throughout the entirety of the semester showed the students displayed a decrease in value for these qualities. These constructs thus suggest that as the students became more familiar with the requirements for DT, they realized the difficulty with achieving their interaction goals.

General Observations

It is evident that although short-duration interdisciplinary events can have a significant impact on design thinking, the guided interaction and iteration through a curriculum is important to cement design thinking. While the responses show sustained improvements in attributes of the design thinking mindset as the course progressed, it is also important to realize the influence of the pandemic on student engagement. Wildman et al. found that changes to team communication, tasks, and roles generally led to negative consequences.¹⁶ The reduction in the number of participants who completed the survey, in addition to the relatively small changes that were observed, suggest that the pandemic affected outcomes especially with respect to team-work and collaboration.

Limitations

The COVID pandemic reduced in-class learning for the latter half of the course and many students did not appear as engaged in virtual learning. Although all teams were successful in developing a design solution for the POC device, they chose not to complete the DT mindset survey even when provided 30 minutes during class time to complete the survey. Some statistical improvements were observed relative to the baseline for both sessions 2 and 3, but it is hard to draw any conclusions relative to improvements in DT between sessions 2 and 3 based on the limited number of students who completed the final survey. Future research should repeat this study, possibly tracking individual students directly (rather than allowing anonymous responses), through an in-person design scenario to evaluate potential improvements when provided classroom support in developing their DT skills.

Conclusions

DT generally improves with the opportunity to collaborate, communicate, and design for a specific outcome. Situational factors can influence the ability to enhance DT mindset attributes. These factors include access to diverse teams, instructors intimately familiar with the problem, incentives, and focused time to inspire, ideate, iterate, and implement. However, by providing environments in which students can learn experientially, be engaged in reflections of successful and unsuccessful design, and a safe environment within which to risk and fail, students can improve or sustain their comfort with design methods.

Implications for Educational Practice

Teaching a design course can provide significant opportunities for enhancing DT. The course should be designed around identifying those key aspects of design-thinking taught each week and providing the skillset to engage in those aspects. Future iterations of the course will include increased team-building exercises, for example, beginning the semester with a team contract that identifies roles and responsibilities. Ensuring that students are aware of the key components of design-thinking – explicitly linking the attributes to learning outcomes would be valuable in ensuring that the students are engaged in DT. The course should be designed to include many opportunities for failure and iteration to allow students to become comfortable in taking risks without the repercussions of grade loss. The short-term Health Challenge provided a better opportunity for students to take these risks without suffering consequences as the results were not graded. Ideally, diversity of team members from different disciplines and different faculties would also be integrated^4,5 as also occurred during the Health Challenge. Including diverse populations of students on teams provides students with additional insight into how to empathize, be human-centered, experience diversity, and increase curiosity. However, improvements in other constructs of DT can also be gained by providing dedicated opportunities to ideate, iterate, and implement within a setting in which the students have experiential learning opportunities.