VR and AR virtual welding for psychomotor skills: a systematic review

Abstract

Virtual reality (VR) and augmented reality (AR) continue to play an important role in vocational training in the current pandemic and Industrial Revolution 4.0 era. Welding is one of the highly demanded vocational skills for various manufacturing and construction industries. Students need to undergo many practical sessions to become skilful welders. However, conventional training is very costly in terms of material, time, and infrastructure. Hence, we explore the intervention of VR and AR in welding training, which includes the research purposes, VR and AR technologies, and welding concepts and activities. We performed a comprehensive search of articles from the year 2000 to 2021. After filtering through inclusion criteria and full-text assessment, a total of 42 articles were coded and evaluated. While there has been growth in VR and AR welding training research, there is little discussion in their effectiveness for supporting distance learning, and most studies targeted entry-level students. Our main contributions are classifying primary functions in the virtual welding workshops and their adaptation to the psychomotor domain. We hope these results can empower the research community to develop and improve the VR and AR system and evaluation instruments to support vocational training, especially during this pandemic.

Introduction

Welding is one of the essential and highly demanded skills in the manufacturing sector. It is a sculptural process that combines two or more materials with high heat to melt and fuse the pieces. The materials usually are metals or thermoplastics. The welders applied a filler material to the materials’ joint to create a melted material called a weld pool that cools down to form a joint. While often an industrial process, welding may be performed in various environments, including open-air, underwater, and outer space. Figure 1 shows the real-life welding environment with spark and ultraviolet light generated from welding, the safety equipment and the welding torch.

Fig. 1.

The welder is wearing safety equipment and holding a welding torch to do the welding task

As shown in Fig. 1, welding is a hot, challenging, and physically tasking job but necessary to various products and infrastructures. Its applications form an essential part of everyday life, from cars to high-rise office buildings, aeroplanes to rockets, pipelines to highways. Students need to undergo many practical sessions to become high-qualified welders to fill in the workforce in the market. However, the number of new generation welders is declining alarmingly [8, 56]. Furthermore, there are several problems reported in the conventional welding practice session that involves real welding training, as shown in Table 1.

Table 1.

Problems found in conventional welding training

Problem and issues	Description
Economic factors	1. Training is costly [16, 51]. 2. Training requires a lot of consumable materials and energies [35, 43]. 3. Training equipment needs to be maintained frequently [50].
Technical problems	1. Contain harmful gases and contaminations [54, 56]. 2. Accident-prone [28, 57]. 3. Environment unfriendly [35, 43]. 4. Training requires specialised space or workshops for practice [43, 50].
Social factors	1. The welding profession suffers from societal prejudice due to the harsh and high-risk working environment [56]. 2. There is low motivation among the students [57].
Training issues	1. Difficult to learn and master [4, 50]. 2. Complexity in implementing educational processes and assessment [4, 28, 57]. The assessment includes self-assessment by the trainees. 3. Welding training requires a lot of mentorships [35, 53, 54]. However, the chain-reaction effect from the lack of welders caused the shortage of qualified instructors. 4. Training requires a long training period [28, 43].

These problems are necessitated to be solved to avoid driving the prospective welders away. Additionally, the COVID-19 pandemic has also increased the difficulty to perform the hands-on training effectively. Hence, virtual simulation is the potential and timely solution for these problems. Examples of virtual simulation are virtual reality (VR) and augmented reality (AR). In the VR application, the users immerse in the virtual world and interact with the virtual objects. The AR application displays the virtual information in front of the users’ vision when viewing the real-world environment. It allows them to interact with virtual objects by using real objects. Past research stated that the VR and AR intervention in welding training could improve knowledge transfer, save many consumables’ costs, and support a new pedagogical approach and learning experiences. Besides, previous studies showed that the integration of VR and AR technologies could bring various benefits to education and increase students’ performance [2, 3, 12, 17]. In parallel to the Industrial Revolution 4.0 (IR 4.0), we anticipated that the investigation on VR and AR technologies in vocational welding training could provide an important framework and guidelines for instructors and policymakers to design practical learning content and improve the psychomotor skills of students.

Hence, the rationale of our work is to conduct a systematic review of the use of VR and AR techniques for delivering psychomotor welding skills. By summarising and synthesising the research evidence, the readers can gain a comprehensive understanding of the research purposes, trends, and scopes of the VR and AR applications in welding education. Another main finding is the exploration of available technologies, including custom and off-the-shelf equipment, to provide a cost-effective solution. We also systematically mapped VR and AR welding training workshops with the psychomotor domain based on the classified training system’s functionalities. Therefore, this systematic review answered the following three research questions (RQ):

1. RQ 1: What were the research purposes and methodologies in the VR and AR welding workshops?

2. RQ 2: What were the VR and AR technologies used in the studies of welding education?

3. RQ 3: What were the welding concepts and activities applied in VR and AR simulations?

Methodology

The methodology of this systematic review consisted of two critical phases: the searching and analytical phases. Figure 2 outlines the systematic review procedure, as inspired by the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) [2, 17, 33, 39].

Fig. 2.

PRISMA flow chart

Search phase

Before the searching process, we established the study characteristics as the eligibility criteria by using the PICOS search element, which represents the population (P), intervention (I), comparison (C), outcome (O), and study design (S), as shown in the following:

1. Populations: The types of participants were welder trainees, such as students or expert welders. There are no age limitations.

2. Intervention: The usage of VR or AR technologies in conventional welding training.

3. Comparison: The trials compare the advantages and disadvantages of the controlled group that used the traditional method and the experimental group that used the VR or AR intervention.

4. Outcome: There were two types of outcome measurements, which were the quantitative and qualitative metrics.

5. Study design: There were three kinds of study. The first study concerned the design of the welding training system. The second study focused on the user trial to identify the effectiveness of virtual welding training. The third study was the review paper of VR or AR welding systems.

The article records were identified by searching the electronic databases, including Scopus, Web of Science, IEEE Xplore, Google Scholar, and Dimensions. The search strings used were the different combinations of keywords to search in all the databases, as shown in Fig. 3. The search started on 26 October 2019 and ended on 27 November 2019. Two additional search phases were conducted on 31 March 2020 and 10 August 2021 to collect the latest publications.

Fig. 3.

The keywords used for the search strings

Analytical phase

We performed the study selection through two stages, the screening stage and the eligibility stage. Two authors conducted the screening process independently to check the articles’ titles, abstracts, and keywords according to the inclusion and exclusion criteria, as shown in Table 2. If the article’s abstract did not provide sufficient information, the introduction, discussion, and conclusion sections were reviewed additionally. The included article must satisfy all the inclusion criteria. In contrast, the article was excluded if it had at least one exclusion criterion.

Table 2.

Inclusion and exclusion criteria

Inclusion criteria	Exclusion criteria
English language publication. Paper focused on welding in education or training. Paper that satisfied the PICOS. Paper with bibliography. Paper can answer at least one research question. Journal articles, conference proceedings, reports, theses.	Non-English language publication. Paper focused on welding in industry. Paper that does not satisfy the PICOS. Paper without bibliography. Identical research (only include the latest paper). Patents.

Next, the eligibility stage consisted of the qualitative assessment, as shown in Table 3. In the assessment, two authors conducted a full-text review and applied the grading to each article. There are three grading values, which are 1, 0.5, and 0. The grading value 1 indicates that the article has satisfied the assessment item; the grading value of 0.5 means that the article has satisfied the assessment item but are not sufficient; the grading value 0 means that the article does not satisfy the assessment item. Then, the grading of the six assessment items was accumulated and recorded. The article is considered unsuitable and unrelated for further analysis if it scored below 2. A third author was involved in both stages if the two authors’ consensus was not made and a voting session was conducted. The majority vote resolved the disagreements between authors.

Table 3.

Qualitative assessment questions

No.	Question
1.	Were the research objectives stated clearly?
2.	Was the methodology appropriately explained?
3.	Was the design and implementation of the experiment conducted appropriately?
4.	Was the evaluation metric applied in the result and discussed clearly?
5.	Did the paper provide a solution to a problem?
6.	Was the literature reviewed?

Results

Study selection

Firstly, the searching phase provided 475 records from the selected electronic databases. After excluding the duplication, 260 records remained. Then, the screening stage excluded 188 records as either they did not satisfy the inclusion criteria or had at least one exclusion criteria. We conducted a full-text review from the remaining 72 screened records to apply the grading to each article and extract the data. Next, the authors discarded 30 articles because their full-text was not available or their assessment grading scored below two. Finally, this systematic review included 42 articles for the analysis and review. Figure 4 shows the flowchart of the study selection.

Fig. 4.

Flowchart of study selection

RQ 1: Research purposes and methodologies

Research purposes

We classified the included articles into three main study design types: user study, system design, and review. The user study focused on the user’s perception of the virtual welding training system and the effectiveness of improving the welding skills of the trainees. The system design centred on the design and development of the virtual welding system to improve the user experience and perception to support knowledge transfer. The review consisted of the literature review that analysed previous VR and AR welding training systems studies. Then, this paper categorised the research purposes into six types as followings:

1. User Study 1 (US1): This user study contained the evaluation of the VR or AR intervention effects, such as welding performance scores, certificate rate, psychomotor and cognitive skills.

2. User Study 2 (US2): This user study included comparative studies between non-intervention and intervention training groups.

3. User Study 3 (US3): This user study investigated the usability and user acceptance of the system.

4. System Design 1 (SD1): This system design research focused on designing and improving system framework and architecture, including devices used and new interaction methods.

5. System Design 2 (SD2): This system design research implemented advanced computer graphics algorithms to improve the simulation and immersion level of the VR or AR welding system, including weld bead visualisation and realism improvement.

6. Review (R): This research involved using secondary data sources and literature reviews.

Table 4 shows the categorisation of the collected studies in terms of the six types of research purposes and the classification of three study design types. Based on the results, designing and improving the VR or AR welding system was the most common research purpose with twenty-four studies, followed by investigating the system’s usability with seventeen studies. Eleven studies evaluated the VR or AR intervention effects on welding performance and psychomotor and cognitive skills. In contrast, there had been a lack of effective instrumental strategy to compare the performances between the VR or AR intervention group and the traditional non-intervention group, with a total of six studies. Furthermore, ten studies related to improving the simulation and immersive experience of VR or AR systems. Lastly, only six articles provided a comprehensive literature review.

Table 4.

Research purposes of the VR and AR welding workshops studies, arranged by year

Studies	User Study			System Design		Review
	US1	US2	US3	SD1	SD2	R
Kobayashi et al. (2003) [29]	X	X		X
Liang et al. (2008) [32]				X	X
Wang et al. (2009) [52]				X	X
Mellet-d’Huart (2009) [36]						X
Dalto et al. (2010) [16]			X	X
Hillers et al. (2010) [20]				X
Yang et al. (2010) [55]			X	X	X
Jo et al. (2011) [25]				X	X
Stone et al. (2011) [47]	X		X
White et al. (2011) [53]				X	X
Chambers et al. (2012) [13]				X	X
Fast et al. (2012) [19]			X	X	X
Stone et al. (2013) [48]	X	X	X
Yap et al. (2014) [57]			X	X
Xie et al. (2015) [54]				X	X
Baskoro and Haryanto (2015) [5]				X
Bickerstaff (2015) [8]	X	X
Byrd et al. (2015) [11]	X	X
Liu and Zhang (2015) [34]				X
Okimoto et al. (2015) [41]			X
Benson et al. (2016) [6]				X
Bharath and Patil (2017) [7]				X		X
Torres et al. (2017) [50]	X		X
Yao et al. (2017) [56]				X	X
Liyang (2018) [35]			X	X
Baglow and Lovegrove (2019) [4]		X	X
Ko et al. (2019) [28]				X
Price et al. (2019) [43]			X
Torres-Guerro et al. (2019) [51]	X	X
Agrawal & Pillai (2020) [1]			X	X
Chibani et al. (2020) [14]				X
Chung et al. (2020) [15]			X
Huang et al. (2020) [21]	X		X
Isham et al. (2020) [23]	X		X	X	X
Karstensen & Lier (2020) [26]	X
Knoke (2020) [27]						X
Lavrentia et al. (2020) [31]						X
Papakostas et al. (2020) [42]			X
Angel-Urdinola et al. (2021) [3]						X
Dujovic et al. (2021) [18]				X
Ipsita et al. (2021) [22]	X		X	X
Katheria et al. (2021) [30]						X
COUNT	11	6	17	24	10	6

The user study research aimed to evaluate the effectiveness of the VR or AR welding training simulators to close the gap between the pedagogical approach and computer science. As explained in the previous list, there were three types of user studies. The studies in US1 evaluated the VR or AR technologies intervention effects in the training process by comparing the welding scores before and after the training [21]. Some studies also compared the welding performances across a specific time interval, such as several user trials [22, 23]. The studies in US2 typically separated the participants into two groups [47, 51]: the conventional or controlled group using the actual welding training method; and the intervention or experimental group using the VR or AR training method. Both groups took the same course’s contents and syllabus in a specific timeframe. After that, they performed an actual welding examination to compare the welding results according to the evaluation metric of the examination body, such as the American Welding Society (AWS). The studies in US3 evaluated the impression and usefulness of the software by actual users, including the usability study and user acceptance study (UAT). The users tested the software to complete the provided tasks under real scenarios, and the feedbacks were collected to improve the software.

For US1 and US3 user study types, we also considered the comparative studies to evaluate different media, such as between monitor screen and VR and between different VR or AR devices. An example of experiments comparing different VR or AR devices is between Oculus Quest head-mounted display (HMD) and mobile phone [23]. In contrast, the US2 studies analysed the effectiveness of knowledge transfer by comparing the performance between non-intervention and intervention training groups. For example, the results showed that the VR or AR welding simulator was suitable for training the welders’ cognitive development and psychomotor skills [47, 48]. The system can also improve their confidence and preparedness before conducting actual welding. With physical and mental resilience, they could produce fewer mistakes and be more efficient in handling the welding torch, reducing the materials used. For more information, subsection 3.2.2 explained the research methodologies and instruments used by the previous studies.

Next, the system design research focused on designing and improving the efficiency and realism level of the training simulator. The studies in SD1 provided the system framework and architecture designs that are important to the software engineers to reproduce the virtual welding systems. We elaborated on these studies on two main topics: subsection 3.3 described the devices used in building the virtual welding simulators; subsection 4.1 explained the functional requirements and interaction methods used by the previous studies. The studies in SD2 usually improved the realism of the virtual welding simulation [24, 32, 55], such as the weld bead visualisation, particle effect to simulate the welding spark, audio effects and realistic haptic feedback. Past research stated that simulating the virtual training environment with a high level of realism and using real-life welding tools as user input could increase the cognitive and psychomotor skills of the trainees [48]. Thus, the trainees learn to recognise these sensory perceptions without conducting the real welding process. However, the SD2 studies were not within our research scope.

Based on the collected articles, there were six related literature review papers. The first is the work by Bharath and Patil [7] which provided a review on VR systems for metal arc welding and presented a design concept. However, they only gave a brief review of the topic for arc metal welding simulation, such as the VR technologies used, welding tasks and types of the user interface to improve the realism. Mellet-d’Huart [36] presented complementary information and specifications concerning adult learning and training using VR technologies. This work presented various learning theories and paradigms applied in the previous research, including welding. Lavrentieva et al. [31] reviewed the commonly used hardware and software solutions for the VR and AR welder simulation training and analysed the advantages and disadvantages of these technologies. They also described the standard procedure to conduct the virtual welding workshop. Both Mellet-d’Huart [36] and Lavrentieva et al. [31] focused on the usage of commercialised VR and AR welding training systems, such as CS WAVE [16], VRTEX [49], and Soldamatic [44].

In Knoke et al. [27], the authors classified the training simulators into several manufacturing processes: primary shaping, forming, separating, joining, coating and finishing, and changing material properties. The welding training was classified under the joining process. The authors also presented the research trends of human-machine interface (HMI) technologies for the training simulators and their impact and limitations, including desktop, VR and AR technologies. Katheria et al. [30] provided a short overview of welding skills development using VR and AR technologies. They also described the vital welding performance metrics that need to cultivate in training, such as welding speed, working angle and travel angle. Besides discussing the commercialised VR or AR welding system and its advantages in training, they proposed five requirements to create an ideal welding training simulator. For example, realism, hand-eye coordination features, welding posture versatility, and various welding positions and joints [30]. Angel-Urdinola et al. [3] conducted a meta-analysis of the experiment results that evaluate the impact of VR training on skills development, which includes welding training. This paper included all the articles that evaluated the performance metrics and had randomised control trials, like US1 and US2. This study found that the VR training simulators effectively developed students’ technical, practical, and emotional skills [3].

Therefore, this paper aimed to contribute to the knowledge in research purposes and methodologies, VR and AR technologies, and welding concepts and activities in the context of virtual welding training. In addition, we also classified the main functional requirements in VR and AR welding workshops based on the data extracted from the included evidence and systematically mapped them with the psychomotor domain. This systematic mapping could provide a guideline for the instructors and software developers to design and implement a VR or AR welding simulator to ensure the effective knowledge transfer of psychomotor skills.

Research methodologies

In this section, we organised the research methodologies that evaluate the effectiveness and impact of the VR or AR welding training workshops on the learning experience and performance of the welders. Table 5 shows the two methodologies identified: qualitative and quantitative methods. Please note that some research conducted a mixed method.

Table 5.

Research methodologies and instruments used by the included studies

Methodologies	Description and Intervention Technology Reported	Instruments	References
Quantitative	To determine the welding certification score based on the real welding results after the training. The evaluation criteria are the weld bead quality and stress testing. Intervention technology: VR	Accredited professional examination: i. American Welding Society (AWS) ii. The Welding Institute (TWI)	[8, 11, 21, 48]
	To determine the psychomotor skill of the trainees based on welding performance metrics. Intervention technology: VR	Collecting the welding performance metrics: i. Welding speed ii. Arc length iii. Contact-to-work-distance (CTWD) iv. Travel angle v. Work angle vi. Penetration depth	[4, 6, 16, 18, 29, 32, 51]
	To determine the psychomotor skill of the trainees based on physical skills, such as muscle region usage. Intervention technology: VR	Collect physical skills using electromyography (EMG) electrodes attached to the several muscle regions of trainees.	[47, 48]
	To determine the learning time of the trainees and the effectiveness of the virtual welding simulator in knowledge transfer. Intervention technology: VR	Time-based analysis: i. Execution time ii. Adaptation time iii. Interaction time iv. Total training time	[22, 23, 47, 50]
	To determine the mental development of the trainees based on cognitive, emotional and workload elements of the participants. For example, comprehension, stress level, concentration, mental workload and cybersickness. Intervention technology: VR	Bloom’s taxonomy in the cognitive domain	[47, 48]
		NASA Task Load Index (TLX)
		Biometric test, eye-tracking test	[50, 51]
		Simulation Sickness Questionnaire (SSQ)	[23]
	To determine the system’s usability and user behaviour or intention to use the system. Intervention technology: AR, VR	Nielsen’s heuristics	[50]
		System Usability Scale (SUS)	[22, 23]
		Technology Acceptance Model (TAM)	[42]
		Questionnaires on learning effectiveness, learning satisfaction	[15, 21]
	To determine the suitability and effectiveness of a virtual welding system based on the opinion of the experts by using quantitative methods. Intervention technology: VR	Expert questionnaires	[15, 21]
Qualitative	To determine the user perception toward VR or AR intervention in the welding workshops. It is also to observe how the trainees interact with the intervention technology. Intervention technology: AR, VR	Pre- and post-test questionnaires	[6, 19, 23, 41, 43, 55, 57]
		Interview	[4, 15, 21, 26]
		Case studies	[16, 21]
	To determine the suitability and effectiveness of welding knowledge transfer based on the opinion of the experts. Intervention technology: AR	Expert review	[1]

Based on the finding, the quantitative approach can use the accredited welding examination from professional organisations, such as the American Welding Society (AWS) and The Welding Institute (TWI), to ensure that welding skills evaluation is standardised and better prepare the trainees for the certification test. Stone et al. [48] stated that the quality of welds is evaluated based on the dimensions of the weld bead and its bend tests. After the virtual training session, the researchers can use this examination to evaluate the weld quality of the trainees in a real welding test. In addition, the software developers can also work with the welding instructors to design and implement the learning contents for the VR or AR welding training workshops consistent with the certified examination. In [47], they found out that the VR intervention group performed better than the traditional group, with an increase of 41.6% in overall certifications earned.

Besides that, Stone et al. [48] conducted a study by comparing the effect of full VR and VR-integrated training in welding in terms of cognitive and physical skills. The VR-integrated training group consisted of 50% training time in real welding training and 50% training time in VR training. Meanwhile, the full VR training group incorporated 100% training time in VR training. Both groups were required to complete the actual welding examination at the end of the training. The results found that the VR-integrated training group performed better in difficult welding positions than the full VR welding group. Therefore, virtual and real practical training must complement and integrate to achieve the best welding skills [35, 48]. Byrd et al. [11] also compared the quality score of welding between the experienced and novice groups while using the VR welding simulator. They reported that the experienced group scored an average of ten points higher than the novice group in terms of welding quality scores. Hence, the industries could use the VR welding training simulation to evaluate and upgrade their professional welders continuously.

The welding performance assessment in VR and AR welding training workshops can also collect the responding variables or welding performance metrics during the welding evaluation process, as presented in Table 5. For example, welding speed, arc length, contact-to-work-distance (CTWD), travel angle, work angle, and penetration depth. The virtual welding systems can track these variables to generate a time-based database to calculate average and standard deviation and form a time series graph for further evaluation. Kobayashi et al. [29] analysed the electrode’s welding angle and feed rate before and after the virtual welding training session. Benson et al. [6] calculated the accuracy rate of their subjects to maintain the correct CTWD and welding speed. Although many studies mentioned the welding performance metrics for post-assessment [4, 18, 32, 51], only a few studies evaluated these metrics’ accuracy by comparing them with ideal values. Since there are ideal performance metrics for every welding position level, future researchers could utilise these ideal metrics to calculate the error rate of the trainees.

Besides that, the time-based analysis can determine the learning time of the trainees when using the virtual welding training system, including execution time, adaptation time, interaction time, and total training time. The virtual welding systems can also collect these time-based metrics for evaluation. Torres et al. [50] defined execution time as the total time to complete welding; adaptation time represents the time for the trainees to master the control of the welding system; and lastly, the interaction time is the time to complete a task by the trainees. Stone et al. [47] compared the total training time between the VR-integrated and traditional training groups and found that the VR intervention reduced the training time. Meanwhile, [50] conducted a time-based analysis study to investigate how male and female students performed in the VR welding training workshop. Furthermore, [22, 23] found that the execution time of the participants decreased as the number of trials increased when using the VR welding training system. Isham et al. [23] also compared the execution time difference between Oculus Rift and mobile phone welding training systems. These results showed that the participants felt more confident and familiar to perform the welding with each trial.

Additionally, the researchers can compare the physical skills using electromyography (EMG) electrodes attached to the several muscle regions of trainees [47, 48]. Stone et al. [47] collected the physical skills data from experts. They generated a model to use as a ground truth dataset to find the error rate of the trainees’ performance in terms of kinesthetic memory and welding skills developments.

For measuring mental development, several researchers conducted experiments on how to evaluate the cognitive elements of the trainees. The work of [47, 48] measured the level of cognitive learning and development by using Bloom’s taxonomy, including knowledge, comprehension, application and analysis on each welding position. Their results showed that the VR welding training workshop helped improve the trainees’ cognitive development.

Besides, mental strength is an important factor in determining the quality of welding results and the confidence of the welders, which includes the concentration level, stress level, cybersickness and mental workload. Both [50, 51] used the eye-tracking test based on the thermal image analysis to evaluate the attention and immersion levels. The emotions of the welders can also capture by using a biometric headset to record the electroencephalography (EEG) signal. The work of [51] also showed that the VR-integrated group had less stress and more concentration than the traditional training group. Besides that, the NASA Task Load Index (TLX) assessment can also be employed to compare the mental workload of the trainees between VR-integrated training and traditional training [47]. The Simulation Sickness Questionnaire (SSQ) can measure the cybersickness level of the virtual welding systems. Isham et al. [23] researched the cybersickness levels between VR devices using SSQ and concluded that the results of mobile phone VR were close to the high-end Oculus Quest. This result showed that the low-cost mobile VR welding simulators could perform effectively as the high-end products with low cybersickness levels. In summary, the eye-tracking, mental workload tests and cybersickness test can discover the usability and immersion level of the VR and AR welding training workshops. Additionally, the emotion test can evaluate the adaptation level of the welders with or without previous VR training when conducting the actual welding.

Meanwhile, user acceptance or usability studies can examine the usefulness and intuitiveness of the system to promote better human-computer interaction (HCI). The method used can be quantitative or qualitative analysis. In quantitative analysis, the correlation analysis between welding scores and different graphical user interface (GUI) layouts can find the most suitable GUI layout during welding training [48]. Besides, ten Nielsen’s dimensions of usability and the System Usability Scale (SUS) were utilised to explore the quantitative usability and user’s level of attention when using the VR welding training workshop [22, 23, 50]. Furthermore, researchers can determine the user acceptance and intention to use the virtual welding simulators using Technology Acceptance Model (TAM) and measure the trainees’ perceived learning effectiveness and satisfaction [15, 21, 42].

Various questionnaires and surveys were used in the qualitative analysis to obtain how the users felt and perceived when using the VR or AR welding training systems [6, 19, 41, 43, 55, 57]. They are also known as structured interview methods because they collect the subjective perception and intentions of the participants based on the multiple-choice questions. The reporting styles are arranged thematically rather than statistically. Example questions are the perceived usefulness of the system, acquisition of welding knowledge, level of realism, and impression on the user interface (UI).

Besides that, the data collected from the interviews and case studies can also generate qualitative analysis results. In [16], they conducted a case study in the Matrai Welding Technique and Vocational Training Ltd. using the CS WAVE VR welding system to study the effect of VR intervention in the welding training. They interviewed and observed the trainers and trainees and summarised the results of VR intervention’s benefits and good practices for welding learning and teaching processes. Baglow and Lovegrove [4] also oversaw the qualitative approach by using semi-structured and focus group interviews with the vocational training students. The results were the students’ opinions regarding the potential advantages of VR intervention to develop their skills and motivate peer learning, and they also gave valuable suggestions for system improvement.

Furthermore, [15, 21] interviewed the experts and instructors to collect the learning outcomes. Agrawal and Pillai [1] also observed the experts when welding to collect the software requirements. The collected requirements help develop the virtual welding system, suitable course contents and questionnaires for student evaluation. In addition, [1] conducted an expert review with the instructors to determine the suitability and effectiveness of welding knowledge transfer. The welding instructors tested the prototype and acknowledged that the AR mobile welding system was an excellent idea to practice hand stability during welding.

In summary, several studies showed that the participants criticised the realism of the virtual welding systems [4, 19, 43]. Hence, improving the realism of the virtual training system is necessitated to ensure the system’s usefulness in successful knowledge transfer. Moreover, there is no research comparing AR and VR intervention effects on welding training. With the increased popularity of VR and AR technologies, we expect comparative studies between VR and AR training can contribute to welding and other vocational training. Future researchers could compare and evaluate the effect of these interventions based on certification rates, welding performance metrics, usability, level of realism and immersion.

RQ 2: VR and AR devices

Table 6 shows the classification of virtual welding training workshops based on the reported studies. Figure 5 illustrates the stacked bar chart of the number of reported studies by types of welding training workshops and year. We excluded the four review papers in the stacked bar chart in Fig. 5. The results showed the research trend based on the VR and AR technologies. It also presented the research frequency in terms of the number of studies per year. The VR method was primarily used in developing virtual welding training workshops with a total of thirty studies. The AR method was investigated and implemented by eight studies. In addition, five review papers discussed the VR or AR techniques in welding training. In 2020, the total number of research studies of VR or AR welding training workshops had achieved the highest with seven studies, followed by 2015 with five studies. This result shows that the VR and AR techniques are recently attaining more attention in vocational welding training, especially during the COVID-19 pandemic.

Table 6.

Classification of virtual welding training workshops

No.	Method	Number of Reported Studies	References
1.	Virtual Reality (VR)	30	[4, 6–8, 11, 13–16, 18, 19, 21–23, 25, 26, 28, 32, 35, 43, 47, 48, 50–57]
2.	Augmented Reality (AR)	7	[1, 5, 20, 29, 34, 41, 42]
3.	Review articles of VR or AR welding training workshops	5	[3, 27, 30, 31, 36]
	TOTAL	42

Fig. 5.

Stacked bar chart of the number of studies by types of welding training workshop and year, excluding the review papers

In addition, there is a gap of no related publication between 2004 to 2007. The first reason is the omission of duplicated research as there are several follow up works published in the latest year. Secondly, we anticipated that the lack of hardware and technological advancement hindered the development of VR or AR welding simulators between 2004 and 2007. As reported in the following subsections, the studies using customised virtual welding systems started in 2010 [16], such as CS WAVE and VRTEX. The commercialised VR headset and gadget, such as Oculus Rift and HTC VIVE, also featured in later publications, which began in 2017 [50, 56].

Types of VR devices

There are three types of VR devices reported in the studies, as shown in Fig. 6. Most of the studies focused on the head-mounted display (HMD), with twenty studies using HMD to develop the new interaction system or user experiments. Next, six studies utilised conventional peripherals, such as a monitor. Lastly, six studies reported the usage of specific devices. The specific devices used the conventional monitor for display and an additional installation, such as foldable workbench and 3D glasses, to achieve a semi-immersive virtual simulation. Figure 8 shows one of the examples of a foldable welding workbench.

Fig. 6.

Types of VR devices

Fig. 8.

CS WAVE [16]. The monitor is mounted on a specialised device, and the user can change its position to practice various welding techniques

Figure 7 shows the classification of VR devices reported in the studies. In general, the types of VR welding training workshops are non-immersive and immersive. The immersive VR application allows the user to view and interact with the virtual objects in a simulated virtual environment. The objects in the virtual environment do not need to be present in the actual world. Using an HMD, the user can see the virtual environment 360 degrees around the user.

Fig. 7.

Classification of VR devices reported in the studies

In contrast, the non-immersive VR application displays the virtual object on a flat screen for users to interact. The non-immersive VR commonly uses conventional peripherals such as a monitor display, and it is an affordable option to set up. The conventional peripheral can be installed on a specific device to optimise the user interaction, such as a workbench. The CS WAVE allows the trainees to practice different welding positions by adjusting the workbench [16], as shown in Fig. 8. Besides, 3D vision glasses can also enhance the user’s perception by producing a sense of depth and changing the monitor display’s perspective based on the user’s head position.

Both VR applications utilised a controller to simulate the welding torch motion as performed by the trainees. The controller consists of the position tracking sensor to physically realise the interaction of the welding process in the virtual environment. In some cases, an additional position tracking sensor was fitted in the HMD to track the head motion of the trainees, especially in the semi-immersive [13, 55] and immersive VR [22, 23] applications. The visualisation results allow the trainees to feel like they are real-life. Besides that, the VR welding training workshops used gesture-based haptic feedbacks, such as inertia, drag force and vibration, to support kinesthetic learning with physical activity and tangible experience. Tables 7 to 9 shows the article references and their VR system, position tracking system and controller, and haptic feedback devices.

Table 8.

The different types of non-immersive specific devices VR welding training workshops based on position tracking and haptic systems

Reference	VR System	Position Tracking System and Controller	Haptic Feedback
Non-Immersive – Specific Device
Dalto et al. (2010) [16]	Non-immersive VR using specific devices (CS WAVE monitor workbench)	Yes (Welding torch controller)	–
Yang et al. (2010) [55]	Semi-immersive VR using monitor workbench +3D stereoscopic display (Nvidia 3D vision shutter glass)	Yes, (Optical tracking by using Optitrack)	Yes (Torch fitted with vibrator)
White et al. (2011) [53]	Semi-immersive VR using monitor workbench +3D stereoscopic display (Nvidia GeForce 3D Vision)	Yes (Optical tracking by using Optitrack FLEX V100)	–
Chambers et al. (2012) [13]	Semi-immersive VR using workbench +3D stereoscopic display (Nvidia GeForce 3D Active Glass)	Yes (Optical tracking by using OptiTrack FLEX)	–

Table 7.

The different types of non-immersive conventional peripheral VR welding training workshops based on position tracking and haptic systems

Reference	VR System	Position Tracking System and Controller	Haptic Feedback
Non-Immersive – Conventional Peripheral
Wang et al. (2009) [52]	Non-immersive VR using conventional peripheral (Monitor)	Yes (6DOF PHANTOM, SensAble Technologies)	Yes (6DOF PHANTOM, SensAble Technologies)
Yap et al. (2014) [57]	Non-immersive VR using conventional peripheral (Monitor)	Yes (PHANTOM Omni, SensAble Technologies)	Yes (PHANTOM Omni, SensAble Technologies)
Benson et al. (2016) [6]	Non-immersive VR using conventional peripheral (Monitor)	Yes (Gloves fitted with sensors)	Yes (SensAble Technologies haptic device)
Chibani et al. (2020) [14]	Non-immersive VR using conventional peripheral (Monitor)	Yes (SPIDAR’s effector as welding torch)	Yes (SPIDAR haptic device)
Chung et al. (2020) [15]	Non-immersive VR using conventional peripheral (guideWeld VR Welding Simulator)	Yes (Welding simulator gun fitted with electromagnetic position tracking sensors)	–
Huang et al. (2020) [21]	Non-immersive VR using conventional peripheral (guideWeld VR Welding Simulator)	Yes (Welding simulator gun fitted with electromagnetic position tracking sensors)	–

Table 9.

The different types of immersive HMD VR welding training workshops based on position tracking and haptic systems

Reference	VR System	Position Tracking System and Controller	Haptic Feedback
Immersive – HMD
Liang et al. (2008) [32]	Immersive VR using HMD	Yes (Optical marker)	–
Jo et al. (2011) [25]	Immersive VR using HMD	Yes (Optical camera tracking)	Yes (Torch fitted with vibrator)
Stone et al. (2011) [47]	Immersive VR using HMD (VRTEX)	Yes (Electromagnetic sensor)	Yes (Built-in controller)
Fast et al. (2012) [19]	Immersive VR using HMD	Yes (Optical tracking by using OptiTrak V120:SLIM)	Yes (Vibrator device in the welding torch controller)
Stone et al. (2013) [48]	Immersive VR using HMD (VRTEX)	Yes (Electromagnetic sensor)	Yes (Built-in controller)
Xie et al. (2015) [54]	Immersive VR using HMD (ONEW360)	PS Eye and inertia Sensor	–
Bickerstaff (2015) [8]	Immersive VR using HMD (VRTEX) and mobile (VRTEX Mobile)	Yes (Electromagnetic sensor)	Yes (Built-in controller)
Byrd et al. (2015) [11]	Immersive VR using HMD (VRTEX)	Yes (Electromagnetic sensor)	Yes (Built-in controller)
Torres et al. (2017) [50]	Immersive VR using HMD (Oculus Rift)	Yes (Kinect)	–
Bharath and Patil (2017) [7]	Immersive VR using HMD	Yes (Gloves and sensors)	–
Yao et al. (2017) [56]	Immersive VR using HMD (HTC VIVE)	Yes (HTC VIVE controller)	–
Liyang (2018) [35]	Immersive VR using HMD	Yes (Welding torch controller)	–
Baglow and Lovegrove (2019) [4]	Immersive VR using HMD	–	–
Torres-Guerrero et al. (2019) [51]	Immersive VR using HMD (Acer Mixed Reality Headset)	Yes (Acer Mixed Reality Controllers)	–
Ko et al. (2019) [28]	Immersive VR using HMD	Yes (Magnetic position tracking sensor)	Yes
Price et al. (2019) [43]	Immersive VR using HMD (VRTEX)	Yes (Electromagnetic sensor)	Yes
Isham et al. (2020) [23]	Immersive VR using mobile-based HMD (Smartphone)	Yes (3D printed torch with optical marker)	–
Karstensen and Lier (2020) [26]	Immersive VR using HMD (Fronius Virtual Welding)	Yes (Welding simulator gun fitted with electromagnetic position tracking sensors)	–
Ipsita et al. (2021) [22]	Immersive VR using HMD (Oculus Quest)	Yes (Oculus Touch controller)	Yes (Built-in Oculus Touch controller).
Dujovic et al. (2021) [18]	Immersive VR using HMD	Yes (GMAW welding gun)	–

Types of AR devices

There are three types of AR devices identified in the literature, as shown in Fig. 9. The HMD was the most utilised device in the research with five studies. Both conventional peripheral and glass displays recorded one study each.

Fig. 9.

Types of AR devices

Figure 10 shows the classification of AR-based welding training workshops. The first type of AR is the monitor display which uses the conventional peripheral to display the virtual objects that blend with the real world. The second type is the optical-see-through method, where the AR objects overlay in front of the users’ vision. Examples are HMD and glass display. The users wear the HMD and move around freely when viewing the AR contents in the real-world environment. The glass display utilises a projector that overlays the AR objects on the glass in front of the real-world object [34]. The HMD has better functionality than the glass display in terms of mobility, as the users can move their heads and body around while the glass display is static.

Fig. 10.

Classification of AR devices identified in the studies

There are many AR tracking methods [45], such as optical tracking and geolocation tracking. The optical tracking method was the most utilised in the AR welding training workshops. The AR application detects the optical markers and displays the AR contents on the location of those markers, such as the base metal. The gesture-based welding torch controller also built-in the markers or sensors to detect its position and orientation when the users handle the controllers [41]. The AR application uses the position and orientation information to determine the distance of the tip of the welding torch to the base metal to simulate the welding process. Tables 10 to 12 shows the article references and their AR display system, type of AR, and haptic feedback devices.

Table 11.

The different types of optical-see-through (glass or projection display) AR welding training workshops based on the type of AR and haptic system

Reference	AR Display System	Type of AR	Haptic Feedback
Optical-See-Through – Glass or Projection Display
Liu and Zhang (2015) [34]	Optical-see-through using a glass display	Motion sensor tracking (Leap Motion)	–

Table 10.

The different types of monitor display AR welding training workshops based on the type of AR and haptic system

Reference	AR Display System	Type of AR	Haptic Feedback
Monitor Display – Conventional Peripheral
Baskoro and Haryanto (2015) [5]	Monitor display	2D optical marker	Yes (Stepper motor)

Table 12.

The different types of optical-see-through (HMD or wearable glass) AR welding training workshops based on the type of AR and haptic system

Reference	AR Display System	Type of AR	Haptic Feedback
Optical-See-Through – HMD or Wearable Glass
Kobayashi et al. (2003) [29]	Optical-see-through display using HMD	2D optical marker (Infrared photo sensor)	–
Hillers et al. (2010) [20]	Optical-see-through display using HMD (ISmaS)	2D optical marker	–
Okimoto et al. (2015) [41]	Optical-see-through display using HMD (Soldamatic)	2D optical marker	–
Agrawal and Pillai (2020) [1]	Optical-see-through display using mobile (Smartphone)	2D optical marker.	–
Papakostas et al. (2020) [42]	Optical-see-through display using HMD (Soldamatic)	2D optical marker	–

Based on the research trends, most of the research used specialised VR welding equipment for testing, such as VRTEX, CS WAVE and guideWeld VR. Several researchers conducted the VR training using consumer-level VR equipment, which proved the efficiency of the off-the-shelf equipment to train the welding skills [50, 51]. Besides, some research utilised mobile phone technologies to create virtual welding systems [1, 23]. Consumer-level VR equipment can help lower the price for virtual welding training system setup, and they are readily available in the market. In contrast, the results show that AR-based training is still under study compared to VR-based training. We expected AR technology to provide a promising welding training interaction and experience because it allows users to interact with real-world objects. Therefore, more system design and feasibility studies of AR-based training and comparative studies between VR and AR training are required in the future.

RQ 3: Welding concepts and activities

This section presents the various concepts and activities reported in the VR and AR welding training workshops, such as the base materials, welding processes, and welding positions.

Types of base materials

Learning to select the correct welding parameters and techniques based on the welding material is one of the crucial skills of welders. Figure 11 summarises all the mentioned base materials used to conduct the research. Steel and metal were the most studied base material types with seventeen and sixteen studies, respectively. Next, aluminium recorded a total of thirteen studies. In contrast, titanium, magnesium, copper, and zinc-plated steel were less concentrated, with one study for each type.

Fig. 11.

Types of base metal studied in the previous literature

The studies on the types of base material used in the simulation are helpful for the trainees to learn the correct welding parameters and methods [4, 35]. In system design research, the VR or AR welding training workshops need to simulate the correct welding behaviours, such as the weld pool formation, metal spark effects, thermodynamics, and audio effects. The weld bead visualisation is also one of the critical aspects. Byrd et al. [11] argued that the human welder response model enables the welders to react and make necessary changes based on the appearance of the weld pool during the welding. Therefore, we think that more base materials, their properties, and how the weld beads formation based on users’ skills and welding parameters can potentially and effectively improve the functionality of the VR or AR welding training workshops. Table 13 summarises all the mentioned base materials and their references.

Table 13.

Types of base metal and their references in the previous literature

Type of Base Materials	Number of Reported Studies	References
Steel, low-carbon steel, stainless steel	17	[6, 8, 11, 15, 16, 18, 19, 21, 34, 35, 41–43, 47, 48, 56, 57]
Aluminium	13	[8, 11, 13, 15, 19, 21, 41–43, 47, 48, 53, 56]
Titanium	1	[56]
Magnesium	1	[56]
Copper	1	[56]
Zinc-plated steel	1	[56]
Metal / not specified	16	[1, 5, 7, 14, 20, 22, 23, 25, 26, 29, 32, 50–52, 54, 55]

Types of welding processes

Figure 12 shows the various welding processes studied by previous research. Different welding processes have their welding methods and procedures. The VR and AR welding training workshops concentrated on arc welding techniques, which include gas metal arc welding (GMAW), shielded metal arc welding (SMAW), gas tungsten arc welding (GTAW) and flux-cored arc welding (FCAW). GMAW is also known as metal inert gas (MIG) welding and metal active gas (MAG) welding. For SMAW, it is sometimes referred to as manual metal arc (MMA) welding or stick welding, which involves using an electrode. Lastly, GTAW can also be called tungsten inert gas (TIG) welding.

Fig. 12.

Types of welding process in the literature

Among the processes, GMAW was the most popular welding process, with nineteen studies. It was followed by SMAW and GTAW, with twelve and seven studies, respectively. Meanwhile, FCAW had minimum attention from the research community with only three studies. Seven studies did not report the specific welding process type.

Most of the research focused on the GMAW process. We suggest implementing more welding processes in virtual workshops to better prepare the trainees for the certified examination. Also, many previous works studied the effectiveness of VR or AR intervention for entry-level welders [4, 8], which results in the concentration on GMAW processes and low to intermediate welding levels as mentioned in subsection 3.4.3. Therefore, future studies expect additional investigation and design of the learning contents and activities for intermediate-level welders and experienced welders to improve their skills and support lifelong learning. Table 14 shows the various welding processes and their related articles.

Table 14.

Types of welding processes and their references in the previous literature

Type of Welding Process	Number of Reported Studies	References
GMAW, MIG, MAG	19	[4, 7, 13, 18–23, 28, 35, 41–43, 53–57]
SMAW, MMA	12	[1, 6, 7, 11, 21, 29, 41, 42, 47, 48, 52, 56]
GTAW, TIG	7	[25, 28, 32, 34, 41, 42, 56]
FCAW	3	[8, 25, 55]
Not specified	7	[5, 14–16, 26, 50, 51]

Types of welding positions

There are many ways to classify welding positions. Based on the welding certification [9, 37], there are two main welding structure categories: the plate weld and pipe weld. In plate weld, the four prominent positions are (1) flat, (2) horizontal, (3) vertical, and (4) overhead. On the other hand, the four positions for the pipe weld are (1) auto-rolled position, (2) horizontal fixed position, (5) vertical fixed position, (6) inclined position. Then, the plate weld contains fillet weld (F) and groove weld (G), while the pipe weld consists of pipe-plate or fillet weld (F) and groove weld (G). Figure 13 shows the classification of welding positions.

Fig. 13.

Types of welding position

For the plate weld structure, we observed that most studies focused on the level 1 welding position (1F and 1G) with a total of twenty-seven studies. Both level 2 (2F and 2G) and level 3 (3F and 3G) had received fourteen and thirteen studies, respectively. In contrast, there were three studies in horizontal fixed positions (2F and 2G) in pipe welding. Two studies recorded the inclined pipe welding positions (6F and 6G). Auto-rolled and vertical pipe welding positions only had one study each. By comparing the welding structure, plate weld was the most studied welding position type with an accumulation of sixty-two studies compared to only seven studies in the pipe weld. Figure 14 shows the bar charts of the reported welding positions.

Fig. 14.

Types of welding positions found in the included articles and their number of studies

Based on the results, the VR and AR welding training workshops helped train the low to intermediate welding positions from levels 1 to 3, and the plate weld was the most studied activity. Consistent with subsection 3.4.2, most research targeted the effectiveness of VR or AR welding training workshops for the entry-level trainees, and therefore beginner and intermediate levels were focused. According to Stone et al. [48], the current issue in welding training workshops is that the higher difficulty levels are challenging to evaluate. Besides, there is software complexity to simulate the overhead welding positions properly. More physics principles such as gravity are needed to properly simulate the welding process and weld beads formation. In the future, research is required to study the effect of the virtual welding workshops on the learning of difficult welding positions and improve the virtual welding system to support complex welding positions. Table 15 shows the classification of welding positions and their reference articles.

Table 15.

Types of welding positions and their references articles

Welding Structure	Type of Welding Process	Number of Reported Studies	References
Plate	1F, 1G	27	[1, 5, 6, 8, 11, 13–15, 18, 20, 21, 23, 25, 26, 29, 32, 35, 41–43, 47, 48, 53–57]
	2F, 2G	14	[8, 11, 15, 19, 21, 23, 25, 41, 42, 47, 48, 52, 55, 57]
	3F, 3G	13	[11, 14, 15, 18, 21, 23, 25, 41, 42, 47, 48, 52, 55]
	4F, 4G	8	[15, 18, 21, 25, 41, 42, 52, 55]
Pipe	1F, 1G	1	[34]
	2F, 2G	3	[26, 35, 56]
	5F, 5G	1	[50]
	6F, 6G	2	[41, 42]

Discussion

Main functions in VR and AR welding workshop

There were several functions designed and implemented in the VR and AR welding workshops to promote psychomotor skills. We generalised the functions into three components of Welding Training, Post Training Assessment and Teachers’ or Instructors’ Assistance, as shown in Fig. 15.

Fig. 15.

Types of function in the VR and AR welding training workshop

Function 1: Welding training

The appealing part of using VR and AR in welding training workshops is providing multimodal and sensory interactions that promote psychomotor skills. The multimodal interaction gives the users multiple input and output experiences [40], such as obtaining haptic and auditory feedback when the torch touches the base metal. Figure 16 shows the three types of UI implemented in the welding training function: gesture-based, visual-based, and auditory-based.

Fig. 16.

Types of UI implemented in the welding training function

The gesture-based UI consists of input control and haptic feedback. The users use a controller to perform the tasks as described in the welding concepts and activities. The position tracking system is utilised to capture the input from the users to control the welding process, such as the welding torch’s orientation and speed. The previous research commonly adopted optical-based, magnetic-based, and six-degree-of-freedom (DoF) position tracking systems [5, 23, 43, 57]. There was also the use of gloves’ sensors to track the hand position of trainees [7]. The system calculated these input parameters and updated the welding torch’s position in the virtual world.

The haptic feedback can improve the cognitive and psychomotor skills of the trainees by supporting tangible experience. Inertia and vibration when handling the welding torch can increase the realism and familiarity of trainees with the actual welding process [48]. The haptic guidance and feedback can also inform the trainees if they perform the welding incorrectly [52]. Besides, the sense of touch is an essential feedback, especially in the VR environment. Participants reported they felt more natural and aware of their movement in VR welding training through the tactile experience [23]. Hence, the haptic feedback can provide a realistic experience of welding for the trainee to familiar with and become a teaching tool to guide the trainees when they perform incorrect actions.

GUI is also a requisite visual-based UI method. The design of GUI adopts overlay information on top of the users’ field of view through the display [16, 48, 53]. The information was displayed in various forms, as shown in Fig. 17, such as the performance graph of welding speed and travel angle [25, 55], visual guidance on the welding travel path [16, 55] and ghost metaphor for movement guidance [25, 55]. The work of Stone et al. [48] stated that the overlays of arc length and work-travel angle as GUI improved the trainees’ passing rate effectively. Meanwhile, the VR-trained welders also showed more concentration and less stress when conducting the actual welding session [51]. Based on the previous studies on visualisation [10, 38], the visual embellishment and metaphoric design on the GUI can significantly improve the learners’ short-term and long-term memory. The proper usage of visual guidance can also help the learners to grasp the complex key concept, such as in [48]. Therefore, the GUI can provide an efficient way for the beginner to learn and acquire basic psychomotor skills based on visual guidance and adapt the learned skills in the actual welding.

Fig. 17.

Types of visual guidance: a directional-arrow [20]; b colour guidance within the zone of tolerance [16]; c ghost metaphor for the motor skills imitation [55]

The categories of the auditory-based UI are sound effects and audio guidance. Realistic sound effects aim to generate the real welding sound to familiarise the trainees with the actual welding situation through hearing cognition, for example, when the welding torch touches the base metal and different welding speeds [19, 29]. The audio guidance [6] activates when the trainees perform the motion incorrectly, such as speed exceeding the optimal values and out of bound. Like haptic feedback, audio feedback is a helpful guide for trainees to improve their psychomotor skills during welding sessions.

Function 2: Post training assessment

Besides hands-on welding practice, the VR and AR welding training workshops also provide other activities to improve the trainees’ understanding and motivation. The ability to record the welding performance is one of the advantages of the VR and AR welding training workshops compared to traditional training. The trainees’ performance was recorded in the database in terms of welding speed, arc length, work angle, and travel angle. The system also stored the video record of the welding process in the database [25, 29, 55]. The students and instructors can use this information for the post welding assessment and readily improve their performance in the next training session. The students can playback the animation or view their performance in a time-based graph for self-evaluation [55], while the instructors can monitor their students’ progress [51, 56]. Through the collection of performance data, the welding training can become an interactive video game that can motivate the trainees to improve themselves continuously and compete with their peers to obtain the highest scores. This function is also known as the gamification of learning activities, where the learning process becomes more enjoyable to encourage the students’ engagement with the learning content.

Function 3: Teachers’ or instructors’ assistance

Additional functions were also developed for the instructors to further improve the trainees’ learning experience. The works of [16, 41] presented software for teachers and instructors to monitor and analyse their students during or after the practice sessions. Besides, Yao et al. [56] provided the instructors with online capabilities to assist and correct their students in real-time. These functionalities provide a new way for the instructors to guide and interact with their students to improve their motor skills.

The three primary functions can guarantee the success of the VR and AR welding training workshops. In contrast, there is also a lack of discussion and formal user study on its capability in distance learning. With the advent of mobile technologies, more affordable commercial products are available and owned by consumers. This condition allows the trainees to learn and train psychomotor skills anywhere and anytime. Hence, it is crucial to study how distance learning functions can help them to improve their welding skills.

Mapping VR and AR welding activities with the psychomotor domain

Welding requires an intensive level of psychomotor skills. However, we observed no established standard or framework to evaluate the psychomotor domain when VR or AR intervention was implemented in the welding training. Therefore, this work synthesised previous research results by systematically mapping Simpson’s consideration of Bloom’s taxonomy in the psychomotor domain [46] with the suitable functions in the virtual welding training system, as shown in Fig. 18.

Fig. 18.

The classification of Simpson’s psychomotor domains with the functions of the virtual welding training workshop

The psychomotor domain categories arranged from perception to origination depicted the increase of complex action levels. The related keywords and training outcomes were extracted from previous studies to show the relationship between the psychomotor domain categories with the VR or AR functions. Besides, we also classified the main functions in VR and AR welding workshops by the psychomotor domain categories based on the extracted data, as shown in Table 16.

Table 16.

Classification of virtual welding training workshops by Simpson’s psychomotor domain and its reported training outcomes

Main Function	Psychomotor Domain Category	Reported Training Outcomes
Gesture-based input control	Perception	• Adjust the welding torch to the base metal and navigate the torch along the line to stick the metal while maintaining proper speed and angle [57]. • Differentiate the effect of different machine settings (voltage, amp, gas flow) and material properties (plate thickness) [4, 15, 22].
	Guided Response	• The welding equipment and system encourage the trainees to repeat the exercise and motivate trial and error [4, 15, 21, 41, 47].
	Mechanism	• Use the welding torch to perform the welding confidently and get good quality scores [47].
	Complex overt response	• Reduced the training time [21, 22, 47].
	Adaptation	• Adapts the similar muscle exertion as the expert welder [47, 48]. • Adapts to real welding more efficiently with higher certification scores [4, 47, 48].
	Origination	• Develops a new and alternative posture to maintain stability during welding [4, 48].
Gesture-based haptic feedback	Perception	• Detects the vibration when the welding torch touches the base metal [14, 57].
	Guided Response	• Responds to the vibration when the student welds out of place [52].
Visual-based guidance (GUI)	Set	• Estimates the welding speed, work angle, travel angle, and welding direction based on the overlay guidance [23, 43, 48].
	Guided Response	• Follows the guidance lines for welding, such as directional line, colour guidance, and ghost metaphor, to correct the posture, welding speed, and working angle [15, 16, 18, 19, 23, 26, 55]. • Follow the overlay GUI guidance in complex welding positions and improved the performance of trainees [15, 16, 18, 19, 23, 26, 48, 55].
Realistic sound effect	Perception	• Distinguishes the welding progress (speed, distance) based on auditory input [19, 29].
Audio guidance	Guided Response	• Follows the audio guidance and instruction during welding [6].
Welding performance review	Set	• Recognises own abilities and limitations based on the performance graph to readily adjust the welding posture on the next training [19, 55].
	Origination	• Revises own welding skills (hand position, speed) to get higher scores and compete with peers [4, 26].
Video playback	Set	• Reorganises own welding skills (hand position, speed) to perform better welding results by evaluating own performance [26, 57].
	Adaptation	• Recognises own abilities and limitations by viewing video playback and volunteer to learn new welding skills [41, 57].
Teaching assistance function	Complex Overt Response	• Operates the welding torch more efficiently [29].
	Guided Response	• Follows the teacher’s instruction during welding. Assists by the teacher when welding incorrectly [56].

Perception is the ability of learners to use their sensory organs to guide themselves in purposeful motor action. By yielding a gesture-based input control such as the position tracked welding torch controller, the trainees can adjust the welding torch to the base metal and translate the torch along the line to stick the metal while maintaining proper speed and angle [57]. In addition, trainees learn to differentiate the effect of different welding machine settings and material properties, such as voltage, amp, gas flow, and plate thickness, when performing the welding [4, 15, 22]. This awareness can aid them in setting the correct welding settings before welding. The gesture-based haptic feedback can also help the trainees use their tactile sensory organs to detect the effect of their actions. For example, the welding torch vibrated when it touched the base metal [14, 57]. Besides, a realistic sound effect in the virtual welding system allows the trainees to distinguish the welding status, such as their speed and distance between the mouth of the welding torch and base metal [19, 29]. These sensory stimuli can let the trainees familiarise themselves with the welding environment and help them learn how to respond to these auditory cues.

Set is a preparatory adjustment or readiness for a learner to act. The GUI overlay guidance displays the current welding performance metrics, such as welding speed, work angle, and travel angle [23, 43, 48]. The knowledge of ideal performance metrics for every welding position level prepares the trainees to adjust their actions if the parameters in the GUI are not within the ideal ranges. Post-training assessment is also the best way to assist the trainees in recognising their weaknesses and trying new welding posture in the next training session. The trainees can access the welding performance review function that displays the performance graphs of their performance metrics to evaluate their performance [19, 55]. Furthermore, they can also watch the video playback of the welding session to evaluate their welding skills, such as hand position and speed, and mentally prepare themselves if the same situation occurs [26, 57].

Guided response is an early level of learners to develop complex psychomotor skills. It includes imitation and trial and error to build up the learners’ skillset [46]. The visual-based guidance GUI, such as directional line, colour guidance, and ghost metaphor, can allow the learners to follow the welding guidance line [15, 16, 18, 19, 23, 26, 55]. Gesture-based haptic feedback can also notify the trainees by vibrating the controller when they weld out of place [52]. In addition, audio guidance is a great tool to instruct the trainees during welding if their performance metrics are not ideal [6]. Instead of guiding by the virtual system, instructors can assist their trainees when they weld incorrectly [56]. In this case, the instructor can stand beside the trainees or monitor remotely and thus, support the distance learning development. For trial and error, the virtual welding training can motivate the trainees to repeat the exercise [4, 15, 21, 41, 47]. Hence, the trainees could discover the best welding techniques while experimenting.

Mechanism is a basic proficiency level of the learners to perform an action task with confidence and respond to more complex situations. The outcome of this intermediate stage is that the trainees can handle the welding torch to perform the welding confidently and obtain good quality scores in a real-life welding environment after training with the VR system for a time [47]. This outcome shows that the virtual welding systems can transfer the basic knowledge of the psychomotor domain in welding training.

Complex overt response is an expert level of learners to perform the task more efficiently and exhibit more complex posture or movement. In Stone et al. [47], the integration of the VR welding training system helped reduce the training time compared to the traditional training method. The trainees can operate the welding torch more efficiently during the welding simulation training using the teaching assistance function. They scored higher than the trainees without teaching assistance [29]. Besides, the result also showed the vast improvement of electrode feed rate where the results were closed to the ideal values [29]. By learning to perform better quickly, the trainers may incorporate more complex welding skills to improve the current learning outcomes.

Adaptation is a high motor skill level of the learners where they can modify their learned skills to adapt to different situations. The most common method to implement adaptation is by providing an actual welding evaluation session after training with the VR welding simulation. Past results showed that the trainees adapted to real welding more efficiently and got a higher certification rate than the traditional training method [4, 47, 48]. The trainees displayed the ability to adapt the similar muscle exertion as the expert welder in some welding positions. However, the results are not robust as the trainees may also use the wrong muscle region for certain welding positions [47, 48]. As mentioned, the video playback function allows the trainees to recognise their abilities and limitations by viewing video playback. It can also trigger the trainees to voluntarily modify their welding posture and actions on the next training session [41, 57].

Origination is the ability of the learners to create a new set of actions to complete a specific task. Several studies [4, 48] observed that the trainees developed their own and alternative posture to maintain stability during welding. In addition, the welding performance review in terms of performance graphs and a high score list can motivate the trainees to compete with their peers by developing new techniques. The trainees stated in their interviews that they constantly revised their welding skills as shown in the graphs, such as hand position and speed, to get higher welding scores and win against their peers [4, 26]. These outcomes showed that the virtual welding system could help and encourage the trainees to discover their abilities and at the same time cultivate collaboration and competitiveness among the trainees.

The systematic mapping shows that the virtual welding systems can realise all seven levels of Simpson’s psychomotor domain. This result also positively shows VR and AR’s potential to acquire welding skills through simulation. Additionally, they can aid teachers and instructors to achieve the educational objectives and measure psychomotor skills to evaluate their students’ achievement.

Threat of validity

We recognised some limitations reported in this systematic review that may affect its validity. The potential limitations are related to the risk of bias and limitations of the review process.

The systematic review collected the data across studies to report the VR and AR applications in the welding training workshops. The risk of bias is that the subject population, the demographics, and the number of hours spent in the training is not the same across studies. According to the PICOS, the population also did not distinguish the experience level and age of the subjects. The future recommended work could review the effectiveness of VR and AR intervention for supporting entry-level and intermediate-level students to learn psychomotor skills and providing the platform for expert welders to upgrade their skills.

Besides that, the limitations of the review process include the constraint of search and study selection. We did not gain access to all related papers because of the limitation of the search strings and the exclusion of non-English publications. Thus, the interpretation and synthesis of the included studies may be hampered to a small degree.

The study selection constraint is the threat of informativeness, where a small number of articles present only the conceptual and complete system design without user study. The latter mostly discussed the implementation of the algorithm to improve the realism of the VR or AR system. Therefore, the analytical phase in subsection 2.2 used a set of qualitative assessments to determine the acceptance of each article based on the scores.

Conclusion

This paper presents a systematic review of VR and AR techniques for delivering welding skills. The VR and AR welding training workshops’ current state was analysed and synthesised based on the evidence from 42 studies.

The instructors and trainers could consider using this result to develop practical learning experiences. The results contain various innovative learning materials in the VR and AR welding training workshops. Thus, they could decide what type of interaction, welding position, and welding concepts to cultivate for the training. Besides that, we classified various user study designs for the researchers to choose suitable instruments, including the usability study of VR and AR welding training workshops and the effectiveness of the simulation in helping the students acquire psychomotor skills. These could produce standardised, evidence-based results that can contribute new knowledge to the pedagogical approach of welding and other vocational education.

Furthermore, we anticipated that this systematic review could be helpful for vocational educators, computer scientists and software engineers. The current state of VR and AR welding training workshops and incorporating a pedagogical approach will benefit them in designing, implementing, and improving the system. The multi-discipline collaboration between computer science and engineering education could realise and improve VR and AR intervention effectiveness in vocational training. Another contribution is the focus of the psychomotor domain and its measurable outcomes to evaluate this hands-on education, which provides a standard guideline to design and implement the virtual system for other vocational education. Lastly, the application of digital technology in engineering education to deliver practical technology-based skills is timely, particularly in the current pandemic situation and IR 4.0 era.

Funding

This work was supported by the Ministry of Education (MOE) through Fundamental Research Grant Scheme (FRGS/1/2018/TK03/UTM/02/16) with vote no: R.K130000.7856.5F114.

Availability of data and material

NA

Code availability

NA

Declarations

Ethical and consent statements

The production of this article does not involve any human participants or animals.

Conflict of interest

The authors declare that they have no conflict of interest.