Skip to main content
PMC home page
Research logoLink to Research
. 2023 May 25;6:0154. doi: 10.34133/research.0154

A Wearable Multidimensional Motion Sensor for AI-Enhanced VR Sports

Zi Hao Guo 1,2,3,†, ZiXuan Zhang 3,†, Kang An 4,†, Tianyiyi He 3, Zhongda Sun 3, Xiong Pu 1,2,*, Chengkuo Lee 3,*
PMCID: PMC10211429  PMID: 37250953

Abstract

Regular exercise paves the way to a healthy life. However, conventional sports events are susceptible to weather conditions. Current motion sensors for home-based sports are mainly limited by operation power consumption, single-direction sensitivity, or inferior data analysis. Herein, by leveraging the 3-dimensional printing technique and triboelectric effect, a wearable self-powered multidimensional motion sensor has been developed to detect both the vertical and planar movement trajectory. By integrating with a belt, this sensor could be used to identify some low degree of freedom motions, e.g., waist or gait motion, with a high accuracy of 93.8%. Furthermore, when wearing the sensor at the ankle position, signals generated from shank motions that contain more abundant information could also be effectively collected. By means of a deep learning algorithm, the kicking direction and force could be precisely differentiated with an accuracy of 97.5%. Toward practical application, a virtual reality-enabled fitness game and a shooting game were successfully demonstrated. This work is believed to open up new insights for the development of future household sports or rehabilitation.

Introduction

Innovating new technologies brings about a revolution in terms of life quality. In the past few decades, the rapid development of wearable electronics and the internet of things (IoTs) has brought us into the digital and intelligent health era [1–7], radiating to applications involving rehabilitation [8,9], health monitoring [10], disease diagnosis [11], etc. This also has a substantial impact on the sports field.

Regular exercise has been considered a key to open the door for a longer health span. It could delay the onset of 40 chronic conditions/diseases, such as cardiovascular disease, diabetes, or some mental diseases [12]. However, traditional sports events (e.g., running, hiking, football, etc.) are dominated by outdoor activities, which are significantly influenced by weather conditions (rainy, windy, temperature, etc.) or other special situations (e.g., coronavirus disease 2019 [COVID-19] pandemic) (Fig. 1A). In this case, doing exercises indoor or even working out at home is highly desired; this therefore promotes new commercial opportunities, typified by the Nintendo switch. They have developed a series of motion-sensing games through portable designs and achieved great commercial success, especially during the COVID-19 pandemic. One key component of these human–machine interface (HMI) systems should be the direction sensors that are integrated into the potable external devices (e.g., Joy-con) and could respond to angular velocity or accelerations. Unfortunately, relevant commercial angular velocity or acceleration sensors generally adopt piezoresistive or capacitive mechanisms, and power consumption issues seriously hinder their scalable utilizations [13]. In this regard, because of their miniaturized configurations, easy fabrication protocols, high output, and cost-effectiveness, the emerging triboelectric nanogenerators (TENGs) provide promising energy solutions [14–23]. Efforts have been devoted to developing self-powered TENG-based direction sensors for human health monitoring fields. For example, Xu et al. [24] have reported a spring-like TENG for vibration energy collection and acceleration sensing, Xie et al. [25] developed a self-powered angle sensor utilizing a rotation TENG, and Shi et al. [26] have constructed a self-powered gyroscope ball for both acceleration and angular velocity detection. However, the above-mentioned direction sensors mostly react to either vertical motion and acceleration, or planar motion and angular velocity. Limited detection dimensions as well as poor wearable comfort block their further applications. More importantly, existing strategies extract the motion information from obtained triboelectric signals by simply differentiating the signal amplitudes or peak numbers [27–29], which are not perfectly adapted to our unpredictable and complex daily movements. The burgeoning Artificial Intelligence of Things (AIoTs, the combination of artificial intelligence technology with IoT technology) provides a new strategy to unlock the shackles of technology [30–33]. Specifically, wearable electronic devices in IoT collect and transmit sensory data to the cloud, and the data are then remotely analyzed by artificial intelligence algorithms (e.g., machine learning or deep learning algorithms), which is an efficient way to learn higher-level features of the raw input signals, especially for the triboelectric featured dynamic sensing signals [34–36]. Besides, with the onset of virtual reality (VR) and augmented reality (AR), various gaming-based HMIs are developed [37–41]. Such functionality-enhanced HMI systems facilitate the user to absorb the virtual space that will pave a new way to construct the immersive VR sport system (Fig. 1B).

Fig. 1.

Fig. 1.

Open in a new tab

The schematics of the multidimensional sensor. (A) The influence of weather conditions on conventional sports events. (B) AIoT-enabled future household intelligence sports. (C and D) The structure of the multidimensional sensor. (E to G) Potential applications of the multidimensional sensor regarding gait monitoring, movement recognition, and virtual reality.

In this work, we proposed a self-powered wearable multidimensional motion sensor that can both sense vertical acceleration and planar angular velocity. Each part of the sensor (e.g., acceleration and angular velocity part) is sensitive to the mechanical stimuli and holds excellent durability (over 10,000 cycles). The multidimensional motion sensor was designed and developed to be integrated onto a belt for gait and waist motion sensing. By leveraging the Support Vector Machine (SVM) model and a classical machine learning algorithm for classification, the smart belt could identify various motion patterns including walking, running, waist spinning, and turning with an accuracy of 93.8%. Then, a VR fitness game was developed based on the smart belt. The virtual character was controlled by different movements and the calorie consumption of the controller during the exercise could be calculated instantaneously. Moreover, the multidimensional motion sensor could also be adopted to collect data with more abundant information. As evidence of the concept, we further wear the sensor at the ankle position to obtain the signals generated by shank motions that are more complicated than waist motions. Aided by the Convolutional Neural Network (CNN) model, kicking from different directions and forces could be effectively recognized with an accuracy of 97.5%. Hence, supported by the AI technique, the multidimensional motion sensor presented in this work could be used for gait monitoring, movement recognition, and VR, which not only shows its prospective application in the intelligence sports area but also exhibits great potential in the rehabilitation, smart home, and healthcare field (Fig. 1E to G).

Results

Design, sensing mechanism, and characterization of the multidimensional motion sensor

The configuration of the multidimensional motion sensor is schematically demonstrated in Fig. 1C and D. Generally, to detect motion along arbitrary directions, the multidimensional motion sensor is designed to equip with 2 parts: an acceleration sensor (Fig. 1D(i)) and an angular sensor (Fig. 1D(ii)). The acceleration sensor is assembled on the top of the whole structure, to detect vertical acceleration. It consists of an upper shell (the internal surface of which is attached with the upper electrode), a middle shell, an Eco-flex pad, a tray, steel balls, a weight plate, a guide rod, and a spring. As for the angular sensor, it is composed of a pedestal and a lower shell (inside of which contains 4 individual lower electrodes and an independent Eco-flex ball) for planar movement sensing or angular velocity sensing.

The working principle of the acceleration sensor is mainly based on the electrostatic induction effect, as shown in Fig. 2A. Due to the electronegativity, the Eco-flex naturally carried abundant negative charges. When the Eco-flex pad approaches (or even contacts with) the upper electrode under an external vertical acceleration, electrons will flow from the upper electrode to the ground. In contrast, when the Eco-flex moves away from the upper electrode, electrons will flow back to the upper electrode. To characterize the output performance of the acceleration sensor, an electrometer was adopted to measure the voltage output. The typically generated voltage signal is illustrated in Fig. 2B. The Eco-flex pad was fabricated through a mold casting method; the detailed process can be found in Fig. S1 and Materials and Methods. To increase the surface charge quantity of the Eco-flex pad, ball-shaped patterns were further created on its surface and the output could be increased to 125% of the initial state (Fig. S2). We also found that under the same acceleration, the increment of weight value (e.g., the number of steel balls) is beneficial to the sensitivity of acceleration detection (e.g., a higher output of sensor), as shown in Fig. 2C. Due to the limitation of the size of the weight plate, the maximum mass of steel balls could be loaded to 5 g. Then, the relationship between the acceleration value and sensor output was determined, and the results are demonstrated in Fig. 2D. As the applied acceleration increased, the outputs of the sensor also increased. A larger acceleration will create a smaller gap between the upper electrode and Eco-flex pad and therefore enhance the electrostatic induction effect. Besides, the durability of the acceleration sensor was also measured. After 10,000 cycles under the acceleration of 10 m/s2, the output of the sensor shows no obvious decrease (Fig. 2E).

Fig. 2.

Fig. 2.

Open in a new tab

The mechanism and characterization of the multidimensional sensor. (A) The working principle of the acceleration sensor. (B) The typical voltage signal of the acceleration sensor. (C) Voltage outputs of the acceleration sensor when loaded with various weights. (D) The relationship between sensor output and acceleration value. (E) The durability of the acceleration sensor. (F) The working mechanism of the angular velocity sensor. (G) The typical voltage signal of the angular velocity sensor. (H) The independent output signals from 4 channels. (I) Outputs of the angular velocity sensor under spinning conditions. (J) The relationship between the angular velocity and sensor output. (K) The durability of the angular velocity sensor.

The working principle of the angular sensor is similar to the acceleration sensor, as exhibited in Fig. 2F. When the Eco-flex ball rolls on the lower electrode, positive charges will be induced on the electrode, and electrons will flow from the electrode to the ground. When the Eco-flex ball leaves the electrode, electrons will flow back from the ground to the electrode to equilibrate the positive charges. Since the angular sensor contains 4 individual channels, we further adopted a multi-channel oscilloscope to simultaneously measure its voltage signals from each channel. The output waveform of the angular sensor from one single channel is demonstrated in Fig. 2G. The contact between the Eco-flex ball and the electrode will generate a negative peak and the Eco-flex ball generates a positive peak as it leaves the electrode. Besides, as shown in Fig. 2H, when the Eco-flex ball makes contact with each lower electrode from the original position (e.g., bottom of lower shell), respectively, every channel could work independently, and no obvious crosstalk was observed. Then, we further measured the output performance of the angular sensor under the spinning situation, as illustrated in Fig. 2I. If the angular sensor is spinning in an anticlockwise direction, the voltage waveform of channel 1 will be one-, two-, and three-quarters faster in phase than channels 2, 3, and 4, respectively. In contrast, in a clockwise direction, the phase of channel 1 will be slower than the other channels. Moreover, the relationship between the angular velocity and sensor output was investigated, as shown in Fig. 2J. The linear voltage output increases as the angular velocity increases. Figure 2K depicts the output voltage of the angular sensor after 10,000 cycles of spinning, showing the robustness of the sensor for longterm applications.

Development of artificial intelligence-enhanced wearable sport system

Considering the fact that wearable sensors that can be used to precisely gather motion signals from the human body during exercise and conventional sports events are susceptible to weather conditions, we built an indoor wearable intelligence sport system based on our multidimensional motion sensor. Since the waist is the joint that connects the upper and lower limbs, motions from different body parts will reflect on the waist motion [42,43]. In this regard, we integrated the multidimensional sensor with a belt and measured the signals at the waist position generated by the movement of walk, run, left turn, right turn, clockwise spin, anticlockwise spin, go upstairs, and go downstairs (marked from 0 to 7 in sequence), as shown in Fig. 3A to H. The data acquisition system is illustrated in a schematic diagram in Fig. 3I. For the instantaneous multi-channel data acquisition, each sensor channel would be connected to Arduino MEGA 2560 with 8 integrated circuits of amplifier. The channel of acceleration sensor was defined as channel 1, and the channels of angular sensor were set as channels 2 to 5. It could also be noticed that the collected triboelectric signals (Fig. 3A to H) have much information that was difficult to distinguish by the naked eye or simply differentiated by signal amplitude or peak numbers. Fortunately, as an emerging technique for extracting subtle features, machine learning has been commonly used in triboelectric signal analysis. The SVM is considered as a strong classification machine learning algorithm with the features of high efficiency, small sample input, and direct use [44–46]. Therefore, the raw data from the multidimensional sensor were further fed into the SVM model, which is constructed by Python for recognition tasks. A dataset containing 800 samples (each motion collected 100 samples) was constructed, where 640 samples were used for training (80%) and 160 samples were used for testing (20%). The data point length from each channel is set as 300; hence, there will be 5 channels × 300 = 1,500 features for each sample as the input of SVM. For the visualization, we utilized the t-distributed stochastic neighbor embedding (t-SNE) algorithm, a nonlinear dimensionality reduction technique for embedding high-dimensional data visualization in a low-dimensional space of 2 or 3 dimensions [47,48]. The clustered results of the dataset are successfully visualized as exhibited in Fig. 3J. After the training process, the new clustered results are demonstrated in Fig. S3, and it could be found that all data points from different motions are well classified. The recognition accuracy could reach up to 93.8%, and the confusion map of the classified result is shown in Fig. 3K.

Fig. 3.

Fig. 3.

Open in a new tab

Recognition of waist motions. (A to H) Signals collected through Arduino that are generated by walking, running, left turning, right turning, clockwise spinning, anticlockwise spinning, going upstairs, and going downstairs. (I) The schematics of data collection and machine learning system. (J) The visualized clustered results of the dataset. (K) The confusion map of the classified result.

However, for practical application, it is well known that triboelectric output will be susceptible to environmental variations (e.g., humidity, temperature) [49–51]. Also, we cannot ensure that the sensor will be worn at the exact same position as the last time used. All these issues will affect the signal waveform generated from the multidimensional sensor and decrease the recognition accuracy. To improve reliability, we attempted to extend a more comprehensive dataset from one single day to multiple days (Fig. S4), and the proportion of training and testing datasets is also kept at 80% and 20%. The dataset used in Fig. 3J and K was labeled as Day 0. When fed the dataset of Day 1 into the SVM model that was trained by Day 0, the accuracy decreased to 31.7% due to the different environmental conditions and positions (Fig. S4A). Nevertheless, as the number of days increases, the accuracy has significantly improved to 90% (Fig. S4B to F).

Benefiting from the rapid development of VR/AR technologies, diversified immersive HMIs have been widely used in gaming, digital twin, social media, surgical training, etc. [52,53]. Here, we demonstrated a VR fitness game (constructed by Unity 3D) controlled by the multidimensional sensor integrated belt. The schematic diagram of the whole game system is exhibited in Fig. 4A to C. First, the triboelectric signal generated from waist motions will be gathered by Arduino. Then, the equivalent analog signal spectrum collected by Arduino will be transmitted to the PC through wireless transmission. Based on the received data, the SVM model that was trained by multiple days’ dataset could recognize the feature and send a command to Unity based on TCP/IP communication. This compels the virtual character to do corresponding actions. A photograph of the whole system and the VR interface is shown in Fig. 4D. In this fitness game, we choose 5 motions to control the virtual character. Motions of walking, anticlockwise spinning, running, clockwise spinning, and left turning were used for controlling the character to walk forward, go right, run forward, go right, and turn left, respectively (Fig. 4E to I). In the meantime, according to the data in Fig. S5, the calorie consumption during the exercise process was calculated and recorded simultaneously (Movie S1). In addition, this VR sport system also could have a rough distinction of waist-spinning velocity. For instance, we further measured the signals of fast/slow anticlockwise waist spinning and fast/slow clockwise waist spinning (labeled by 0 to 3, respectively), as shown in Fig. S6, and built a dataset following the same rules as before (e.g., each motion contains 100 samples, and the proportion of training and testing datasets is 4:1). After being trained by the SVM model, the recognition accuracy could reach 95% and the confusion matrix is depicted in Fig. S7. Based on the machine learning results, when we spin the waist at a low velocity, the character in VR space will have no reactions (Fig. S8A and C). In contrast, when spinning the waist at a high velocity, the character will move to the right or left according to the spinning directions (Fig.S8B and D). The whole process can be found in Movie S2.

Fig. 4.

Fig. 4.

Open in a new tab

The VR fitness game based on the multidimensional sensor. (A and B) The schematic diagram of the control system. (C) The flowchart for the control process. (D) The photograph of the whole system and VR interface. (E to I) Demonstrations of virtual character controlling through gait and waist movement. Scale bars in (D) to (I) represent 35 cm.

Excellent motion sensors are expected to be available for arbitrary trajectories, not only for simple straight or circle trails but also for composite trails. The motions of the shank are the results of hip joint motions and knee joint motions, which should be more complicated than the waist motions [54,55]. Our multidimensional sensor could also be used to detect this complex movement. To prove it, we further wear the sensor at the ankle position to collect signals generated from shank motions. As demonstrated in Fig. 5A to D, signals of left kick, right kick, straight kick, and slightly straight kick (labeled by 0 to 3) are successfully obtained. Then, the collected data (each type contains 100 samples) were fed to the SVM model for machine learning (training set 80%, test set 20%), and the confusion matrix with a recognition accuracy of 80% is demonstrated in Fig. S9. The relevant low accuracy could be attributed to the abundant details of signals generated from shank motions and the limited computability of the SVM model.

Fig. 5.

Fig. 5.

Open in a new tab

The application of the multidimensional sensor to detect shank movements. (A to D) Signals collect from motions of left kick, right kick, straight kick, and slightly straight kick. (E) The structure of the 1D CNN model. (F) The visualized clustered results of the dataset. (G) The accuracy variation of the training set and validation set, (H) The confusion map of classified results. (I and J) The demonstration of the VR football shooting game. Scale bars in (I) to (L) represent 35 cm.

Owing to the high level of performance across various types of data, deep learning technology has become a favored subset of machine learning, which could achieve a higher accuracy of complex recognition [56,57]. In particular, CNN technology of deep learning has been proven to be very effective for the overall dataset and suitable for analyzing time sequences of triboelectric sensor data [58,59]. In this regard, we constructed a one-dimensional (1D) CNN model for precisely recognizing shank motions. The detailed structure of the 1D CNN model is depicted in Fig. 5E, which includes 3 convolution layers, 3 max-pooling layers, and 1 fully connected layer that outputs the predicted result of 4 motions. Each type of dataset is separated into a training set (60%), a validation set (20%), and a testing set (20%). After 200 epochs, the accuracy of the training dataset could reach 100%, and the accuracy of the validation set could reach 98.3%, as shown in Fig. 5G. Then, the trained 1D CNN model could assist the testing set to achieve 97.5% accuracy of recognition, and the confusion map of classified results is exhibited in Fig. 5H. The t-SNE-aided visualization results of the dataset before and after training are shown in Fig. 5F and Fig. S10, respectively. Similar to the SVM model, to improve reliability, we further extend the dataset from 1 day to 5 days, and the accuracy could be increased from 77.5% to 95%, as illustrated in Fig. S11. As a practical demonstration based on the deep learning results, a football shooting VR game is designed and performed (Movie S3). In this VR scenario, the left, right, and straight kicks are set as the command for kicking the football to the left, right, and center areas of the goal (Fig. 5I to K). Besides, by determining the force of kicking, the distance of football trajectory can also be controlled. For instance, a slight kick will shoot the football to a closer distance (Fig. 5I).

Discussion

In summary, a self-powered wearable multidimensional motion sensor is developed for intelligent indoor sports applications. Due to the integration of a vertical acceleration sensor and a planar angular velocity sensor, the multidimensional motion sensor could have a sensitive response to arbitrary movement trajectories. By combining with a belt, the multidimensional sensor can be used to detect waist or low limb motions, including waist spinning, turning, walking, and running. With the aid of the SVM machine learning algorithm, 93.8% accuracy in differentiating the above movements has been achieved. Furthermore, a VR fitness game controlled by the multidimensional motion sensor was demonstrated, and the calorie consumption during the whole exercise process was calculated and recorded. In addition, this multidimensional motion sensor is also competent to collect signals that contain more abundant information or details. Accordingly, the sensor was further worn at the ankle position to track the motion of the shank, and a high accuracy (97.5%) of identifying 4 motions including left, right, straight, and slight straight kicking was obtained through 1D CNN-based deep learning analytics. Looking forward, the multidimensional sensor offered an energy-saving and intelligent solution for movement trajectory tracking, which exhibits great potential in future intelligent home-based sports, VR gaming, and rehabilitation.

Materials and Methods

Fabrication of the multidimensional motion sensor

The upper shell, middle shell, tray, weight plate, pedestal, and lower shell were designed by 3D max and directly printed by a 3D printer (ANYCUBIC 3D-Printer 4Max Pro) using polylactic acid filament. Nickel conductive textiles were selected to be pasted on the inner surface of the upper and lower shells as upper and lower electrodes, respectively.

The Eco-flex pad and ball were fabricated by a mold-casting method. First, part A and part B of Eco-flex 0020 were mixed with a 1:1 weight ratio. Then, the obtained Eco-flex gel was poured into 3D-printed molds with patterned and ball shapes, respectively. After drying in an oven at 60 °C, the Eco-flex pad and ball were obtained.

Measurement and characterization

The acceleration stimuli were provided by an exciter WA-0308. The angular velocity was controlled by a Spin Coater KW-4A. The output from the acceleration sensor was measured by a Keithley 6514 electrometer. Outputs of the angular velocity sensor were tested by an oscilloscope (DSO-X3034A, Agilent). Analog voltage signals generated from the multidimensional sensor were collected by Arduino MEGA 2560. The 1D CNN models were constructed in Python with Keras and TensorFlow backend.

Acknowledgments

Funding: This work was supported by the China Scholarship Council (No. 202006790091), the RIE Advanced Manufacturing and Engineering (AME) programmatic grant A18A4b0055 “Nanosystems at the Edge” at NUS, Singapore, the National Key Research and Development Program of China (Grant No. 2019YFB2004800, Project No. R-2020-S-002), the National Natural Science Foundation of China (52173274), the National Key R & D Project from Minister of Science and Technology (2021YFA1201603), and the Strategic Priority Research Program of the Chinese Academy of Sciences (XDA16021101). Author contributions: Z.H.G. conceived the idea and designed the experiments. Z.H.G., Z.Z., K.A., T.H., and Z.S. conducted the experiments. Z.H.G., X.P., and C.L. contributed to the writing of the manuscript. Competing interests: The authors declare that they have no competing interests.

Data Availability

All data needed to evaluate the conclusions in the paper are present in the paper and/or the Supplementary Materials.

Supplementary Materials

Supplementary 1

Fig. S1. The fabrication process of the Eco-flex pad.

Fig. S2. The outputs of vertical mode sensors that adopted the patterned or original Eco-flex pad.

Fig. S3. Cluster results of 8 movements’ datasets.

Fig. S4. The accuracy of 6 movements (0: walking, 1: running, 2: left turn, 3: right turn, 4: clockwise spinning, 5: anticlockwise spinning) versus different days.

Fig. S5. The calorie consumption of 4 movements.

Fig. S6. The waveform of motions: (0) Fast anticlockwise spin, (1) Slow anticlockwise spin, (2) Fast clockwise spin, and (3) Slow clockwise spin.

Fig. S7. The confusion matrix of SVM mode for (0) Fast anticlockwise spin, (1) Slow anticlockwise spin, (2) Fast clockwise spin, and (3) Slow clockwise spin.

Fig. S8. The corresponding screenshot of slow anticlockwise spinning, fast anticlockwise spinning, slow clockwise spinning, and fast clockwise spinning in VR space of Unity.

Fig. S9. The confusion matrix of SVM mode for (0) left kick, (1) right kick, (2) straight kick, and (3) slightly straight kick dataset.

Fig. S10. Cluster results of 4 movements’ datasets (0: left kick, 1: right kick, 2: straight kick, 3: slightly straight kick) by SVM after training.

Fig. S11. The accuracy of 4 movements (0: left kick, 1: right kick, 2: straight kick, 3: slightly straight kick) versus different days.

Click here for additional data file. (8.4MB, docx)
Supplementary 2

Movies S1 to S3

Click here for additional data file. (10.1MB, zip)

References

  • 1.Zhang Z, Wen F, Sun Z, Guo X, He T, Lee C. Artificial intelligence-enabled sensing technologies in the 5g/internet of things era: From virtual reality/augmented reality to the digital twin. Adv Intell Syst. 2022;4(7):2100228. [Google Scholar]
  • 2.He T, Lee C. Evolving flexible sensors, wearable and implantable technologies towards bodynet for advanced healthcare and reinforced life quality. IEEE Open J Circuits Syst. 2021;2:702–720. [Google Scholar]
  • 3.Liu L, Guo X, Lee C. Promoting smart cities into the 5g era with multi-field internet of things (IoT) applications powered with advanced mechanical energy harvesters. Nano Energy. 2021;88: Article 106304. [Google Scholar]
  • 4.Yang Y, Guo X, Zhu M, Sun Z, Zhang Z, He T, Lee C. Triboelectric nanogenerator enabled wearable sensors and electronics for sustainable internet of things integrated green earth. Adv Energy Mater. 2023;13(1):2203040. [Google Scholar]
  • 5.Su L, Xiong Q, Wang H, Zi Y. Porous-structure-promoted tribo-induced high-performance self-powered tactile sensor toward remote human-machine interaction. Adv Sci. 2022;9(32):2203510. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Zhang Q, Xin C, Shen F, Gong Y, Zi Y, Guo H, Li Z, Peng Y, Zhang Q, Wang ZL. Human body IoT systems based on the triboelectrification effect: Energy harvesting, sensing, interfacing and communication. Energy Environ Sci. 2022;15(9):3688–3721. [Google Scholar]
  • 7.Fu J, Song Z, Xu G, Wang H, Li X, Wang J, Ding W, Ren W, Lei IM, Zi Y. Optical emission from triboelectric gas discharge toward self-powered gas sensing. Adv Opt Mater. 2023;11(9):2202697. [Google Scholar]
  • 8.Zhang Q, Jin T, Cai J, Xu L, He T, Wang T, Tian Y, Li L, Peng Y, Lee C. Wearable triboelectric sensors enabled gait analysis and waist motion capture for IoT-based smart healthcare applications. Adv Sci. 2022;9(4):2103694. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Gao S, He T, Zhang Z, Ao H, Jiang H, Lee C. A motion capturing and energy harvesting hybridized lower-limb system for rehabilitation and sports applications. Adv Sci. 2021;8(20):2101834. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Wang HL, Guo ZH, Pu X, Wang ZL. Ultralight iontronic triboelectric mechanoreceptor with high specific outputs for epidermal electronics. Nano-Micro Lett. 2022;14(1):86. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Son D, Lee J, Qiao S, Ghaffari R, Kim J, Lee JE, Song C, Kim SJ, Lee DJ, Jun SW, et al. . Multifunctional wearable devices for diagnosis and therapy of movement disorders. Nat Nanotechnol. 2014;9(5):397–404. [DOI] [PubMed] [Google Scholar]
  • 12.Ruegsegger GN, Booth FW. Health benefits of exercise. Cold Spring Harb Perspect Med. 2018;8(7): Article a029694. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Roy AL, Sarkar H, Dutta A, Bhattacharyya TK. A high precision SOI MEMS–CMOS±4g piezoresistive accelerometer. Sensors Actuators A Phys. 2014;210:77–85. [Google Scholar]
  • 14.Guo ZH, Jiao YC, Wang HL, Zhang C, Liang F, Liu JL, Yu HD, Li CM, Zhu G, Wang ZL. Self-powered electrowetting valve for instantaneous and simultaneous actuation of paper-based microfluidic assays. Adv Funct Mater. 2019;29(15):1808974. [Google Scholar]
  • 15.Guo ZH, Wang HL, Shao J, Shao Y, Jia L, Li L, Pu X, Wang ZL. Bioinspired soft electroreceptors for artificial precontact somatosensation. Sci Adv. 2022;8(21): Article eabo5201. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Wang HL, Guo ZH, Zhu G, Pu X, Wang ZL. Boosting the power and lowering the impedance of triboelectric nanogenerators through manipulating the permittivity for wearable energy harvesting. ACS Nano. 2021;15(4):7513–7521. [DOI] [PubMed] [Google Scholar]
  • 17.Bai Z, He T, Zhang Z, Xu Y, Zhang Z, Shi Q, Yang Y, Zhou B, Zhu M, Guo J, et al. . Constructing highly tribopositive elastic yarn through interfacial design and assembly for efficient energy harvesting and human-interactive sensing. Nano Energy. 2022;94: Article 106956. [Google Scholar]
  • 18.Zhang P, Guo W, Guo ZH, Ma Y, Gao L, Cong Z, Zhao XJ, Qiao L, Pu X, Wang ZL. Dynamically crosslinked dry ion-conducting elastomers for soft iontronics. Adv Mater. 2021;33(31): Article e2101396. [DOI] [PubMed] [Google Scholar]
  • 19.Le X, Shi Q, Sun Z, Xie J, Lee C. Noncontact human–machine interface using complementary information fusion based on mems and triboelectric sensors. Adv Sci. 2022;9(21):2201056. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.He T, Guo X, Lee C. Flourishing energy harvesters for future body sensor network: From single to multiple energy sources. IScience. 2021;24(1): Article 101934. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Zhu M, He T, Lee C. Technologies toward next generation human machine interfaces: From machine learning enhanced tactile sensing to neuromorphic sensory systems. Appl Phys Rev. 2020;7(3): Article 031305. [Google Scholar]
  • 22.Zhu M, Yi Z, Yang B, Lee C. Making use of nanoenergy from human–nanogenerator and self-powered sensor enabled sustainable wireless IoT sensory systems. Nano Today. 2021;36: Article 101016. [Google Scholar]
  • 23.Sun Z, Zhu M, Lee C. Progress in the triboelectric human–machine interfaces (HMIs)—Moving from smart gloves to AI/haptic enabled HMI in the 5G/IoT era. Nanoenergy Adv. 2021;1(1):81–120. [Google Scholar]
  • 24.Xu M, Wang P, Wang YC, Zhang SL, Wang AC, Zhang C, Wang Z, Pan X, Wang ZL. A soft and robust spring based triboelectric nanogenerator for harvesting arbitrary directional vibration energy and self-powered vibration sensing. Adv Energy Mater. 2018;8(9):1702432. [Google Scholar]
  • 25.Xie X, Chen Y, Jiang J, Li J, Yang Y, Liu Y, Yang L, Tu X, Sun X, Zhao C, et al. . Self-powered gyroscope angle sensor based on resistive matching effect of triboelectric nanogenerator. Adv Mater Tech. 2021;6(10):2100797. [Google Scholar]
  • 26.Shi Q, Wu H, Wang H, Wu H, Lee C. Self-powered gyroscope ball using a triboelectric mechanism. Adv Energy Mater. 2017;7(22):1701300. [Google Scholar]
  • 27.Xie L, Chen X, Wen Z, Yang Y, Shi J, Chen C, Peng M, Liu Y, Sun X. Spiral steel wire based fiber-shaped stretchable and tailorable triboelectric nanogenerator for wearable power source and active gesture sensor. Nano-Micro Lett. 2019;11:1–10. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28.Zhang B, Wu Z, Lin Z, Guo H, Chun F, Yang W, Wang ZL. All-in-one 3D acceleration sensor based on coded liquid–metal triboelectric nanogenerator for vehicle restraint system. Mater Today. 2021;43:37–44. [Google Scholar]
  • 29.Pu X, Liu M, Chen X, Sun J, Du C, Zhang Y, Zhai J, Hu W, Wang ZL. Ultrastretchable, transparent triboelectric nanogenerator as electronic skin for biomechanical energy harvesting and tactile sensing. Sci Adv. 2017;3(5): Article e1700015. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Guo X, He T, Zhang Z, Luo A, Wang F, Ng EJ, Zhu Y, Liu H, Lee C. Artificial intelligence-enabled caregiving walking stick powered by ultra-low-frequency human motion. ACS Nano. 2021;15(12):19054–19069. [DOI] [PubMed] [Google Scholar]
  • 31.Sun Z, Zhu M, Zhang Z, Chen Z, Shi Q, Shan X, Yeow RCH, Lee C. Artificial intelligence of things (AIoT) enabled virtual shop applications using self-powered sensor enhanced soft robotic manipulator. Adv Sci. 2021;8(14):2100230. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Huang M, Zhu M, Feng X, Zhang Z, Tang T, Guo X, Chen T, Liu H, Sun L, Lee C. Intelligent cubic-designed piezoelectric node (icupe) with simultaneous sensing and energy harvesting ability toward self-sustained artificial intelligence of things (AIoT). ACS Nano. 2023;17(7):6435–6451. [DOI] [PubMed] [Google Scholar]
  • 33.Yang Y, Shi Q, Zhang Z, Shan X, Salam B, Lee C. Robust triboelectric information-mat enhanced by multi-modality deep learning for smart home. InfoMat. 2023;5(1): Article e12360. [Google Scholar]
  • 34.Wen F, Sun Z, He T, Shi Q, Zhu M, Zhang Z, Li L, Zhang T, Lee C. Machine learning glove using self-powered conductive superhydrophobic triboelectric textile for gesture recognition in VR/AR applications. Adv Sci. 2020;7(14):2000261. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35.Zhu J, Ji S, Yu J, Shao H, Wen H, Zhang H, Xia Z, Zhang Z, Lee C. Machine learning-augmented wearable triboelectric human-machine interface in motion identification and virtual reality. Nano Energy. 2022;103: Article 107766. [Google Scholar]
  • 36.Sun Z, Zhu M, Shan X, Lee C. Augmented tactile-perception and haptic-feedback rings as human-machine interfaces aiming for immersive interactions. Nat Commun. 2022;13(1):5224. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 37.Wen F, Zhang Z, He T, Lee C. AI enabled sign language recognition and VR space bidirectional communication using triboelectric smart glove. Nat Commun. 2021;12(1):5378. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38.Zhang Z, He T, Zhu M, Sun Z, Shi Q, Zhu J, Dong B, Yuce MR, Lee C. Deep learning-enabled triboelectric smart socks for IoT-based gait analysis and VR applications. Npj Flex Electron. 2020;4(1):29. [Google Scholar]
  • 39.Shi Q, Sun Z, Le X, Xie J, Lee C. Soft robotic perception system with ultrasonic auto-positioning and multimodal sensory intelligence. ACS Nano. 2023;17(5):4985–4998. [DOI] [PubMed] [Google Scholar]
  • 40.Zhu M, Sun Z, Lee C. Soft modular glove with multimodal sensing and augmented haptic feedback enabled by materials’ multifunctionalities. ACS Nano. 2022;16(9):14097–14110. [DOI] [PubMed] [Google Scholar]
  • 41.Bai Z, Xu Y, Lee C, Guo J. Autonomously adhesive, stretchable, and transparent solid-state polyionic triboelectric patch for wearable power source and tactile sensor. Adv Funct Mater. 2021;31(37):2104365. [Google Scholar]
  • 42.Lan K-C, Shih W-Y. Using simple harmonic motion to estimate walking distance for waist-mounted PDR. Paper presented at: 2012 IEEE Wireless Communications and Networking Conference (WCNC); 2012 Apr 1–4; Paris, France.
  • 43.Misiaszek JE. Early activation of arm and leg muscles following pulls to the waist during walking. Exp Brain Res. 2003;151:318–329. [DOI] [PubMed] [Google Scholar]
  • 44.Auria L, Moro RA. Support vector machines (SVM) as a technique for solvency analysis. Berlin (Germany): DIW Berlin; 2008.
  • 45.Pan Y, Jiang J, Wang R, Cao H. Advantages of support vector machine in QSPR studies for predicting auto-ignition temperatures of organic compounds. Chemom Intell Lab Syst. 2008;92(2):169–178. [Google Scholar]
  • 46.Anguita D, Ghio A, Greco N, Oneto L, Ridella S. Model selection for support vector machines: Advantages and disadvantages of the machine learning theory. Paper presented at: The 2010 international joint conference on neural networks (IJCNN); 2010 Jul 18–23; Barcelona, Spain.
  • 47.Van der Maaten L, Hinton G. Visualizing data using t-SNE. J Mach Learn Res. 2008;9(86):2579–2605. [Google Scholar]
  • 48.Zhang Z, Shi Q, He T, Guo X, Dong B, Lee J, Lee C. Artificial intelligence of toilet (AI-Toilet) for an integrated health monitoring system (IHMS) using smart triboelectric pressure sensors and image sensor. Nano Energy. 2021;90: Article 106517. [Google Scholar]
  • 49.Bui V-T, Huynh ND, Chau NM, Kim W, Kim H, Oh I-K, Choi D. High-temperature operatable triboelectric nanogenerator using microdome-patterned polyimide for self-powered sensors. Nano Energy. 2022;101: Article 107612. [Google Scholar]
  • 50.Wang J, Wu H, Fu S, Li G, Shan C, He W, Hu C. Enhancement of output charge density of TENG in high humidity by water molecules induced self-polarization effect on dielectric polymers. Nano Energy. 2022;104: Article 107916. [Google Scholar]
  • 51.Zhou Q, Lee K, Kim KN, Park JG, Pan J, Bae J, Baik JM, Kim T. High humidity-and contamination-resistant triboelectric nanogenerator with superhydrophobic interface. Nano Energy. 2019;57:903–910. [Google Scholar]
  • 52.Shen S, Yi J, Sun Z, Guo Z, He T, Ma L, Li H, Fu J, Lee C, Wang ZL. Human machine interface with wearable electronics using biodegradable triboelectric films for calligraphy practice and correction. Nano-Micro Lett. 2022;14(1):225. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 53.Zhu M, Sun Z, Chen T, Lee C. Low cost exoskeleton manipulator using bidirectional triboelectric sensors enhanced multiple degree of freedom sensory system. Nat Commun. 2021;12(1):2692. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 54.Liu T, Inoue Y, Shibata K, Morioka H. Development of wearable sensor combinations for human lower extremity motion analysis. Paper presented at: Proceedings 2006 IEEE international conference on robotics and automation, 2006. ICRA; 2006 May 15–19; Orlando, FL.
  • 55.Leardini A, O’Connor JJ, Giannini S. Biomechanics of the natural, arthritic, and replaced human ankle joint. J Foot Ankle Res. 2014;7(1):1–16. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 56.Wang Y, Hu Z, Wang J, Liu X, Shi Q, Wang Y, Qiao L, Li Y, Yang H, Liu J, et al. . Deep learning-assisted triboelectric smart mats for personnel comprehensive monitoring toward maritime safety. ACS Appl Mater Interfaces. 2022;14(21):24832–24839. [DOI] [PubMed] [Google Scholar]
  • 57.Dong B, Zhang Z, Shi Q, Wei J, Ma Y, Xiao Z, Lee C. Biometrics-protected optical communication enabled by deep learning–enhanced triboelectric/photonic synergistic interface. Sci Adv. 2022;8(3): Article eabl9874. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 58.Hasan M, Ullah S, Khan MJ, Khurshid K. Comparative analysis of Svm, ANN and Cnn for classifying vegetation species using hyperspectral thermal infrared data. Int Arch Photogramm. 2019;42:1861–1868. [Google Scholar]
  • 59.Baranwal SK, Jaiswal K, Vaibhav K, Kumar A, Srikantaswamy R. Performance analysis of brain tumour image classification using CNN and SVM. Paper presented at: 2020 Second International Conference on Inventive Research in Computing Applications (ICIRCA); 2020 July 15–17; Coimbatore, India.

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplementary 1

Fig. S1. The fabrication process of the Eco-flex pad.

Fig. S2. The outputs of vertical mode sensors that adopted the patterned or original Eco-flex pad.

Fig. S3. Cluster results of 8 movements’ datasets.

Fig. S4. The accuracy of 6 movements (0: walking, 1: running, 2: left turn, 3: right turn, 4: clockwise spinning, 5: anticlockwise spinning) versus different days.

Fig. S5. The calorie consumption of 4 movements.

Fig. S6. The waveform of motions: (0) Fast anticlockwise spin, (1) Slow anticlockwise spin, (2) Fast clockwise spin, and (3) Slow clockwise spin.

Fig. S7. The confusion matrix of SVM mode for (0) Fast anticlockwise spin, (1) Slow anticlockwise spin, (2) Fast clockwise spin, and (3) Slow clockwise spin.

Fig. S8. The corresponding screenshot of slow anticlockwise spinning, fast anticlockwise spinning, slow clockwise spinning, and fast clockwise spinning in VR space of Unity.

Fig. S9. The confusion matrix of SVM mode for (0) left kick, (1) right kick, (2) straight kick, and (3) slightly straight kick dataset.

Fig. S10. Cluster results of 4 movements’ datasets (0: left kick, 1: right kick, 2: straight kick, 3: slightly straight kick) by SVM after training.

Fig. S11. The accuracy of 4 movements (0: left kick, 1: right kick, 2: straight kick, 3: slightly straight kick) versus different days.

Click here for additional data file. (8.4MB, docx)
Supplementary 2

Movies S1 to S3

Click here for additional data file. (10.1MB, zip)

Data Availability Statement

All data needed to evaluate the conclusions in the paper are present in the paper and/or the Supplementary Materials.

Articles from Research are provided here courtesy of American Association for the Advancement of Science (AAAS) and Science and Technology Review Publishing House

RESOURCES