Reliability and validity of muscle activity analysis using wearable electromyographs

Abstract

[Purpose] The aim of this study was to develop a novel wearable surface electromyograph called NOK, and compare its reliability and validity to an existing electromyograph. [Participants and Methods] The study participants were 23 healthy university students (Seven males and 16 females; age 20.3 ± 1.1 years [mean ± standard deviation]; height 162.0 ± 6.7 cm; weight 58.4 ± 10.1 kg) who all gave informed written consent. The newly developed electromyograph (NOK) features a rubberized skin contact surface that requires no electrodes and allows the acquisition of up to 10 channels of muscle waveforms on a portable personal computer. After measuring maximal isometric elbow extension and flexion, we examined muscle waveforms during isometric contractions of elbow joint flexion and extension at approximately 50% of maximal voluntary contraction using both NOK and Delsys electromyographs and compared the results of the two devices. [Results] We found a significant moderate correlation between the measurements by the two devices for biceps and triceps. The measurements by the two devices also showed strong measure-retest reliability. Systematic errors were observed for elbow flexion and extension in the two measurements, indicating limited agreement between the two measurement methods. [Conclusion] Although the new device also has high repeatability and reliability, it is unsuitable for analyzing detailed muscle activity. However, since it can measure up to 10 channels of muscle activity, it is expected to be used in the rehabilitation and sports field in the future.

INTRODUCTION

The development of wearable devices has been remarkable and their worldwide use was projected to show a compound annual growth rate of 38% from 2017 to 2025¹⁾. In 2019 alone, an estimated 225 million portable devices were sold²⁾. Physiological signals, such as blood pressure, oxygen saturation, electrocardiography, and electromyography (EMG) signals constitute essential data collected by wearable devices like watches, pendants, bracelets, and various accessories³⁾.

In the context of evaluating sports performance, numerous efforts have been made to quantify physical functions using EMG, motion analysis, and muscle strength devices. Measurement of muscle activity using EMG is useful to assess the level of central nervous system excitation and explain specific physical movements^{4, 5)}. Furthermore, muscle fatigue can be measured⁶⁾, and exercise guidance can be provided based on fatigue levels. Additionally, EMG data on muscle activity during exercise or sports performance can be effective of visual feedback, even to assess sports accessories such as golf gloves⁷⁾.

In conventional surface EMG, electrodes are affixed directly to the skin necessitating skin preparation. When measuring multiple muscles, much time is required to prepare the skin sites and apply multiple electrodes. This imposes physical burdens on users and demands much time.

In recent years, wearable EMG devices have been developed to reduce the burden on users without the need for skin preparation. Specifically, accessories like bracelets or armbands have been designed to capture EMG signals, offering valuable non-invasive information. The armbands typically feature multiple EMG sensors arranged radially on a flexible band, making them easy to apply and adaptable for various applications⁸⁾, including hand gesture recognition⁹⁾ and therapy monitoring¹⁰⁾.

Eliminating the drawbacks of conventional EMG devices is expected to benefit both users and researchers by facilitating measurement convenience. We have developed an EMG that is useful for both the participant and the measurer. The skin contact area is made of rubber to soften the sensation against the skin. We have also proposed a device that can be connected via Bluetooth to a portable device, such as an iPad, and has up to 10 channels of EMG, making it easy to use clinically. However, whether wearable EMG devices can reliably measure muscle activity data remains unclear. To establish wearable EMG devices as a versatile tool in society, scientific verification is essential. This study aimed to assess the reliability and validity of muscle activity measurements by a newly developed wearable EMG device.

PARTICIPANTS AND METHODS

Twenty-three healthy right-handed college students (7 males, 16 females; aged 19 to 22, mean ± SD: age 20.3 ± 1.1 years.; height 162.0 ± 6.7 cm, weight 58.4 ± 10.1 kg) were enrolled. Juntendo University medical ethical committee approved the study, which adhered to the tenets of the Declaration of Helsinki (ID 21-015). All participants provided written informed consent. The new wearable EMG device (NOK, Co., Kanagawa, Japan) is equipped with a rubber sensor cover designed to minimize skin discomfort. It can measure up to 10 channels of EMG and transmit the data to a portable device such as an iPad using Bluetooth technology. It operates at a sampling frequency of 500 Hz, with a low-pass filter set at 480 Hz and high-pass filter at 10 Hz (Fig. 1).

Fig. 1.

New device.

The existing wearable EMG device for comparison was the Delsys wireless electromyograph (InterRehab Co., Tokyo, Japan), a well-established and reliable wireless EMG device^{11, 12)}. It operates at a sampling frequency of 2,000 Hz, with a low-pass filter set at 850 Hz and high-pass filter at 10 Hz. Maximal voluntary contraction (MVC) muscle force for elbow joint extension and flexion was measured in the study participants. Muscle activity during these each joint movement was simultaneously recorded by EMG (NOK) and EMG (Delsys) devices. Additionally, muscle strength was assessed using a hand-held dynamometer (Mobie, Sakai Medical Co., Tokyo, Japan). Measurements were taken twice for the upper limb biceps and triceps on the dominant hand side. Participants were then instructed to perform isometric contractions of elbow joint flexion and extension exercises using an adjustable dumbbell weight, set at approximately 50% of their MVC. For elbow flexion, the participants were seated on a chair, with the elbow joint in a 90-degree flexed position. For elbow extension, participants were in a supine posture with the weight held in slight elbow joint flexion. Isometric contractions were measured twice with each contraction lasting 5 s.

Mean, standard deviation and variance were calculated for the two measurements obtained by each device (Fig. 1). To assess whether the results obtained by EMG (NOK) could be predicted using the data obtained by EMG (Delsys), linear regression analyses were conducted, with data from EMG (NOK) as the independent variables and data from EMG (Delsys) as the dependent variables. Spearman’s correlation was examined. Reliability for both systems was determined using a single measure of a two-way random model. As per Munro¹³⁾, reliability coefficients were categorized as follows: (0.90–1.00) very high correlation, (0.70–0.89) high correlation, (0.50–0.69) moderate correlation, (0.26–0.49) low correlation, and (0.00–0.25) little to no correlation.

Additionally, to assess agreement between the data obtained by the two devices, intraclass correlation coefficients (ICCs) (2, 1) were calculated and categorized as follows: (<0.5) poor agreement, (0.5–0.75) moderate agreement, (0.75–0.9) good agreement, (>0.90) excellent agreement. Cronbach’s α coefficients corrected item–total correlation, and the Spearman item correlation matrix was produced to evaluate the internal consistency of the measurement using 2 device. Agreement between the data obtained by the two devices was further evaluated using Bland–Altman analysis. In the Bland–Altman analysis, the Y-axis difference was calculated by subtracting the sEMG (DELSYS) data from the sEMG (NOK) data (Fig. 2). Statistical analyses were performed using SPSS (Version 24.0, IBM, Tokyo, Japan), with p<0.05 deemed significant (Table 1).

Fig. 2.

Bland–Altman analysis of elbow flexion.

Table 1. Muscle activity by EMG device: mean, SD, and ICC values.

		Correlation	1st session mean	2nd session mean	ICC (95% CI)	Cronbach’s α
			activity & SD (mV)	activity & SD (mV)
sEMG (NOK)	Elbow flexion	r=0.59*	0.723 ± 0.378	0.712 ± 0.407	0.959 (0.959–0.982)	0.979
	Elbow extension	r=0.42*	0.226 ± 0.214	0.224 ± 0.239	0.971 (0.934–0.988)	0.985
sEMG (DELSYS)	Elbow flexion		1.617 ± 1.059	1.543 ± 0.821	0.835 (0.650–0.927)	0.910
	Elbow extension		1.214 ± 1.734	1.322 ± 1.727	0.727 (0.457–0.874)	0.842

p<0.05 sEMG (NOK) vs. sEMG (DELSYS).

EMG: electromyography; ICC: intraclass correlation coefficient; SD: standard deviation; CI: confidence interval.

RESULTS

Mean ± SD maximum muscle strength measured by EMG (NOK) was 15.0 ± 4.4 kgf in elbow flexion and 10.6 ± 4.4 kgf in elbow extension, while it was 13.9 ± 4.3 kgf in elbow flexion and 10.7 ± 3.9 kgf in elbow extension when measured by EMG (Delsys). Analysis by Student’s t-test found no significant between device difference in measurements for elbow extension but a significant difference for elbow flexion. Regarding correlation between the two devices, a significant correlation was found for biceps (r=0.59, p=0.003) and triceps (r=0.42, p=0.05). All of these values were significant and high Cronbach’s α. ICC tended to be higher in EMG (NOK) than in EMG (DELSYS).

The fixation errors for elbow flexion and extension by both measurement devices ranged from −1.17 to −0.55 and −1.70 to −0.39, respectively, with none of them including zero, indicating a fixation error. Additionally, the plots showed non-random variation and proportional errors. Specifically for elbow flexion (Fig. 2), the Y-axis varied slightly below 0. EMG (Delsys) measurements were slightly smaller than EMG (NOK) measurements, contributing to the proportional errors. The plots were not randomly scattered within the margin of error (LOA; limits of agreement), indicating low agreement between the two measurement methods. The regression coefficient for the two measurements of elbow flexion was −0.93, signifying a significant difference and ruling out a regression coefficient of zero, suggesting the presence of a proportional error.

In the case of elbow extension, the measurements were also slightly scattered below 0 on the Y-axis (Fig. 3). Similar to the findings for elbow flexion, EMG (Delsys) measurements were slightly smaller than EMG (NOK) measurements, leading to proportional errors. The plots were not randomly scattered within the margin of error (LOA), indicating low agreement between the two measurement methods. The regression coefficients for elbow flexion and extension were 0.78 and 0.99, respectively, significantly different from zero, suggesting the presence of a proportional error.

Fig. 3.

Bland–Altman analysis of elbow joint extension.

DISCUSSION

In this study, we measured muscle activity in the biceps and triceps of 23 university students using two devices, EMG (NOK) and EMG (Delsys), and compared the reliability and validity of the measurements of the two devices. There was a significant between device difference in maximum muscle force measured by the two devices during elbow flexion. However, since dumbbell measurements using sEMG (NOK) and sEMG (DELSYS) used dumbbells of the same mass, we consider that the muscle force values when measuring muscle activity at about 50% muscle output are equivalent. Therefore, there is any difference not attributable to maximum muscle force in this case.

Both devices exhibited high ICCs between tests and Cronbach’s α coefficients were also high, indicating that both devices have high reliability and validity. Within device accuracy was assured with highly reliable re-measurement.

In another study, the reliability and validity of a newly developed low-cost EMG device compared with that of an established system for measuring muscle activity and fatigue levels was reported by Bawa et al¹¹⁾. This study develops an inexpensive wearable EMG and examines its performance. They found mean Spearman’s correlation was 0.76 with a range of (0.71–0.85) at peak level contraction, while the mean level contraction average was 0.71 at a range of (0.62–0.81).

Jang et al.¹⁴⁾ compared a newly designed surface electromyography device (PSL-EMG-Trl) with a conventional surface electromyography device (BTS-FREEMG1000). The signals obtained from the rectus femoris (RF) and biceps brachii (BB) muscles were compared using Pearson’s correlation. The results indicated an excellent agreement with a high reliability for RF (ICC=0.81–0.96) and BB (ICC=0.83–0.94). The present study also showed high ICC values, indicating high reliability within the device and a strong correlation with the existing Delsys system, suggesting the validity of the new EMG device as a reliable measurement tool.

Regarding limitations, both devices showed systematic errors in elbow flexion and extension. The EMG (NOK) and EMG (Delsys) devices have a large discrepancy in acquisition frequency, with 500 Hz and 2,000 Hz, respectively. This difference may have contributed to errors in data resolution. Additionally, the distance between the metal and rubber electrodes differs between devices, with EMG (NOK) having a considerably larger distance, potentially affecting the collection of muscle fiber activity. Besides the distance between electrodes on the respective sensors, they were concurrently placed on the body so the sites slightly differed which may have potentially introduced a systematic error.

Regarding future prospects, globally, simplified electromyographs, including dry electrodes¹⁵⁾ and semi-dry electrodes¹⁶⁾, have already been developed and their effectiveness has been verified¹¹⁾. The 500 Hz acquisition frequency, although considered low compared to general electromyographs, is suitable for determining approximate muscle activity. Improvements in technology and AI resources may render cheaper devices sufficiently useful.

To recap, we assessed the reliability of a portable, wearable surface EMG device capable of measuring up to 10 channels of muscle activity. The device records muscle activity data, which can be accessed and processed using a dedicated software application running on a portable device such as an iPad. Our findings demonstrate that the new EMG device provides accurate and consistent measurements, akin to existing devices. Differences in measurement accuracy, attributed to distance between electrode placement and measurement frequency, were minimized by the new EMG device. Its low acquisition frequency and suitability for large movements are considered beneficial to determine approximate muscle activity, making it a suitable choice for applications where precise muscle activity measurements are not the primary concern but where mobility and convenience are crucial considerations.

Funding

This research was funded by NOK Corporation.

Conflict of interest

None.