Magnetized joystick with dual-anisotropy for high-capacity wearable human-machine interactions

Abstract

One of the urgent demands for flexible and wearable sensors is to achieve single-channel, less-sensor, and multi-output for effective communications. However, expansion of signal capacity on a single interface often requires sensor arrays, which significantly increases the complexity in electrode layout and multiple device design. In this work, we developed a flexible magnetized joystick (FMJ) with structurally induced eigenfrequency anisotropy by applying laser cutting process to a raw magnetized cubic. Using an eigenfrequency-polarity dual-anisotropy addressing method, we show that single FMJ can achieve four identifiable outputs based on the direction of in-plane mechanical inputs. According to the cantilever beam model, the eigenfrequency library can be expanded by adjusting cutting parameters, dimensions, and material properties, laying the foundation for further expanding signal capacity through the combination of multiple FMJs. The advantages of FMJs in terms of high capacity and multi-functionality are demonstrated through a range of human-machine interaction (HMI) applications, including game control, PowerPoint slides, robot car control, and a fingertip calculator. The increased signal requirement would not impose the complexity of electrical connection thanks to the interference-free parallel design of the electrodes. We expect that the development strategy will provide valuable insights for further advancement of high-capacity wearable HMI.

Introduction

Human-machine interaction (HMI) has transformative potential in virtual reality^1,2, smart healthcare^3–5, and remote robotic control^6,7. The pursuit of further advancements in HMI has led to increasingly higher demands in all aspects of the overall HMI system. Flexible wearable sensors offer exceptional surface adaptability, enabling them to conform to various irregular surfaces, including the human body, and capture signals for effective communication. Compared with conventional rigid sensors, they provide significantly improved comfort, making them a promising alternative. Recently, various flexible wearable sensors based on principles such as capacitive^8,9, resistive¹⁰, triboelectric^11,12, and electromagnetic^13,14 have successfully achieved electrical signal sensing of diverse information, including mechanical force¹⁵, temperature^16,17, sweat^18,19, sound²⁰, direction²¹, and surface property²². This has greatly expanded the application scope of flexible wearable sensors in the field of HMI.

However, as the application range of flexible wearable sensors broadens, performance requirements for such sensors continue to rise. One key objective is to achieve broader signal capacity, enabling the entire system to sustain greater command capability while maintaining an appropriate overall size²³. Numerous studies have reported sensors with dual-signal capability. For instance, based on the resistive principle, bending the sensor varies its resistance, yielding two distinct output signals by distinguishing whether the resistance increases or decreases relative to the baseline²⁴. Another example is based on triboelectric nanogenerators, where long and short presses are used to encode the dot and dash of Morse code²⁵. However, these switch-like devices based on “bending” or “pressing” often face various challenges when further increasing signal capacity. For example, array-based sensing strategies inherently pose challenges such as a multi-fold increase in electrode interface channels, elevated power consumption, multiple wiring and packaging complexity²⁶, making it difficult to achieve the desired portability. By augmenting the coding framework with a defined idle state, these sensors can be easily adapted to ternary systems. However, this approach inevitably raises concerns about signal transmission efficiency. Furthermore, attempts to output multiple distinct signals through threshold setting also will suffer from inaccuracies owing to variations in perceptual thresholds across individuals.

“Sliding” is a widely used input modality in modern smart electronic devices. By mapping distinct directions to specific commands, a single module can deliver significantly more functions than achievable via “pressing” or “bending” alone. Introducing “sliding” into HMI systems has the potential to significantly expand signal capacity. Following this concept, Feng et al. developed a triboelectric sensor with precisely designed electrode patterns to achieve accurate sensing of four sliding directions¹², while the impact of ambient humidity on the triboelectric stability has to be considered^27,28. In recent studies of wearable HMI based on electromagnetic induction, the systems exhibit stable and accurate communication results across different environmental conditions, such as relative humidity from 43.6% to 96.4%, and temperature between 20 ^oC and 80 ^oC^29,30. Even with surrounding magnetic field, e.g. up to 20 mT, the signals by the defomration of magnetoelastomers can maintain with environmental stability, durability, and accuracy. Thanks to the built-in alignment of magnetized moments, magnetoelastomers are also ideal for establishment of the interface where sliding operation is required. When deformed along different directions, the induced electrical signals can provide distinguishable readouts according to the direction of mechanical stimuli and related variation tendency of magnetic flux. Previous related work employed three magnetized micropillars and one non-magnetized micropillar, successfully distinguishing four sliding directions along orthogonal axes by analyzing the polarity and number of electrical signals²¹. However, when distinguishing more directions is required (such as eight directions), this approach may require far more than eight sensors, posing challenges in terms of overall size and portability. When vibrating, elastomers exhibit an eigenfrequency determined by their intrinsic properties³¹. In flexible sensors based on electromagnetic induction, the signal is typically an oscillating and decaying electrical signal that conceals the eigenfrequency of the triggered flexible sensor. By establishing a library of eigenfrequencies for different sensors, the number of sensors required for high-capacity HMI interfaces can be significantly reduced if employing signal eigenfrequency instead of signal quantity addressing^29,30. In the case of a sensor array, signals can be output through single channel without concerns of signal overlapping if multiple eigenfrequencies are introduced. If structural design can further enable a single sensor to output distinct eigenfrequencies when exposed to different sliding directions, the signal capacity of individual sensor can be enhanced in a highly integrated and portable behavior. To date, however, it is still challenging to realize multiple eigenfrequency production within a single piece of magnetized structure.

In view of this, we herein report a wearable HMI interface that can perceive mechanical stimuli from four orthonormal directions with non-overlapping reactions. Via establishing structural anisotropy on a three-dimensional (3D) magnetized device, a single device is capable of producing different eigenfrequencies when deformed from two in-plane axial directions. Furthermore, mechanical stimuli applied in different directions (X + , X − , Y + , Y − ) can be precisely converted to identifiable signals when polar differences were incorporated. Analysis confirms that the anisotropic 3D structure is essential to provide eigenfrequency differences for axes, while the intrinsic magnetized orientation serves as foundation to identify the positive and negative orientations along the same axis. Experimental results demonstrate that the FMJ exhibits reliability, robustness, and anti-interference from environments that takes advantages from the principle of electromagnetic induction. The eigenfrequencies of flexible magnetized joystick (FMJ) can be accurately tuned by varying cutting parameters, material properties, and dimensions, laying the groundwork for integrating multiple FMJs to further expand signal capacity. As a proof-of-concept, we showcase the immense potential of the FMJ, whether used independently or integrated, in diverse scenarios such as daily entertainment, machine interactions, and intelligent control. We expect that the capability of direction-aware reaction and high-capacity communications will provide a valuable reference for the design and optimization of future wearable HMI systems.

Results and discussion

Design and working principles

Human skin contains various sensory receptors that can transduce diverse mechanical stimuli on the skin surface into discernible physiological signals. These signals are then transmitted via the nervous system to the neural center, enabling the formation of distinct perceptions, as shown in Fig. 1a. The sensory process is efficient, rapid, and precise, laying a robust and reliable foundation for rapid subsequent responses. As HMI systems continue to evolve, there is a parallel need for efficient and precise interaction processes. Based on the mentioned human perceptual processes, developing high-capacity HMI interfaces that output non-overlapping signals, thereby enabling machines to accurately discern different human intentions, is an attractive approach. Motivated by this, a fingertip-scaled wearable interface was developed using the principle of electromagnetic induction (Fig. 1b). With a customized polymeric glove, the interface can be tightly attached to the fingertip to establish a functional interaction channel between human and the electric terminal. Such interface not only identifies the in-plane mechanical stimuli from four different directions, but also produces non-overlapping signals to enable subsequent coding for efficient HMI process. As shown in Fig. 1c, the interface consists of two parts: a flexible magnetized joystick (FMJ) and a flexible coil underneath. PDMS and Ecoflex form the flexible substrate of the FMJ, while NdFeB particles embedded within the PDMS/Ecoflex matrix endow the FMJ as a permanent and flexible magnet. The detailed fabrication process of the FMJ is illustrated in Fig. S1, and a corresponding description can be found in the “Methods” section. Microscopic images of the FMJ and NdFeB particles are presented in Figure S2. The corresponding energy-dispersive spectroscopy (EDS) maps of the selected elements reveal that the elastomer and the NdFeB particles are uniformly mixed during the preparation period. The magnetic field simulation results are shown in Fig. S3, which indicates the variation of localized magnetic field when the 3D FMJ was deformed along different orientations. A typical optical image of the FMJ, along with its dimensions, is shown in Fig. 1d. After laser cutting, the four side faces of the raw body exhibit two distinct sets of cutting patterns. The inclined pattern is shown on the two faces which are parallel to the X direction (denoted as Face 1) and the interlaced pattern is shown on the two faces which are parallel to the Y direction (denoted as Face 2). This structural anisotropy results in larger moments of inertia ($I$) when bending along the X direction (as shown in Fig. S4). Therefore, the flexural strength ($EI$, where $E$ is Young’s modulus) along the X direction is higher when compared with the perpendicular direction. Figure 1e further shows the geometrical changes when the device was bent along the directions of Y and X. As revealed during the bending experiment in Fig. S5, a larger force is required to bend the FMJ along X direction when compared with Y direction if the top of the FMJ is displaced by the same distance. Furthermore, an increased cut depth can also cause a larger difference of $I$ between two directions, resulting in more obvious difference in the force required to bend the same angle along X and Y directions. According to the formula for the eigenfrequency of a cantilever beam ($f=\frac{{1.875}^2}{2\pi l^2}\sqrt{\frac{EI}{\rho A}}$, where $l$ is the length of the cantilever beam, $\rho$ is the material density, and $A$ is the cross-sectional area), a larger flexural strength corresponds to a higher eigenfrequency. Therefore, triggering the FMJ along different axes produces different vibration frequencies. Based on this mechanism and the polarity differences caused by the inherent magnetic poles after magnetization, the working principle of this HMI interface is shown in Fig. 1f. When the FMJ is activated, oscillation of magnetized architecture will induce periodical electrical signals in the coil layer due to the variation of localized magnetic flux. Among these signals, those generated by activating the FMJ along different axes (e.g., X or Y) exhibit different eigenfrequencies, which stems directly from the axis-related eigenfrequency differences of the FMJ’s vibration. In addition, signals produced by activating the FMJ along the positive and negative directions of the same axis (e.g., X+ and X − ) display different peak voltage polarities owing to the variations in magnetic polarity. Thus, by analyzing the eigenfrequency and peak voltage polarity of the collected signals, specific operational direction (X + , X − , Y + , Y − ) leading to signal generation can be instantly and accurately identified. By encoding such distinguishable electrical signals into distinct operational commands (Signal 1 to Signal 4), sliding operations from four different directions can be precisely identified and applied to trigger diverse functional responses within an individual HMI system.

Fig. 1.

Overview of the magnetized joystick with dual-anisotropy for high-capacity wearable human-machine interactions. a Schematic diagram of human’s sensory receptors and the perceptual process. b Optical images of a wearable HMI interface in fingertip scales. c Composition of FMJ and the diagram of localized magnetic field distribution. d Optical images of the miniature FMJ and the side faces with different patterns after laser cutting. e Optical imaging reveals the geometrical changes when the device was bent along Y (left panel) and X (right panel) directions. The arrows of X and Y indicate the axis directions. f Schematic diagram of the sensing principle for HMI applications based on an individual FMJ

While the FMJ’s anisotropic structure across different directions endows it with direction-dependent eigenfrequencies, our observations reveal that the magnetization direction has a profound effect on whether the electrical signals produced by activating the FMJ along different directions display direction-dependent eigenfrequency disparities. As shown in Fig. 2a, when the FMJ was magnetized along the X direction (noted as X-magnetized FMJ), the Face 2 served as the equivalent N and S poles of a magnet. At this point, when the X-magnetized FMJ was bent along the X direction, a greater change in magnetic field intensity was observed at the bottom coil (Figure S6). As a result, although oscillatory decaying electrical signals were generated when activated along both directions, the electrical signals produced by activating the X-magnetized FMJ along the X direction exhibited higher intensity (Fig. 2b). Then, Fast Fourier transform (FFT) analysis was performed on the collected electrical signals from the two directions, and the results are presented in Fig. 2c. The sensed electrical signals exhibit a clear difference in eigenfrequency values when the X-magnetized FMJ is activated along different directions. The signals generated by activating the FMJ along X direction (noted as X-magnetized-X) all show an eigenfrequency of ~138 Hz, while those generated by activating the FMJ along Y direction (noted as X-magnetized-Y) all exhibit an eigenfrequency of ~84 Hz. Thus, when the FMJ is magnetized along the X direction, it can output two distinct vibration frequencies, which is consistent with our expectations. However, when the FMJ was magnetized along the Y direction (noted as Y-magnetized FMJ in Fig. 2d), the Face 1 served as the equivalent N and S poles of a magnet. Consequently, bending the Y-magnetized FMJ along the Y direction resulted in a greater change in magnetic field intensity at the bottom coil (Fig. S7) with stronger electrical signals (Fig. 2e). And the FFT analysis of the electrical signals (Fig. 2f) shows the eigenfrequency of signals generated by Y-direction activation remained low at ~96 Hz. But the frequency spectrum of signals from X-direction activation exhibited two characteristic frequency peaks. The first peak was identical to the eigenfrequency of the signals generated by Y-direction activation (~96 Hz), while the second peak was ~130 Hz. Since the peak of ~96 Hz had a larger amplitude, the eigenfrequency of the induced electrical signal also manifests as ~96 Hz when the Y-magnetized FMJ vibrates along the X direction. Therefore, when magnetizing the FMJ along the Y direction, the FMJ can only output a single vibration frequency.

Fig. 2.

The effect of magnetization direction on the frequency of the FMJ in the orthogonal direction. a Schematic diagram of the FMJ magnetized along X direction. b Voltage profiles of the X-magnetized FMJ activated along X and Y directions. c FFT analysis of the voltage profiles in (b). d Schematic diagram of the FMJ magnetized along Y direction. e Voltage profiles of the Y-magnetized FMJ activated along X and Y directions. f FFT analysis of the voltage profiles in (e). g The time-displacement curves of the unmagnetized FMJ, X-magnetized FMJ and Y-magnetized FMJ after activation along the X and Y directions. h FFT analysis of the time-displacement curves in (g). i Eigenfrequency values of the unmagnetized FMJ, X-magnetized FMJ and Y-magnetized FMJ after activation along the X and Y directions. j The processes of the X-magnetized FMJ’s first vibration cycle after activated along the X and Y directions

To investigate this magnetization direction-dependent eigenfrequency, we first studied the time-displacement curves of the unmagnetized FMJ, X-magnetized FMJ and Y-magnetized FMJ after activation along the X and Y directions. The results from Laser Doppler Vibrometer (LDV) are presented in Fig. 2g. It can be observed that after release, three FMJs all exhibited oscillatory decay when moving along different directions. Furthermore, those vibrations along the X direction show faster amplitude decay with a shorter duration. FFT analysis was further performed on the time-displacement curves of the three FMJs (Fig. 2h). The results indicate that the three FMJs exhibit similar eigenfrequencies when vibrating along the same direction (see Fig. 2i for the corresponding eigenfrequency values). Moreover, regardless of the direction of vibration, only one peak is observed in the frequency spectrum. From this perspective, the direction of magnetization has insignificant effect on the vibration eigenfrequency of the FMJ.

Then, we used a high-speed camera to observe the actual vibration behavior of the three FMJs. For the X-magnetized FMJ, the processes of its first vibration cycle when activated along the X and Y directions are shown in Fig. 2j. For the Y-magnetized FMJ, the processes of its first vibration cycle when activated along X and Y directions are presented in Fig. S8. The complete vibration cycles of each FMJ when activated along different directions were recorded in Videos S1–S4. As indicated by the red arrows in Fig. S9, a sub-vibration along Y direction was observed when the Y-magnetized FMJ vibrates along the X direction. To clearly illustrate this phenomenon, Videos S5 and S6 record the vibration of the X and Y-magnetized FMJs after X-direction activation from top-view. In the video, we applied two lines to mark the range of moving displacement in the Y direction. In principle, the FMJ should move in the area between the two lines when activating the FMJ along the X direction. However, it can be seen that for the Y-magnetized FMJ, its movement along the X direction will extend beyond the white line region. The reasons for this phenomenon can be attributed as follows. First, the FMJ inherently tends to move along the Y direction owing to the lower flexural strength along this specific direction. Consequently, the direction of maximum magnetic field intensity for the Y-magnetized FMJ is also along the Y direction. When vibrating along the X direction, the Y-magnetized FMJ would easily generate a sub-vibration along the Y direction due to the inherent magnetic force within the matrix. This results in the sensed electrical signal containing eigenfrequency information from vibration in both X and Y directions. However, due to the stronger magnetic field intensity in the Y direction, the eigenfrequency peak of the Y-direction vibration is more significant as presented in the FFT profiles. Thus, when vibrating along the X direction, it also exhibits the eigenfrequency of vibration along the Y direction instead of exhibiting the anisotropic behavior. Figure S10 further illustrates the variations in magnetic field lines, magnetic flux, voltage signals, and frequency profiles corresponding to FMJs with different magnetization directions and bending directions. From the schematic diagrams, it can be observed that magnetization direction is also playing an important role in determining the production of distinguishable frequencies. Along with the real-time vibration records (Fig. S9), the Y-direction magnetized FMJ is thus incapable of generating anisotropic eigenfrequencies. As the eigenfrequency anisotropy is dependent on the magnetized direction, the X-magnetized FMJ will be applied for the following studies.

Modeling and customization

As mentioned before, a single FMJ can output four distinguishable signals. However, in many cases, merely four signal channels cannot meet the requirements of HMI systems. In such cases, to combine multiple FMJs with non-overlapping eigenfrequencies is a convenient approach to achieve a multiplicative increase in signal capacity. Since the FMJ has a spring-like structure²², we investigated the eigenfrequency of the FMJ based on the model of the cylindrical helical spring. For a cylindrical helical spring (Fig. 3a), a cylindrical cantilever beam can be introduced as equal model to determine the intrinsic eigenfrequency as illustrated in Fig. 3b. When bent under a transverse force ($F$), the spring and the cantilever beam should possess the same bending strain energy. For a cylindrical helical spring with an average spring diameter of $D$, a wire diameter of $d$, an effective coil number of $N_c$, and an elastic modulus of $E$, the overall bending deformation can be divided into the bending deformation of each turn of wire, which can be regarded as the bending of a circular arc beam with a radius of $\frac{D}{2}$ under a transverse force. In this case, the bending strain energy of a single turn is $U_{single}=\frac{M^2\pi D}{2EI}$, where $M$ is the bending moment of a single turn ($M=F\cdot \frac{D}{2}$), and $I$ is the cross-sectional moment of inertia of the wire ($I=\frac{\pi d^4}{64}$). Thus, the total bending strain energy $U_{total}=N_c\cdot U_{single}=\frac{8F^2{D}^3{N}_c}{E{d}^4}$. And for a cylindrical cantilever beam with a diameter of $d_b$ and a length of $L_b$, its bending strain energy $U_b=\frac{F^2{L}_b^3}{6EI}=\frac{32F^2{L}_b^3}{3\pi E{d}_b^4}$. Based on $U_{total}=U_b$ and $d_b=d$, we obtain $L_b=\sqrt[3]{\frac{3\pi D^3{N}_c}{4}}$. So, according to the formula for the eigenfrequency of the cylindrical cantilever beam ($f_b=\frac{{\omega }_b}{2\pi }=\frac{{1.875}^2}{2\pi L_b^2}\sqrt{\frac{EI}{\rho A}}$, $A$ is the cross-sectional area), the eigenfrequency of the cylindrical helical spring can be expressed as $f=C\frac{d^2{E}^{\frac{1}{2}}}{N_c^{\frac{2}{3}}{D}^2{{\rho }^{\frac{1}{2}}A}^{\frac{1}{2}}}$, where $C$ is a constant. Furthermore, when the helix angle α is greater than 10^o, $N_c$ needs to be corrected to $N_c^{\prime }=\frac{H}{\pi D\tan \alpha }$. Therefore, the influence of spring parameters on its eigenfrequency is $f=C\frac{d^2{E}^{\frac{1}{2}}{\left(\tan \alpha \right)}^{\frac{2}{3}}}{H^{\frac{2}{3}}{D}^{\frac{4}{3}}{{\rho }^{\frac{1}{2}}A}^{\frac{1}{2}}}$. Thus, the eigenfrequency of the FMJ can be tuned by adjusting the parameters of its equivalent spring. For the developed FMJ, its geometric parameters are illustrated in Fig. 3c. Among these, cut cycle (C), cut space (S), and cut angle (θ) are the laser cut parameters; the length (L), width (W), and height (H) of the FMJ are the dimensional parameters. Additionally, there are unlabeled material parameters, such as Young’s modulus (E) and density (ρ). As discussed below, such parameters allow us to flexibly customize the eigenfrequencies and establish the frequency library for subsequent HMI applications.

Fig. 3.

Parameter-dependent frequency performance of the FMJ. a Schematic diagram of the cylindrical helical spring model with key parameters that affect the eigenfrequency: average spring diameter (D), wire diameter (d), helix angle (α) and height (H). b Schematic diagram of the equivalent cylindrical cantilever beam: diameter (d_b) and length (L_b). c Geometric parameters of the FMJ, including cut angle (θ), cut space (S), cut cycle (C) and length (L), width (W) and height (H). d Variations of the average cut depth with different cut cycles (C). e Eigenfrequencies for the X and Y directions of the FMJ with different cut cycles (C). f Images show the structure changes of the FMJ and equivalent spring model with different cut space (S). g Eigenfrequencies for the X and Y directions of the FMJ with different cut spaces (S). h Images show the structure changes of the FMJ and equivalent spring model with different cut angles (θ). i Eigenfrequencies for the X and Y directions of the FMJ with different cut angles (θ). j Eigenfrequencies for the X and Y directions of the FMJ with different Ecoflex contents. k Eigenfrequencies for the X and Y directions of the FMJ with different heights (H). l Optical images of FMJ with different lengths (L). m Eigenfrequencies for the X and Y directions of the FMJ with different lengths (L). n Optical images of FMJ with different widths (W). o Eigenfrequencies for the X and Y directions of the FMJ with different widths (W)

Based on the aforementioned model, increasing the cut cycle (C) will increase the number of laser scans per cut line that in turn increases the cut depth (Fig. 3d). This leads to a larger overlapping area formed by the cut patterns on Face 2, which is equivalent to the increase of average spring diameter ($D$) of the equivalent spring. Consequently, the eigenfrequencies of the FMJ in both directions decrease gradually. As shown in Fig. 3e, the eigenfrequencies for the Y direction decreases from ~144 Hz to ~95 Hz, while that for the X-direction decreases from ~242 Hz to ~161 Hz. As the space between two adjacent cut lines on Face 1 increases (Fig. 3f), the number of cut lines on Face 1 decreases accordingly. This is equivalent to increasing the wire diameter ($d$) of the equivalent spring while reducing the effective coil numbers ($N_c$). Therefore, the eigenfrequencies of the FMJ increases gradually with the increase in cut space (Fig. 3g). Specifically, the eigenfrequency for the Y direction rises from about 91 Hz to ~225 Hz, and that for the X direction increases from around 145 Hz to 308 Hz. Increasing the cut angle raises the helix angle of the equivalent spring, which also leads to a reduction in the effective coil number. On the other hand, as shown in Fig. 3h, a larger cut angle also increases the number of cut lines on Face 1 that does not connect the two Face 2 (as marked with yellow arrows), resulting in a further reduction in the effective coil number. Thus, the eigenfrequencies of the FMJ for the Y direction increases from approximately 88 Hz to around 275 Hz, and that for the X direction rises from roughly 125 Hz to 375 Hz (Fig. 3i).

According to the governing principle, the eigenfrequency of the spring is proportional to the ratio of Young’s modulus to density. In view of this, we also modified the eigenfrequencies of the FMJ by adjusting the ratio of raw materials (PDMS, Ecoflex, and NdFeB). Here, increasing the ratio of NdFeB increases both the Young’s modulus and the density. However, the change in the ratio of the Young’s modulus to the density is minimal under this adjustment (see Table S1). Since increasing the mass ratio of NdFeB can enhance the signal intensity, we fixed the mass of NdFeB to be twice of the total mass of the PDMS and Ecoflex. We then adjusted the ratio of PDMS to Ecoflex to modify the ratio of Young’s modulus to density of the FMJ, with the results presented in Table S2. It can be observed that increasing the ratio of Ecoflex significantly reduces the FMJ’s Young’s modulus while slightly increasing its density. Consequently, the ratio of Young’s modulus to density decreases with increasing Ecoflex content, which in turn leads to a reduction in the eigenfrequencies. The corresponding eigenfrequency results (Fig. 3j) for FMJs with different Ecoflex contents activated along the two directions further confirm this trend: as Ecoflex content increases from 0 wt.% to 100 wt.%, the eigenfrequency of the FMJ for the Y direction decreases from approximately 147 Hz to around 44 Hz, and the eigenfrequency for the X direction decreases from roughly 281 Hz to 78 Hz.

Finally, the impact of dimensional parameters on the eigenfrequency is investigated. Figure 3k illustrates the effect of the FMJ’s height on its eigenfrequencies along the two directions. It can be observed that as the height increases, the eigenfrequencies continuously decrease. Since the eigenfrequency is proportional to the negative two-thirds root of the height, the magnitude of the decrease in eigenfrequency becomes smaller as the height increases. It should be noted that in previous studies, the length and width of the FMJ were both 3 mm. In this case, the initial eigenfrequencies for the X and Y directions are equal, and the difference in eigenfrequencies between the two directions is caused by differences in their internal structures. When the length or width of the FMJ is adjusted such that the two dimensions are no longer equal, it is necessary to account for the difference in initial eigenfrequencies for the two directions. Referring to the formula for the eigenfrequency of a cantilever beam with a rectangular cross-section ($f=\frac{{1.875}^2}{2\pi H^2}\sqrt{\frac{EI}{\rho A}}$), the initial eigenfrequency for the Y direction should equal to $\frac{{1.875}^2}{2\pi H^2}\sqrt{\frac{E{W}^2}{12\rho }}$, while the initial eigenfrequency for the X direction should be $\frac{{1.875}^2}{2\pi H^2}\sqrt{\frac{E{L}^2}{12\rho }}$. Thus, when the length of the FMJ is increased (as shown in Fig. 3l), the initial eigenfrequency of the FMJ for the X direction should increase significantly, while the initial eigenfrequency for the Y direction remains essentially unchanged. After laser cut is applied, the eigenfrequencies for both directions are shown in Fig. 3m. It can be observed that the eigenfrequency for the Y direction also increases, which may be due to the increase in length that leads to a decrease in the average spring diameter ($D$) of the equivalent spring as the cut depth remains unchanged. When the width of the FMJ is increased, the cut cycle is also increased proportionally. However, as shown in Fig. 3n, the overlapping area of the cut lines decreases gradually with increasing width. Especially, the cut lines no longer overlap when the width equals to 4.5 mm. Thus, although the eigenfrequencies of the FMJ for both directions increase gradually with a larger width, the eigenfrequencies for both directions are relatively lower if the width is small, and become significantly higher with a larger width (Fig. 3o). Here, we note that the eigenfrequency of some FMJs overlaps with common power-line frequencies. We therefore tested the influence of environmental electromagnetic interference on eigenfrequency identification. As shown in Fig. S11, both the voltage signal and the frequency peak of the noise are much weaker when compared with the signals generated by the low-eigenfrequency FMJ. This indicates that the environmental noise would not bring significant interference on the frequency identification of the low-eigenfrequency FMJ. In addition, during routine manual testing, we set a signal acquisition trigger threshold (as described in “Methods”) higher than the noise to avoid collecting noise frequency information during idle periods.

Robustness and reliability

To function as a reliable HMI interface, the FMJ must exhibit strong robustness, ensuring it accurately conveys human intent to electrical terminals under various conditions. Therefore, as illustrated in Fig. 4a and 4b, we first evaluated the stability of the FMJ’s sensing signals through bending it to specific vertical angles along the two directions (noted as θ_V-X and θ_V-Y) and then releasing. The corresponding optical images are shown in Figure S12, and real-time voltage profiles are presented in Fig. 4c and 4d. It can be observed that as the vertical bending angle gradually increased from 10° to 90°, the sensed signals exhibited only an increase in the amplitude, while the polarity of the peak voltage remained unchanged. This is because a larger vertical bending angle only causes more significant magnetic field intensity variation at the bottom coil, without altering the trend of magnetic field variation (see Fig. S13). Increasing the vertical bending angle also did not affect the distinguishability of signals along X and Y directions. An FFT analysis of the voltage profiles in Fig. 4c and 4d was carried out and the results are displayed in Fig. 4e and 4f. It can be observed that as the vertical bending angle increases, the eigenfrequencies of the voltage signals along the two directions remained stable at ~183 Hz (X direction) and ~107 Hz (Y direction), respectively. For vertical bending angles smaller than 10^o, the test results are presented in Fig. S14. In principle, a smaller bending angle would cause a lower signal intensity as reflected from the voltage profiles. However, the captured eigenfrequencies can maintain at the same level to ensure the reliability of this eigenfrequency-based interface. Thus, varying the vertical bending angle has negligible impact on the stability of the FMJ’s sensing performance. Additionally, using the setup illustrated in Fig. 4g, we evaluated the effect of different horizontal angles (noted as θ_H) on the stability of the sensing signals. Here, the FMJ was placed on a rotatable platform to adjust the horizontal within the range of 0° to 350° (Fig. S15). Afterward, a baffle plate would move linearly to trigger the bending process. The voltage signals corresponding to horizontal angles from 0° to 90° are shown in Fig. S16. Note that when θ_H = 0°, the bending was performed along the positive Y direction, while the bending was along the positive X direction at θ_H = 90°. As the horizontal bending angle increased, the bending direction gradually aligned with the magnetic pole direction of the FMJ. As a result, the variation in magnetic flux at the bottom coil became more pronounced, leading to a continuous increase in the intensity of the voltage signal. While the FFT analysis results of the corresponding voltage curves (Fig. 4h) show that when θ_H = 0°, the voltage signal exhibited an eigenfrequency of ~107 Hz. As the horizontal bending angle increased, the intensity of the peak corresponding to ~107 Hz decreased steadily, while a peak representing ~183 Hz emerged and grew stronger. When θ_H = 40°, the intensity of the peak representing ~183 Hz exceeded the peak representing ~107 Hz, causing the eigenfrequency of the FMJ’s sensing signal shifted to ~183 Hz. With a further increase in θ_H, the intensity of the peak representing ~183 Hz continued to rise, while that representing ~107 Hz continued to diminish and finally disappeared completely when θ_H = 70°. The eigenfrequencies of the FMJ’s sensing signals across the horizontal bending angle range of 0° to 360° are presented in Fig. 4i. When θ_H falls within the ranges of 330° to 30° and 150° to 210°, the eigenfrequency of the sensing signal is ~107 Hz; when θ_H is within 40° to 140° and 220° to 320°, the sensing signal eigenfrequency is ~183 Hz. The results confirm the tolerance of bending orientation which might occur during the real applications when the FMJ was manually triggered. Additionally, we evaluated the stability of frequency generation by periodically triggering the FMJ under different bending speeds and bending positions at the transition regions of θ_H = 30^o and 40^o. As presented in Figure S17, the voltage peak values vary accordingly when triggering speeds were increased from 5 mm/s to 50 mm/s. However, the prominent characteristic peaks under the transition angles remained at ~107 Hz (θ_H = 30^o) and ~183 Hz (θ_H = 40^o), respectively. Similarly, the dominant peaks at transition angles remained unaffected when the FMJ was deformed at different positions (Fig. S18), indicating the stability of the signal production based on the eigenfrequency methodology.

Fig. 4.

Orthogonal anisotropic response and long-term sensing reliability. a Schematic diagram of activating the FMJ along X direction with different vertical angles (θ_V-X). b Schematic diagram of activating the FMJ along Y direction with different vertical angles (θ_V-Y). c Voltage profiles generated by FMJ after being activated along X direction with different vertical angles. d Voltage profiles generated by FMJ after being activated along Y direction with different vertical angles. e FFT results of the voltage curves in (c). f FFT results of the voltage curves in (d). g Schematic diagram of the experimental setup for activating the FMJ with different horizontal angles. h FFT results of the voltage curves obtained by activating the FMJ with a horizontal angle between 0^o and 90^o. i The eigenfrequencies of the FMJ with a horizontal angle between 0^o and 350^o. j Schematic diagram of the experimental setup for stability evaluation. k Long-term stability performance of the FMJ via periodical deformation up to 10,000 cycles. l Eigenfrequencies obtained from different samples prepared by the same templates and fabrication parameters. m Induced electrical signals generated by activating the FMJ along X directions with different trigger speeds. n Induced electrical signals generated by activating the FMJ along Y directions with different trigger speeds. o Eigenfrequencies corresponding to the voltage profiles in (m) and (n)

Subsequently, we tested the long-term stability of the FMJ when bent along X and Y directions using the setup illustrated in Fig. 4j. As presented in Fig. 4k, the eigenfrequencies of the FMJ remained stable at ~107 Hz and ~183 Hz after over 10,000 cycles along both directions, demonstrating the reliability of the FMJ for long-term use. In addition, since the FMJ is a spring-like structure formed by cutting an elastomer, its internal structure may fracture under extreme conditions (e.g., high-force stretching, large-angle twisting). We further performed additional fatigue tests under large-angle bending, stretching, and twisting. The results are shown in Fig. S19. Upon up to 10000 cycles, the FMJs can exhibit stable production of the eigenfrequencies to showcase the robustness of the flexible system. Even though slight frequency variation was observed upon significant mechanical deformation, e.g. stretched to 200% or twisted to 180°, such extreme conditions can normally be avoided in most situations to maintain a stable eigenfrequency output for real-world applications. We therefore also assessed the fabrication stability of the FMJ. As shown in Fig. 4l, FMJs fabricated with the same process parameters exhibit nearly identical eigenfrequencies in both directions. From this, it follows that if the original FMJ in an HMI system is damaged, it can be directly replaced with another FMJ of the same type without adjusting the system’s configuration parameters that ensures ease of use. Finally, considering that the activation speed of the FMJ is not consistent in daily use, we also used the setup illustrated in Fig. 4j to test the effect of activation speed on the stability of the FMJ’s sensing signals. When the activation speed was increased from ω to 10ω, the signals generated by bending the FMJ along the X and Y directions are shown in Fig. 4m and 4n, respectively. It can be observed that apart from causing a slight increase in voltage amplitude, the activation speed does not alter the polarity of the signals (see Fig. S20). FFT analysis was performed, and the results are presented in Fig. 4o. It is evident that the activation speed also does not change the eigenfrequency of the sensed signals. In addition to above results, we characterized the effects of coil parameters (number of turns and wire diameter) as well as the distance between the coil and the FMJ on the voltage signal and the eigenfrequency. As shown in Fig. S21, when the number of turns and the wire diameter decrease, and the distance between the coil and the FMJ increases, the voltage signal intensity gradually decreases, while the eigenfrequencies of the voltage signals remain unchanged. In summary, the FMJ can stably distinguish operations in different directions across various operating scenarios, exhibiting excellent robustness and reliability towards wearable applications. The direction-dependent production of unique eigenfrequency and peak polarity allows the interface to avoid strict control on manual triggering process, e.g. applied force, deformation speed and angle, during the real-world HMI communication.

Demonstrations of wearable interaction

As mentioned earlier, the FMJ-based interface can output signals with distinct eigenfrequencies and peak voltage polarities based on the operation directions (X + , X-, Y + , Y-). The high signal capacity, coupled with simple and intuitive operation, enables the realization of various HMI applications using a single FMJ. Here, we first combined a single FMJ with a custom-designed coil (Fig. S22) to form the displayed HMI interface, and then attached this assembly to the index finger for proof-of-concept demonstration (Fig. 5a). Subsequently, we could activate the FMJ by sliding or toggling it along different directions to enable the control of HMI applications. As shown in Fig. 5b, four distinct electrical signals were produced when the FMJ was activated along different orthogonal directions. Among these signals, those triggered along the X direction have an eigenfrequency of 161.7 Hz, while those activated along the Y direction exhibit an eigenfrequency of 90.8 Hz. Additionally, the electrical signals corresponding to the positive (+) and negative (−) directions display opposite peak voltage polarities. As illustrated in Fig. 5c, these electrical signals are then encoded based on two criteria: the eigenfrequency value and the relationship between the absolute values of the maximum and minimum peak. Thanks to the non-overlapping behavior, the encoding maps four operational directions to different commands in the HMI system (e.g., up, down, left, right). Here, we demonstrate three distinct application scenarios: a maze interface (Fig. 5d), an airplane game (Fig. 5e), and control of PPT slides (Fig. 5f). For the maze game (Fig. S23 and Video S7), the FMJ is activated via sliding motions. When sliding the FMJ along different directions, the indicator light on the HMI interface moves one grid in the same direction. This enables the indicator light to navigate along a desired path through intuitive controls until it escapes the maze. For the airplane game (Video S8), the FMJ is activated by toggling. By encoding the movement direction of the user’s airplane to the electrical signals corresponding to different toggling directions, we achieved control over the airplane to move to specific positions and destroy enemy airplane. While for the PPT control demonstration (Fig. S24 and Video S9), the electrical signals generated by toggling the FMJ along X + , X − , Y + , and Y− directions are encoded as “next slide”, “previous slide”, “start presentation”, and “end presentation”, respectively. It can be observed that when the FMJ is activated by toggling along different directions, the PPT program can accurately execute the corresponding functions. By combining FMJs with non-overlapping eigenfrequencies, the signal capacity can be further expanded to enable more complex HMI applications. As a proof-of-concept, two FMJs were integrated to form a flexible controller for robotic car (Fig. 5g). In the flexible controller, each FMJ contributes to control of four directions, collectively enabling 8-directional movement of the target. Since the entire controller consists of a substrate made of PDMS/Ecoflex and flexible coils of polyimide (PI) and copper (Cu) layers (Fig. S25), it can be folded manually with ease as demonstrated in Fig. S26. In the specific demonstration (Fig. 5h and Video S10), the robotic car successfully moved in eight directions after the FMJs were activated along different orientations. This is attributed by the absence of eigenfrequency overlap between two FMJs and the stability of eigenfrequency and peak voltage polarity (see Table S3 for eigenfrequencies and peak polarity corresponding to each moving direction). Coupled with increased command capacity, this intuitive correspondence allows rapid and convenient achievement of object controls in scenarios such as navigation in obstacle-ridden paths (Fig. 5i).

Fig. 5.

Human-machine interaction applications enabled by single- and dual-FMJ sensors. a Schematic diagram and optical image of the wearable HMI interface assembled from one FMJ. The operations of slide or toggle can generate four signals according to the directions. b Four different electrical signals generated by activating the FMJ along four orthogonal directions. c The programming process based on the eigenfrequencies and peak voltage polarities. Demonstrations of d maze control, e airplane game and f manipulation of PowerPoint (PPT) slides. g Schematic diagram of the flexible controller assembled from two FMJs. h The process of successfully controlling the robotic car to move along eight directions using the flexible controller with two FMJs. i Schematic diagram of using the flexible controller for remotely controlling a robotic car to navigate a path filled with obstacles and avoid collision

It can be expected that with more FMJs in a system, the interactive capacity will be simply enhanced without burdening the wiring complexity. By using four FMJs with non-overlapping eigenfrequencies, we further established a wearable calculator interface on fingertip (Fig. 6a). Benefiting from the dual-parameter addressing methodology, the interface can be connected to the HMI system using single-channel electrode by the well-designed coils (Fig. S27). Even though all FMJs share the same coil layer, the identifiable eigenfrequency/polarity allows accurate addressing of the mechanical stimuli to ensure operational reliability. For practical operation, four FMJs are attached on fingertips from the ring finger to the pinky finger, and activation is achieved by manually triggering these FMJs with the thumb. The activation directions of each FMJ and their corresponding functions in the calculating system are illustrated in Fig. 6b. The non-overlapping eigenfrequencies and the polarity of the peak voltages corresponding to each function of the calculator are displayed in Table S4. The LabVIEW interface of the calculator is shown in Fig. 6c. When an FMJ is activated, the corresponding calculator key lights up, with the input displayed in real time in Region 1 and the calculation result shown in Region 2. Upon completion of former calculation, triggering the FMJ4 to the right will clear all results, preparing the system for next cycle of calculation. Specific examples of addition, subtraction, multiplication, and division operations are presented in Fig. 6d, and the full operation demonstrations are recorded in Videos S11–14. It can be observed that throughout the operation process, the input can be accurately generated with fast production of the calculation results (~150 ms according to the frames in the videos). Upon successful demonstration of such wearable calculating system, it can be anticipated that similar interactive interfaces can also be established using the platform. For example, Fig. 6e illustrates the concept of an FMJ-based flexible miniaturized keyboard. This can be realized by using more FMJs (e.g. seven) with non-overlapping eigenfrequencies by adjusting the key parameters as discussed above. Thanks to the dual parameter addressing, all FMJs can be integrated into a single coil, enabling electrical signals to be generated from a single pair of electrodes without mutual interference. The competitive size of the FMJ lays the foundation for portability and wearability. Owing to the stability of eigenfrequency and magnetic orientation, multi-platform interface can be easily achieved by developing a unified control system (Fig. 6f). This allows us to use a portable HMI interface to control a broad spectrum of HMI systems. Figure 6g summarizes the capacity per electrode, capacity per channel and capacity per sensor of recently developed HMI interfaces with multi-command functions. It can be observed that the FMJ exhibits superiorities among these works. As a compact and high-capacity communication interface, the FMJ holds great development potential and promising application prospects for wearable and portable HMI systems.

Fig. 6.

FMJ-based fingertip calculator and multi-scenario interaction. a Optical image of the fingertip calculator assembled from four FMJs with underlying coil layer. b Mapping of different calculator functions based on the fingertip calculator with detailed input-output relationship. Each FMJ can allocate up to four functions. c LabVIEW interface for the fingertip calculator. d Specific examples of addition, subtraction, multiplication, and division operations using the fingertip calculator. e Schematic diagrams of the concept of an FMJ-based flexible miniaturized keyboard assembled from seven FMJs. f Schematic diagrams of multi-platform controlling system using the FMJ-based HMI interface. g Comparison of capacity/electrode, capacity/channel and capacity/sensor of recently developed HMI interfaces with multi-command functions^32–41

Conclusion

In summary, we demonstrated a high-capacity HMI interface based on flexibly magnetized joystick (FMJ). Using laser cutting approach, the structural anisotropy was realized to enable the anisotropy of FMJ’s vibration eigenfrequency upon mechanical deformation. By integrating a polarity determination mechanism, the FMJ can output four non-overlapping electrical signals depending on the direction of activation (X + , X-, Y + , Y-). The eigenfrequencies of FMJ can also be systematically adjusted based on different processing parameters, device dimensions and material properties. This finally provides the foundation for arrayed FMJs within the HMI interface to further explore the communication capacity. Through a pair of electrodes and a single channel, we demonstrated the application of FMJs (from one to multiple units) in various HMI scenarios, including game entertainment, PPT manipulation, directional robotic car, and fingertip calculators. We have also illustrated the potential of this high-signal-capacity FMJ in other HMI applications. With the robustness, we expect that constructing structural eigenfrequency anisotropy and adopting the dual-parameter addressing strategy can provide a concise solution for future wearable HMI systems with high capacities.

Methods

Materials

Polydimethylsiloxane (PDMS) base and curing agent (Sylgard 184 kit) were purchased from Dow Corning, USA. Ecoflex (model of 00-50) was introduced from Smooth-On, Inc., USA for the modulation of elastic modulus. Neodymium-Iron-Boron (NdFeB) particles were from Magnequench, China and copper substrates (Chengdu Do-itc New Material Co., Ltd., China) were used for coil fabrication.

Fabrication of FMJ and the coil electrodes

The fabrication of FMJ consists of three steps: preparation of the raw body, construction of the anisotropic structure, and magnetization. During the preparation of the raw body, an engraving machine (CNC-3020, JingYan Instruments & Technology Co., China) was used to form a predefined three-dimensional (3D) cavity in a polymethylmethacrylate (PMMA) slab. Then, polydimethylsiloxane (PDMS), Ecoflex, and NdFeB magnetic particles were mixed in a beaker with a preset mass ratio. After stirring thoroughly, the homogeneous mixture was degassed in a vacuum chamber. The degassed mixture was subsequently injected into the PMMA cavity and cured in an oven at 80 °C for 30 min. The cured PDMS/Ecoflex/NdFeB composite was then peeled off from the PMMA mold, yielding the raw body. The anisotropic structure was created through a laser cutting process. A commercial laser engraving system (LPKF ProtoLaser U4, LPKF Laser & Electronics AG, Germany) was used to cut predefined patterns on two opposite surfaces of the raw body, thereby introducing structural anisotropy. At last, the laser-patterned raw body was exposed to an external magnetic field with intensity of ~3 T to complete the fabrication of the FMJ. All coils used in this work were fabricated using the aforementioned laser engraving machine on a sandwich-structured copper-clad substrate, consisting of two 18 μm-thick copper films separated by a 75 μm-thick polyimide layer. Depending on the application scenario, the overall dimensions of these coils varied, but the loop width and the spacing between adjacent loops were both fixed at 100 μm.

Characterizations

Scanning electron microscopy (SEM, Sigma FE-SEM, Zeiss Corporation, Germany) equipped with an energy-dispersive spectrometer (EDS) was used to characterize the surface morphologies and element distributions of the FMJ. Vibration behavior was studied by a Laser Doppler Vibrometer (LDV, VibroOne, Polytec Inc., USA) to record the free-end displacement. The real-time motion was recorded by a high-speed camera (VEO 710S, Phantom, USA) with frame rate of 10,000 fps (frames per second). Magnetic field distribution was measured by a three-axis commercial Gauss-Tesla meter (DX-360, Dexin Mag, China). The voltage signals induced by FMJ vibration and generated in the coil were amplified by low noise current preamplifier (Model SR570, SRS, USA), captured by multifunctional I/O device (USB-6341, National Instruments, USA), recorded and displayed by LabVIEW software (National Instruments, USA). All data were analyzed and processed by OriginLab software. All wearable demonstrations were carried out by research volunteers with ensured safety and informed consent during the characterization.

Simulation of magnetic field

COMSOL Multiphysics 6.1 was employed to simulate the magnetic field around the FMJ under a steady state. The material of the FMJ was set as sintered NdFeB, with a magnetic field intensity of 1.0186e6 A/m. The air atmosphere was set as a cube with a side length of 20 mm. “Magnetic field, no current” was selected as the physical field for the simulation model. The governing equations were $H=-\nabla V_m$, $\nabla \cdot B=0$, $B={\mu }_0{\mu }_{r}H$, and $B={\mu }_0\left(H+M\right)$, where $H$ is the magnetic field vector, $V_m$ is the magnetic scalar potential, $B$ is the magnetic flux density vector, ${\mu }_0$ is permeability of vacuum, ${\mu }_r$ is relative magnetic permeability and $M$ is the magnetization vector. The initial value of $V_m$ was set as 0. The boundary conditions were set as $n\cdot B=0$.

Configuration of the LabVIEW interface

The built-in DAQmx series, subVIs, in LabVIEW were used to configure the acquisition parameters of the induced signals with sampling rate of 10,000 Hz, sampling number of 300 and sampling trigger condition of >0.01 V (relative to baseline). Using the peak-seeking and Fast Fourier Transform (FFT) procedure, the polarity of peak voltage and eigenfrequency of the voltage signals can be distinguished and calculated for further communications.

Author contributions

B.Z. supervised the project. Y.L. and B.Z. conceived the concept. Y.L., M.L. and B.Q. conducted the experiments. Y.L. analyzed the data and wrote the original draft. Y.Z., Z.D. and D.F. helped design the LabVIEW programs. Y.Z., Z.D., D.F., J.W. and J.L. commented on the organization of the figures. All authors discussed the results, commented on the paper and revised the manuscript.

Data availability

Data supporting the findings of this study are available within the article (and its Supplementary Information) and from the corresponding authors upon reasonable request.

Competing interests

The authors declare no competing interests.

Supplementary information

The online version contains supplementary material available at https://doi.org/10.1038/s41378-026-01459-y.