Frequency-selective acoustic and haptic smart skin for dual-mode dynamic/static human-machine interface

Abstract

Accurate transmission of biosignals without interference of surrounding noises is a key factor for the realization of human-machine interfaces (HMIs). We propose frequency-selective acoustic and haptic sensors for dual-mode HMIs based on triboelectric sensors with hierarchical macrodome/micropore/nanoparticle structure of ferroelectric composites. Our sensor shows a high sensitivity and linearity under a wide range of dynamic pressures and resonance frequency, which enables high acoustic frequency selectivity in a wide frequency range (145 to 9000 Hz), thus rendering noise-independent voice recognition possible. Our frequency-selective multichannel acoustic sensor array combined with an artificial neural network demonstrates over 95% accurate voice recognition for different frequency noises ranging from 100 to 8000 Hz. We demonstrate that our dual-mode sensor with linear response and frequency selectivity over a wide range of dynamic pressures facilitates the differentiation of surface texture and control of an avatar robot using both acoustic and mechanical inputs without interference from surrounding noise.

A frequency-selective and noise-independent triboelectric sensor is demonstrated for acoustic and haptic human-machine interface.

INTRODUCTION

Human-machine interfaces (HMIs) play a key role in the interaction between humans and machines by allowing the facile and intuitive control of machines for nondisabled and disabled people, who may or may not have knowledge regarding complicated software and hardware. Most of the existing wearable HMI devices use low-frequency (1 to 10 Hz) touch or hand motions such as tapping, bending, and flutter to deliver simple commands to machines (1, 2). Recent developments in HMIs requiring high-frequency signal detection for robots, virtual reality (VR), augmented reality (AR), and Internet of Things (IoTs) demand precise and intuitive control of HMIs to deliver various senses and biosignals from humans. In addition to the low-frequency tactile mapping of objects (3), the perception of roughness and surface texture based on high-frequency vibration (80 to 300 Hz) detection will be required for a robotic skin to precisely perceive and manipulate objects (4). The mechano-acoustic sensing of electrophysiological signals such as cardiac operation and vocal vibration from skin requires a frequency bandwidth of 10 to 2000 Hz (5). A dynamic sensor with a high signal-to-noise ratio (SNR), a broad linear response range, and a wide frequency bandwidth is essential for the accurate perception of high-frequency dynamic signals using HMIs. Among the various candidates for dynamic sensors, triboelectric sensors (TESs) instantly generate high power in response to dynamic stimuli without additional power supply. Thus, TESs are considered self-powered voice recognition devices for biometric identification (6), hearing aids (7), and skin-attachable microphones (8). In addition, TESs recognize multiple physical touch (9), motion (10, 11), fine texture (12), and displacement of objects (13), demonstrating great potential for dynamic HMI applications (14). Moreover, the electrical power generated due to mechanical deformation can be used to power luminescence devices (15) and actuators (16) that can offer feedback on dynamic information to the user through user-interactive devices.

Dynamic HMIs require selective recognition of desired frequency information without interference from surrounding noises. Hence, developing dynamic sensors with controllable resonance frequency that perceive the dynamic response with a specific frequency while canceling undesirable noise frequency is necessary. In the human ear, thousands of basilar membranes in the cochlea provide gradient stiffness depending on width, thickness, and location, allowing the selective detection of an individual acoustic vibration from a broad range of frequencies, thus enabling us to recognize only the desired sound from the complex noise (17). Inspired by the frequency tunability of the inner structure of cochlea, various frequency-selective acoustic sensors have been created by modulating the form factors of channel materials including sensor size, membrane length, and location or by introducing diaphragms with different hole shapes, sizes, and thicknesses on the top layer of acoustic sensors (18–21). However, narrow tunable range of resonance frequencies and difficulties in miniaturization are the main limitations of these sensors. Considering the frequency selectivity using vibrational behavior depending on different modulus, control of the modulus or stiffness of active materials would result in the variation of resonance frequency. Recently, porous, crack, and biophotonic nanostructures of active materials have been studied for acoustic identification and improvement in acoustic sensitivity (22–25). Improved designing of materials and structural parameters would broaden the tunable range of frequency and increase the sensitivity, providing a new platform for the demonstration of advanced dynamic sensor that can cover the human voice frequency (100 to 4000 Hz) and dynamic tactile sensation (<100 Hz).

In contrast to conventional HMIs based on single dynamic signals, dual-mode dynamic HMIs based on multiple dynamic signals from acoustic and tactile/physical stimuli with high sensitivity and frequency selectivity enable facile control of machines without surrounding interferences. However, dual-signal perceptive dynamic HMIs based on a single device have not been reported thus far. Therefore, we propose a frequency-selective acoustic and haptic smart glove for dual-mode HMIs. For the dual-mode HMIs, we suggest a hierarchical ferroelectric composite comprising surface macrodome (MD) and inner micropore (MP) structures decorated with nanoparticles (NPs) to develop a self-powered frequency-selective TES with high sensitivity and linear response (Fig. 1A). Upon pressure application, the gradual deformation of hierarchical ferroelectric composite results in the linear electrical output over a wide dynamic pressure range when external pressure is applied (Fig. 1B). In the hierarchical structure, the surface MD structure provides a larger variation in contact area, leading to increased sensitivity within a low-pressure range. The high deformability of inner MP structure prevents the rapid saturation of sensitivity. This extends the detectable pressure over a wide range. In addition, the NP in MP structure increases the dielectric property of the film and the localized stress under mechanical deformation, resulting in enhanced pressure sensitivity over a wide pressure range. The porosity and pore size of hierarchical structures, which affect the density and mechanical modulus of ferroelectric composites, are easily controllable, resulting in the active frequency selectivity (Fig. 1C). Furthermore, modulating form factors such as thickness and area of ferroelectric composites significantly increase the dynamic range of frequency selectivity (145 to 9000 Hz), covering the audible frequency range. As a proof of demonstration, we developed multichannel acoustic sensor arrays with different resonance frequencies capable of canceling unexpected noise and enabling the fabrication of high-accuracy voice recognition devices (Fig. 1D). The qualitative comparison of voice recognition can be performed using a machine learning technique based on the artificial neural network (ANN)–based training method that facilitates higher accuracy and selectivity in the proposed sensor than a commercial microphone. Last, we fabricated a texture-perceptive smart glove integrated with TESs on the tip of each finger that can detect and differentiate between various surface textures. Combined with a wireless platform, our smart glove can be used as a dual-signal perceptive HMI device with dual-mode operation using both mechanical and acoustic signals for the control of avatar robotic arms, thus providing a next-generation platform of HMI devices.

Fig. 1. Hierarchically designed ferroelectric composites for dynamic interfacing applications.

(A) Schematic of the TES using the hierarchical architecture of macrodome (MD), micropore (MP), and nanoparticles (NPs). (B) Schematic of the dependency of each structural component of the hierarchical ferroelectric composite on the pressure sensitivities of TESs. (C) Graph showing the frequency selectivity of TESs depending on the structural designs of hierarchical ferroelectric composites. (D) Applications of TESs in various dynamic interfacing devices including noise-independent voice recognition, texture perception, and dynamic motion detection and interfacing using robotic hands.

RESULTS

Fabrication of TESs with hierarchical ferroelectric composites

We prepared hierarchical ferroelectric composites with MD/MP/NP structures by micromolding and solvent exchange processes for a TES with high sensitivity and linear response (fig. S4). The surface MD structure was fabricated by casting polyvinylidene trifluoroethylene (PVDF-TrFE)/barium titanate NP (BTNP) solution onto dome-patterned silicon molds with different diameters of 80, 120, and 150 μm for a constant diameter/pitch ratio of 0.66 (fig. S5). The MPs within the PVDF-TrFE/BTNP composite were formed due to the phase separation between solvent and nonsolvent components during the solvent exchange process (fig. S4). When the composite solution is exposed to humid conditions, water droplets condense on its surface during the solvent evaporation and get soaked into the composite polymer solution. During subsequent drying, inner MPs are formed within the water-insoluble polymer composite matrix. The inner walls of the MPs are dotted with hydrophilic BTNPs after water evaporation. In this fabrication process, the porosity and pore size can be easily tuned using the solidification time that is determined by the solubility and evaporation rate of solvent (26). With an increase in solidification time, inner MPs coalesce together, leading to larger pore size and greater porosity. We used different polar solvents including dimethyl sulfoxide (DMSO), dimethylformamide (DMF), acetonitrile (ACN), and methyl ethyl ketone (MEK) to tune porous structures as necessary (fig. S6). The pore size decreases from 14, 10, 8, to 2.5 μm, depending on the solubility of polar solvents (DMSO > DMF > ACN > MEK) (fig. S7A). With the decrease of pore size, the porosity decreases from 80, 70, 40, to 20% (fig. S7B). In the hierarchical ferroelectric composite, PVDF-TrFE polymer induces the polarization of composite and can be easily solution-processed for the micropatterning, leading to the formation of microstructured ferroelectric composites. In addition, the BTNPs can enhance the ferroelectric properties of composites and concentrate the applied stress around the hard BTNPs in the composite under pressure.

The flexible TES comprises a hierarchical ferroelectric composite and an aluminum (Al)–coated counter electrode as negative and positive triboelectric layers (fig. S8). Through the triboelectrification and electrostatic induction between these surfaces with opposite triboelectric charges, TESs can generate spontaneous electrical power in response to mechanical pressure, resulting in self-powered dynamic force sensors (27–31). The performance of TESs is based on the surface charge density of ferroelectric materials. The surface charge density is proportional to the ferroelectric polarization that couples with the surface polarization through triboelectrification (32, 33). Therefore, the piezoelectric output current was measured on the basis of the annealing temperature to optimize the crystalline β-phase of ferroelectric PVDF-TrFE/BTNP composites (fig. S9). Above the 100°C of annealing temperature, TESs exhibit significantly increased piezoelectric outputs. This indicates that the crystalline β-phase of ferroelectric PVDF-TrFE/BTNP composite is highly enhanced near the Curie temperature of PVDF-TrFE (≈100°C) (34). The counter triboelectric electrode is optimized, and a thin (50-μm) Al–polyethylene terephthalate (PET) electrode is used to achieve the best TES performance (fig. S10). Compared with the indium tin oxide (ITO)–PET electrode, the Al-PET electrode exhibits enhanced triboelectric current due to the relatively larger positive triboelectricity of Al than ITO in the triboelectric series (35). Moreover, thin (50-μm) Al-PET electrode results in larger TES performance than the performance thick (200-μm) Al-polyethylene naphthalate and Al-PET electrodes. This is due to the larger triboelectrification area induced by the high deformability of the relatively low modulus of the thin PET substrate under mechanical pressure.

Structural effect of hierarchical ferroelectric composites on TES performance

The material and structure of ferroelectric composites affects the stress-induced polarization and capacitance of ferroelectric composites. This affects the surface charge density and output performance of TESs (32). We studied the pressure sensitivities of TESs with different structural components, such as BTNPs, MPs, and MDs (Fig. 2A). Compared with bare PVDF-TrFE films, a PVDF-TrFE/BTNP composite film enhances the stress-induced polarization because of the stress concentration at the interface of polymers having different mechanical moduli increasing the triboelectric output current. In addition, the high-k property of BTNPs enhances the dielectric polarization and capacitance of ferroelectric composites (fig. S11A). This increases the surface charge density and thereby results in the enhancement of triboelectric output performances (the details are in note S1) (32, 36, 37). The triboelectric current increases with the concentration of BTNP until 3 weight (wt %) but decreases for concentrations over 3 wt % due to the reduced deformability caused by the increased stiffness of composite films (fig. S12). The introduction of a porous structure in the PVDF-TrFE/BTNP composite increases the output current more than the introduction of a planar structure due to greater variation in film capacitance caused by improved compressibility (fig. S11A) that enhances the surface charge density (note S1) (37). The compressibility increases with the thickness of porous films, resulting in enhanced output current (fig. S13). Furthermore, the localized stress at the interface of MP-NP structure enhances the stress-induced polarization (38). Compared with other flat surfaces, the surface MD structure enhances the triboelectric currents in the low-pressure range (<3 kPa) more (Fig. 2A, green) due to greater variation in surface contact area and concentrated stress at the contact point of surface MD structures (39). The increase in MD size improved the output current due to larger deformability of surface MD structures (fig. S14). Therefore, owing to those structural contributions of gradual NP, MP, and MD structures, the sensitivity and linear response of hierarchical structure significantly improved without electrical saturation in a wide pressure range (~70 kPa; Fig. 2A, yellow). Furthermore, the sufficient gap distance between the opposing surfaces in the hierarchically structured composite enables a high-power generation even without any bulk spacers that are required in conventional TESs. As shown in fig. S15, the output performance of hierarchical TESs is inversely proportional to the thickness of spacer as opposed to the output performance of TESs with planar and flat-surfaced porous structures. The spacer-free device based on hierarchical structures offers the design for ultrathin and wearable TESs.

Fig. 2. Pressure sensitivity and linearity depending on the structural components and porosity of hierarchical TESs.

(A) Comparison of the pressure sensitivities of TESs based on the different structures of ferroelectric composites [porosity (P) = 70%]. (B) Comparison of the pressure sensitivities of the hierarchically structured TESs depending on different porosities. (C) FEA-based simulation results for the stress-induced polarization of the ferroelectric composites with different structures and the hierarchically structured composites with different porosities under 70-kPa applied pressure. (D and E) Simulated results of the ferroelectric polarization combined with piezo- and tribo-polarizations based on different structures of the ferroelectric composites (D) and different porosities of the hierarchically structured composites (E).

The pressure sensitivity and linearity of hierarchically structured TESs significantly increased with an increase in the porosity and pore size of the ferroelectric composite (Fig. 2B). This is attributed to the enhanced stress-induced polarization and a greater variation in the capacitance due to the higher compressibility of composites (fig. S11B). In addition, the increased crystallinity and well-aligned dipole of ferroelectric composite due to the use of a higher dipole moment solvent induces enhanced surface potential, further improving the pressure sensitivity (40). The improvement in crystallinity depending on solvents can be observed from x-ray diffraction (XRD) results (fig. S16), which results in enhanced piezoelectric currents (fig. S17). The higher porosity results in a gradual increase in surface contact area in a low-pressure region and a consistent variation in film capacitance in a high-pressure range, which leads to an increase in the linearity of pressure response and extension of dynamic pressure range. The surface contact area in the low-pressure region is decreased and the deformation range of MD in the high-pressure region is increased, corresponding to an increase in the size of the MD structure. This leads to gradient deformability of ferroelectric composites, enabling a high linear response over a wide pressure range (fig. S18). The dynamic pressure sensitivity (36 nA/kPa, 1 V/kPa) and wide range of linear response (~70 kPa) of the proposed hierarchical TESs with a porosity of 80% are significantly higher than those of the existing polymer-based TESs (table S5) (4, 9, 32, 41–50). While a few TESs show outstanding performances in terms of either linearity or sensitivity, none of the existing TESs exhibit excellent linearity and sensitivity simultaneously. The pressure-induced output voltages of TESs show high sensitivity and linearity (fig. S19). The maximum power density of 66.5 μW/cm² was achieved for an external load resistance of ~22 megohms (fig. S20). This is the highest power density among the power densities achieved by any other flexible TESs (table S5). While the pressure sensitivities of our TESs decrease as the sensor size is reduced, the high linearity of our TESs could be achieved regardless of sensor size (fig. S21).

To study the stress concentration effect of hierarchical structures and the relationship between stress-induced polarization and TES performance, we performed the finite element analysis (FEA) of structure-dependent stress distribution and the resultant ferroelectric polarization of composites (note S2). Stress is concentrated near the NP, MP, and MD structures (fig. S22), resulting in the significantly higher stress-strain (S-S) slope (2057) of hierarchical MD/MP/NP structures than the S-S slope (0.14) of the planar structure (fig. S22A). Furthermore, the stress concentration increases with the porosity. The S-S slope increases from 166 to 2314 with the increase of porosity from 20 to 80% (fig. S22B). Therefore, the hierarchical MD/MP/NP structure with 80% porosity exhibits a high S-S slope of 2314, indicating 16,528 times improved stress concentration compared to the stress concentration in the planar structure (slope of 0.14). Figure 2C shows the simulated stress-induced polarization of composites based on the stress distribution results (fig. S22) and the experimentally measured charge density values (fig. S23). Unlike the conventional TESs with the triboelectric surface polarization (tribo-polarization), the proposed TES based on ferroelectric composites exhibits the piezo-polarization in the interior of the composite in addition to the tribo-polarization (the details are in note S3). The introduction of BTNPs with high piezoelectric and dielectric coefficients simultaneously improves the piezoelectric and dielectric properties of PVDF-TrFE/BTNP composites. This enhances both piezo- and tribo-polarizations (fig. S24, A and B, red), resulting in the higher polarization than the polarization in the planar PVDF-TrFE films ( Fig. 2D, red). The pressure response by the tribo-polarization is rapidly saturated in the low-pressure range. However, the enhanced piezo-polarization due to the concentrated stress at the interface between the polymer matrix and BTNPs improves the pressure sensitivity in the high-pressure region. The MP structure with the large surface area and high compressibility improves the surface charge density increasing the tribo-polarization. Moreover, piezo-polarization in the porous structure is increased due to the stress concentration at the interfaces of air void, PVDF-TrFE polymer, and BTNPs (Fig. 2D, blue, and fig. S24A, blue). Furthermore, the MD surface structures with the large surface areas enhance the tribo-polarization more than the piezo-polarization in the low-pressure range. The simulated total polarization values in Fig. 2D agree with experimental results in Fig. 2A, indicating that MD, MP, and NP structures enhance pressure sensitivities in the low-, medium-, and high-pressure regions, respectively. Further increase in porosity significantly increases piezo-polarization by increasing the sites of localized stress and tribo-polarization by increasing the surface contact area (fig. S25). Notably, the piezo-polarization with the linear response to mechanical stress is more dominant than the tribo-polarization, leading to the linear increase in total polarization of composites (Fig. 2E). The simulated results of porosity-dependent ferroelectric polarization agree with the experimental results in Fig. 2B, indicating that stress-induced polarization heavily influences the linearity and sensitivity of TESs.

High-sensitivity and frequency-selective acoustic sensing of hierarchical TESs

The proposed self-powered and flexible TESs with excellent dynamic force sensing capability can be used as wearable acoustic sensing devices. To evaluate the acoustic sensing capability of the proposed hierarchical TESs, we recorded the output voltages of TESs in response to acoustic sound at different frequencies and sound pressure levels (SPLs) tuned by a sound controller (Fig. 3A). We used sound waves with different SPLs at a constant frequency of 100 Hz to obtain the acoustic sensitivity and minimum detection level of the TES. The proposed hierarchical TES with a highly linear response to dynamic pressure can record sound pressure with a linear voltage signal and high sensitivity (S) of 607 mV/Pa, which is comparable with that of a commercial microphone (582 mV/Pa) (Fig. 3B). On the contrary, a planar-structured TES exhibits a low sensitivity (~53.1 mV/Pa) and nonlinear response. The proposed TESs exhibit a higher SNR values [SNR = 20log (V_output/V_noise)] than the SNR values of a commercial microphone over the entire sound pressure range (Fig. 3C) and SPLs at the fixed frequencies of 0.1, 1, and 10 kHz (fig. S26, A to C). The highest acoustic sensitivity of the proposed TESs is ~33 dB at the frequency of 200 Hz (fig. S26D).

Fig. 3. Acoustic sensing capability and frequency selectivity of the hierarchical TESs.

(A) Photograph of the experimental setup for measuring the acoustic sensing capability. The distance between TESs and speaker is 5 cm. The inset shows variable parameters for tuning the resonance frequency. (B) Output voltages of the commercial microphone and hierarchical TESs as a function of sound pressure. (C) SNRs of the commercial microphone and hierarchical TESs as a function of sound pressure (sound frequency, 100 Hz). (D) Frequency-dependent output voltages of the hierarchical TESs with a porosity of ~70% and a thickness of 500 μm for sound sources with a pressure intensity of 0.5 Pa. (E to G) Acoustic sensing capabilities of the hierarchical TESs for different thickness, porosity, and area of the hierarchical composites for sound sources with multiple frequencies between 80 and 10,000 Hz. (E) Resonance frequency as a function of the thickness of hierarchical composites (porosity, 80%). (F) Resonance frequency as a function of the porosity of hierarchical composites. (G) Resonance frequency as a function of the area of hierarchical composites (porosity, 20%). (H) FEA-based simulated results of the displacement of the ferroelectric composites for different mechanical modulus. (I) Experimental and theoretical resonance frequencies of hierarchical TESs as a function of porosity. (J) Comparison between the tunable frequency range of the proposed hierarchical TES and existing frequency tunable acoustic sensors.

The TES exhibits a specific resonance frequency at 200 Hz (Fig. 3D). This aids the development of a frequency-selective acoustic sensor to amplify specific sounds close to the resonance frequency and minimize the disturbance from harmonic noise oscillation. The tunability of resonance frequency of the proposed TESs is expressed by the equation for the resonance frequency of a rectangular plate as follows (51)

$$\text{Resonance frequency} (\text{Hz}),f=\frac{{\mathrm{\lambda }}^2}{2\mathrm{\pi }{a}^2}\sqrt{\frac{{\mathit{Eh}}^3}{12\mathrm{\gamma }(1-{\mathrm{\nu }}^2)}}$$ (1)

where a, h, E, ν, γ, and λ are the length, thickness, elastic modulus, Poisson’s ratio, mass per unit area, and dimensionless parameter as a function of vibration mode, respectively. The detailed explanation on the theoretical calculation and experimentally measured mechanical modulus is provided in note S4 and fig. S27, respectively. Equation 1 indicates that the resonance frequency increases with the mechanical modulus that is inversely proportional to the porosity. Moreover, the resonance frequency is directly proportional to the thickness and inversely proportional to the length of TESs (i.e., sensor area). This agrees with the experimental results. We analyzed the recorded waveforms (fig. S28, A to C) and fast Fourier transform (FFT) results (Fig. 3, E to G, and fig. S28, D to F) for TESs with different porosities, thicknesses, and sensor sizes (Fig. 3A, inset) over a wide range of sound frequencies (80 to 10,000 Hz). Compared to the nonporous structure with the resonance frequency approximately equal to 600 Hz, the porous composites show red shift of resonance frequency toward 200 Hz due to the decrease in film modulus caused by the increase in inner porosity (Fig. 3F). FEA-based simulation verified (Fig. 3H) that the shift in resonance frequency is attributed to different displacements of the film depending on its mechanical modulus. The decrease in thickness further shifts the resonance frequency from 195 to 145 Hz at 80% porosity (Fig. 3E) due to the increase in displacement of thinner films. For the porosity of 20% with the high resonance frequency, decreasing the sensor size increases the resonance frequency from 530 to 9000 Hz (Fig. 3G) due to the decrease in displacement of the film caused by the increased effective modulus of the film. Therefore, the resonance frequencies of the proposed TESs are tuned by varying the thickness, porosity, and sensor size to achieve the frequency ranges of 145 to 195 Hz, 195 to 626 Hz, and 530 to 9000 Hz, respectively. This is determined using the calculated frequencies from Eq. 1 (Fig. 3I and fig. S29).

The proposed TESs exhibited significantly increased sensitivity, dynamic range, and frequency tunability than the existing acoustic sensors (table S6) (6–8, 18, 20, 21, 23, 52–60). The acoustic sensitivity (607 mV/Pa), SNR (33 dB), and relative sensitivity (−56 dB) of the proposed TESs are higher than the acoustic sensitivity, SNR, and relative sensitivity of the existing self-powered acoustic sensors based on piezoelectric and triboelectric properties. Crack-based resistive sensors (53) and microhole-patterned capacitive sensors (55) have achieved higher sensitivities than the sensitivity of TESs. However, the higher sensitivities were obtained by direct contact with the human neck surface and measuring the skin vibration. The dynamic bandwidth range (145 to 9000 Hz) of tunable resonance frequency for the proposed TESs is considerably wider than the bandwidth of the existing self-powered frequency-selective acoustic sensors (Fig. 3J). The frequency ranges of human voice (85 to 255 Hz), telephone (300 to 3400 Hz), and public noise (20 to 200 Hz) that lie within the tunable frequency range (145 to 9000 Hz) of the proposed TESs are suitable for practical applications, unlike the inorganic acoustic sensors with frequency tunability in the high-frequency region only (>10 kHz). Furthermore, our TESs have the flexibility to maintain the frequency-selective acoustic sensing capability under different bending curvature, which provides a great potential for wearable electronics (fig. S30).

The tunable resonance frequency of the proposed TESs enables a noise-independent and frequency-selective multichannel acoustic sensor array with multiple resonance frequencies that accurately records the target sound without interference from specific frequency noise. For the multichannel acoustic sensor, four acoustic sensing channels with different resonance frequencies of 200 Hz (channel 1: t, 500 μm; p, 80%; a, 2.0 cm × 1.5 cm), 600 Hz (channel 2: t, 100 μm; p, 20%; a, 2.0 cm × 1.5 cm), 4500 Hz (channel 3: t, 100 μm; p, 20%; a, 1.2 cm × 0.9 cm), and 9000 Hz (channel 4: t, 100 μm; p, 20%; a, 0.4 cm × 0.3 cm) are integrated onto a flexible PET substrate (Fig. 4, A and B), which is supported by the STFT results under frequency sweep tests (fig. S31). To study the noise-independent acoustic sensing capabilities of the multichannel acoustic sensor, we recorded the target sound wave for different noises with mono and multiple frequencies (Fig. 4C) and analyzed the short-time Fourier transform (STFT) of the sound wave. Without noise condition, the commercial microphone (MIC) exhibits STFT results matching with the STFT results of the original sound. However, the STFT image is significantly distorted for a single-frequency noise (200 Hz) due to the harmonic oscillation of noise signal (Fig. 4D). On the contrary, the impact of noise disturbance is negligible on the proposed multichannel acoustic sensors because of one specific resonance frequency for each pixel (Fig. 4E and fig. S32). The effectivity of the channels of a multichannel acoustic sensor is less affected than the effectivity of a commercial MIC by interference from high-frequency noise of 2000 Hz (fig. S33). In addition, the multichannel acoustic sensors show different STFT images for each channel, allowing the selective detection of the desired acoustic wave by the amplification of sound near the resonance frequency ranges and thus minimizing the noise influence. The proposed multichannel acoustic sensors show clearer STFT images than the STFT images of commercial MICs under noise conditions with multiple frequencies, for example, from the sound source of Beethoven’s 5th Symphony (Fig. 4, D and E, bottom).

Fig. 4. Noise-independent acoustic sensing capability of multichannel TES array.

(A) Schematic of the experimental setup for measuring acoustic sensing capability of commercial microphone and multichannel TES array under noisy environment. (B) Photograph of multichannel TES array for noise-independent acoustic sensor. (C) STFT images of original sound “Frequency-selective acoustic sensor” (top) and multiple noise sound (Beethoven, Symphony No. 5) (bottom). (D) STFT results of the acoustic waveforms recorded using the commercial microphone without noise (top), with single-frequency noise (middle), and with multiple noise (bottom). (E) STFT results of the acoustic waveforms recorded using the multichannel TES arrays without noise (top), with single-frequency noise (middle), and with multiple noise (bottom). The magnified STFT images show the noise-independent acoustic sensing capabilities near the resonance frequency range of each channel. (F) Confusion matrix of acoustic sensing capability of the commercial microphone and channel 1 of multichannel TESs under the noise environment of 200 Hz. The row of the confusion matrix represents the predicted class, and the column represents the true class to discern whether test data are matched with the predicted class or not. The first and second columns are the classification instances on the test data without and with acoustic waveform, respectively. The diagonal cells (green) represent the correct instances between the predicted class and true class. (G) Acoustic sensing accuracy of the commercial microphone and the multichannel hierarchical TESs for different frequency noises.

For qualitative comparison of noise-independent acoustic sensing capability, we evaluated the accuracy of voice recognition using a machine learning technique. The voice recognition by ANN is conducted through the process of data acquisition, feature extraction, and pattern recognition. We selected the shallow feed-forward neural network for pattern recognition as well as the frequency and amplitude responses to the voice as the key features of the TES. The resulting confusion matrices of voice recognition results for the commercial MIC and multichannel TESs are shown in Fig. 4F and fig. S34 (detailed explanation on confusion matrix based on ANN is included in note S5). The proposed TES exhibits a higher accuracy of 99.2% than the accuracy of 86.7% for commercial sensors under 200-Hz noise environment. The frequency noise of 200 Hz has a negligible impact on the high accuracies of other channels of TESs with resonant frequencies above 200 Hz (fig. S34). The proposed TES exhibits the higher accuracy for voice recognition under single-frequency noise environment that ranges from 100 to 8000 Hz than the accuracy of commercial microphone. Moreover, each channel displays accuracy greater than 95% for different frequency noises except for the resonant frequency noise (Fig. 4G). Notably, although the accuracy of each channel is rather decreased due to simultaneous amplification of noise and sound near the resonance frequency range, a high capability of voice recognition is still maintained and the accuracy of each channel is much larger than that of commercial sensor. This result indicates that the proposed multichannel TES can be used in wearable voice recognition devices, cochlear implants, and HMIs.

Dual-signal perceptive HMI

The noise-independent and frequency-selective dynamic sensing capability of the proposed TESs facilitate texture perception, remote control of dynamic robotic hand motion, and fabrication of dynamic HMI devices. We fabricated a smart glove by integrating TESs onto the tips and joints of fingers for the detection of surface texture and finger movements, respectively (Fig. 5A). To evaluate the texture perception capability, the proposed TES was placed on the three-dimensional printed target surfaces with regular line patterns and scanned using a homebuilt system that controlled the scanning speed and micro-unit displacement (fig. S35A). In particular, mimicking the surface texture perception by fingerprints on human fingertips (4), fingerprint-like patterns with parallel ridges (pitch, 470 μm; width, 163 μm) are configured on the TES for amplifying the tactile sensing signals (fig. S35B). When TESs scan the surface textures with regular intervals of 1500, 1250, 1000, and 750 μm (Fig. 5B), periodic spike peaks are generated (fig. S36). The FFT spectra obtained from regular spike peaks exhibit fundamental frequencies that depend on the scanning speed (fig. S37). The STFT results display the spatial distribution images corresponding to texture patterns (Fig. 5C). The high-accuracy texture perception capability of the proposed TES enables the classification of various surface textures with different materials and roughnesses using the machine learning techniques. We collected large quantities of time-dependent current data (fig. S38A) corresponding to the type of surface textures (for example, polyester, cotton, nylon, silk, glass, paper, and sandpaper) for ANN training and transformed the data into FFT spectra for extracting frequency features (fig. S38B). The featured three frequency peaks based on the different roughness and regularity of surface textures enabled the clear classification of various materials. The resulting classification matrix indicated a high positive predictive value of 92.7% for the proposed training model, which provides a simple platform for the robotics requiring sophisticated works.

Fig. 5. Applications of the hierarchical TESs for dynamic HMI.

(A) Schematic of the smart glove for HMI applications, enabling the texture perception and remote control of robotic hands. The inset shows each pixel composition with sensor, encapsulation layer, and adhesive tape. (B) Optical images of the target surface texture with different line pattern widths of 1.5, 1.25, 1, and 0.75 mm. (C) STFT of the output currents generated by scanning the surface textures. (D) ANN to recognize different textures and surface roughness. (E) Control block diagram and electronic circuit in wireless network system of HMI applications. (F) Output voltages of hierarchical TESs on the smart glove as a function of bending angle. a.u., arbitrary units. (G) Photographs of motion control of robotic hand using different bending motions of single-pixel smart glove. (H) Photographs of motion control of robotic hand using the multipixel smart glove. (I) Photograph of sound-driven HMI application. (J) Demonstration of motion control of the robotic hand using frequency acquisition within the separated working ranges.

We further demonstrated a TES-based smart glove for HMIs to remotely control various motions of robotic hands (fig. S39). The wireless platform consisted of a Bluetooth-based signal transmitter with an analog signal conditioning circuit to adapt the output signals and a wireless data receiver connected with actuators for controlling the robotic hands (Fig. 5E, detailed explanation on experimental setup is included in note S6). The high sensitivity and linearity of the proposed TESs enables precise control of versatile motions of avatar robot hands without complex signal processing and operating algorithm. Figure 5F shows that the triboelectric output voltages increase proportionally with the increase in the bending degree of human fingers from 30° to 120°. The polarity and intensity of output voltages allow the accurate control of the flexion-extension motions and bending degree of robotic hands (Fig. 5G and movie S1). In addition, our TESs show the clear changes in the voltage depending on the bending movement (bending curvature, 0.46) with different frequencies (0.16 to 5 Hz) that are similar with the usual frequency range of hand gesture, which proves the availability of our TESs for the glove-based HMI applications (fig. S40). Figure 5H shows the real-time manipulation of versatile gestures of robotic hands using the smart glove. The motion of robotic hands was controlled using various motions of smart gloves (movie S2), providing a great potential for various applications including humanoid and surgical robotics and AR/VR. The conventional HMIs are primarily driven by mechanical stimuli (61). However, the proposed TES with frequency-selective acoustic sensing capability enables sound-driven HMI devices for remote control and continuous real-time manipulation (Fig. 5I). For the acoustic sound-driven HMIs, we use four different acoustic frequency ranges (100 to 300 Hz, 300 to 500 Hz, 500 to 700 Hz, and 700 to 900 Hz) to operate the robotic hands with different finger grasping motions. As shown in Fig. 5J and movie S3, versatile robotic hand motions can be controlled wirelessly using specific input sound frequency. Furthermore, the sound-driven HMIs with the high acoustic frequency selectivity can be only operated by the designated frequency, which permits the accurate manipulation without interference from complex frequency inputs (movie S4). Similar to the mechanical pressure-induced bending motion control, the hierarchical TESs with an improved acoustic sensing capability (fig. S41) and a high linearity in the low-pressure range (<1 Pa) (fig. S42, A and B) also enable to manipulate the bending degree of the robotic hand by the changes in the amplitude of acoustic sound, which provides a great possibility for the development of precisely controllable HMI devices (fig. S42C and movie S5). The proposed TESs with dual-mode dynamic/static HMI capabilities exhibit improved control of complicated tasks by various devices and robots than the existing glove-based HMI devices based on single mechanical signals even under noise disturbances.

DISCUSSION

We developed highly linear and sensitive TESs based on the ferroelectric composites with a hierarchical architecture comprising MD, MP, and ceramic NP. The hierarchical geometry induced stress concentration at the interface of heterogeneous materials with different moduli and high deformability, providing linear gradient of stress-induced polarization. This facilitated the high sensitivity (36 nA/kPa) and linearity (1 V/kPa) of the proposed TESs over a wide dynamic pressure range (0 to 70 kPa). We demonstrated the capabilities of the proposed TESs by using them in dynamic interfacing devices used to recognize the acoustic wave, surface texture, and dynamic movements. The facile tunability of the resonance frequency for TESs using the structural designs of hierarchical TESs allows the realization of a high acoustic selectivity over a wide frequency range (145 to 9000 Hz). This leads to a high accuracy of over 95% in the case of noise-independent voice recognition devices. Furthermore, the high flexibility and linear responsivity of TESs assist in detecting and distinguishing between the fine textures of surfaces and versatile motion of robotic hands. Thus, hierarchical TESs exhibit great potential as the next-generation sensor for dynamic interfacing applications. The capabilities of hierarchical TESs based on ferroelectric composites provide a solid platform for improving the existing conventional sensors and application of TESs in humanoid robots, wearable devices, and biometrics.

MATERIALS AND METHODS

Fabrication of ferroelectric composite with hierarchical structure

For preparing the ferroelectric composite solution, PVDF-TrFE (70/30) powder (Piezotech, France) was initially dispersed in various organic solvents (DMSO, DMF, ACN, and MEK). The dispersed PVDF-TrFE solution (15 wt %) was then mixed with polyvinyl pyrrolidone (PVP)-coated BaTiO₃ NPs (U.S. Research Nanomaterials Inc., USA) in the desired mixing ratio. To form the MD surface of the ferroelectric composite, the PVDF-TrFE/BT solution was poured onto the micropatterned polydimethylsiloxane molds replicated by the silicon master molds with different feature sizes (diameter, 80, 120, and 150 μm; pitch/diameter, 1.5; height/diameter, 0.5). The casted solution was stored in the water-filled desiccator for 12 hours under vacuum condition for MP development. After drying the ferroelectric composite solution, the dried film was moved to another desiccator for removing the residual solvent. The composite films were finally annealed in a vacuum oven at 100°C for 2 hours to crystallize the PVDF-TrFE polymer.

Fabrication of flexible TES devices with hierarchical ferroelectric composite

To compose the TES devices, an Au electrode was coated onto the top of the ferroelectric composite by thermal evaporation. The Al-based counter electrode with positive surface charge was fabricated by thermal evaporation onto the PET substrate with a thickness of ~100 nm. The two films with positive and negative charges faced each other and were attached using the polyimide (PI) tape at both ends for fixing the neighboring films. We fabricated a smart glove by adding an encapsulation layer (PI tape) and a stretchable adhesive layer (3M VHB tape) at the bottom of the sensor to reduce the damage under repetitive usage and to tighten the sensor onto the glove.

Characterization

The structures and morphologies of the hierarchical ferroelectric structure were characterized using field-emission scanning electron microscopy (S-4800, Hitachi, Japan). Depending on the organic solvents, the crystallinity of the ferroelectric composites was analyzed using XRD spectroscopy (D8 ADVANCE, Bruker AXS, USA) with Cu Kα radiation, and the mechanical properties were analyzed using a dynamic mechanical analyzer (TXA-TM, Yeonjin Corp., South Korea).

Evaluation of the dynamic sensing capabilities of TESs

The pressure-dependent electrical currents and voltages of TESs were evaluated using a source meter (S-2400, Keithley, USA) and oscilloscope (DPO 2022B, Tektronix, USA), respectively. The frequency of the input signals of dynamic pressure was maintained at ~5 Hz using a pushing tester (JIPT, JUNIL Tech, South Korea). For evaluating the acoustic sensing capability, TESs and condenser-type reference microphone (frequency range, 10 to 20,000 Hz; dynamic range, 33 to 135 dB) (40PH, GRAS) were positioned on a stand 5 cm away from the speaker. The output voltages were measured depending on the acoustic waveforms controlled by a loudspeaker test oscillator at a high-frequency range (80 to 20,000 Hz) (SG-3428B, Sigma Electronic Technology Co. Ltd., South Korea). To analyze the texture perception capability of TESs, we used a laboratory-built micro stage system (Micro Motion Technology, South Korea) for repeated displacement under constant speed (1 to 5 mm/s) and loading weight (10 g) and subsequently measured the electrical output from the source meter.

Finite element calculation

The theoretical simulations on the stress distribution and stress-induced polarization of the ferroelectric composites with different structures and porosities were performed using COMSOL 5.4 Multiphysics. The FEA of the structural effect on the resonance frequency of TESs was performed using Abaqus. The resonance frequency of TESs for different porosities, thicknesses, and areas was numerically calculated by analyzing the vibrational behavior of the films under external vibrations.

Artificial neural network

For the voice and texture recognition analysis, three strongest frequencies and its total amplitudes in the FFT spectra of the digital data were selected for feeding the ANN. In addition, an optimization process was applied to prevent the overfitting modeling error that occurs when a function is too closely fit to a limited set of data points. Our ANN model comprises 6 nodes for the input layer, 10 nodes for the hidden layer, and 2 nodes for the output layer. Therefore, at least 120 samples are used to train the ANN.

Supplementary Materials

This PDF file includes:

Notes S1 to S6

Figs. S1 to S42

Tables S1 to S6

Other Supplementary Material for this manuscript includes the following:

Movies S1 to S5