Bioinspired flexible sensing-processing-visualizing integrated system towards tactile-visual signal recognition

Abstract

The rapid growth of flexible Internet of Things devices and multisensory human-computer interaction necessitates advanced systems capable of multimodal data sensing, processing, and feedback in an intelligent and energy-efficient manner. However, conventional architectures face limitations in energy inefficiency and interface mismatch due to the fragmented architecture of sensing, processing, and feedback units. Inspired by biological sensory systems, we introduce an MXene-based flexible dual-mode sensing-processing-visualizing integrated system into a single wearable device through a hierarchical MXene platform combining piezoelectric nanogenerators (mechanosensation), optoelectronic synapses (visual processing), and color-shifting quantum dot light-emitting diodes (optical feedback) with optimized interfaces. The wearable integrated system demonstrates tactile-visual signal recognition capabilities, biological self-protection behavior to adapt to environmental stimuli, dynamic trajectory recognition, and spatial positioning for motion recognition, which are highly desired in multisensory interaction. The bio-inspired material-architecture-function co-design strategy and integrated system promote the development of wearable neuromorphic hardware, edge computing, and intelligent human-machine interaction.

The authors present a wearable MXene-based sensing-processing-visualizing integrated system for tactile-visual recognition, multisensory interaction, trajectory recognition, and real-time feedback.

Introduction

The proliferation of flexible Internet of Things (IoT) devices and the emergence of multisensory interaction applications have heightened the demand for optical and mechanical data processing to meet the diverse needs across various user scenarios^1,2. The acquisition of large volumes of data has further fueled the growing need for integrated edge computing solutions that can facilitate information perception, processing, visualization, and human-computer interaction. However, in traditional von Neumann architecture, the physical separation of multimodal sensing units and computing units presents significant challenges in data transmission efficiency, energy consumption, communication bandwidth, and flexibility^3–5.

Highly evolved biological sensory systems enable humans to sense multimodal information and then process and respond to information efficiently and intelligently, which can be attributed to their highly-physical-integrated one-body architecture. Inspired by the biological sensory system, researchers have focused on emulating its functions to enhance the efficiency of information perception and processing. Notable advancements have been achieved in mimicking the fundamental nervous system functions by functionally integrating sensors^6–8, neuromorphic devices^9–12, actuators^13–15, and peripheral circuits^16,17. Integration of sensory memory processing devices has been partially achieved to reaching in-sensory/memory computing system relying on the serial connection of peripheral circuits^7,18–22. Most existing approaches mainly rely on serial connections between distinct components for multi-mode sensory system, which is limited in complicated system design and flexibility of wearable devices due to device interface mismatch at both mechanical and electrical levels^18,23. Integration of tactile and visual sensory modalities and the corresponding processing and feedback systems—with matched mechanical and electrical interfaces between different functional units—remains a key challenge.

Living organisms have effectively addressed these challenges by utilizing shared flexible quasi-two dimensional (2D) cell membranes that not only allow for outstanding mechanical interface matching but also enable efficient ionic-electronic signal transfer and conduction, essential for coordinated sensory processing and feedback. In detail, the cell membranes—as a component-shared universal platform that enhances nutrient utilization and interface compatibility for both mechanical and electrical interfaces—serve multiple roles including tactile sensation via the inserted Piezo2, vision sensation through the inserted rhodopsin, conversion of the sensory stimuli into ion-electron signals, and efficient transportation of these signals for interacting and processing, thereby facilitating diverse physiological processes (Fig. 1a).

Fig. 1. Design of MXene-based flexible sensing-processing-visualizing integrated system.

a Schematic of a biological multimodal neural system connected through neural conduction occurring on the cell membrane. Left: Piezo2 ion channel protein allows ions to pass through the cell membrane and generates action potentials when subjected to pressure. Center: rhodopsin allows ions to pass through and generate an action potential, when exposed to light, due to the detachment of the prosthetic group affected by the conformational change of retinal. Right: nociception and visualization formed in the brain region due to external stimulation. b Schematic of MXene-based flexible sensing-processing-visualizing integrated system connected through electron conduction occurring on the L-MXene. Left: MXene-dual-functional PENG with M-MXene additives (polarization locking) and L-MXene electrodes. Center: OSMX-based (charge trapping) AOS with L-MXene electrodes. Right: CS-QLED with L-MXene (high electrical conduction). c Top: scanning electron microscopy (SEM) images of M-MXene (left), OSMX (center), and L-MXene (right); scale bars: 1 μm. Bottom: schematic of the minimally intensive layer delamination (MILD) etching and mechanical delamination process.

The working principle and structure of a quasi-2D multi-functional cell membrane-based multisensory perception-processing-feedback system provide valuable inspiration for designing fused material and architecture in the field of bio-inspired electronics to achieve multimode, wearable and interface-matching functions. Advances in 2D materials, particularly MXenes, may open new avenues for overcoming these challenges. Among them, Ti₃C₂T_x MXene—known for its high electrical conductivity (>25000 S cm^-1), excellent mechanical flexibility (single-layer thickness of approximately 1 nm), excellent bendability (by sliding the MXene layers to disperse stress), rich and tunable physicochemical properties (by tailoring sizes and surface terminations), solution processability, and high transparency at thin film^24–27—outperforms conventional materials (more detailed comparisons can be seen in Supplementary Table 1) and have attracted intensive interests and demonstrated varied applications^28–32. Due to these merits, MXene is promising in constructing the bio-inspired integrated multisensory perception-processing-feedback system, yet unexplored.

Here, we present a bioinspired MXene-based flexible sensing-processing-visualizing integrated system (FSPVS) featuring mechanically and electronically matched interfaces towards wearable intelligent sensory systems (Fig. 1b). By tailoring the size and surface states, MXene acts multiple roles including a shared high-efficiency electrical conduction platform, tactile-sensation-enhanced material, visual sensation and synaptic material, and preferable charge-injection layer with minimized energy barrier, thereby achieving the conversion of the sensory stimuli into electron signals and their efficient transportation (Fig. 1b). Large-sized MXene (L-MXene) serves as a shared platform encompassing the following integrated unites: mechanosensation via medium-sized MXene (M-MXene) interfacial polarization-locking regulated piezoelectric nanogenerator (PENG); visual sensation and synaptic functionalities through oxidized small-sized MXene (OSMX)-based artificial optoelectronic synapse (AOS); optical feedback by integrated color-shifting quantum dot light-emitting diode (CS-QLED). Through the integration of these functions, the FSPVS demonstrates significant advancements in tactile-visual signal intelligence recognition and its applications in (1) bioinspired self-protection behaviors for environmental adaptation; (2) multimode image recognition task; (3) dynamic strike type and trajectory recognition via dual-mode input reservoir computing (RC) and high-precision spatial positioning for motion recognition—functionalities that are highly desired in the multisensory interaction scenarios. This work shows the potential of MXene in bioinspired flexible electronics and offers a promising solution for next-generation wearable intelligent devices, human-computer interaction, and multisensory interaction applications.

Results

Design of MXene-based flexible sensing-processing-visualizing integrated system

After millions of years of natural selection, biological organisms have developed highly matching interfaces between various tissues, optimizing compatibility in terms of material properties, and mechanical and electrical characteristics (Fig. 1a). (1) The cell membrane of biological cells is mainly composed of a quasi-2D phospholipid bilayer (thickness: 5-10 nm) and various transmembrane proteins inserted on it, which is a key structure for maintaining important functions such as cell morphology (mechanical stability) and signal transduction (neural electrical signals). (2) The mechanical properties (e.g., elastic modulus and tensile strength) of cell membranes in different tissues are similar, maintaining tissue structural and functional stability. (3) Neural conduction characteristics are consistent across different sensory perceptions. For example, tactile perception relies on Piezo2 channels that let cations flow into the cell upon pressure, triggering an action potential (Fig. 1a left)³³. Visual perception uses rhodopsin; light-induced changes in retinaldehyde conformation alter membrane potential (Fig. 1a center)³⁴. Neural transmission between synapses occurs when neurotransmitters bind to the postsynaptic membrane, causing a potential change (Fig. 1a, center and right). The consistency in material composition, mechanical properties, and ionic transport characteristics allows organisms to synthesize the necessary substances using minimal diversity during physiological processes, thereby significantly enhancing inter-tissue transport efficiency and information transmission.

Inspired by the cell membrane-based biological organisms, a material-architecture-function co-design strategy based on Ti₃C₂T_x MXene (2D material with a thickness of only a few atomic layers) is proposed for constructing a flexible sensing-processing-visualizing integrated system (Fig. 1b) with mechanical and electrical interface compatibility. By tailoring the size and surface states, MXene serves multiple functional roles: (1) a tactile-sensation-enhanced material in PENG through M-MXene with abundant surface −OH terminations that can form hydrogen bonds with the C − F bonds in poly(vinylidene fluoride-co-trifluoroethylene) (P(VDF-TrFE)) to improve piezoelectric phase (Fig. 1b left); (2) a visual sensation and synaptic material in AOS via OSMX photoresponse and defect-mediated charge trapping for synaptic conductance modulation (Fig. 1b center); (3) a preferable charge-injection layer in CS-QLED through L-MXene with engineered energy barriers (Fig. 1b right); and (4) a shared flexible high-efficiency electrical conduction platform in the FSPVS (preparation process and optical images are shown in Supplementary Fig. 1). During the system operation, the piezoelectric potential formed by polarized charges is transmitted through L-MXene to the AOS during the PENG compression process, thereby modulating its conductive state. Meanwhile, light stimulation generates a photocurrent in the AOS. The EPSCs triggered jointly by pressure and light stimulation are subsequently transmitted to the CS-QLED through L-MXene, inducing its luminescence. The use of homogeneous MXene-based materials in system integration not only simplifies the fabrication process but also enhances the mechanical flexibility and electrical contact of individual components (Supplementary Note 1 for a more detailed analysis of interface matching)³⁵.

The morphology and phase characterization of these MXene are shown in Fig. 1c top from left to right (corresponding to Fig. 1b) and Supplementary Fig. 2. Fortunately, the above-mentioned MXene can be obtained during our designed single synthesis and post-processing flow (Fig. 1cbottom). Firstly, the Al-rich Ti₃AlC₂ MAX (MXene precursor) was etched using the “MILD” method to obtain multilayer-MXene with minimal defects. Subsequently, the multilayer-MXene was exfoliated using the mechanical force generated by vortex oscillation to obtain L-MXene and M-MXene, thereby maintaining the lateral size and minimizing in-plane defects. M-MXene rich in -OH terminations and large-lateral-sized L-MXene with minimal defects were further separated by centrifugal force to respectively match the device applications of PENG, AOS, and CS-QLED. In addition, to address both material utilization and AOS application requirements, the mechanically unexfoliable multilayer-MXene was first ultrasonically fragmented into S-MXene (Supplementary Fig. 3), then subsequently oxidized and surface-functionalized to create photoresponsive OSMX containing abundant defect sites (Supplementary Fig. 4). All the reaction products of MXene precursor can be efficiently utilized in the system, preferable for low-cost environmentally-friendly production.

MXene-dual-functional PENG

Self-powered MXene-dual-functional PENGs, similar to the tactile sensing unit of Piezo2, are fabricated by depositing M-MXene/P(VDF-TrFE) (MXP) on shared L-MXene conductive layer (Fig. 2a) with exceptional electrical conductivity and mechanical strength. Additionally, MXene, with abundant and tunable surface terminations, facilitates solution-processability, enabling uniform dispersion in various solvents, including water, polar and non-polar organic solvents (N,N-dimethylformamide, dimethyl sulfoxide, chloroform, etc.)³⁶. These characteristics of MXene enable dual-functionalization in PVDF-based ferroelectrics: (1) flexible mechanically-interface-matched conductive channels via solution-processable fabrication, in contrast to conventional Cu/Al tapes (where interfacial adhesion deterioration upon repeated bending cycles causes performance degradation, Supplementary Figs. 5 and 6); (2) enhanced tactile sensing through electroactive phase engineering enabled by interfacial polarization locking, achieved via hydrogen-bond interactions between rich −OH in MXene surfaces (Supplementary Fig. 7) and the C − F dipoles in P(VDF-TrFE) polymer matrices.

Fig. 2. Characterization of MXene-dual-functional PENG.

a Schematic of the MXene-dual-functional PENG fabrication process. Schematic illustrating the working principle of the MXene-dual-functional PENG in both b initial and c pressurized states. d Piezoelectric output of MXene-dual-functional PENG at different MXene sizes at 11 N. e Short current of MXene-dual-functional PENG at different M-MXene proportions at 11 N. f Short-circuit of MXP-0.5 at different compression forces. g Current density of our work with MXene electrodes versus reported works with other electrodes (Supplementary Table 2). h COMSOL simulation results of piezopotential distribution with varying MXene proportions: i) 0%, ii) 0.1%, iii) 0.5%, iv) 1.0%.

Figures 2b and c illustrate the working mechanism of MXene-dual-functional PENG in initial and pressurized states, respectively. The interfacial polarization between the M-MXene and the P(VDF-TrFE) matrix results in a relatively uniform orientation of electric dipoles within the composite material (Fig. 2b). Upon compression, the self-polarization effect, coupled with the formation of deformed dipoles, induces the generation of polarized charges on the surface of the film. These charges are subsequently attracted to and accumulate on the charged surfaces, thereby establishing a piezoelectric potential (Fig. 2c). Consequently, external free charges migrate to the electrodes and accumulate to counterbalance the piezoelectric potential. The impact of MXene with varying flake sizes on the piezoelectric performance of PENG was initially examined (Fig. 2d). When M-MXene with a suitable flake size was used as an additive, the resulting MXene-dual-functional PENG demonstrated the highest output, which can be attributed to efficiently enhanced interface polarization and minimized leak current. Subsequently, the effect of M-MXene content on P(VDF-TrFE) was further investigated (Supplementary Fig. 8 and Supplementary Note 2). X-ray diffraction (XRD) and Fourier transform infrared (FT-IR) spectra analyses revealed that at an M-MXene loading of 0.5 wt% (the optimal sample is labeled MXP-0.5), the β-phase content reached its highest level, coinciding with the maximum dielectric constant. The piezoelectric output performance was further evaluated, including measurements of open-circuit voltage and short-circuit current (Supplementary Fig. 9 and Fig. 2e). Both metrics indicated that the output performance of MXene-dual-functional PENG peaked at an MXene content of 0.5 wt%, consistent with the aforementioned characterization results. The enhancement in piezoelectric output can be attributed to the increased β-phase content due to interface polarization locking. Fig. 2f and Supplementary Fig. 10 show that the short-circuit current of MXP-0.5 increases linearly with applied pressure ranging from 1.5 to 13 N, demonstrating a linear relationship between output current and pressure (piezoelectric sensitivity line slope: 0.419 ± 0.015 μA N^-1), which allows for precise signal calibration and accurate pressure estimation.

The dual-functional roles of M-MXene as an additive and L-MXene as an electrode in the PENG improved interface contact, significantly enhancing charge collection efficiency (Fig. 2g and Supplementary Table 2). This led to a substantial increase in current density under pressure stimulation, which was notably higher than that observed in piezoelectric films using other electrodes. The prepared MXene-dual-functional PENG exhibits excellent mechanical interface matching and stable output voltage over more than 9900 cycles (as confirmed by the long-term fatigue test in Supplementary Fig. 11), showcasing superior mechanical durability and reliability in practical applications. The results of the COMSOL Multiphysics finite element analysis simulation further supported the experimental data (Fig. 2h).

Synaptic properties of OSMX-based AOS

Visual sensation and synaptic functionalities are obtained by integrating OSMX-based AOS on a shared flexible and transparent L-MXene thin conductive layer (Fig. 3a). To achieve the light perception capability and the charge trapping ability required for AOS while improving the efficiency of the MXene synthesis process, we implemented a two-step modification procedure: (1) S-MXene obtained via ultrasonic fragmentation of waste multilayer-MXene was controllably oxidized into TiO_x-based OSMX (Supplementary Fig. 12), simultaneously enhancing light responsivity and defect density for charge trapping toward synaptic property (Supplementary Fig. 13); (2) Surface functionalization with n-dodecylphosphonic acid (DDPA) improved colloidal stability in organic solvents (verified DDPA successful grafting via the emergence of P 2p peaks in Fig. 3b and Supplementary Fig. 14). Transmission electron microscope (TEM) analysis of solution-processed ZnO nanocrystals (NCs) in Supplementary Fig. 15 confirms their uniform NCs morphology with a narrow size distribution, and atomic force microscopy (AFM) characterization (Supplementary Fig. 16) reveals a smooth surface morphology (RMS = 2.57 nm), ensuring uniform layer formation in the AOS device architecture. Furthermore, the exceptional electrical conductivity, robust mechanical strength, and superior flexibility of L-MXene render it a promising candidate to replace commercial indium tin oxide (ITO). This substitution could potentially address several critical limitations of ITO, including its inherent brittleness, high manufacturing costs, and insufficient electrical conductivity on flexible substrates such as polyethylene terephthalate (PET)³⁷. We investigated the effects of L-MXene concentration on transparency and conductivity by spin-coating solutions with varying concentrations onto PET substrates. In Supplementary Fig. 17, at a concentration of 7 mg mL^-1, the film exhibited 80.4% transparency and a sheet resistance of ~64 Ω □^-1. Meanwhile, we evaluated the bending resistance performance of the L-MXene/PET and ITO/PET electrodes. The results show that L-MXene/PET demonstrates stable electrical resistance (Supplementary Fig. 18) and superior surface integrity (Supplementary Fig. 19) after 5000 bending cycles compared to ITO/PET, thereby enhancing its potential in flexible electronic devices.

Fig. 3. Characterization of synaptic properties of OSMX-based AOS.

a Schematic illustrating the structure of the optoelectronic synapse. b High-resolution X-ray photoelectron spectroscopy (XPS) P 2p spectra of OSMX. c I-V scanning curves of AOS (0 → + 1.2 V → 0 V for 10 times). d Continuous conductance modulation by 15 positive electrical pulses with gradually increasing amplitudes (1.6 ~ 1.9 V) and 15 negative electrical pulses at a fixed amplitude (–0.3 V), with both pulse width and interval set to 0.37 s. EPSC of the AOS triggered by PENG at different e pulse numbers and f frequencies. Photoresponse of the AOS under illumination with varied g time, h pulses, and i frequencies at a fixed wavelength of 365 nm. j Variation of the PPF index as a function of different light interval times (Δt). k Experimental outputs of 16 different 4-bit optical pulse trains, ranging from (0000) to (1111). l Surface potential image of the ZnO NCs/OSMX-PMMA/ZnO NCs thin film before and after illumination; scale bars: 500 nm.

To evaluate the synaptic plasticity of the device, we conducted electrical tests on the AOS. The input voltage pulse, device conductance, and response current were used as analogs for the synaptic pre-spike, synaptic strength, and synaptic post-current, respectively. The analog resistance switching behavior was characterized by measuring the I-V curve, which demonstrated good resistance switching capability (Supplementary Fig. 20). A continuous voltage scanning mode was employed to apply a positive voltage (0 V → + 1.2 V → 0 V) to the L-MXene electrode, resulting in a gradual increase in current (Fig. 3c). Conversely, applying a negative voltage (0 V → − 1.2 V → 0 V) led to a continuous decrease in current (Supplementary Fig. 21). The progressive change in current induced by electrical signals mimics the alteration of synaptic weight (detailed calculation formula is provided in Supplementary Note 3) triggered by neural impulses. Supplementary Fig. 22 illustrates that 16 electrical pulse patterns, ranging from (0000) to (1111), correspond to 16 distinct response states, highlighting the potential of this approach in RC applications.

To elucidate the electrical mechanism of AOS, we performed a detailed fitting analysis of the I-V curve (Supplementary Fig. 23). The results demonstrated that the behavior was consistent with the space charge limited current (SCLC) theory³⁸. Figure 3d illustrates the modulation process of device conductance (synaptic weight) via voltage pulse stimulation. By applying fixed-width voltage pulses with varying amplitudes, the device conductance increased with the amplitude of positive pulses. Supplementary Figure 24 further demonstrates that both increasing pulse amplitude and stimulation number lead to an enhancement in current. Precise regulation of the device conductivity can be achieved by deliberately tuning the pulse amplitude and number. As a result, the piezoelectric pulse signal generated by PENG can be utilized to modulate the conductive state of the AOS, thereby enhancing its conductance level. During the testing process, the PENG was directly connected to the AOS. PENG can directly generate electrical pulses with varying amplitudes and pulses through its piezoelectric effect, without using an external pulse power source. When both the number and frequency of the generated electrical pulses increase (Fig. 3e and f, detailed EPSC gain analysis results are presented in Supplementary Note 4), the AOS demonstrates an enhanced current response, mimicking the heightened reactivity of biological systems under external stimulation. From the perspective of the resistance switching mechanism, an increase in the number of pulses leads to further accumulation of charges captured at defect sites, thereby enhancing the electrical conductivity of the device. Moreover, when the pulse frequency increases, charges trapped during the previous pulse cannot be fully released before subsequent pulses are applied, resulting in a progressive charge accumulation over time. We also evaluated the energy consumption of the OSMX-based AOS under single-pulse stimulation. The results reveal a low energy consumption of 1.094 pJ (Supplementary Fig. 25), indicating high energy efficiency among the existing two-terminal MXene-based neuromorphic devices (Supplementary Table 3).

In the human visual system, the intensity and duration of optical signal irradiation are crucial parameters for image information. The detection and recognition of invisible ultraviolet (UV) light can effectively avoid visible light interference, which is beneficial in artificial visual systems³⁹. The photocurrent response behavior of the AOS under different UV (365 nm) light durations, pulse numbers, and light intensities is presented in Fig. 3g and h, and Supplementary Fig. 26a, respectively, demonstrating the regulatory effect of optical stimulation parameters on EPSC. The alteration of illumination conditions not only directly determines the response amplitude but also effectively modulates the subsequent relaxation process. When a negative bias is applied, AOS shows a similar trend to that under a positive bias at different illumination times (Supplementary Fig. 26b). Figure 3i further illustrates the EPSC when the AOS is subjected to continuous light pulse sequences at varying frequencies. As the spike frequency increases from 0.2 Hz to 0.67 Hz, the cumulative effect becomes increasingly significant, leading to a pronounced EPSC response. To investigate the potential of photo-induced paired-pulse facilitation (PPF) behavior, two consecutive light pulses with a density of 13.2 mW cm^-2 and a duration of 1 s each were applied to the AOS (Fig. 3j). As shown in the inset of Fig. 3j, the response current to the second light pulse was significantly greater than that of the first pulse. The time constants τ₁ and τ₂ for the photo-induced PPF were fitted to 0.241 s and 3.059 s, respectively (R² = 0.998, calculation and fitting methods are described in Supplementary Note 5). Notably, τ₂ is approximately one order of magnitude larger than τ₁, which aligns well with the characteristics observed in biological synapses⁴⁰. Furthermore, we fabricated AOS devices with varying OSMX-to-PMMA mass ratios to modulate their photoelectric properties (Supplementary Fig. 27). We also conducted systematic tests under different bending radii to investigate the mechanical flexibility of the AOS. When the bending radius ranged from 10 to 3 mm, the electrical pulse conductance modulation performance (Supplementary Fig. 28a) and the optical pulse response current (Supplementary Fig. 28b) of the AOS did not show significant changes. In addition, at the bending radius of 3 mm, the electrical conductance modulation performance and the optical pulse response maintain stability after 1000 bending cycles (Supplementary Fig. 29), further demonstrating the bending resistance of the device. We further analyzed the overall energy consumption of the PENG and AOS system (Supplementary Fig. 30). The result demonstrated that the PENG and AOS system can operate in a self-powered mode (0 J) under pressure stimulation via PENG, and achieve a single-pulse energy consumption as low as 1.437 fJ under optical stimulation. Compared to existing PENG and memristor integrated systems, our PENG and AOS integrated system exhibits distinct advantages in terms of device integration, multimodal sensing capabilities, and versatility in functional applications (Supplementary Table 4).

To evaluate its capability to incorporate sequential optical inputs, we subjected the memristor to sequences of optical pulses. Each sequence comprised four pulses, with “1” and “0” denoting light pulses of 13.2 and 0 mW cm^-2 intensity, respectively, and each pulse lasting for 1 s. In Fig. 3k, single-mode 4-bit optical pulse input varying from (0000) to (1111) yielded 16 distinct states. Moreover, the AOS exhibits multimodal perception capabilities, as demonstrated by processing a 4-bit binary input stream under the “EEOO” stimulation mode, in which the first two pulses were designated as electrical stimuli and the following two as optical stimuli (Supplementary Fig. 31). This dual-mode response facilitates the implementation of a mixed-input RC system. The findings indicate that OSMX-based AOS exhibits rich biomimetic dynamics in response to diverse electrical and optical stimuli, positioning them as a promising component for neuromorphic computing systems.

The nonlinear behavior of the AOS under light can be attributed to the following mechanism: light illumination generates a large number of electron-hole pairs in ZnO NCs and OSMX, thereby enhancing the conductivity of the device. Under an applied voltage, these photogenerated carriers migrate toward the electrodes and may become trapped at the interface between ZnO NCs and OSMX. Upon removal of the light source, the trapped carriers are gradually released, leading to short-term memory effects. This is consistent with the results characterized by Kelvin probe force microscopy (KPFM) (Fig. 3l). Under illumination, the surface potential of the film is higher than in its initial state, indicating charge accumulation at the interface. Meanwhile, the role of OSMX in AOS was investigated through in-situ KPFM characterization (Supplementary Fig. 32). The results demonstrated that OSMX not only significantly enhanced the response to UV light but also facilitated the charge trapping processes at the ZnO NCs/OSMX interface.

Visualized dual-mode sensory for environment-adaptive application

Sensory feedback functionality is obtained by integrating CS-QLED on the shared L-MXene conductive layer (Fig. 4a). Utilizing CS-QLED as feedback units leverages their high color purity and tunability in the visible spectrum, along with their solution-processability on diverse substrates, thereby facilitating the development of flexible electronic skin^41,42. Different wavelengths of light emission can be achieved by mixing red and green quantum dots (R/G QDs). In addition to its excellent electrical conductivity, good transparency, and outstanding bending resistance, L-MXene exhibits a work function of approximately 4.9 eV, which closely matches that of the PEDOT:PSS charge injection layer (approximately 5.1 eV). This alignment facilitates highly efficient charge injection and enables optimal electrical interface matching (Fig. 4b). As the voltage of the CS-QLED increases (2.8 ~ 7.4 V, Fig. 4c), the green emission intensity at ~543 nm gradually increases, while the red emission intensity at ~632 nm gradually decreases. Supplementary Fig. 33 presents the current density–voltage–luminance characteristic curves of the CS-QLED. To investigate the mechanical flexibility of the CS-QLED, we performed bending tests under varying radii and bending cycles (Supplementary Fig. 34). The results demonstrate that the current density–voltage characteristics remain nearly constant as the bending radius decreases from 10 mm to 3 mm (Supplementary Fig. 34a). Furthermore, after 1000 cycles of dynamic bending at a radius of 3 mm (Supplementary Fig. 34b), the current density–voltage characteristics curve does not show marked degradation, indicating preferable mechanical flexibility and bending tolerance.

Fig. 4. Visualized dual-mode sensory for environment-adaptive application based on FSPVS.

a Schematic illustrating the structure of the CS-QLED. b Energy-level diagram and charge carrier transfer process of the CS-QLED. c Normalized electroluminescence (EL) spectrum at various voltages. d Schematic of signal acquisition and processing in an FSPVS. Pressure stimulation from PENG and optical stimulation from AOS can modulate the weighting of AOS, thereby altering the voltage output of CS-QLED, which subsequently enables the avalanche photodiode to detect light with varying intensities and wavelengths. e Schematic of the octopus’s color change upon light and tactile pressure stimulation. f Schematic of visualized dual-mode sensory for environment-adaptive application. g Light-emitting intensity change of the CS-QLED in response to different pressure and optical intensity stimuli. h Commission Internationale de L’Eclairage (CIE) diagram of CS-QLED at various voltages.

Inspired by the unique sensory and neural characteristics of octopuses, we developed a flexible bionic tactile-visual-optical feedback neural network system to harness the potential of PENG, AOS, and CS-QLED (Supplementary Fig. 35). The signal acquisition and processing workflow of the system is illustrated in Fig. 4d. Since CS-QLED requires a substantial driving voltage to induce color changes from red to green, a current amplification module was incorporated to fulfill its operational demands. The designed circuit can function adequately when receiving only visual signal input without tactile input. The decision logic of the FSPVS operates as follows: the PENG detects pressure signals, while the AOS receives optical inputs. The combined pressure and light stimuli act synergistically on the AOS to trigger the generation of EPSCs. These EPSCs are then captured by a conversion unit in the peripheral circuit and transformed into voltage with varying amplitudes, which effectively drive the CS-QLED to produce visual output and enable color switching. Furthermore, the light emitted by the CS-QLED is detected by an avalanche photodiode connected to an oscilloscope.

The skin of some organisms possesses unique additional functions that enable them to perform physiological activities efficiently. For example, the skin of an octopus can rapidly change color in response to external stimuli (such as pressure or light stimuli), allowing efficient disguise (Fig. 4e)⁴³. Figure 4f illustrates the schematic of our system, which is designed to mimic the environment-adaptive dual-mode perception and visualization capabilities of the octopus. When light and piezoelectric stimuli are simultaneously applied, photon excitation and piezoelectric regulation synergistically enhance conductivity. Consequently, as the resistance of AOS decreases, an enlarged current is generated and subsequently converted into an enlarged driving voltage, which effectively induces pronounced color changes in the CS-QLED. As shown in Fig. 4g, in the absence of additional light stimulation, increasing pressure can induce a transition (threshold 1 of light emitting intensity) in the emission of the CS-QLED from red light (state 1: non-defense mode) to yellow light (state 2: environmental monitoring mode). Similarly, without external pressure stimulation, an increase in light intensity can also trigger the same emission transition. When both light and pressure stimuli are applied simultaneously, with the light input being dominant (20 mW·cm^-2 light stimulus and 1.5 N pressure stimulus), the CS-QLED emits yellow light that exceeds threshold 1 (state 2). Conversely, when the pressure stimulus dominates (5 N pressure stimulus and 6 mW·cm^-2 light stimulus), the CS-QLED initially emits yellow light above threshold 1 and gradually transitions to green light surpassing threshold 2 (state 3: alarm mode) as the number of stimuli increases. When moderate pressure and light intensity are concurrently applied, the CS-QLED emits yellow light exceeding threshold 1 (state 2). When both high pressure and high light intensity are simultaneously applied, the CS-QLED exceeds threshold 2 (state 3) and emits green light. The luminescence color of the CS-QLED changes in response to the intensity of stimuli, reflecting the severity of the harmful stimulus, and this variation can be intuitively displayed through visible light-emitting signals (Fig. 4h).

Multi-mode information perception and recognition

To exhibit the advantages of our integrated system, we implemented multimodal image recognition tasks. When identifying contaminated images (Fig. 5a), the left side—being covered by opaque materials—can only be perceived through tactile signals (electrical signals), while the right side remains perceivable via visual signals (optical signals). Therefore, a multimodal mixed-input “EEOO” mode (comprising two electrical pulses followed by two light pulses) is employed. This mode demonstrates distinct functional spatial partitioning: the “EE” segment selectively responds to the left half of the image (with the “OO” segment inactive), whereas the “OO” segment specifically responds to the right half (with the “EE” segment inactive). The original 28×28 pixel images were restructured into a 196×4 pixel format. Subsequently, the processed images were read using a mixed-input “EEOO” mode. A total of 196 input channels were fed into corresponding memory units for processing. Supplementary Fig. 36a illustrates the distribution of weights in the fully connected layer. After the training process, the synaptic weights change from a random to a normal distribution, indicating that the neural network has been trained effectively. The confusion matrix (Fig. 5b) and the achieved accuracy of 91% (Supplementary Fig. 36b) show the effectiveness of the mixed-input strategy in multimodal recognition tasks.

Fig. 5. Multi-mode information perception and recognition.

a Schematic of recognizing the multimodal Fashion-MNIST in complex environments. b Confusion matrix for classifying seven categories of images. c Schematic of strike type recognition, dynamic racket swing trajectory recognition, and position recognition implemented by FSPVS in a simple badminton sports scene towards multisensory interaction scenarios (R denotes the distance between the light-emitting point and the receiver detector). d Schematic of the in-sensor RC system for dynamically recognizing the strike types and racket swing trajectories. e Confusion matrix for classifying distinct striking types. f and g Readout function (F) value of the unit of the read-out layer, representing the probability of the corresponding trajectory. h Initial and final weight distributions before/after training. i Cross-entropy loss and accuracy for racket swing trajectory recognition. j Schematic of the principle of position detection. k Estimated point coordinates and actual point coordinates. l Training loss and validation loss of position detection.

In addition to static image recognition, we further evaluated potential of the system in dynamic information perception and recognition by using a simplified badminton scenario as a example (Fig. 5c). In this application scenario where the CS-QLED is only required to emitting light rather than achieving color-changing function (that is, the driving voltage is small), we have designed a highly integrated FSVPS (based on L-MXene-integrated PENG, AOS, and CS-QLED); it can operate effectively without an additional amplification module, merely by providing a turn-on voltage (Supplementary Fig. 37). This wearable system can achieve several key functions including multi-modal perception, integrated perception-processing architecture, trajectory recognition, and accurate spatial positioning (Supplementary Fig. 38), which is highly desired for multisensory interaction applications. Specifically, User A and User B each wore an FSPVS on their wrists. Even when located in different spaces, these two users were able to engage in online interaction (Fig. 5c). For instance, when User A swung the racket to strike the virtual badminton shuttlecock, the FSPVS could accurately detect four fundamental strike types: forehand heavy shot (F-HS), forehand light shot (F-LS), backhand heavy shot (B-HS), and backhand light shot (B-LS). Simultaneously, the FSPVS identified the racket swing trajectories in eight fundamentally different directions. In addition, the induced CS-QLED light-emitting signal was captured by two photodetectors (PD) to determine the positional data. These strike types, trajectory recognition, and spatial positioning enabled User B to respond appropriately and promptly.

During the badminton scenario, joint movements (representing piezoelectric signals) and racket swings (representing optical signals) occur simultaneously and are separately input into the AOSs working in electrical and optical modes (serving as RC units), respectively (Fig. 5d and Supplementary Fig. 39). For strike types recognition, one-dimensional piezoelectric data (2000) was converted into a four-row pixel format (4×500) through cropping and binarization. Subsequently, the strike type data are fed into the AOS reservoir in electrical mode for computation. Finally, a single-layer fully connected neural network is employed to perform the strike type recognition task. For racket swing trajectory recognition, optical pulses are essential for information processing. To integrate the spatial and temporal information from consecutive frames into a compact representation, we divide each frame image into four consecutive subframes (t₀, t₁, t₂, and t₃), representing four distinct moments of the racket swing. When the racket (yellow pixels) is detected, an optical pulse is applied to the AOS corresponding to each pixel. When no racket movement (blue pixels) is detected, no optical pulse is applied. For each pixel, its temporal evolution is compressed into a 4-bit sequence s(t), composed of a 16-bit binary vector. The sequence of light pulses (from (0000) to (1111)) is fed as input to the physical reservoir of the memristor, causing changes in the EPSC value of the device. This process embeds the temporal evolution of each pixel into the conductance of individual memristors, thereby characterizing the temporal features of object motion.

The confusion matrix of the target labels and output labels in the test set was verified (dataset can be seen in Supplementary Fig. 40), and the results indicated that the system could effectively identify different strike types (Fig. 5e). In addition, the internal repository of the sensor using custom datasets (Supplementary Fig. 41) accurately recognized racket motion including eight different directions (Fig. 5f and g, and Supplementary Fig. 42). During training, the readout function (F) value for target inputs continuously increases, while the F values for non-target inputs decrease. Fig. 5h shows the initial and final weight distributions before/after training. Cross-entropy loss and accuracy for racket swing trajectory recognition are shown in Fig. 5i.

The generated CS-QLED light-emitting signal can be used for position detection. Two PDs are placed at appropriate positions to ensure they can receive the light information emitted from any point within the plane (Fig. 5j). The light intensity data acquired from two PDs are utilized as feature vectors. These vectors are subsequently processed into input features (R₁, θ₁, R₂, θ₂) and output label coordinates (X, Y). Figure 5k illustrates the comparison between the estimated position coordinates and the actual positions. The results demonstrate that the average positioning error for each point in the validation set is 0.0048, indicating the high precision of the proposed method (Fig. 5l). These results underscore the significant potential of the FSPVS with multi-parameter information parallel output in multimodal recognition for human-computer interaction.

Discussion

In summary, a bioinspired flexible sensing-processing-visualizing integrated system has been introduced via a material-architecture-function co-design strategy. Through functionally tailored MXene, the flexible system unifies mechanosensation through a piezoelectric nanogenerator, visual sensation and processing through an artificial optoelectronic synapse, and optical feedback via color-shifting quantum dot light-emitting diodes in a shared flexible platform. Relying on this system, the intelligent integration of tactile and visual signal processing demonstrates its capability in environmental adaptation by emulating bioinspired self-protection behaviors that enable responsive actions to environmental stimuli. This system achieves a classification accuracy of 91% in multimodal image recognition tasks based on the Fashion-MNIST dataset; furthermore, it is capable of identifying dynamic strike types and trajectories, as well as providing high-precision spatial positioning—functionalities that are desired in the multisensory interaction scenarios. The ability to process tactile and visual data efficiently highlights the potential of MXenes in bioinspired function-integrated flexible electronics. Our work demonstrates a promising pathway toward intelligent wearable devices and multisensory human-computer interaction.

Methods

Materials

Poly(3,4-ethylenedioxythiophene)/poly(styr-enesulfonate) (PEDOT: PSS, Clevios AI4083) and poly (9,9-dioctylfluorene-alt-N-(4-sec-butylphenyl)-diphenylamine) (TFB) was purchased from Xi’an Polymer Light Technology Co., Ltd. Red and green CdSe/ZnSe QDs were procured from Xingshuo Nano Technology Co., Ltd. N, N-dimethylformamide (DMF, ACS), KOH (AR), methanol (AR), Poly(methyl methacrylate) (PMMA, Mw ∼ 350000), n-butanol (AR) and LiF (AR) were purchased from Aladdin. Chloroform (AR), HCl acid (AR, 36-38%), acetone (AR) and absolute ethanol (99.9%) were purchased from Sinopharm Chemical Reagent Co., Ltd. Al-rich Ti₃AlC₂ MAX (400 mesh, Foshan Xinxi Technology Co., Ltd.), zinc acetate dehydrated (99%, Sigma-Aldrich), n-dodecylphosphonic acid (DDPA, 98.5%, Macklin), P(VDF-TrFE) (80:20, Solvay). All materials were used as received without any purification.

Preparation of S/M/L-Ti₃C₂T_x MXene aqueous suspension

The MXene suspension is prepared using a minimally intensive layer delamination (MILD) method⁴⁴. To obtain L-MXene and M-MXene, we centrifuged the MXene dispersion at 240 g for 15 min to remove unetched MAX. The liquid was subsequently centrifuged at 3210 g for 30 min. The resulting supernatant was identified as M-MXene, while the precipitate was designated as L-MXene. To prepare defect-rich S-MXene for subsequent oxidation treatment, multi-layered MXene was ultrasonicated in a water bath for 2 h. The resulting dispersion was then centrifuged at 1300 g for 1 h to isolate the supernatant.

Preparation of Ti₃C₂T_x MXene DMF suspension

To replace the solvent in the MXene aqueous dispersion, a solvent substitution method was used. The MXene aqueous dispersion was centrifuged at 10610 g for 30 min to obtain the MXene precipitate. After that, the precipitate was dispersed into 20 mL of DMF. The operation was repeated 3 times to prepare MXene DMF dispersion.

Preparation of OSMX chloroform suspension

To make OSMX well dispersed in organic solutions, we grafted it with organofunctional groups. First, DDPA was added to chloroform and stirred until completely dissolved. At the same time, the pH of the OSMX aqueous solution (1 mg mL^-1) was adjusted to below 2.5 with HCl. Afterward, the aqueous OSMX solution was added to the DDPA chloroform solution and stirred vigorously for 3 h. At the end of the reaction, the stirring was stopped, and the reaction was allowed to stand and wait for stratification, and the lower organic phase solution was taken. To remove the unreacted DDPA, the solution was centrifuged at 6790 g and washed 3 times with chloroform. Finally, the precipitate was dispersed in an appropriate amount of chloroform to obtain the OSMX dispersion. To enhance the smoothness of the spin-coated film, a small amount of PMMA was added to the OSMX dispersion.

Preparation of ZnO NCs suspension

The synthesis of ZnO NCs was optimized based on previous reports and dispersed in n-butanol⁴⁵. Typically, 0.59 g of zinc acetate dehydrated was dissolved in 25 mL of methanol and heated at 65 °C in a water bath. Then, 13 mL of 4.6 mmol KOH methanol solution was dropped into the zinc acetate solution with continuous stirring for 2.5 h. The precipitated ZnO NCs were centrifuged, washed with methanol three times, and then dispersed in n-butanol solution.

Device fabrication

Fabrication of MXene-dual-functional PENG

Piezoelectric films were prepared using a drop-casting method. Firstly, P(VDF-TrFE) powder was added to MXene DMF dispersion containing different mass fractions of MXene and stirred for 12 h to ensure its complete dissolution to obtain the MXene/P(VDF-TrFE)-n (MXP-n, n represents the mass fraction of MXene relative to P(VDF-TrFE)) solution. Afterward, the MXP-n solution was dropped on a clean and flat glass plate placed in a blast oven at 70 °C and heated for 7 h to allow complete drying. Then, the MXP-n films were annealed in a vacuum oven at 130 °C for 1 h to improve the crystallinity. The MXene electrode was spin-coated to the surface of the MXP-n films on both sides. The electrodes were used to lead the wires using silver adhesive and then covered with Kapton tape to complete the encapsulation.

Fabrication of AOS

The PET substrate was sequentially cleaned in deionized water, acetone, and absolute ethanol for 10 mins each, followed by drying with flowing nitrogen. To enhance surface wettability, the PET substrate underwent plasma treatment. L-MXene was initially deposited onto the PET substrate and allowed to rest for 3 mins. It was then subjected to spin-coating at 2000 rpm for 60 s, followed by an additional spin-coating process at 6000 rpm for 90 s. Subsequently, the L-MXene/PET was annealed at 100 °C in a tube furnace for 12 h to eliminate adsorbed impurities and improve electrical conductivity. The ZnO NCs solution was spin-coated at 2000 rpm for 40 s and heated at 80 °C for 10 min. The OSMX/PMMA solution (mass ratio of 4:6) was spin-coated at 4000 rpm for 40 s and heated at 80 °C for 10 min. After that, another layer of ZnO NCs was spin-coated with the same parameters as before. Finally, the Al electrode with a thickness of 50 nm was thermally evaporated through a mask plate.

Fabrication of CS-QLED

The preparation of L-MXene/PET was carried out as described above. PEDOT:PSS and TFB layers were deposited onto the substrate via spin-coating at 3000 rpm for 40 s, after which the samples were annealed at 100 °C for 20 min. The red and green QDs are mixed at a mass ratio of 1:15. For the QDs and ZnO layers, the spin-coating parameters were 2000 rpm for 60 s, followed by annealing at 80 °C for 10 min.

Fabrication of FSPVS

To fabricate the FSPVS, a thin patterned PET mask was first attached to a PET substrate (3 × 4.5 cm²), and its hydrophilicity was enhanced via oxygen plasma treatment. Subsequently, an L-MXene film was spin-coated on the substrate and annealed at 100 °C for 12 h. Following this, the MXP solution was drop-cast onto the PENG region of the PET substrate, dried at 70 °C for 7 h, and vacuum-annealed at 100 °C for 2 h to improve crystallinity. Then, L-MXene was deposited onto the upper surface of MXP film using a similar procedure. Next, the PENG and AOS regions were masked, and PEDOT:PSS, TFB, R/G QDs, and ZnO layers were sequentially spin-coated onto the CS-QLED region. Afterward, the CS-QLED region was re-masked, and ZnO, OSMX/PMMA, and ZnO layers were sequentially deposited. Finally, the top electrode was thermally evaporated, and the entire device was encapsulated. To ensure an efficient electrical conduction connection between the devices, MXene can be further sprayed in the conduction pathway.

Device characterization

The sample morphologies were characterized by SEM (Hitachi SU-8600), TEM (JEOL JEM-2100F), AFM, and KPFM (Bruker Multimode 8). Physical phases of samples were characterized by XRD (Bruker D8 advance, Cu Kα radiation, λ = 1.5406 Å), FT-IR (Thermo Fisher Nicolet IS50), and ultraviolet-visible-near infrared absorption spectroscopy (Hitachi UH5700). The surface elemental distribution of the samples was characterized using XPS (Thermo Scientific ESCALAB Xi + ) instrumentation provided by eceshi (www.eceshi.com). The output signals of the PENG were measured by an oscilloscope (Siglent SDS1202X-C) and an electrometer (Keithley 6514). The optical and electrical properties of the AOS were characterized using a source meter (Keithley 2400 and Tonghui 1992) by connecting to both ends. The AOS is stimulated by 365 nm UV light sources, and the distance of the UV light source was fixed to ensure consistent light intensity across all generated pulses. The transient electroluminescence characteristics of the CS-QLED were measured using an avalanche photodiode and an oscilloscope (Rigol DS1202Z-E). The luminance and chromaticity of the integrated FSPVS were measured using a specialized radiance colorimeter (Everfine, SRC-600). The overall FSPVS relies on an external power source to ensure the reliable operation of all components. Informed consent was obtained from the human research participant (Zhen Wang). The sheet resistance was measured by the digital four-probe tester (Suzhou Jingge Electronic Co., Ltd., ST-2258C).

Network training

For multimodal Fashion-MNIST recognition, the original 28×28 pixel images were binarized and restructured into a 196×4 pixel format. Subsequently, the processed images were read by using a mixed-input “EEOO” mode. The covered yellow pixels on the left half could only be sensed via electrical pulses from tactile signals, whereas the purple pixels represent their absence; all pixels on the right half indicate the absence of an electrical pulse. The yellow pixels on the right half correspond to light pulse inputs. A total of 196 input channels were fed into corresponding memory units for processing. Then, a single-layer fully connected neural network is employed to perform the multimodal Fashion-MNIST recognition task.

For strike types recognition: the experimental participant swung the racket with varying types, and raw data was collected. The one-dimensional strike-type data (2000) was converted into a four-row pixel format (4×500) through cropping and binarization. Subsequently, the strike type data are fed into the AOS reservoir in electrical mode for computation. The AOS reservoir in the electrical mode consists of 500 storage devices that serve as reservoir nodes. Then, a single-layer fully connected neural network with 500 input nodes was designed to perform the strike types recognition task. For racket swing trajectory recognition: image sequences of each racket trajectory were captured, and the frame difference method combined with binarization processing was applied to generate image sequences. Each input image (56×56) was divided into four consecutive pixel rows (4×3136), spanning from the leftmost to the rightmost column in a top-down manner. Subsequently, the swing trajectory information was fed into the AOS reservoir in optical mode for computation. The AOS reservoir in the optical mode consists of 3136 storage devices that serve as reservoir nodes. Their conductance values are programmed and calibrated based on experimentally measured data from individual devices. Then, a single-layer fully connected neural network with 3136 input nodes was designed to perform the racket swing trajectory recognition task. The readout layer was implemented in software, and the model was trained using the Adam optimizer’s gradient descent algorithm (with an initial learning rate of 0.001) to minimize classification cross-entropy loss. Finally, the data were divided into training and test sets to evaluate the performance of the model.

Author contributions

Y.L., Y.Z., and W.H. supervised the project. Y. L. had the idea, and Y.L., Z.W., and Y.Z. designed the experiment. Z.W. and J.L. performed the studies. Z.W. and Y.L. prepared the manuscript. R.Z., Q.L., X.H., J.S., Y.J., K.Y., H.F., X.G., Y.W., D.L., L.L., and F.L. participated in analyzing the results and discussing the manuscript.

Peer review

Peer review information

Nature Communications thanks Xiangshui Miao, and the other, anonymous, reviewer(s) for their contribution to the peer review of this work. A peer review file is available.

Data availability

All data are available from the corresponding authors upon request. Source data are provided with this paper.

Code availability

The code that supports the theoretical plots within this paper is available from the corresponding author upon request.

Competing interests

The authors declare no competing interests.

Supplementary information

The online version contains supplementary material available at 10.1038/s41467-025-67316-0.