Fingertip-scale six-axis tactile interface with high-precision force sensing and position localization for dexterous human–machine interactions

Abstract

Developing microsystem-based tactile sensors that can simultaneously decode multidirectional forces and precise contact locations remains a formidable challenge, limiting robotic dexterity and human–machine interaction. To bridge this gap, we present HexaTouch, a fingertip-scale (15 × 15 × 8 mm) sensor that synergizes the deformation-encoding principle of vision-based sensors with the miniaturization and rapid response of capacitive sensing. The core is a bioinspired bilayer elastomer monolithically incorporating a graded micropillar array, which creates spatially heterogeneous stiffness to enhance sensitivity and load tolerance while generating rich deformation patterns in response to mechanical stimuli. These local deformations are directly transduced into high-resolution capacitive images via a dense capacitive micro-array. A dedicated machine learning framework decodes these images into six-axis force/torque vectors (Fx, Fy, Fz, Mx, My, Mz) and three-dimensional contact coordinates (x, y, z). Experimental results demonstrate exceptional performance, with force measurement errors under 1.6%, contact localization precision of up to 0.1 mm, and inference latency of only 1.5 ms. The system maintains high stability across 0–40 °C and 40–90% relative humidity, with mechanical robustness confirmed through 10-day cumulative cyclic loading tests. The versatility of this sensing system is further validated through extensive applications, including dexterous grasping with stability assessment, precise peg-in-hole assembly under misalignment, and intuitive human-machine interaction in drone flight control and virtual gaming. HexaTouch therefore provides a robust, adaptable micro-tactile sensing platform that significantly advances robotic manipulation and environmental interaction.

Introduction

The application scope of robotics has broadened from traditional industrial settings to increasingly complex, human-centered scenarios over the past two decades¹ (Fig. 1). Within these contexts, effective object manipulation requires precise perception of when, where, and how physical interactions occur, a capability humans exhibit. This progression has catalyzed the development of robotic hands featuring both highly dexterous actuation and advanced tactile sensing^2–4. Notably, the integration of tactile sensors at the finger or fingertip level has been recognized as a promising approach to enhancing the perceptual acuity of robotic manipulators^5–7.

Fig. 1. A conceptual illustration of the multiaxial tactile perception (HexaTouch) for robotic manipulation.

a Integration of sensing system on robotic fingertip for dexterous manipulation. b Inspiration from human skin: mechanoreceptors convert force into neural signals for brain processing and action feedback. c Explanations of working principle: conductive micropillars (CP) interact with the capacitance of the array (CA), producing contact images that are decoded by a neural network into multiaxial forces (Fx, Fy, Fz, Mx, My, Mz) and positions (x, y, z). The insulating layer (IL) provides protection for the sensing system

Rigid force sensors, such as those based on strain gauges⁸, have achieved considerable advancements in sensitivity and miniaturization. Nonetheless, their inherent rigidity limits their suitability for safe and adaptive interactions within human-centric environments. In comparison, flexible tactile sensors address this issue by exploiting mechanical compliance and conformability to establish safer interactive interfaces^9–13. Various transduction mechanisms, including resistive¹⁴, capacitive^15,16, and piezoelectric¹⁷ effects have been explored to develop flexible devices, both as individual elements and as sensor arrays^10,18. Despite these developments, most current flexible tactile sensors remain limited to detecting uniaxial loads, predominantly normal pressure^19–22, limiting their applicability in tasks requiring fine-grained contact discrimination, such as precision assembly, texture recognition, and medical assistance. Recent progress in multiaxial force sensing^22–25 and torque measurement^26,27 using flexible devices represents an important step forward; nonetheless, these devices generally lack high-fidelity spatial localization of load application points, which is also a critical requirement for dexterous robotic manipulation^23,28.

Vision-based tactile sensors, exemplified by systems such as GelSight²⁹, TacTip³⁰, Insight²³, and others^3,30–32, adopt fundamentally different mechanisms. By capturing elastomer deformation through embedded cameras, these systems provide rich spatial contact maps and enable the reconstruction of force vectors with high precision^3,33. However, their reliance on optical setups results in bulk that exceeds typical fingertip dimensions, a limited depth of field that restricts the measurable deformation range, and a suspended elastomer design prone to structural failure under large loads, collectively hindering their integration into robotic fingers. On the other hand, capacitive tactile sensors are structurally simple, compact, and capable of quick response^34,35. By patterning electrodes into dense arrays, capacitive tactile sensors can also provide adequate spatial resolution^36–38, although they generally lack the deformation-rich information characteristic of vision-based approaches.

Collectively, current tactile sensing technologies have yet to realize an optimal balance among compactness, robustness, multidimensional force perception, and precise localization. This critical gap motivates the development of a novel strategy that integrates these requirements within a fingertip-scale platform. The complementary strengths and weaknesses of the above sensors suggest that integrating the deformation-encoding capability of vision-based sensors with the compact form factor of capacitive devices could yield an optimal solution. In this study, we demonstrate a microsystems-based approach to tactile sensing, in which a bioinspired microstructure is monolithically integrated with a high-density capacitive microarray to achieve high-precision force sensing and position localization (Fig. 1). In particular, a bilayer elastomer incorporating graded micropillar structures directly interfaces with the high-density capacitive electrode array. Local deformations induced by external loads are transduced into distributed variations in capacitance that preserve both the force magnitude and spatial characteristics. Subsequently, machine learning algorithms decode these features into six-axis force/torque vectors and three-dimensional contact positions with high fidelity.

Results

Hybrid design and sensor architecture

Flexible materials, valued for their intrinsic ability to conform to complex object geometries, endow safe human–robot interactions and effectively dissipate impact energy during unintended collisions^39–41. These characteristics render them particularly suitable for constructing compliant and sensitive perceptual interfaces. The very softness that enables conformability, on the other hand, imposes fundamental constraints, including limited load-bearing capacity and susceptibility to large deformation under gravitational and inertial forces^41–43. Such limitations partly explain the shortcomings of conventional vision-based tactile sensors when applied to tasks involving high contact loads and demanding load tolerance.

To concurrently achieve surface conformability, high contact sensitivity, robust load tolerance, and precise spatial localization, we introduce a hybrid sensor architecture that integrates principles of vision-based tactile sensing with capacitive sensing within a fingertip-scale form factor (15 × 15 × 8 mm, Fig. 1c). The device structurally comprises a fingertip-shaped bilayer elastomer, an FPC1020AP (192 × 192, Zhupu Core Technology Co., Ltd., Shenzhen, China) chip serving as a miniaturized capacitor array, a printed circuit board (PCB) providing signal-conditioning circuitry, including filtering and amplification modules, and a rigid supporting substrate, which provides structural stability.

The elastomer features a bilayer composite inspired by the layered architecture of human skin and incorporates a fingertip-like curvature of 1/15 mm⁻¹. Unlike flat geometries, this curved surface promotes conformal contact with external objects, reducing interfacial gaps and enabling more deterministic, efficient load transfer. In addition, geometric congruence enhances the stability of contact centers, thereby improving the accuracy of load localization under varying force conditions. The outer insulating silicone rubber layer (0.5 mm thickness; Shore hardness 30) ensures mechanical compliance and conformal contact with objects. Beneath it, a carbon-black–impregnated conductive rubber layer (Shore hardness of 50, volume resistivity of 4.3 Ω cm) functions as an active electrode.

This conductive layer is further patterned into an array of micropillars, as detailed below, which interface directly with the microcapacitor array to form distributed capacitive sensing units. Local deformation of the elastomer under applied loads modulates the contact area at the micropillar tips, thereby altering site-specific capacitances. The FPC1020AP chip transduces these capacitance variations into grayscale values, enabling reconstruction of both the magnitude and spatial distribution of applied loads via machine-learning-supported grayscale pattern analysis.

To minimize batch-to-batch variations and enhance durability, we implemented three strategies. A dedicated metal mold ensured dimensional consistency of elastomers across batches (Fig. S1, variation < ±0.05 mm). Positioning structures on the rigid support maintained assembly alignment within 0.1 mm. During packaging, the FPC1020AP chip was first mounted on the PCB, which was then mounted on the stiff support. Copper foils were attached for connection, the elastomer was aligned using positioning slots, and conductive silicone adhesive was applied and cured for sealing. These measures ensure reliable electrical and mechanical stability.

In contrast to vision-based tactile sensors employing suspended elastomers and bulky optical components (e.g., GelSight²⁹, GelTip⁴⁴, Minsight³²), such a hybrid design achieves a compact size (15 × 15 × 8 mm) and enhanced structural stiffness, facilitating convenient fingertip integration while maintaining robustness under substantial loads.

Bioinspired structured elastomer design

Further inspiration was derived from the intermediate ridge structures located between the dermis and epidermis in human skin, which amplify tactile stimuli through microleverage effects and transmit them to mechanoreceptors at the ridge apices⁴⁵. Emulating this principle, we implemented a 7 × 7 micropillar array on the underside of the elastomer. Each pillar measures 1 mm in diameter with a center-to-center pitch of 1.50 mm, resulting in an active span of 9 mm that aligns with the sensing window of the FPC1020AP chip (Fig. 2a). This multipoint contact configuration potentially enhances the signal-to-noise ratio, as previously explored by Zheng et al.^46,47, probably by exploiting both microleverage amplification and local stress concentration at the pillar tips.

Fig. 2. Design and mechanical response of the bioinspired structured elastomer.

a Schematic bottom-view illustration of the spatial distribution and height variation of the micropillars. The color bar indicates the initial clearance between the pillar tips and the supporting substrate. b Comparative illustration of the deformation and contact behavior under different loads for both uniform-height and graded-height pillars. c Representative contact images displaying the interfacial contact patterns for pillars with different height distributions. d Displacement of contact centroids of the elastomer micropillars under tangential loading. e Evolution of the total contact area of the elastomer micropillars as a function of normal load. f Rotational response of the elastomer micropillars under applied torque

Capacitance variations were observed at the micropillar tip–FPC1020AP interface, enabling the FPC1020AP output to be directly interpreted as contact maps of the hemispherical micropillar heads. According to Hertzian contact theory, the growth rate of the contact area generally decreases with increasing normal load, leading to reduced sensitivity at high loads. To overcome this limitation, we introduced graded-height micropillar arrays as an alternative to uniform-height designs. Two configurations were initially considered. The first features the tallest pillars at the center and periphery, with heights progressively decreasing toward intermediate regions (termed M-shaped array), as illustrated in Fig. 2a, b. The second configuration is the inverse, where the central and peripheral pillars are the shortest, and heights increase gradually toward intermediate regions (termed W-shaped array). As shown in Fig. S3, the W-shaped array exhibits slightly higher initial sensitivity than the M-shaped counterpart, due to fewer pillars contacting the substrate during early loading. However, the spacing between the tallest (first-contacting) pillars in the W-shaped configuration is smaller than that in the M-shaped configuration, resulting in substantially lower resistance to torque or off-axis loads. Consequently, the W-shaped design was excluded from further consideration. Under normal loading, the tallest pillars make initial contact with the capacitor array, in contrast to uniform-height pillars, which engage simultaneously. As the load increases, these pillars undergo further deformation, and the shorter ones subsequently make contact in sequence. This progressive engagement distributes mechanical interactions over a broader force range (Fig. 2b).

A direct comparison between the M-shaped and uniform-height arrays is presented in Fig. 2c, highlighting their distinct contact behaviors under identical loading conditions. Further investigation using the finite element method showed that micropillars made of silicone rubber with a moderate elastic modulus exhibit approximately linear, stable increases in total contact area under load, avoiding early saturation across a wide load range (Fig. S4). After screening a range of material formulations, we identified that an elastic modulus of approximately 2.14 MPa offers an optimal balance. This choice mitigates the excessive deformation and limited effective operating range associated with lower‑modulus materials (which tend to deform too easily and saturate quickly), while also addressing the insufficient contact response of higher‑modulus materials (which require excessive force to achieve measurable contact regions). Additionally, we investigated the influence of the maximum height difference within the micropillar array on sensing performance (Fig. 2d–f).

Contact shifts under Fx produced directional patterns. Displacement–force analysis confirmed that graded arrays exhibited larger displacements and higher tangential sensitivity than uniform arrays, with sensitivity increasing as height difference increased (Fig. 2d). Under Fz loading, uniform micropillars deformed simultaneously, yielding circular contact patterns, whereas graded arrays engaged sequentially, producing heterogeneous morphologies. For graded arrays, increasing the height difference from 0 mm to 0.5 mm reduced the contact area under equivalent loads, enhancing load tolerance. However, at 0.75 mm, excessive deformation of the tallest pillars interfered with sequential contact of shorter pillars, degrading performance (Fig. 2e). Under moment about the x-axis, contact distributions became asymmetric, with graded arrays showing more pronounced features. Under torsion about the z-axis, contact centers rotated, and shapes became anisotropic. At 30 N·mm torque, graded arrays achieved ~8° rotation versus ~2° for uniform arrays (Fig. 2f). Experimental observations have confirmed the results of the finite element analysis described above (Fig. S5). All these comparisons demonstrate that, relative to uniform micropillar arrays and simple micro-hemisphere arrays⁴⁶, graded-height micropillars achieve greater sensitivity, richer directional signatures, and more distinctive contact patterns across multiple loading modes. Such load-dependent contact images generated by a graded array with a height difference of 0.5 mm facilitate more accurate extraction of both load magnitude and spatial information through image-based analysis.

Imaging and data transmission

As previously described, the micropillars, composed of conductive rubber, serve as additional electrodes that interact with the capacitive array. When the tips of the micropillars contact the upper electrodes of the capacitive array, the capacitances change, and the FPC1020AP chip converts the capacitance of each unit to normalized grayscale values by dividing it by the full-scale range of the capacitive array. This conversion yields original 192 × 192-pixel tactile images that directly reflect the distributed contact states across the array (Fig. 3a). The tactile image stream is then transmitted to the core processing unit (Raspberry Pi 4B) via a high-speed SPI interface. The underlying system runs Ubuntu 20.04 LTS, providing a stable environment for communication between the FPC1020AP and the Raspberry Pi 4B. The images are subsequently delivered to the host computer via TCP/IP over Ethernet sockets, ensuring reliable, low-latency connectivity and enabling immediate further processing (Fig. 3a).

Fig. 3. Experimental setup, learning framework, and performance evaluation of the HexaTouch tactile sensor.

a Schematic of the robotic experimental platform integrating HexaTouch sensor, an ATI Nano 17 reference sensor, a robotic arm, a Raspberry Pi controller, and a data acquisition system. b Data processing pipeline: raw tactile images are processed by DenseNet-121 and global average pooling, then fed into a fully connected layer for six-axis force/torque regression (Fx, Fy, Fz, Mx, My, Mz); contact positions (x, y, z) are localized using a separate KNN classifier. c Correlation plots between measured and ground-truth six-axis forces/moments, with RMSE and R² values indicating high accuracy across all components. The cyan represents the training dataset, and the pink represents the test dataset. d Contact localization accuracy over a 7 × 7 grid, with recognition rates exceeding 98% across the sensing surface. e Contact depth estimation results, showing accurate reconstruction of z-axis displacement with submillimeter precision. f Cycle tests at 24-h intervals for 10 days

On the host computer side, a dedicated tactile acquisition program developed within the ROS2 (Robot Operating System 2) framework manages the entire pipeline, including command issuance, data acquisition, and real-time visualization. Specifically, acquisition commands issued by the host are relayed through the socket connection to the Raspberry Pi 4B, which forwards them to the FPC1020AP via the SPI link. Upon receiving the command, the FPC1020 initiates imaging and streams the grayscale data back to the Raspberry Pi 4B. Lightweight preprocessing steps, such as data format conversion, buffering, and error checking, are performed before transmitting the processed data back to the host computer for real-time rendering and further analysis.

Beyond ensuring stable and efficient communication, the architecture supports a many-to-one topology. Leveraging the topic-based asynchronous message-passing mechanism in ROS2, a single node can publish data to a designated topic, while multiple nodes subscribe to the same topic to receive the data stream. This enables simultaneous data acquisition from multiple fingertip sensors via a unified host interface (Fig. S6). Such scalability not only provides robust adaptability for multi-sensor robotic platforms but also facilitates seamless integration with robotic operating systems, establishing a foundation for real-time, coordinated multi-finger manipulation tasks.

Mapping images to load information

Figure 3b (left column) displays typical original tactile images output directly from the FPC1020 sensor. Median filtering, background subtraction, and grayscale stretching⁴⁶ were adopted to enhance contrast and reduce noise, yielding clearer representations of micropillar–capacitor interactions. The grayscale distribution of tactile images encodes spatial information about micropillar–capacitor deformation under external loading. Localized forces alter the contact area and geometry at the micropillar tips, resulting in intensity variations in the image. To train the model, tactile images were further processed using Gaussian filtering and Z-score normalization. In addition, ground-truth force signals from the Nano17 sensor were denoised using a 30 Hz Butterworth low-pass filter (covering the dominant tactile frequency band while removing robot vibrations and high-frequency environmental noise), followed by mean–standard deviation normalization. This preprocessing pipeline improved robustness against interference and enhanced both prediction accuracy and convergence during model training.

A hierarchical learning framework combining a convolutional neural network (CNN) and a fully connected layer (FC) was implemented (Fig. 3b). Specifically, DenseNet-121 was employed to extract multiscale spatial features from tactile images, with its cross-layer connectivity preserving both local contact details and global deformation patterns. These features were then mapped to six-dimensional force/torque vectors via a customized Fully Connected Layer (FC), enabling nonlinear regression. This hybrid design took advantage of the CNN’s strengths in spatial encoding and the FC’s capacity for high-dimensional regression. A dataset of 25,000 image–force pairs was obtained (see Fig. S7 for the data distribution) and divided into training (80%) and validation (20%). The model, optimized with Adam and MSE loss under standard hyperparameter settings (learning rate scheduling, gradient clipping, batch size 64), converged within 80–300 epochs to MSE values of 0.000688172 (training) and 0.0007017544 (validation), indicating robust fitting and generalization. Test results (Fig. 3c and S8) showed that the predicted values clustered tightly along the diagonal, indicating minimal bias. Compared with other models, such as EfficientNet, ResNet50, and SENet, DenseNet-121 achieved the best performance in this study for six-dimensional loads, in terms of both RMSE and MAE (Table S1). The responses of the test data set are shown as orange points in Fig. 3c. The variation in sensor responses among different samples is minimal. Consequently, the machine learning models trained on data from different samples exhibited near-perfect consistency, with R² values varying by less than 0.05 and RMSE varying by less than 0.5. This high degree of reproducibility confirms that our fabrication and assembly processes are well controlled and that sensor performance is consistent.

For fast prototyping of load position recognition, a k-nearest neighbor (KNN) algorithm was also implemented. Unlike deep networks, KNN requires no explicit training, making it well-suited for smaller datasets and quick evaluations. The algorithm predicts outputs by measuring distances between samples and using the labels or values of the k nearest neighbors. Using 5000 contact-position samples (70% for training and 30% for testing), the model achieved a classification accuracy of 99.7% on the test set with k = 3 (Table S2) and Euclidean distance adopted (Fig. 3d). Finally, the positional resolution along the z-axis reached 0.1 mm (Fig. 3e).

As shown in Figs. S9, 10, over the temperature range of 0 °C to 40 °C and the relative humidity range of 40% to 90%, the results of random loading tests closely matched the predictions of the initial training model (calibrated at 25 °C and 60% relative humidity). To evaluate the durability and repeatability of the sensing system, a 240-h continuous test was conducted. Every 24 h, the sensor was subjected to about 10 min of cyclic normal loading of 3.6 N (Fig. 3f). The output remained stable over 10 days, with a variation of less than 0.1 N. Post-10-day-cycle tests (Fig. S11) further demonstrated the sensor’s long-term durability. The different sensor samples also showed high coincidence levels (Fig. S12). Such durability and stability can be attributed to several factors, including the mechanical stability of the silicone rubber micropillars under varying environmental conditions and the electrical stability of the overall system. First, as shown in Fig. S4, the carbon black-impregnated conductive rubber used here exhibits linear elasticity over a strain range of more than 400%. Additionally, the material properties of silicone rubber remain largely unchanged within the tested environmental ranges. Moreover, the elastomer layer consists of an outer protective layer and an inner micropillar layer. The outer silicone layer serves as an effective barrier against heat and moisture. Finally, the interface between the elastomer layer and the base is reinforced and sealed with silicone adhesive. These design features collectively ensure stable sensor performance under the tested environmental conditions.

Table 1 compares the common performance metrics of the proposed sensor with those of representative vision-based devices. The results demonstrate that our device outperforms existing systems across several key metrics, particularly in multiaxial accuracy and localization fidelity. After calibration, the reliable measurement ranges were determined as follows: Fx ± 5 N, Fy ± 5 N, Fz 0 to –25 N, Mx ± 0.1 Nm, My ± 0.1 Nm, and Mz ± 0.05 Nm. On an RTX 3060 Ti platform, the model required only 1.5 ms per inference, with GPU utilization below 15% and a maximum power consumption of 20 mW, indicating the system’s capability for real-time, fast measurements.

Table 1.

Comparison of sensor performance parameters

Sensor	Load Range				Load localization Accuracy		Size (mm3)	Accuracy (%FS)
	Fx/Fy (N)	Fz (N)	Mx/My (N·mm)	Mz (N·mm)	x/y (mm)	z (mm)
GelSight29	−5 ~ 5	−25 ~ 0	/	−300 ~ 300	/	/	80×40×40	3.3%
GelSlim49	−2 ~ 2	0 ~ 4	/	/	/	/	80×37×20	8.1%
Tenon/mortise structure26	−1 ~ 1	−3 ~ 0	−4 ~ 4	−4 ~ 4	/	/	7×7×7	4.1%
Dotview46	−3 ~ 3	−12 ~ 0	−50 ~ 50	−10 ~ 10	/	/	25.5×47×14.4	4.0%
DotTip47	−3 ~ 3	−10 ~ 0	/	/	0.53	/	21×32×34	2.8%
GelStereo50	−5 ~ 5	−20 ~ 0	−400 ~ 400	−50 ~ 40	0.04	0.104	30×30×29	1.3%
Insight23	0.03 ~ 2	0.03 ~ 2	/	/	0.6	/	1/3π×402×70	4.1%
HexaTouch (This work)	−5 ~ 5	−25 ~ 0	−100 ~ 100	−50 ~ 50	2	0.1	15×15×8	1.6%

Robust robotic manipulation with the assistance of HexaTouch

To validate the performance of the sensor in real-world tasks, we first constructed a multimodal grasping platform (Movies S1, S2) comprising a five-finger dexterous hand (model RH56BF3, Inspire Robots, China), a UR5 robotic arm (model UR5, Universal Robots, Denmark), the above tactile sensor, and custom connectors. By embedding the HexaTouch sensors into the fingertips of the robotic hand, multimodal feedback was achieved by fusing force and positional information. Specifically, a pre-grasping stage, during which tactile cues are used to assess contact states and grasp stability, was incorporated into multimodal grasping. During this stage, the fingertip sensors continuously capture the contact position and contact force, enabling real-time analysis of grasp stability and dynamic adjustment of finger pose, contact location, and applied force, thereby significantly improving grasp reliability.

In the first case, a three-finger pinch grasp was evaluated. During the pre-grasping stage, tactile images indicated that the object primarily contacted the frontal regions of the fingertips (Fig. 4a, left). A pronounced tangential force was detected, and considering the curved surface geometry of the fingertips, this configuration implied that the contact plane between the fingertips and the sphere was located above the sphere’s cross-section. Under these conditions, the grasp was highly unstable, and simply increasing the gripping force could not secure the object (Movie S1). In response, the system adapted the grasp configuration based on real-time touch feedback. Specifically, the hand was slightly relaxed, the entire manipulator was lowered, and the fingers were then reclosed while reassessing the contact locations and forces. Once the contact region shifted toward the centers of the fingertips (Fig. 4a, right), and the measured ratio of tangential to normal force decreased below 0.1, the contact plane approached the sphere’s cross-section. Under this condition, an increased gripping force enabled stable and reliable grasping (Movie S1). In the second case, a two-finger pinch grasp was examined (Movie S2). During the pre-grasping stage, tactile images indicated that the object was primarily contacted by the lateral regions of the index finger and thumb (Fig. 4b, top). This configuration created an asymmetric contact plane and unstable grasping conditions. If executed directly, the object tended to slip or rotate, resulting in grasp failure. Unlike the three-finger case, where the entire hand was adjusted, here the system modified the grasp by laterally reorienting the fingers themselves. This adjustment shifted the contact points and progressively corrected the in-hand orientation of the object (Fig. 4b, middle and bottom). Once the contact plane aligned with the object’s cross-section and the force distribution stabilized, the pinch became geometrically centered. Under these conditions, the grasp achieved stability, and continuous force monitoring further enabled fine regulation of the normal force, thereby preventing slippage during subsequent manipulation. A series of 100 repeated trials demonstrated that force-feedback control dramatically increased the grasping success rate from 48% to 98% for three-finger grasps and from 40% to 93% for two-finger grasps.

Fig. 4. Multimodal sensing-based adjustment strategies for stable grasping and precise peg-in-hole assembly.

a Example of three-finger grasping with sensing-based hand adjustment: tactile feedback from fingertip sensors identifies unstable contact states during the initial grasp, enabling adaptive pose correction and secure grasping. b Example of two-finger pinch grasping with sensing-based finger adjustment: contact localization guides thumb repositioning to achieve stable object holding. c Demonstration of the USB insertion task, including searching, inserting, and successful connection phases. d Representative force–moment curves during the insertion process, where Fx, Fy, Fz, Mx, My, and Mz provide real-time feedback for contact state recognition and adaptive control

Beyond grasping, we investigated assembly tasks using multiaxial tactile feedback, focusing on peg-in-hole operations, a canonical challenge in precision robotics. USB insertion was selected as a representative case, typically involving four stages: peg positioning, hole searching, in-hole adjustment, and insertion⁴⁸. This study focused on the latter three stages under specific misalignment conditions, including (1) lateral offset leading to edge collisions and (2) angular deviation between the plug and socket axes in clearance-fit conditions, which can cause jamming.

High-resolution force sensing and closed-loop control are essential for robust insertion. Compared to flat tactile sensors, the fingertip-curved design enhances compliance during shaft rotation, reduces stress concentration, and maintains image fidelity under load, thereby improving adaptability. A two-finger gripper equipped with curved-surface sensors was used to insert a USB cable (Movie S3). During the hole-search phase, vertical force feedback and compliant control enabled coarse reorientation of the plug, achieving preliminary alignment. In the in-hole adjustment phase, six-axis force cues guided micromotions to correct axes, mitigating impacts and preventing jamming (Fig. 4c). Figure 4d presents representative force trajectories and state transitions during USB insertion. The system monitored end-effector displacement along the z-axis, along with Fx, Fy, Mx, and My signals, as key indicators. From t₀ to t₁, grasping and coarse alignment were completed, during which Fz exhibited noticeable variation. From t₁ to t₂, the initial plug–socket contact caused a gradual rise in Fy and the emergence of a torsional moment Mz, indicating a residual pose deviation. Grid-based hole searching under a constant −2 N contact force transformed the interaction from surface contact to line contact, leading to the convergence of Mz toward zero and the stabilization of Fy and Mx.

From t₂ to t₃, insertion began with remaining y–z misalignment, and the torque feedback continuously corrected the plug orientation. Accumulated misalignment produced increases in Fy and Mx, together with fluctuations in Fx, signaling the onset of jamming. The force-control strategy subsequently refined the alignment, after which admittance control enabled smooth insertion. Once the operation was completed, all six-axis force/torque signals stabilized, and the plug was fully seated, as verified by the indicator activation.

HexaTouch for human‒machine interactions

To establish a unified tactile interaction paradigm, we first demonstrate the general directional-input capability of HexaTouch (Fig. 5a), in which the sensor enables intuitive, multi-directional control via fingertip contact. By mapping the location and direction of touch to corresponding commands, HexaTouch provides a natural alternative to traditional keyboard-based directional inputs, forming the basis for downstream human–machine interaction tasks. Building on this general input scheme, we further validated the feasibility and advantages of HexaTouch for interactive ground mobility control (Fig. 5b and Movie S4). In intelligent vehicle operations, a lightweight haptic interaction scheme was developed that leverages sensor miniaturization and rapid response to regulate motion. Specifically, the sensor surface was divided into a 3×3 sensing grid. The eight peripheral zones corresponded to forward, backward, lateral, and diagonal motions, while the central zone served as an emergency stop, enhancing operational safety and controllability. Compared with conventional controllers, this approach eliminates the need for complex remote-control interfaces or additional hardware, allowing intuitive command input via fingertip touch. At the control logic level, a command redundancy filtering mechanism was implemented to automatically suppress repeated inputs, thereby alleviating the communication bandwidth demands and enhancing system responsiveness. Users can issue directional movement or immediate stop commands by simply tapping different regions on the sensor surface. The system exhibits smooth transitions even during rapid switching, evidencing its high sensitivity and strong anti-interference capability. Furthermore, this framework was extended to interactive virtual gaming applications (Fig. 5c and Movie S5). In the Thunder Fighter game, the sensor surface was also partitioned into a 3×3 grid. The peripheral zones corresponded to directional movements (forward, backward, left, right, and diagonal), with speed proportional to the local load magnitude. The central zone was designated as the firing command, successfully enabling immersive game interaction.

Fig. 5. HexaTouch supports multimodal human–machine interactions across ground mobility, virtual gaming, and aerial control.

a Tactile input mapping enabling intuitive multi-directional commands. b Ground mobility operation shown in Movie S4, using a 3 × 3 grid for directional input and emergency stopping with stable responsiveness. c Virtual gaming interaction in Movie S5, where local load on the 3 × 3 grid produces directional motion and firing control. d Drone control in the PX4 environment is presented in Movie S6, with tangential force, normal force, and torque Mz assigned to translation, altitude regulation, and yaw motion under threshold gating. e–g Relationships between force inputs and flight responses, including the dependence of height on Fz, horizontal velocity on tangential force, and yaw rate on Mz. h Time-resolved six-axis force signals govern sequential altitude adjustment, planar motion at approximately 1.8 m/s, and yaw rotation at 22 N·mm with an angular velocity of 1.5 rad/s

Building on these multi-modal interaction schemes, we developed a haptic interaction-based flight control strategy that integrates the HexaTouch sensor within the PX4 simulation environment, enabling full-range drone maneuvering (Fig. 5d and Movie S6). Two representative control modes were investigated, including (1) horizontal translation governed by the magnitude of the tangential force and contact position, and (2) altitude and yaw regulation mapped to the normal force and torque about the z-axis, respectively. In this framework, intuitive yet precise flight control relies on the accurate perception of forces and contact locations, together with real-time mapping of these signals into motion commands. In horizontal translation, tangential forces determine the forward/backward and lateral velocities, while the load locations determine the direction of motion, enabling smooth, continuous planar trajectories. For altitude and rotational control, the normal force is mapped to the ascent/descent velocity, and the torque about the z-axis is mapped to the yaw angular velocity, thereby coordinating multiple degrees of freedom without command conflicts. To improve robustness and prevent unintended operations, a threshold-based triggering mechanism was introduced. The horizontal velocity control was activated only when the tangential force exceeded 1 N, altitude adjustment was triggered when the normal force exceeded 1 N, and yaw control required an applied torque exceeding 10 N·mm. These thresholds effectively filtered out environmental noise and incidental contacts, ensuring command accuracy and control reliability.

The correlations between the sensor outputs and drone motion parameters are summarized in Fig. 5e–g. Figure 5h illustrates a set of six-axis force signals and their corresponding mapping to flight velocity and direction during control experiments. During the t₀–t₁ stage, altitude control was engaged, with Fz maintaining a linear relationship with the flight height. Between t₂ and t₃, two successive Fz peaks above 5 N stabilized the drone at an altitude of 5 m, preparing it for subsequent maneuvers. In the t₄–t₅ stage, directional control was achieved through the contact position, where the Fx and Fy magnitudes determined the forward/backward and lateral velocities of about 1.8 m/s. Finally, yaw control was realized between t₆ and t₇, where the polarity of Mz dictated the rotation direction and its amplitude scaled the angular velocity (stable Mz = 22 N·mm, angular velocity = 1.5 rad/s).

These demonstrations substantiate HexaTouch as a versatile and generalizable platform for multi-axis tactile sensing. By bridging fine robotic manipulation, precision assembly, aerial and terrestrial mobility control, and immersive human–machine interaction, HexaTouch establishes a robust foundation for next-generation dexterous and adaptive systems.

Discussion

This work introduces HexaTouch, a fingertip-scale tactile sensing microsystem that enables multidimensional perception by integrating deformation-driven mechanical encoding with high-density capacitive sensing. The core architecture comprises a bioinspired bilayer elastomer with a graded micropillar array, interfaced with a 192 × 192 micro-capacitive array that generates tactile images capturing detailed contact mechanics. These images are processed using a combined DenseNet-121 and multilayer perceptron architecture to decode six-axis force/torque vectors and three-dimensional contact positions, delivering full sensing-to-control functionality in a miniaturized form factor.

The graded micropillar configuration addresses fundamental challenges in flexible tactile sensing (i.e., conformability, load tolerance, and spatial fidelity) by producing progressive mechanical engagement under varying loads. This microscale engineered structure enhances directional sensitivity and generates highly distinctive contact patterns across diverse loading conditions. Compared to conventional vision-based or capacitive sensors, HexaTouch achieves a superior balance between compliance, load capacity, and spatial resolution while overcoming typical limitations in structural reliability and in richness of deformation information. Additionally, benefiting from the specialized structural design and encapsulation, the system exhibits superior stability and robustness.

Experimental validation confirms sub-millimeter positioning accuracy (0.1 mm in z-axis), high-fidelity multidimensional force/torque estimation, real-time inference capability (1.5 ms), and high environmental stability and durability in dynamic applications, including dexterous manipulation, precision assembly, USB insertion, and human-machine interaction. These results demonstrate the successful system-level integration of mechanical intelligence and embedded machine learning in a scalable micro-tactile platform.

In summary, HexaTouch establishes a new microstructure-enabled sensing paradigm that combines graded micropillars, capacitive imaging, and data-driven decoding to deliver compact, robust, and high-precision tactile perception. Its validation across robotic manipulation and interactive applications highlights both versatility and real-time performance, while the generalizable design supports future integration into multi-fingered hands, collaborative robots, and wearable systems. Future work will focus on enhancing environmental robustness, improving generalization to unstructured contact scenarios, and deeply integrating with impedance, admittance, and learning-based control frameworks, thereby advancing the development of next-generation adaptive tactile-aware robotic systems.

Methodology

Simulation

To determine the geometric dimensions and material properties of the micropillar array, we first performed finite-element simulations (COMSOL Multiphysics 6.3) to analyze micropillar-substrate contact.

Model simplification

To facilitate computation, the system was simplified into two core components: the micropillar array layer and a rigid support layer representing the capacitive micro-array. A movable rigid plane was also introduced for load application, as illustrated in Fig. S2.

Material model

Given that the conductive silicone rubber exhibits good linear elasticity within the load range of interest (Fig. S4), the micropillar array layer was modeled as a linear elastic material. The elastic modulus was varied between 1 MPa and 5 MPa during parameter screening, with Poisson’s ratio set to 0.48 and density to 1.13 × 10³ kg/m³.

Contact conditions

The normal behavior of the contact between the micropillars and the substrate was defined as hard contact, allowing pressure transmission upon contact and zero pressure upon separation. The tangential behavior was modeled with a friction coefficient of 0.15. A surface-to-surface discretization scheme with finite sliding was employed.

Boundary conditions

The boundary conditions were set to closely replicate the actual assembly state. The rigid support layer was fully fixed, and the outer edges of the micropillar array were also constrained. Various load types, including normal compression, in-plane shear, axial torsion, and transverse bending, were applied via the movable rigid plane to simulate the six-axis loading conditions of the sensor.

Meshing strategy

A variable-density meshing strategy was adopted: the micropillar regions were meshed with finer elements, while the bulk regions were meshed more coarsely. Second-order tetrahedral elements were used throughout. A mesh convergence study showed that refining the micropillar mesh beyond 0.2 mm resulted in a change of less than 2% in the calculated contact area. Therefore, this mesh setting was adopted for all subsequent analyses.

The geometric features of the contact regions served as a basis for estimating the respective contributions of pillar configuration and material properties.

Fabrication

The proposed sensor has overall dimensions of 15 mm × 15 mm × 8 mm and comprises a force-sensing layer, an integrated circuit chip (model FPC1020AP, Zhupu Core Technology Co., Ltd., Shenzhen, China), a signal acquisition printed circuit board (PCB) module, and a bottom support structure. The force-sensing layer features a bilayer design: the outer layer is an insulating silicone rubber that blocks external interference, while the inner layer consists of a conductive silicone-based protuberance array that directly contacts the chip to generate force–capacitance imaging signals. The IC chip and PCB board are integrated into a compact module with a total thickness of 2 mm. The bottom support structure was fabricated using additive manufacturing via a 3D printer (model X1C, Bambu Lab, Shenzhen, China). The support not only provides mechanical stability but also embeds and fixes the integrated acquisition board (including the PCB and IC chip) on its top surface. Conductive copper foil was wrapped around the sidewalls of the support, and electrical connections between the chip bonding wires (model H62, Taizhou Biling Hardware Co., Ltd.), conductive protuberance layer, and copper foil were established via conductive epoxy (model 3701, Hanjing Electronics Co., Ltd., Shenzhen, China). After curing, the conductive adhesive simultaneously ensures electrical continuity and mechanical fixation, maintaining effective coupling between the sensing layer and the chip even under cyclic loading and recovery.

The force-sensing layer was prepared via a two-step molding process to achieve monolithic integration of the two layers. The outer layer was designed as a curved insulating structure, whereas the inner layer was patterned into a gradient-distributed protuberance array. The fabrication process is as follows (Fig. S1):

First, 5.000 g of carbon black–based conductive silicone elastomer (model 50, Hanjing Electronics Co., Ltd.) and 0.100 g of curing agent were weighed via a precision electronic balance (model XPR204S/AC, Mettler Toledo, Switzerland). The mixture was homogenized in a planetary mixer at 200 rpm for 15 min and then spread onto a polyethylene film substrate. Subsequently, 0.55 g of the mixture was precisely placed into the first molding cavity of the mold base and compressed with a molding die for alignment and shaping. The assembled mold was then cured in a constant-temperature oven (model 9070A, Shanghai Shanzhi Instrument Co., Ltd.) at 180 ± 2 °C for 20 min. After demolding, the conductive layer was inspected and trimmed.

For the secondary molding, 5.000 g of silicone rubber (Model 30, Hanjing Electronics Co., Ltd.) and 0.100 g of curing agent were mixed under identical conditions using a planetary mixer (Model SIE-MIX1000, Guangzhou Sinos Technology Co., Ltd.). A measured 0.35 g of the mixture was deposited into the secondary molding cavity, and the pre-cured protuberance layer from the first step was carefully positioned and fixed. The assembly was then cured at room temperature for 6 h. After demolding and surface treatment, the complete bilayer force-sensing structure was obtained.

Calibration

To calibrate the six-axis force and contact position, the sensor was rigidly mounted alongside a high-precision six-axis force/torque sensor (Nano17, ATI Industrial Automation, USA), ensuring both shared the same coordinate frame. The sensing system was installed on a testbed, and a robotic manipulator (UR5, Universal Robots, Denmark, positional accuracy of 0.01 mm), equipped with a calibration indenter, was used, as illustrated in Fig. 3a. Nano17 collected ground-truth six-axis force/torque data, while the proposed sensor simultaneously recorded capacitance images, and UR5 provided accurate 3D contact position data.

The curved sensing surface was uniformly partitioned into 49 subregions (2 mm × 2 mm each), within which random loads of varying magnitudes and orientations were applied across the full working range (Fig. S3). This procedure generated a high-quality dataset of tactile image–force data pairs, providing a solid foundation for machine learning. The advantage of this approach lies in the efficient acquisition-based calibration data; however, the accuracy of load position prediction is jointly constrained by the subregion size, the precision of the robotic manipulator, and the performance of the learning model.

After preprocessing, the resulting measurements were used as labeled datasets. Two datasets were constructed: (1) a tactile image–six-axis force dataset containing 25,000 samples, and (2) a tactile image–contact position dataset containing 5000 samples.

Author contributions

Yi Song: Writing—review & editing, Methodology, Conceptualization, Funding acquisition, Supervision. Junwei Wang: Writing—review & editing, Experiments, Data collection, and Analysis. Zongke Li: Experiments, Data collection, and Analysis. Weilang Hu, Ye Qiu, Ye Tian: Experiments, Discussion. Pei Zhao, Aiping Liu: Discussion. Huaping Wu: Supervision, Project administration, Writing—review & editing.

Data availability

All necessary data have been provided in the main text and supplementary materials. Other supporting information is available upon request from the author.

Conflict of interest

The authors declare no competing interests.

Supplementary information

The online version contains supplementary material available at 10.1038/s41378-026-01292-3.