Skip to main content
NCBI home page
ACS AuthorChoice logoLink to ACS AuthorChoice
. 2022 Jan 2;4(1):308–315. doi: 10.1021/acsaelm.1c01010

Skin-Inspired Flexible and Stretchable Electrospun Carbon Nanofiber Sensors for Neuromorphic Sensing

Debarun Sengupta †, Michele Mastella ‡,§, Elisabetta Chicca ‡,§, Ajay Giri Prakash Kottapalli †,∥,*
PMCID: PMC8793024  PMID: 35098136

Abstract

graphic file with name el1c01010_0007.jpg

During the past few decades, a significant amount of research effort has been dedicated toward developing skin-inspired sensors for real-time human motion monitoring and next-generation robotic devices. Although several flexible and wearable sensors have been developed in the past, the need of the hour is developing accurate, reliable, sophisticated, facile yet inexpensive flexible sensors coupled with neuromorphic systems or spiking neural networks to encode tactile information without the need for complex digital architectures, thus achieving true skin-like sensing with limited resources. In this work, we propose an approach entailing carbon nanofiber–polydimethylsiloxane composite-based piezoresistive sensors, coupled with spiking neural networks, to mimic skin-like sensing. The strain and pressure sensors have been combined with appropriately designed neural networks to encode analog voltages to spikes to recreate bioinspired tactile sensing and proprioception. To further validate the proprioceptive capability of the system, a gesture tracking smart glove, combined with a spiking neural network, was demonstrated. Wearable and flexible sensors with accompanying neural networks such as the ones proposed in this work will pave the way for a future generation of skin-mimetic sensors for advanced prosthetic devices, apparel integrable smart sensors for human motion monitoring, and human-machine interfaces.

Keywords: piezoresistive, sensors, neuromorphic, carbon nanofibers, leaky integrate-and-fire, artificial neurons

Introduction

With the recent progress in flexible electronics and the wearable smart technology, next-generation smart prosthetic devices are becoming a reality. These devices can recreate the sense of touch in prosthetic limbs and create sensing possibilities which assist toward realizing smart apparels for real-time human vital monitoring and soft sensors for human-machine interactions. In particular, skin-like artificial sensors based on polymer–nanomaterial composites are widely researched for wearable and next-generation prosthetic applications. Various conductive nanomaterials such as carbon black, carbon nanotubes (CNTs), graphene, Mxene, and silver nanowires in conjunction with polymer elastomeric materials like ecoflex, polyimide, polydimethyl siloxane (PDMS), and polyurethane have been used for developing skin-like flexible and stretchable sensors.1−6 In addition to the vast variety of two-dimensional (2D) nanomaterials already reported in the literature, inexpensive nanomaterials like electrospun carbon nanofibers (CNFs) have also been employed for developing flexible and wearable sensors.7,8 The electrospinning method provides a facile and cheaper alternative to cumbersome processes like chemical vapor deposition, laser ablation, arc deposition, chemical and mechanical exfoliation, and graphite oxidation–reduction employed for synthesizing carbonaceous nanomaterials like CNTs and graphene. The crucial need of the hour is to develop facile, reliable, and inexpensive skin-like artificial sensors integrated with intelligent systems to achieve somatosensory perception in robots and next-generation prosthetic devices. The present challenges can be solved by adapting a bioinspiration approach entailing a two-pronged strategy, wherein skin-like wearable/skin-mountable sensors could be employed to emulate the skin, and an interface neuromorphic circuit could be used to generate spike patterns, emulating neural firing and mimicking human sense of touch.

The sense of touch is a crucial ability in humans. The exploration of an unknown environment heavily relies on the usage of touch since the first instance of every child’s life.9 This sense enables the recognition of the properties of objects like texture, shape, and softness, which are fundamental for even a simple task like grasping a glass without breaking it. Most of the complex somatosensory abilities observed in human beings lie in the skin, which can be considered as a large-area pressure, tactile, vibration, and temperature sensor. The skin is innervated by mechanosensory afferents which enrich its somatosensory ability as shown by the schematic in Figure 1a. For instance, the glabrous skin of the human hand is populated by low-threshold mechanoreceptors (LTMRs) consisting of a combination of rapid and slow-adapting LTMRs (RA/SA-LTMRs).10 While the SA-LTMRs such as the Ruffini endings and the Merkel cells respond to skin stretch and constant indentation, the RA-LTMRs such as the Pacinian and Meissner corpuscles are more sensitive to dynamic stimuli and (RA/SA-LTMRs).10 The schematic in Figure 1a summarizes the mechanoreceptors innervating the human skin. Robots should be endowed with sensors similar to mechanoreceptors in order to perform complicated tasks like interacting with human beings or unknown environments. While performing simple tasks like object grasping, important information is provided by touch in relation to both the shape of the held object, the position in the hand, and the force exerted by the fingers.10 Specifically, this requires both cutaneous tactile information, like the intensity at which the hand is pressing the object, and haptic proprioception, like the angle of the different fingers or the position of the hand with respect to the arm.11 Therefore, to achieve true skin-like sensing, the system should have sensors capable of detecting pressure on the fingers and strain at the joints and accompanying circuitry to convert the sensor signal to neural impulses. While several biomimetic sensors for cutaneous touch have been proposed in the literature,12−14 few of them considered haptic proprioception and, to our knowledge, the combination between these two is only conceptualized.15

Figure 1.

Figure 1

Open in a new tab

(a) Schematic representation of glabrous and hairy skins with their underlying mechanosensory receptors enabling their somatosensory ability. Reproduced with permission,10 2013, Cell press. (b) Schematic representation of the process steps involved in fabrication of the CNF-PDMS piezoresistive sensors. (c) Photograph of the fabricated CNF-PDMS tactile sensor. (d) Conceptual scheme of the final architecture. The glove will be equipped with tactile sensors on the palm and on the fingertips, with a wrist bending sensor and with stretch sensors able to detect gestures. The data will be then collected using Wheatstone bridges, converted to currents, and fed to a neural network which will use neural coding.

To pursue the combination between these two concepts, in this work, CNF-PDMS-based piezoresistive sensors have been coupled with neuromorphic spiking neural networks to achieve skin-like sensing and proprioception. In this approach, models inspired by biological neurons were used to encode oncoming signals from the sensors and to convert them into digital pulses (called spikes), which preserve the information in the time of the event.16 Several studies have already used this method for converting analog signals from tactile stimuli into spiking activity.12−14,17 Furthermore, the kinds of networks used in this study are increasingly becoming popular due to their versatility and their ability to learn.18 For example, in object tactile orientation detection, several networks have been proposed for decoding the angle at which a bar is pressed using only neurons, synapses, and learning.19,20

In the first part of the paper, the sensing element is investigated. The recorded behavior of a bundled CNF-based piezoresistive sensing element is fed to a simulated neuron. The outcome of the latter is shown to encode information through spike count, similar to an approach proposed for prosthetics application demonstrated earlier.21 In the second (wrist conformation) and third (gesture recognition) parts instead, the strain sensor signals are recorded and coupled with simulated neurons for emulating proprioception. In the case of wrist conformation, the experiment is conducted by sampling the bending angle of a wrist through the fabricated strain sensor. The outcome is fed to a simulated neuron that can encode the bending angle in spike count using the sensor’s voltage as input. For the third case (gesture recognition), a more complex approach has been adapted. By considering several CNF-PDMS strain sensors placed at the finger joints, the simulated network was able to detect specific gestures. This was done by a first layer of neurons, connected in a one-to-one fashion coupled with a second layer, through synapses. The latter was able to indicate which gesture was performed. The interplay of these two layers of neurons, along with their synapses, formed a spiking neural network. Such a combination of networks with flexible, wearable yet facile and inexpensive sensors could be a key factor for future generation of smart apparel with cutaneous and haptic abilities capable of recreating the somatosensory perception in prosthetics and robotic interfaces.

Results

System Description

The ultralightweight, stretchable, and facile CNF-PDMS skin-like sensors utilized in this work have been introduced in our earlier work, which describe the fabrication and characterization of this sensing element.7 The sensor design features a piezoresistive CNF bundle which acts as a sensing element embedded in two layers of PDMS for encapsulation. To demonstrate the capability of mimicking proprioceptive perception and tactile sensing, the CNF sensing elements were configured into various designs suitable for strain and pressure sensing. The details of sensor fabrication are schematically represented in Figure 1b. Further details of the sensor design and fabrication steps are presented in the Experimental Section. The sensing mechanism of the CNF-PDMS strain sensor has been reported in our earlier works.7,8 Within a CNF sensing element, electrons travel through the CNF percolation network embedded in the PDMS elastomer matrix. Any external strain or applied pressure results in the change in overlap area between adjacent conductive domains, leading to an overall change in the resistance of the sensor. In addition, electrons can also tunnel across a thin (less than a nanometer) insulating barrier separating two conductive domains, which leads to associated tunneling resistance which can be predicted using Simmon’s tunneling resistance formula.22,23 The tunneling resistance is extremely sensitive to the interdomain separation, and any changes arising due to external pressure/strain are also manifested as an overall resistance change of the sensor. When a pressure/strain is applied on the CNF-PDMS sensor, its resistance changes proportional to the magnitude of the applied force. Variation of resistance can be sampled by ad hoc integrated circuits coupled with a suitably designed Wheatstone bridge,24 even with the possibility to gather multiple sensors using only one circuit25 which exhibits linear behavior. In this work, an appropriately designed Wheatstone bridge circuit is used to convert the resistance changes in the sensors into readable voltage signal outputs. The realization of the stage converting the pressure applied on a piezoresistive element to the current fed to a neuron is not explored. Instead, an ideal linear dependence between the pressure and the current output is used to run the simulation in this work.

In the following subsections, we demonstrate how current coming from this ideal stage can be used in neuromorphic signal processing instead of traditional digital signal processing. The information in the former is conveyed using a new medium: spikes. In this work, different types of encoding are shown with different properties of touch to highlight how versatile and useful a conversion to spikes can be. The signals, converted into currents, are fed into a leaky integrate-and-fire (LIF) simulated artificial neuron. The LIF model is a very common implementation of a simplified biological neuron,26 composed of a resistor–capacitor network and a threshold, as further explained in the methods. The choice of this type of model is given by the great amount of existing hardware: the literature shows implementations on FPGA,27,28 microcontrollers,29 and VLSI chips.30−32 In the latter especially, the system does not require any overhead for circuitry since the readout and the neurons can be integrated in a single monolithic chip. This paper focuses on showing how, using neurons instead of traditional digital communication, computation can be achieved without using ADC or DSP. In the first experiment, which involves applying different pressures on the sensor, analog voltage response is observed, and the subsequent behavior of neurons is investigated. In the second case, which involves measurement of wrist bending, a similar approach is adopted to convert the sensor’s voltage caused by the movement of a wrist into meaningful neural activity. In the last part, related to identification of various hand gestures, the combined activation of several neurons is exploited to reconstruct the occurrence of different finger configurations.

This technique of converting analog signals into events has lower limitations than traditional digital systems. They can encode analog information in their time interval between spikes, keeping most of the meaning and relaxing the Shannon theorem’s constraint. This means that regardless of the stimulus’s transient speed, a neuron is still able to sample most of the signal’s envelope. Second, neurons are excited and spike only when a current is fed at their input. This ability, called event-driven sampling, is useful when the desired signal is sparse. Sparsity is a property typically observed in nature where useful information is condensed in a small fraction of the active time of a sampling agent. The event-driven nature samples the environment only when necessary and with an almost analog time step (only limited by the physical design of the neuron).

Mimicry of Tactile Sensing

Under the effect of an external pressure stimulus, the resistance of the CNF-PDMS sensor changes, which is manifested as voltage signal output from the accompanying Wheatstone bridge circuit. To demonstrate skin-like tactile sensing capability, the sensor was pressed four times in quick succession, followed by a gap of 10 s and subsequent repetition of the sequence six times. As shown by the sensor voltage response plot in Figure 2a, the sensor can generate different voltages depending on the applied tapping pressure which can be used to stimulate a simulated artificial neuron. To be compatible with the neuron physical mechanism, the Wheatstone bridge output is converted through an analog voltage-to-current converter into current and then injected into the neuron.

Figure 2.

Figure 2

Open in a new tab

(a) Sensor voltage generated by different periodic pressures on the sensor. Different pressure values have been used to explore the response of the sensor with voltages up to 1.21 V. (b) Photo of the sensor along with the sensor readout schematic. (c) Number of spikes emitted by the connected neuron when the piezoresistor is stimulated. (d) The membrane voltage reaches 0.4 V, its value is rebased to 0 V, and a spike is generated. (e) Correspondence between the values of the sensor voltage (encoding the pressure) and the number of spikes emitted by the sensor. The number of spikes is calculated per 100 ms.

Once the current is injected in the neuron, charge begins to accumulate on its membrane capacitor that also has a leaky component. The voltage difference present at the capacitor pins increases with the charge as dVmem/dt = (Isensor/C – Vmem/(RC)), where C is the equivalent capacitive value of the membrane and R is the equivalent resistance of the leaky component. When the voltage reaches a specific value (defined as threshold voltage or Vthr), the neuron emits a digital event (or spike). In Figure 2, the voltage on the membrane (Vmem) is plotted to highlight how the sensor’s voltage (Vsensor) influenced the neuron behavior. In the formula, Isensor is related with the Vsensor by an ideal component with linear conductance G so that Isensor = G × Vsensor (exemplified in Figure 2b by component G). The formula as it is highlights that the membrane voltage acts as a low-pass filter.

Figure 2 also shows the encoding ability of a neuron, which embeds the information about the amplitude of the sensor voltage into its spiking activity. This is visible in 2c and 2d, where spike count and the membrane potential of the neuron are presented. Figure 2e highlights the close relationship between the sensor voltage and the neuron activity by plotting the voltage of the sensor and the equivalent spike response. For low voltages (less than 0.2 V), the neuron does not generate any spikes, while for higher voltages, the neuron generates spikes proportional to the input. This is related to the property of the LIF neuron, which does not reach the threshold if the injected current is not high enough. This property can be efficiently used to set an attention threshold under which the communication stream is not active, avoiding the processing of noise or spurious signals.

Wrist Conformation

SAII-LTMRs of the human skin are extremely sensitive to skin stretch. For instance, the SAII-LTMRs found in humans share their physiological traits with proprioceptors which facilitate kinesthetic perception (finger shape or conformation). The strain sensing mechanism of the CNF-PDMS sensors can also be exploited for mimicking proprioceptors capable of detecting the bending of the wrist. When a uniaxial strain is applied, the resistance of the sensor changes owing to the conductive domain disconnection mechanism. The sensor bending generated tensile and compressive strain, which resulted in a voltage proportional to the applied angle then converted by the accompanying Wheatstone bridge circuit. The sensor’s voltage can be used to stimulate an LIF neuron, once converted into current, which can also be applied for strain measurements. The information about the bending can be encoded in the number of spikes that the neuron emits for a given time step. The number of spikes correlates well to the analog voltage generated by the neuron, as visible in Figure 3a. In the latter, the higher the analog voltage, the denser the spike activity.

Figure 3.

Figure 3

Open in a new tab

(a) Comparison between the response to wrist bending of both the sensor and the neuron. The sensor generates a voltage proportional to the bending of the wrist. This voltage is converted into a current and fed into a neuron, simulated on a computer. The number of spikes per 100 ms is here considered as spike count (or spikes #). Different trials for different angles are shown. The neuron follows the analog voltage with the number of spikes but does not generate any spike when the stimulus is only noise (this is visible in the lower part of the figure, where the spikes are 0 when no bending is performed). (b) Statistical analysis of the analog voltage and spike counts with different bending angles. The statistical deviation of the sensor is given by the human error in bending the wrist at a specific angle. This uncertainty is reflected quite well in the neuron with the spike count, which highlights the direct connection between a neuron spiking activity and a current coming from a piezo-resistive readout. This is to demonstrate the high degree of reproduction that the neuron has with respect to the analog values that it receives.

From Figure 3a, it can also be observed that the neuron employment shows an advantage of filtering of noise in encoding. This phenomenon is possible thanks to two properties of neural coding. First, the neuron’s membrane capacitor acts as a low-pass filter, removing the high-frequency noise components. Second, the neuron spikes only when a threshold voltage is reached. This means that an isolated spurious signal that is injected into the neuron through current will not be enough to generate a spike, while a consistent stimulus is going to excite the neuron, resulting in multiple spikes.

In Figure 3b, the statistical mean and standard variation among trials of both voltage and spikes are plotted. It can be seen from the plot that not only does the neuron have a response proportional to consistent stimulus but also it is able to maintain the same statistical properties observed in the original sensor’s voltage. The variation is due to the psychophysical uncertainty of human movements which leads to subsequent angular discrepancies while bending the wrist at certain prespecified angles and the fact that the neurons transfer; also, this psychophysical uncertainty can be used to process rich information, like fine grain movement.

Gesture Recognition

To further demonstrate applications of the CNF-PDMS sensors involving proprioceptive perception, a smart glove system consisting of five identical sensors secured on a nitrile glove coupled with appropriately designed Wheatstone bridge circuits was developed. Different sensors were placed at the hand joints, while a human volunteer moved the fingers in defined ways. The hand performed several digits with the glove. Nominally, number 5 (composed of the palm completely open), number 4 (composed of the thumb bending inward), number 3 (made of the thumb and the pinky fingers closed), and numbers 2 and 1 shown by bending the middle and the index fingers and only the index finger, respectively.

Each of these finger’s movements led to resistance changes in the sensors, subsequently leading to voltage signals generated by the readout circuits. The simultaneous activation of multiple sensors can be used to spot the number mimicked. As seen in Figure 4a, the succession of the numbers (5, 4, 3, 2, and 1) generates in the five sensors time-varying responses. Specifically, the bending of the finger creates a high value, while the relaxed position gives no response. The signals from the sensors are then fed into a simulated layer of five LIF neurons. The response of different neurons, visible in Figure 4c, highlights that the layer (i.e., a group of neurons with the same task) is able to transfer the analog value of the bending quite faithfully in the number of spikes per time step. This approach, like the one used in the case of tactile sensing mimicry previously, exploits the spike activity of neurons to encode the amplitude of the sensor’s voltage without noise and spurious activity.

Figure 4.

Figure 4

Open in a new tab

(a) Response in voltage output of the five different sensors placed at the joint of the glove. The five sensors are plotted one over the other, from the little finger to the thumb. The five different colors superimposed to the graph highlight the different gestures performed in that moment. (b) Order of execution of five different gesture tasks performed with the glove. (c) Response in spike count of the five different neurons connected to the five sensors, plus the Poisson neuron, responsible for acting on the gesture “Five”. The five different colors superimposed to the graph highlight the different gestures performed in that moment. (d) Schematic describing the network used in this example. (e) Response of the decoding layer to the different gestures.

The coincidence activation of several neurons, connected in a one-to-one fashion to the sensors, encodes the different digits performed with the glove. Each number is obtained with the simultaneous bending of several fingers. Therefore, by analyzing which sensor was active at each moment, we can understand which number was performed. The decoding part (i.e., understanding which number the sensors’ activity represents), while being traditionally obtained using microcontrollers or computers, can be done using neuromorphic components. This approach can reduce the need for complex and power-consuming architectures. The full network architecture is briefly shown in Figure 4d and described in detail in the Experimental Section. The structure can recognize which gesture was performed, without any digital support. In Figure 4e, the outcome of the network is visible for each gesture performed. In these, the network responds with a spiking activity on the neuron carrying the gestures semantic.

The gesture task can be extended even further considering that the neuron’s ability to encode into spikes the analog voltage of a sensor can be used to recognize more complex gestures. Figure 5 shows the correspondence between sensors’ output and spiking activity when the smart glove is used to perform more refined gestures. A more complex neural network could also decode these gestures, avoiding the need for further steps in an acquisition chain.

Figure 5.

Figure 5

Open in a new tab

Response of the five sensors and the five neurons to various hand gestures. The sensor output voltage sensors increase with the increasing curvature of the finger, and the neuron response follows this behavior.

Conclusions

In this work, we proposed a combined approach between novel CNF-PDMS-based piezoresistive sensors and spiking neural networks to encode proprioception and tactile information without the need for digital architectures like analog-to-digital converters or digital signal processing. The fabricated piezoresistive sensor is exploited by accompanying neurons connected afterward, and the resulting system has been used for converting different types of stimuli, from tactile sensors or strain sensors, into spikes, showing how the number of the pulses emitted by a neuron can communicate analog values such as the tactile pressure or the deformation. Furthermore, the potential behind this has been unveiled in this work, showing how neural networks can also decode the information contained in the sensors’ response without the need for any digital processing as in the case of gesture recognition. A final architecture, consisting of a direct interface between sensors and neural network CMOS circuits can greatly improve the abilities of robots, autonomous agents, or prosthetic devices to detect complex stimuli, resulting in low power consumption and low latency conversion,30 typical of embedded approaches.

Experimental Section

Electrospinning and CNF Synthesis

The detailed recipe for electrospinning polyacrylonitrile nanofibers and the subsequent pyrolysis has been reported by us previously.7 Polyacrylonitrile (PAN) powder (150,000 g/mol) and N,N-dimethyl formamide (DMF) obtained from Sigma-Aldrich was used to form 9% (w/v) PAN polymer solution in DMF for electrospinning. An Inovenso NanoSpinner NE300 was used for electrospinning. The polymer solution was fed through a 18G needle using a standard syringe pump (model NE300) at a constant flow rate of 1 mL/h. The electrospinning process was conducted for 30 min at 12 kV (applied between the needle tip and a rotating mandrel collector) while maintaining a constant tip to collector distance of 10 cm. The as-spun nanofibers were transferred from the aluminum foil collector to a ceramic crucible and placed inside a furnace for pyrolization (at 950 °C) to synthesize CNF bundles. Further details regarding the pyrolization step are provided in our earlier work.7

Sensor Fabrication and Data Acquisition

The basic fabrication steps include sandwiching a CNF bundle (bonded to copper tapes at two ends) between two layers of PDMS, thus achieving a complete encapsulation. In specific, the tactile sensor was composed of an electrically bonded 2 × 2 cm CNF bundle encapsulated with the PDMS elastomer. However, the five identical strain sensors used for demonstrating proprioceptive perception and gesture monitoring were composed of 4.5 × 0.2 cm bundled CNF films embedded in PDMS elastomeric layers.

For all the experiments involving tactile sensing and gesture monitoring, appropriately designed resistance-matched Wheatstone bridge circuits were developed, to which the CNF-PDMS sensors were connected. The voltage outputs from the Wheatstone bridge circuit were obtained and logged using a National Instruments data acquisition system (DAQ, NI USB-6009) with National Instruments LabView software. A constant power supply of 9 V was used for powering the Wheatstone bridge circuit during all the experiments. Furthermore, the output signals from the bridge circuit were obtained continuously at a sample acquisition rate of 2 kHz in a differential configuration.

Neural Coding

Neuron Model

The conversion from the sensors’ voltage recorded during the different experiments was conducted on a spiking neural network simulator called Brian2.33 The latter is an open-source python library used for recreating spiking neurons and synapses with custom properties. In all the experiments, voltage has been converted to a linear current Isensor = GVsensor, where G is a transconductance constant, with value = 1 pS. This is made to emulate an ideal voltage-to-current converter, able to stimulate the neuron’s membrane capacitor.

The model used for simulating a neuron response is the LIF one. This model has been chosen due to its simple software implementation and the possibility to easily obtain a closed formula for an analytical computation. It can be expressed with the formula

graphic file with name el1c01010_m001.jpg
graphic file with name el1c01010_m002.jpg
graphic file with name el1c01010_m003.jpg

where Vmem is the voltage difference between the neuron inside and the neuron outside, C is the capacitive value of the membrane capacitor present in biological neurons, τmem = RC is the time constant of the membrane originating from the capacitive term and the leaky component, while Vthr is the threshold voltage that generates a spike. The capacitor integrates the current coming either from the synapse (Isyn) or directly from the sensors (Isensor). Both these currents increase the voltage Vmem. The voltage leaks through the membrane with a time constant of τmem. When the threshold is reached, Vmem is rebased to its resting voltage (Vrest), ready for integrating new current.

Neurons communicate with each other through spikes. Spikes are converted into currents through the synapses. Every time a neuron spikes, an excitatory post-spike current is generated and injected in a neuron. This is typically observed in biological neurons, and this also has a circuit equivalent in the literature. The spike is here expressed as a Dirac δ that is 1 only when the spike comes. Each synapse has a dimensionless weight, here expressed as W

graphic file with name el1c01010_m004.jpg

Encoding

Neurons communicate with each other using different paradigms. In this work, we exploited specifically two different cases: the rate encoding and the spatial encoding. In the rate encoding, the information is condensed in the spike rate of a single neuron. The injected current charges the capacitor voltage and makes the neuron spike. The higher the current, more are the times the neuron spikes with that current. The spike number directly communicates the amount of injected current into the neuron. Another way neurons communicate with each other is spatial coding. Different neurons in a specific layer have different roles, with each neuron representing a specific concept. In the case, for example, of four possible stimuli at the input and four different neurons, each neuron spikes when the stimulus that it is representing appears.

Network for Gesture Recognition

In neural networks, the connection between two layers of neurons through synapses can extract meaningful information from raw data, dividing them in semantic classes. In this work, the neurons that encoded the five different sensors were connected with synapses to five different output neurons in a second layer. Synapses are electrical components that convert the spike of a neuron (which is a voltage difference) into a current. The amount of current into which a spike is converted depends on the synapse strength. In this experiment, the strength of the different synapses was tuned to excite the upcoming neurons only with the right combination of input neurons. The combination was chosen to activate the semantic neurons only when the right gesture was performed. The neuron that must spike for the gesture “FOUR”, for example, has a very strong connection with the thumb sensor. Neurons compete between each other, and only the strongest one wins. In the case “FOUR”, this neuron is the one winning when only the thumb is tilted. Gesture “FIVE” instead needs a detailed description, given that the open palm encodes for it. This means that no sensors are active while performing the gesture. This problem can be solved by introducing a sixth neuron in the input layer that is always spiking with a Poisson distribution. When only the sixth neuron is active, then the network understands the presence of the gesture “FIVE”.

Acknowledgments

The authors would like to express their sincere gratitude to Joshua Romano for his help with sensor fabrication undertaken as a part of his MSc thesis.

Author Contributions

D.S. and M.M. contributed equally to this work. D.S. conducted the electrospinning, CNF synthesis, sensor fabrication, and associated experiments. M.M. conceived and realized the neuromorphic experiments with simulated neurons and neural networks. D.S. and M.M. wrote the manuscript. D.S., M.M., E.C., and A.G.P.K. conceptualized the idea. E.C. and A.G.P.K. supervised the project. All authors contributed to the discussion of the results and to the critical reading of the manuscript.

A.G.P.K. and D.S. acknowledge the financial support of the University of Groningen’s start-up grant awarded to A.G.P.K. E.C. and M.M. acknowledge the financial support of EU H2020 project Neutouch grant no. 813713 as well as of the CogniGron research center and the Ubbo Emmius Funds (Univ. of Groningen).

The authors declare no competing financial interest.

References

  1. Amjadi M.; Yoon Y. J.; Park I. Ultra-Stretchable and Skin-Mountable Strain Sensors Using Carbon Nanotubes–Ecoflex Nanocomposites. Nanotechnology 2015, 26, 375501. 10.1088/0957-4484/26/37/375501. [DOI] [PubMed] [Google Scholar]
  2. Qin Y.; Peng Q.; Ding Y.; Lin Z.; Wang C.; Li Y.; Xu F.; Li J.; Yuan Y.; He X.; Li Y. Mechanically Flexible Graphene / Polyimide Nanocomposite Foam for Strain Sensor Application. ACS Nano 2015, 9, 8933–8941. 10.1021/acsnano.5b02781. [DOI] [PubMed] [Google Scholar]
  3. Pang Y.; Tian H.; Tao L.; Li Y.; Wang X.; Deng N.; Yang Y.; Ren T.-L. Flexible, Highly Sensitive, and Wearable Pressure and Strain Sensors with Graphene Porous Network Structure. ACS Appl. Mater. Interfaces 2016, 8, 26458–26462. 10.1021/acsami.6b08172. [DOI] [PubMed] [Google Scholar]
  4. Boland C. S.; Khan U.; Backes C.; O’Neill A.; McCauley J.; Duane S.; Shanker R.; Liu Y.; Jurewicz I.; Dalton A. B.; Coleman J. N. Sensitive, High-Strain, High-Rate Bodily Motion Sensors Based on Graphene–Rubber Composites. ACS Nano 2014, 8, 8819–8830. 10.1021/nn503454h. [DOI] [PubMed] [Google Scholar]
  5. Sharma S.; Chhetry A.; Sharifuzzaman M.; Yoon H.; Park J. Y. Wearable Capacitive Pressure Sensor Based on MXene Composite Nanofibrous Scaffolds for Reliable Human Physiological Signal Acquisition. ACS Appl. Mater. Interfaces 2020, 12, 22212–22224. 10.1021/acsami.0c05819. [DOI] [PubMed] [Google Scholar]
  6. Huang T.; He P.; Wang R.; Yang S.; Sun J.; Xie X.; Ding G. Porous Fibers Composed of Polymer Nanoball Decorated Graphene for Wearable and Highly Sensitive Strain Sensors. Adv. Funct. Mater. 2019, 29, 1903732. 10.1002/adfm.201903732. [DOI] [Google Scholar]
  7. Sengupta D.; Chen S.-H.; Michael A.; Kwok C. Y.; Lim S.; Pei Y.; Kottapalli A. G. P. Single and Bundled Carbon Nanofibers as Ultralightweight and Flexible Piezoresistive Sensors. npj Flex. Electron. 2020, 4, 9. 10.1038/s41528-020-0072-2. [DOI] [Google Scholar]
  8. Sengupta D.; Trap D.; Kottapalli A. G. P. Piezoresistive Carbon Nanofiber-Based Cilia-Inspired Flow Sensor. Nanomaterials 2020, 10, 211. 10.3390/nano10020211. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Streri A. Tactile Discrimination of Shape and Intermodal Transfer in 2- to 3-Month-Old Infants. Br. J. Dev. Psychol. 1987, 79, 618. 10.1111/j.2044-835x.1987.tb01056.x. [DOI] [Google Scholar]
  10. Abraira V. E.; Ginty D. D. The Sensory Neurons of Touch. Neuron 2013, 79, 618. 10.1016/j.neuron.2013.07.051. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Yi Z.; Zhang Y.; Peters J. Biomimetic Tactile Sensors and Signal Processing with Spike Trains: A Review. Sens. Actuators, A 2018, 269, 41. 10.1016/j.sna.2017.09.035. [DOI] [Google Scholar]
  12. Kim E. K.; Wellnitz S. A.; Bourdon S. M.; Lumpkin E. A.; Gerling G. J. Force Sensor in Simulated Skin and Neural Model Mimic Tactile SAI Afferent Spiking Response to Ramp and Hold Stimuli. J. NeuroEng. Rehabil. 2012, 9, 45. 10.1186/1743-0003-9-45. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Bologna L. L.; Pinoteau J.; Passot J. B.; Garrido J. A.; Vogel J.; Vidal E. R.; Arleo A. A Closed-Loop Neurobotic System for Fine Touch Sensing. J. Neural. Eng. 2013, 10, 046019. 10.1088/1741-2560/10/4/046019. [DOI] [PubMed] [Google Scholar]
  14. Lee W. W.; Cabibihan J.; Thakor N. V.. Bio-Mimetic Strategies for Tactile Sensing. SENSORS, 2013 IEEE, 2013.
  15. Hatzfeld C.; Kassner S.; Meiß T.; Mößinger H.; Neupert C.; Pott P. P.; Rausch J.; Rossner T.; Staab M.; Werthschützkya R. Perception-Inspired Haptic Force Sensor - A Concept Study. Procedia Eng. 2012, 47, 112. 10.1016/j.proeng.2012.09.097. [DOI] [Google Scholar]
  16. Kayser C.; Montemurro M. A.; Logothetis N. K.; Panzeri S. Spike-Phase Coding Boosts and Stabilizes Information Carried by Spatial and Temporal Spike Patterns. Neuron 2009, 61, 597. 10.1016/j.neuron.2009.01.008. [DOI] [PubMed] [Google Scholar]
  17. Lee W. W.; Yu H.; Thakor N. V.. Gait Event Detection through Neuromorphic Spike Sequence Learning. Proceedings of the IEEE RAS and EMBS International Conference on Biomedical Robotics and Biomechatronics, 2014. [DOI] [PMC free article] [PubMed]
  18. Chou T.-S.; Bucci L. D.; Krichmar J. L. Learning Touch Preferences with a Tactile Robot Using Dopamine Modulated STDP in a Model of Insular Cortex. Front. Neurorob. 2015, 9, 6. 10.3389/fnbot.2015.00006. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Parvizi-Fard A.; Amiri M.; Kumar D.; Iskarous M. M.; Thakor N. V. A Functional Spiking Neuronal Network for Tactile Sensing Pathway to Process Edge Orientation. Sci. Rep. 2021, 11, 1320. 10.1038/s41598-020-80132-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Dabbous A.; Mastella M.; Natarajan A.; Chicca E.; Valle M.; Barolozzi C.. Artificial Bio-Inspired Tactile Receptive Fields for Edge Orientation Classification. 2021 IEEE International Symposium on Circuits and Systems (ISCAS); IEEE, 2021; pp 1–5.
  21. Osborn L. E.; Dragomir A.; Betthauser J. L.; Hunt C. L.; Nguyen H. H.; Kaliki R. R.; Thakor N. V. Prosthesis with Neuromorphic Multilayered E-Dermis Perceives Touch and Pain. Sci. Rob. 2018, 3, eaat3818 10.1126/scirobotics.aat3818. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Simmons J. G. Generalized Formula for the Electric Tunnel Effect between Similar Electrodes Separated by a Thin Insulating Film. J. Appl. Phys. 1963, 34, 1793–1803. 10.1063/1.1702682. [DOI] [Google Scholar]
  23. Yu Y.; Song G.; Sun L. Determinant Role of Tunneling Resistance in Electrical Conductivity of Polymer Composites Reinforced by Well Dispersed Carbon Nanotubes. J. Appl. Phys. 2010, 108, 084319. 10.1063/1.3499628. [DOI] [Google Scholar]
  24. Boujamaa E. M.; Alandry B.; Hacine S.; Latorre L.; Mailly F.; Nouet P.. A Low Power Interface Circuit for Resistive Sensors with Digital Offset Compensation. ISCAS 2010: 2010 IEEE International Symposium on Circuits and Systems, Nano-Bio Circuit Fabrics and Systems, 2010; pp 3092–3095.
  25. Saxena R. S.; Bhan R. K.; Aggrawal A. A New Discrete Circuit for Readout of Resistive Sensor Arrays. Sens. Actuators, A 2009, 149, 93–99. 10.1016/j.sna.2008.10.013. [DOI] [Google Scholar]
  26. Abbott L. F. Lapicque’s Introduction of the Integrate-and-Fire Model Neuron. Brain Res. Bull. 1999, 50, 303–304. 10.1016/s0361-9230(99)00161-6. [DOI] [PubMed] [Google Scholar]
  27. Cassidy A.; Andreou A. G.. Dynamical Digital Silicon Neurons. 2008 IEEE-BIOCAS Biomedical Circuits and Systems Conference, BIOCAS 2008, 2008; pp 289–292.
  28. Han J.; Li Z.; Zheng W.; Zhang Y. Hardware Implementation of Spiking Neural Networks on FPGA. Tsinghua Sci. Technol. 2020, 25, 479–486. 10.26599/tst.2019.9010019. [DOI] [Google Scholar]
  29. Floreano D.; Zufferey J. C.; Nicoud J. D. From Wheels to Wings with Evolutionary Spiking Circuits. Artif. Life 2005, 11, 121–138. 10.1162/1064546053278900. [DOI] [PubMed] [Google Scholar]
  30. Chicca E.; Stefanini F.; Bartolozzi C.; Indiveri G. Neuromorphic Electronic Circuits for Building Autonomous Cognitive Systems. Proc. IEEE 2014, 102, 1367. 10.1109/jproc.2014.2313954. [DOI] [Google Scholar]
  31. Rubino A.; Payvand M.; Indiveri G.. Ultra-Low Power Silicon Neuron Circuit for Extreme-Edge Neuromorphic Intelligence. 2019 26th IEEE International Conference on Electronics, Circuits and Systems, ICECS 2019, 2019; pp 458–461.
  32. Indiveri G.; Linares-Barranco B.; Hamilton T. J.; Schaik A. v.; Etienne-Cummings R.; Delbruck T.; Liu S.-C.; Dudek P.; Häfliger P.; Renaud S.; Schemmel J.; Cauwenberghs G.; Arthur J.; Hynna K.; Folowosele F.; Saighi S.; Serrano-Gotarredona T.; Wijekoon J.; Wang Y.; Boahen K. Neuromorphic Silicon Neuron Circuits. Front. Neurosci. 2011, 5, 73. 10.3389/fnins.2011.00073. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Stimberg M.; Brette R.; Goodman D. F. Brian 2, an Intuitive and Efficient Neural Simulator. Elife 2019, 8, e47314 10.7554/eLife.47314. [DOI] [PMC free article] [PubMed] [Google Scholar]

Articles from ACS Applied Electronic Materials are provided here courtesy of American Chemical Society

RESOURCES