Abstract
As the clinical applicability of Peripheral Nerve Stimulation (PNS) expands, the need for PNS-specific safety criteria becomes pressing. This study addresses this need, utilizing a novel machine learning and computational bio-electromagnetics modeling platform to establish a safety criterion that captures the effects of fields and currents induced on axons. Our approach is comprised of three steps: experimentation, model creation, and predictive simulation. We collected high-resolution images of control and stimulated rat sciatic nerve samples at varying stimulation intensities and performed high-resolution image segmentation. These segmented images were used to train machine learning tools for automatic classification of morphological properties of control and stimulated PNS nerves. Concurrently, we utilized our quasi-static Admittance Method-NEURON (AMNEURON) computational platform to create realistic nerve models and calculate induced currents and charges, both critical elements of nerve safety criteria. These steps culminate in a cellular-level correlation between morphological changes and electrical stimulation parameters. This correlation informs the determination of thresholds of electrical parameters that are found to be associated with damage, such as maximum cell charge density. The proposed methodology and resulting criteria combines experimental findings with computational modeling to generate a safety threshold curve which captures the relationship between stimulation current and the potential for axonal damage. Although focused on a specific exposure condition, the approach presented here marks a step towards developing context-specific safety criteria in PNS neurostimulation, encouraging similar analyses across varied neurostimulation scenarios.
Keywords: Electrical Stimulation, Safety, Computational Modeling
I. Introduction
Electrical stimulation of nerves represents a compelling frontier in contemporary medical treatment, offering unique therapeutic avenues for an array of neurological and physiological disorders [1–3]. This technique leverages our growing understanding of the nervous system, using precisely controlled electrical currents to modulate nerve activity, thereby influencing the functions they control [4–7]. From the brain’s intricate network of neurons to the fine nerve endings spread throughout the body, electrical stimulation has demonstrated potential in various contexts, including pain management, motor function restoration, and even cognitive enhancement [8–13].
While the use of electrical stimulation spans the entirety of the nervous system, one particular area of focus is the peripheral nerve stimulation (PNS). The PNS, comprising nerves outside the brain and spinal cord, plays crucial roles in motor control, sensory information processing, and autonomic function regulation [14–17]. Electrical stimulation to the peripheral nervous system—holds significant promise due to its potential for direct, localized intervention. As we delve deeper into understanding the capabilities and safety parameters of this promising technique, peripheral nerve stimulation emerges as an exciting and pivotal area of exploration in modern neurotherapeutics.
A. Existing safety protocol for PNS
Peripheral nerve stimulation (PNS) is a method of medical intervention that has witnessed over six decades of widespread clinical usage. Recognized for its significant therapeutic potential, it is employed as a mode of treatment for a number of medical conditions including chronic pain, lost motor functionality, and epilepsy [18–20].
With the growing recognition of the therapeutic effects of PNS and the resulting expansion in its application, it becomes increasingly vital to understand the parameters that affect its safe utilization. Ensuring patient safety and optimizing treatment outcomes constitute a critical objective of any therapeutic intervention, and PNS is no exception. However, despite its long-standing usage, safe limits of induced electric fields and currents due to PNS remain unclear [21–23].
Currently, both researchers and practitioners largely utilize the ”Shannon Criteria” [24], which is derived from extensive empirical observations and theoretical modeling. The Shannon criteria were developed based on experimental data derived from cat brains, collected by McCreery and colleagues [25]. It provided a safety guideline summarized by the equation log(D) = k - log(Q), which was derived by visually assessing axonal fiber damage. In this equation, D represents the charge density per phase, calculated by dividing the charge per phase (Q) by the electrode surface area. The variable ‘k’, indicating the risk of damage, is typically considered to represent an indicator for safety when it is lower than 1.5. This criterion provides an essential linkage between stimulus charge per phase and charge density per phase, which then relates to the likelihood of inducing nerve damage.
B. Limitations of the existing protocol
Despite its wide acceptance, the Shannon criterion is not without limitations.
The first limitation of the Shannon criterion is that it was originally derived from data obtained from cat brain neurons - not peripheral nerves. Given the substantial physiological and morphological differences between these nerve types, including their structural properties and myelination patterns, the applicability of the Shannon criterion to PNS is questionable. This observation highlights the need for safety guidelines based directly on peripheral nerve data to ensure more precise and relevant safety standards for PNS.
Furthermore, the Shannon criterion assumes uniform charge density distribution across the electrode surface, represented by D equals Q divided by the electrode surface area (A). However, numerous studies have raised concerns about this assumption, owing to the demonstration of non-uniform charge distribution across the electrode surface [26, 27]. For instance, a study by W. Grill et al. employed six electrodes with the same surface area but different perimeters [26]. Contrary to Shannon’s assumption, the study found varying charge densities across the electrodes.
Finally, the damage analysis of the Shannon criterion is performed manually, which may introduce subjectivity and potential inaccuracies. This, coupled with the criterion’s origination from non-PNS data, further underscores the need for the development of more refined, PNS-specific safety guidelines.
While the Shannon criterion has served as a beneficial starting point for safety guidelines, it is clear that its limitations necessitate the formulation of a more robust protocol for safety assessment. The development of an improved criterion is essential for optimizing patient safety, therapeutic outcomes, and advancing the field of PNS.
C. Ideal properties of an updated safety criteria
Firstly, these guidelines must be informed by data from peripheral nerves, rather than extrapolated from other parts of the nervous system. This approach ensures the unique characteristics of the peripheral nerve system are considered, enhancing the accuracy and relevance of the safety thresholds.
Secondly, the evaluation of tissue damage should be algorithm-based, leveraging advances in computational neuroscience and data analysis. By replacing or supplementing manual damage analysis with algorithmic methods, we can minimize human error, ensure consistency, and potentially uncover nuanced patterns or thresholds that may be overlooked in manual assessments. This shift towards a more objective and reproducible analysis is likely to improve the reliability of safety evaluations.
D. Admittance Method
The Admittance Method (AM) solver is a well-established accurate bioelectromagnetic simulation method in which the problem space is discretely partitioned: In this method, the computational model is represented as a network of admittances generated from the dielectric properties of different materials. It has been widely used for computational modeling of bioelectromagnetic problems including safety assessment of wearable or implantable devices and neural response to electric and magnetic field [28–34]. Despite its widespread application, the AM approach has not fully leveraged the capabilities of modern CPUs for multiprocessing and parallel computing, thus limiting its computational speed. Further, the process of postsimulation analysis and the integration of results with other simulation platforms is application dependent.
As a final note, it is crucial that ideal safety guidelines for peripheral nerve stimulation consider accurate current and charge density calculations in lieu of broad approximations. Many existing computational models of peripheral nerves use simplified homogeneous fascicles, which lack accuracy [35–38]. As such, there is a clear need for a more realistic nerve computational model to achieve the necessary precision.
Our study aims to develop a new safety criterion through a comprehensive study integrating experimentation, automated model generation, and simulation. We focus on incorporating the key attributes mentioned earlier to create a comprehensive safety guideline through the integration of direct peripheral nerve data, algorithm-based tissue damage analysis, and current density computations using realistic heterogeneous nerve morphologies. These guiding principles are devised to direct the evolution of safety guidelines, aligning them with the progress made in the field of peripheral nerve stimulation, with the goal of emphasizing patient safety and treatment effectiveness. This study intends to provide the PNS community with improved safety guidelines, embodying a comprehensive, multi-institutional, multi-disciplinary approach that harmonizes experimentation with modeling and simulation methods.
II. Methods
In this study, we began with microscopic optical images of stimulated rat sciatic nerves. From this initial point, our methodology followed two parallel paths, specifically designed to highlight peripheral nerve stimulation (PNS) safety from the point of view of structural damage. The block diagram of our study is shown in Fig. 1.
Fig. 1.
Summary of our methodology involving (1) image segmentation and algorithm-based nerve damage quantification for comprehensive assessment of structural and morphological axonal damage, and (2) computational modeling and neural network training for detailed exploration of cell-specific charge density distribution. The correlation between axon damage and local stimulation charge density leads to a safety criterion for PNS.
The first step focuses on image segmentation and analysis, employing a novel algorithm for automatic nerve segmentation to identify the structural and morphological characteristics of myelin and axons. We developed an algorithm-based nerve damage quantification, which enabled us to identify and measure cellular-level damage within the stimulated nerve samples when compared to healthy and control nerve samples. Complementing this evaluation, we integrated a wide range of nerve damage related metrics, such as fiber packing and fiber density, to provide a comprehensive picture of the localized extent of damage across different nerve samples.
Simultaneously, we utilized computational modeling, via the Admittance Method [32, 34, 39–44]. In this process, we used segmented images to develop realistic computational models of the segmented nerves. These computational models, employed to replicate the experiments (electrodes, nerves, and stimulation levels), unveiled detailed information about current and charge distribution inside the nerve during stimulation. We further leveraged these simulated samples to train a neural network capable of accurately grouping each cell by the simulated charge intensity. This led us to obtain local charge density values for all cells across all nerve samples during stimulation, offering a granular understanding of cell-specific current or charge density distribution within the nerves, using morphologies and structural characteristics of the imaged stimulated nerves.
By combining results from these two approaches, we compiled a detailed dataset for every cell in our nerve sample. This dataset shows both the degree of damage and the local stimulation charge density for each cell. We used this information to explore the relationship between cell damage and charge density. Studying this connection is key to better understanding PNS safety and move towards an updated safety criterion.
A. Experiments and microscopy
To correlate experimentally stimulated samples and corresponding computational models, a series of rigorously controlled animal experiments were designed and executed. Utilizing adult male Sprague Dawley rats, a multi-stage process including implantation and stimulation at different current levels was developed. These procedures adhered to ethical guidelines set by the University of Utah Institutional Animal Care and Use Committee.
During the implantation phase, the left sciatic nerve of each isoflurane-anesthetized rat was fitted with a multi-electrode cuff array (MECA), manufactured by Microprobes for Life Science. The MECA connector was securely affixed to the rat’s skull, with the wire bundle routed transcutaneously to the nerve, and the cuff sutured in place.
One week post-implantation, stimulation was conducted under isoflurane anesthesia. The contacts of the implanted MECA were connected to a stimulator delivering biphasic pulses at different current levels for 100 μs per phase, with a 400 μs interpulse period. The nerve was stimulated at a frequency of 50 Hz for a duration of four hours.
A week later, during the final euthanasia phase, the rat was perfused under deep isoflurane anesthesia, first with phosphate-buffered saline (PBS), followed by a fixative solution of 4% formaldehyde and 2% glutaraldehyde in PBS. The region of the nerve with the implant, along with a similar region from the contralateral, unimplanted side, were carefully excised. The samples were then preserved at 4°C in a solution of PFA+Glut+PBS for a week, subsequently transferred into a vial filled with 0.02% sodium azide in PBS, and sent for histological analysis.
High-resolution microscopy was used to examine the nerve samples. From the obtained high-resolution nerve microscopy images, both imaging analysis and modeling using the quasistatic Admittance Method (AM) were conducted separately.
B. Image segmentation and Damage Analysis
In a first step, this study involved an in-depth analysis of nerve cross-sections; the overview of this component of the work is illustrated in Fig. 2. Leveraging a Convolutional Neural Network (CNN) coupled with image analysis algorithms, we achieved precise segmentation of individual fibers of the nerve images (Fig. 2(a)(b)). The segmentation of axon and myelin tissues was conducted semantically using CNN, supplemented by the watershed algorithm, resulting in individual fiber segmentation of axon and myelin. Detailed procedures for this segmentation are provided in our previous study [45].
Fig. 2.
Block diagram of the image analysis approach. (a) Experimentally derived nerve samples from Sprague Dawley rats, comprising healthy and damaged specimens, are used for high-resolution microscopy. (b) These images are segmented using a Convolutional Neural Network (CNN) and a detailed image damage analysis for computing cell-specific damage indicators. Panel (c) illustrates the process using a small detail of the nerve cross-section. Specifically, (c)(A) shows the microscopy image of the cells, (c)(B) shows the corresponding cell segmentation mask, while (c)(C) and (c)(E) demonstrate the fiber density (both manual and automatic), calculated as the number of cells (fractionally counted for those cut by the window) divided by the window area. Further, (c)(D) indicates the fascicle area, representing the area occupied by each cell; (c)(F) and (c)(G) depict axon and myelin packing respectively, calculated as the total axon/myelin area within the window divided by the window area; and (c)(H) fiber Nearest-Neighbor (NN) area, a crucial metric that assesses the area of each myelinated fiber’s cell within the fascicle partition. Panel (d) shows the obtained damage metrics visualized on the segmented nerve images for clarity.
Upon segmentation of the nerve cross-sectional images, we proceeded to employing these to calculate specific damage indicators at the cellular level. This process is summarized graphically in Fig. 2(c), where the microscopy image of cells, segmented masks, and various fiber damaging metrics are computed (Fig. 2(c)(A-G)). To be more specific, the fiber damaging metrics calculated include fiber density, fascicle area, axon packing, myelin packing, and fiber Nearest-Neighbor (NN) area. Fiber packing was quantified by computing the ratio of cell area to fascicle area within the same window. Double-counting of cells partially intersecting two adjacent windows were mitigated by proportionally attributing cell area to each block. As an example, a cell having one-third of its area in a block was counted as one-third of a cell. Thus, fiber packing is a dimensionless measure, ranging from 0 to 1. Fiber density was deduced by dividing the cell count within the measurement window by the corresponding fascicle area within that window. Only the fraction of fiber area within the window was counted when a window intersected a fiber. Window area values were adjusted to account for overlaps with regions outside fascicles or those occupied by blood vessels, areas typically devoid of neurons. NN area is another important damage metrics that we employed, which calculates the area of the Voronoi cell, of the partition of fascicle area, for each myelinated fiber. We finalized this stage by mapping the resulting damage metrics onto the segmented nerve images as shown in Fig. 2(d), thereby offering a precise visualization of nerve fiber morphology.
Achieving comprehensive segmentations and computing damage-related metrics for each cell across all nerve samples provides the necessary data for the computation of cell-wise charge density values to establish a correlation between induced electrical parameters and damage, and ultimately derive a criterion for the safety analysis of PNS interfaces.
C. Charge density from AM modeling and MLP prediction
Our objective is to apply computational modeling to enable the development of a new safety criterion for PNS based on experimental data. In order to achieve this, both a cell-bycell damage analysis and cell-bycell current density values from computational modeling are needed. However, given the excessive computational cost of the Admittance Method (AM) simulation for extremely high-resolution nerve images, it’s impractical to run AM simulations for each nerve sample.
Thus, an effective solution is to apply a neural networkbased predictive model. As detailed in our previous study (referenced as [45]), we found that three key factors primarily influence the charge density induced in the computational cells of the tissue: stimulation intensity, the distance between the considered location in the tissue and the electrode, and the dielectric properties of the considered cell. Using this knowledge, we performed AM modeling on two selected nerve sample and trained a multi-layer perceptron model which utilizes the information from the simulated datasets to predict charge density values in unknown datasets based on these factors.
1). Nerve Sample selection(Build training dataset):
The first step towards constructing such a model is assembling a suitable training dataset. We have so far gathered 38 control, sham and stimulated nerve samples. Control samples are healthy nerve samples, sham samples are samples where electrodes were planted but not stimulated, stimulated samples are electrically stimulated samples with difference current intensity. Each sample exhibits varying levels of damage. To ensure that our training dataset represents the whole spectrum of possible variations while avoiding bias, we need to select representative samples for AM modeling.
To address possible discrepancies between individual samples, we focused on fiber packing and fiber density, metrics that have been validated as useful indicators of nerve damage in previous studies [46–48]. These indicators were key in ensuring that the chosen images correctly represent their respective classes.
Using these metrics, we chose two illustrative samples from our collection: one healthy (with no significant morphological and structural changes relative to a non-stimulated rat), and one damaged (displaying noticeable morphological and structural changes when compared to a non-stimulated rat). The selected healthy nerve image is sourced from control samples with implanted, yet non-stimulated cuff electrodes, while the selected damaged nerve image is taken from a sample stimulated using a 1.2mA current source, as illustrated in Fig. 3(a)
Fig. 3.
(a)(b) Selection and segmentation of two representative nerve samples, one healthy and one damaged, based on fiber packing and fiber density metrics. (c) Construction and computational modeling of the two selected nerve samples for Admittance Method (AM) simulation. (d) Training of a multi-layer perceptron (MLP) model using the dataset derived from the AM simulation of the two representative samples to predict cell-wise charge density values. The MLP model, consisting of a four-layer fully connected neural network with hidden layers of 512, 256, and 128 neurons respectively, utilizes a softmax activation function for charge density classification.
2). AM Modeling on the selected samples:
Upon selecting our two samples, we employed the admittance method to conduct simulations.
The peripheral nerve stimulation simulations utilized a multi-scale computational model that comprises two principal components: a segmented nerve model and a cuff electrode model. The cuff electrodes are designed to represent typical commercial cuff electrodes, with an inner diameter of approximately 2mm and two metal contact wires, one serving as a source cuff electrode and the other as a ground cuff electrode. The computational model includes only the metallic parts of the electrode, ignoring non-conductive elements not in direct contact with the tissue, such as the surrounding insulation layer. These components have negligible effects on the current distribution in the tissue with our modeling. In the in vivo experiments, the nerve is sutured to the electrode, placing the nerve model in close contact with nerve. The final constructed model is depicted in Fig. 3(c).
The simulation setup remains consistent for both healthy and damaged nerve models, maintaining identical cuff electrodes and position. The models are partitioned into cubic voxels, with each voxel corresponding to a distinct material index. The dimensions of the final model are 800×800×1000 voxels in the x, y, and z directions respectively, with a resolution of 3.1 μm across all three dimensions. The nerve model’s material attributes are extracted from the studies by [49, 50], as outlined in Table 1.
TABLE I.
TISSUE PROPERTIES
| Tissue Type | σx | σy | σz | Unit |
|---|---|---|---|---|
| Perineurium | 0.01 | 0.01 | 0.01 | S/m |
| Myelination | 2 × 10−4 | 5 × 10−9 | 5 × 10−9 | S/m |
| Intracellular space | 0.33 | 0.33 | 0.33 | S/m |
| Axoplasm | 0.91 | 0.91 | 0.91 | S/m |
| Epineurium | 0.1 | 0.1 | 0.1 | S/m |
| Nerve membrane | 0.02 | 0.02 | 0.02 | S/m |
| Saline solution | 1.45 | 1.45 | 1.45 | S/m |
| Extracellular space | 0.33 | 0.33 | 0.33 | S/m |
The AM with its multi-resolution feature detailed in [32, 34, 39, 40], is employed to calculate the electric field values at each computation grid node [41–44]. Briefly, this method relies on the construction of a matrix, G, representing the admittances throughout the model, which encompasses two primary aspects: the admittances connected to each node (depicted by diagonal components of the matrix) and the internodal admittance (expressed by the off-diagonal elements). Owing to the inherent structure of the network model, the admittance matrix, G, manifests as a sparse, symmetric matrix. Admittance values are determined by considering the conductivity and the nodal distance in the , , and axes, as elaborated in Equations (1)-(3). In these equations, the conductivities along the , , directions are represented by , , respectively, and , , stand for the internodal distances along corresponding directions.
| (1) |
| (2) |
| (3) |
For effective matrix generation and equation resolution, we have crafted a multithreaded computational platform in Python, employing a biconjugate gradient method. Upon defining the admittance matrix (G) and the current vector (I), we solve the equation GV = I to determine the vector of induced voltage, V, at every node of the model.
Prior to running electric field simulations, we utilize a 3D multi-resolution meshing method to diminish computational time, while retaining accuracy in the region with relatively higher currents. Given, for the considered set-up, the low currents in the inner parts of the the nerve, the need for fine resolution diminishes in distal regions from the nerve periphery. Hence, the computational mesh resolution is coarser at the nerve’s core and finer near the nerve’s edge (i.e., proximal to the electrodes and fascicle boundaries). This method significantly curtails computational time by reducing the total node count in the model.
Voltage values are computed at every node of the model, situated at the voxel vertices. A trilinear interpolation is applied to estimate the voltage at various points within a voxel, basing calculations on the values at its vertices, as the conductivity value within each voxel is considered constant. Employing these interpolated voltage values, additional parameters such as electric field, charge density, and current density, at any location within the model can be calculated.
3). Neural network training and charge density prediction:
The AM simulation results are used as the training dataset for the predictive model. In the imaging dataset, cell-by-cell current density values are unknown. However, the distance of each cell from the electrode, its material type, and stimulation level are known. With this information, it is possible to train a neural network to predict the current density values for each unknown computational cell.
Our training dataset, based on two sample types, provides us with over one billion simulated voxels, offering a vast volume of training data. From this, our analysis identifies several primary factors contributing significantly to the cellularlevel distribution of current density: the cell proximity to the electrode, the surrounding material type, and the stimulation level. These insights are supported by our prior studies [45][51][52]. Despite the undetermined exact cell-level current density values within our imaging dataset, we leverage these cellular attributes to construct a multi-layer perceptron (MLP) [53]. Our MLP consists of a four-layer fully connected neural network that accepts three attributes as input. It comprises hidden layers with 512, 256, and 128 neurons respectively and uses a softmax activation function in the output layer for charge density value classification, as illustrated in Fig. 3(d). The training data input was pre-sorted into five classes according to the charge density ranges. Through this method, we successfully approximated cell-wise charge density values across various nerve samples, a task that would otherwise be impractical.
Leveraging a dataset of approximately 30,000 individual cells derived from two nerve samples, we divided the data using a 70/30 split for training and validation purposes. The performance of the resulting model was evaluated against an independent nerve model with full-resolution AM simulation.
III. Results
In summary, our methodology centers around a systematic analysis of cell damage in response to varying charge densities. We initiated our process with image segmentation, subsequently carrying out a damage assessment for each image, thereby enabling us to calculate the extent of damage on a cell-by-cell basis across the entire image set. Simultaneously, we utilized the Admittance Method (AM) modeling and neural network prediction on the segmented images, associating each cell with a distinct charge density value. This facilitated the formulation of a relationship between the cell-wise damage indicators and charge density values. Having established this correlation, we were then able to examine the extent of damage across a spectrum, ranging from unstimulated (healthy) cells to cells exposed under varying levels of stimulation, based on their specific cell-wise charge densities. This analysis empowered us to propose safety criteria based on the correlation between a cell’s charge density value and its extent of damage. The results we present include the categorization of nerve cells according to their corresponding charge densities, along with the accuracy of these classifications. By utilizing empirical distribution function (EDF) plots, we demonstrate a range of damage indicators specific to each cell group, distinguished by their unique charge density levels. These EDF plots allowed us to identify the charge density group with the most substantial morphological alterations, thereby enabling us to propose a potential safety threshold. This proposed boundary is rooted in the observation that cells at this specific charge density exhibit an increased likelihood of damage, thereby warranting the establishment of safety criteria centered around this particular level.
In contrast to the traditional Shannon Criteria, our proposed safety limit offers a more comprehensive risk assessment. By providing quantitative data corresponding to different stimulation intensities, we’re able to pave the way for safer practices and strategies. This vital information could also contribute to the design of future nerve stimulation therapies and devices, advancing both efficacy and safety within the field.
A. Cell-wise charge density prediction results
In our study, we selected two representative nerve samples for comprehensive modelling. Through this process, we procured over 100,000 cells, forming a rich dataset for our analysis. We strategically partitioned this dataset into three subsets: 70% allocated for training the model, 15% for validation to fine-tune the model parameters, and the remaining 15% reserved for final testing to assess the model’s performance on unseen data.
Our model, a multi-layer perceptron network, was trained over 1,000 epochs. This prolonged training process allowed the model to learn intricate patterns and relationships within the data. We observed that, following this training regimen, the accuracy for both training and validation datasets plateaued, reaching a commendable level of approximately 90%, as shown in Fig. 4(a). This level of performance indicated that the model was able to generalize well and was not overfitting the data, i.e., the model was not merely memorizing the training examples but effectively learning underlying patterns, enabling accurate predictions on unseen data.
Fig. 4.
Depiction of the neural network model training and performance assessment. (a) Illustrates the training and validation process for our multi-layer perceptron network, performed over 1,000 epochs. This exhaustive training led to a peak model accuracy of approximately 90%, demonstrating its ability to discern underlying patterns and avoid overfitting. (b) Showcases the classification performance of our model on a diverse dataset of over 100,000 cells, represented through a confusion matrix. The matrix elucidates the model’s misclassifications primarily between neighboring classes, with notably fewer misclassifications in the highest charge density range - a critical determinant for gauging potential cellular damage.
Our neural network allows us to organize cells from our large set of nerve samples systematically by calculating current density values. The neural network aids in allocating each cell to one of five categories, each representing a specific range of charge densities: 0–0.5, 0.5–1.5, 1.5–2.5, 2.5–3.5, and 3.5nC/cm2 and above. The five categories are determined based on our simulations as well as the results from our previous study [45].
We further tested the model on the test dataset, revealing a similar accuracy level to the training and validation sets. This consistency underscored the model’s robustness and ability to maintain performance across various data subsets. As we were dealing with a multi-class classification problem involving five distinct classes, we generated a confusion matrix to analyze the classification performance in detail, as shown in Fig. 4(b).
The confusion matrix is a powerful tool for visualizing the performance of a classification model, revealing the precise locations of misclassifications. Our matrix showed that the primary source of misclassifications was between neighboring classes, which is a common occurrence in classification problems due to similarities between adjacent classes.
Remarkably, we found that the class representing the highest charge density range exhibited very few misclassifications. This finding is particularly crucial for our study, as accurate classification of cells within this range is vital for understanding potential cellular damage due to high charge densities.
B. Correlation curves between cell damage and charge density
Following the neural network classification process, we next turned our focus towards the visual representation and analysis of these cell groupings, employing empirical distribution function plots mapped against various damage metrics.
We again categorized the cells according to their charge density values, underscoring the significance of local charge density rather than the original nerve sample for the groupings. Notably, the categorization of cells is independent of their originating nerve sample, as cells from disparate samples can affiliate with the same group if their local charge density values align closely. During the initial segmentation of our dataset, three principal groups are known based on the samples themselves: stimulated cells, control cells, and sham cells. Further exploration within the stimulated cell group unveiled five distinct subcategories, each characterized by their individual charge densities as previously enumerated.
Subsequently, we deployed empirical distribution function plots to visually represent these seven cellular groups against various damage metrics. The interpretation of these plots uncovers intriguing patterns: the distribution curves for control and sham cells predominantly reside on the right, thus representing the archetype of a healthy cell distribution. Conversely, cell groups displaying elevated charge density values display a left-leaning curve, indicative of morphological alterations and potential damage, as shown in Fig. 5.
Fig. 5.
Analysis of cell groupings based on charge density against various damage metrics. (a) Axon packing, (b) Axon size, (c) Fiber packing, and (d) Fiber Nearest-Neighbor (NN) ratio. The empirical distribution function plots visually represent seven distinct cellular groups, including stimulated cells, control cells, sham cells, and five subcategories within the stimulated cell group defined by their charge density. The curves indicate that control and sham cell groups, typically resembling healthy cells, tend to reside on the right, whereas cell groups with higher charge densities skew left, suggestive of morphological changes and potential damage. Most notably, cells with a local charge density per phase exceeding 3.5 nC/cm2 consistently deviate from other categories across all damage metrics, underscoring the potential safety threshold.
These curve distributions across different charge density levels can fluctuate based on the selected damage metrics. However, one consistent feature prevails: the curve corresponding to cells with a local charge density per phase of 3.5 nC/cm2 or greater is consistently positioned far to the left, indicating significant differences from other curves. These cells markedly diverge from other categories across all damage metrics, suggesting an increased risk of damage. Given this result, it appears advisable to introduce a safety threshold for the local charge density per phase of 3.5 nC/cm2 since this could induce substantial morphological changes in the cell.
C. Comparison with Shannon Criteria and future safety criteria
Our proposed safety limit offers a compelling comparative perspective to the traditional Shannon Criteria. To elucidate this, for each nerve sample identified by a unique Shannon k value ascertained at the time of the experiment, we plot the proportion of cells for each sample that exhibit a local charge density per phase exceeding 3.5 nC/cm2.
For the nerve sample characterized by Shannon k=2.11, a remarkable 98.88% of cells display a local charge density value beyond 3.5 nC/cm2. This high proportion signifies a substantial risk of damage for virtually all cells in this nerve sample, aligning with Shannon Criteria that deems a K value of 2 as surpassing the generally accepted safety limit of 1.5.
In contrast, the nerve sample with Shannon k=0.55, generally considered as relatively safe by Shannon Criteria, exhibits an unexpected pattern under our criteria. Over 59% of the cells within this sample exceed the safety limit. A similar trend is observed for the nerve sample with Shannon k=0.11, where 12.42% of cells surpass a local charge density value of 3.5 nC/cm2, implying a potential risk of damage for roughly 12% of cells, a level far from being considered very safe, as shown in Fig. 6.
Fig. 6.
Comparative analysis of proposed safety criteria and traditional Shannon Criteria for nerve stimulation. The figure illustrates the proportion of cells for each nerve sample, identified by a unique Shannon k value, exceeding a local charge density per phase of 3.5 nC/cm2. The plot reveals nerve samples with Shannon k=2.11, k=0.55, and k=0.11 and the proportion of cells within each that surpass the proposed safety threshold. Our criteria unveil potential risks not identified by the Shannon Criteria, providing a more nuanced understanding of the damage risk under stimulation. The visualization underscores the advantages of our proposed method, offering a more granular perspective that can assist experimentalists in making more informed decisions on experimental parameters.
These findings underscore the capability of our proposed criteria to shed light on the potential damaging risk of a nerve under stimulation, presenting a more nuanced understanding than what the traditional Shannon Criteria can offer. Compared to the Shannon Criteria, our proposed stimulation safety limit of 3.5 nC/cm2 provides an enhanced level of detail regarding the safety of nerve stimulation at given intensities. While the Shannon Criteria presents a single value and somewhat ambiguous limit, our proposed method delivers a quantitative measure of damage risk. This granular data enables experimentalists to make more informed decisions regarding their experimental parameters.
To make our safety criteria easier to use, we fitted our stimulated samples to a curve that could use as a model for practitioners in the case of stimulating nerve samples using cuff electrodes and subsequently quantifying the percentage of cells with a charge density exceeding the proposed safety limit, as shown in Fig. 7.
Fig. 7.
Empirical model demonstrating the relationship between stimulation intensity and the risk of excessive charge density to nerve cells. This curvefitting process visualizes the correlation between the controllable stimulation intensity from the source cuff electrode (x-axis) and the percentage of nerve cells exceeding our proposed safety limit (y-axis), providing a tool for practitioners to estimate potential cell damage during peripheral nerve stimulation.
In this figure, the y-axis represents the percentage of nerve cells exceeding our safety limit, while the x-axis corresponds to the controllable stimulation intensity from the source cuff electrode. Utilizing these parameters and data points, we performed curve-fitting procedures to ascertain the relationship between these two variables. The resulting curve presents an empirical model, encapsulating the observed correlation between stimulation intensity and the risk of excessive charge density to nerve cells.
Although still in its early stages, this safety curve provides valuable insight into a potential tool we aim to develop for practitioners, particularly those working with peripheral nerve stimulation via cuff electrodes and more. The usability and significance of this approach are evident even at this preliminary stage. Looking ahead, the proposed approach, with a more expansive dataset and increased variety in experimental procedures, nerve and electrode types, etc., can provide an updated and more robust neural network, paving the way for broader application in the field.
IV. Discussion
The aim of this research is to advance the development of a more accurate and up-to-date safety standard for peripheral nerve stimulation. Despite its long-standing use, the Shannon criteria, a conventional safety guideline often used for such stimulation, is burdened with shortcomings. These include a lack of data specific to the peripheral nervous system, an overreliance on manual damage analysis, which can be prone to inconsistency, and a flawed assumption of uniform charge density. To create an improved safety standard for peripheral nerve stimulation, several elements should be considered: utilization of data that is specific to peripheral nerves, the application of algorithm-based assessments of tissue damage to reduce human (i.e., subjective) errors, and the inclusion of accurate current density computations, rather than assuming a uniform distribution. In our efforts to define this next-generation safety standard, we have incorporated these important considerations.
Evaluating nerve damage induced by electrical stimulation often hinges on the crucial factor of current density. However, obtaining accurate estimations of current density distribution within a nerve poses a significant challenge. To overcome this, we utilize a high-resolution computational approach, the Admittance Method (AM), which constitutes a significant part of our multi-scale computational platform. This approach is capable of producing exceptionally detailed results for current density calculations. Despite its efficacy, applying the AM to the current high-resolution nerve models necessitated the development of a multi-resolution adaptive meshing algorithm to reduce the computing complexity along with an averaging filter to improve the accuracy.
The computational complexity of conducting Admittance Method (AM) modeling for each nerve sample necessitates an alternative approach. The neural network prediction approach described in this study constitutes a viable alternative; it allowed us to identify several important factors such as distance from the electrode, material type, and stimulation level that influence cell-by-cell current density distribution. These characteristics can assist us in predicting current density on cells without conducting full AM simulations. Despite the promising results, our current predictive model exhibits limitations rooted in the homogeneity of the nerve samples under study. Our dataset primarily consists of identical nerve types subjected to similar experimental procedures, conditions under which the neural network thrived. However, if the experimental environment or nerve sample characteristics vary significantly, the performance of our current model might waver. Further, while the study indicates that fiber packing and myelination levels impact micro-dosimetric exposure and correlation between altered packing and myelination, further studies will be required to establish causality. Additionally, the study relies on high-resolution models, which may not capture the variability in tissue properties between electrodes and cells, the impact of scar tissue formation and capacitive effects at the nerve-electrode interface. To overcome these limitations and enhance our methodology, we intend to leverage larger, more diverse datasets in future studies. Incorporating variables such as differing experimental procedures, various nerve and electrode types will enrich our data collection, introducing a new dimension of complexity to our model. Consequently, a robust and versatile neural network will emerge, capable of delivering more reliable predictions with a wider range of applicability. The potential of our approach lies in its inherent adaptability to new datasets and conditions, signifying its capability to evolve with expanding data resources. As we advance and refine our models, we are optimistic about unveiling further insights into the intricate relationships between charge density induced by Peripheral Nerve Stimulation (PNS) and nerve cell health.
We chose the rat sciatic nerve as the initial test bed for this study due to its well-documented anatomy and accessibility for experimental manipulation, making it a standard model in peripheral nerve research. The sciatic nerve’s relatively large size and mix of myelinated and unmyelinated fibers provide a robust foundation for developing and validating our computational models. Analyses involving nerves with different proportions of myelinated versus unmyelinated fibers will have to be performed to determine the generalizability of the proposed predictive algorithms to account for differences in electrical conductivity and tissue response. Our research marks a significant but still initial step toward developing comprehensive safety guidelines for peripheral nerve stimulation. Although our safety criteria are in their early stages, we have a solid framework in place to build upon. The present study primarily utilizes data from the sciatic nerves of rats. While this has provided essential insights, it may limit the applicability of our results to other nerve types or species. Therefore, we plan to incorporate data from the pig vagus nerve in future studies to include a wider range of species and nerve types. This expansion is crucial to our overarching goal: enhancing the practical utility of our research, and ultimately developing robust safety criteria applicable to PNS stimulation in human subjects.
HIGHLIGHTS:
one main highlight is our approach, which uniquely integrates three steps: experimentation on rats, the creation of a high-resolution nerve model through direct experiments, and predictive simulation. Another highlight is the updated PNS safety criteria that this study proposes.
Acknowledgment
This work was supported by the NIBIB of the National Institute of Health Grant No. R01EB029271, and an unrestricted grant to the Department of Ophthalmology from Research to Prevent Blindness, New York, NY.
Biographies
Michael Shell Biography text here.
John Doe Biography text here.
Jane Doe Biography text here.
References
- [1].Yue Lan et al. “Retinal stimulation strategies to restore vision: Fundamentals and systems”. In: Progress in retinal and eye research 53 (2016), pp. 21–47. [DOI] [PubMed] [Google Scholar]
- [2].Benabid Alim Louis et al. “Deep brain stimulation of the subthalamic nucleus for the treatment of Parkinson’s disease”. In: The Lancet Neurology 8.1 (2009), pp. 67–81. [DOI] [PubMed] [Google Scholar]
- [3].Nelson Timothy S et al. “Closed-loop seizure control with very high frequency electrical stimulation at seizure onset in the GAERS model of absence epilepsy”. In: International journal of neural systems 21.02 (2011), pp. 163–173. [DOI] [PubMed] [Google Scholar]
- [4].Weiner Richard L. “The future of peripheral nerve neurostimulation”. In: Neurological Research 22.3 (2000), pp. 299–304. [DOI] [PubMed] [Google Scholar]
- [5].Doucet Barbara M, Lam Amy, and Griffin Lisa. “Neuromuscular electrical stimulation for skeletal muscle function”. In: The Yale journal of biology and medicine 85.2 (2012), p. 201. [PMC free article] [PubMed] [Google Scholar]
- [6].Stein Richard B, Peckham P Hunter, and Popovic Dejan´. Neural prostheses: replacing motor function after disease or disability. Oxford University Press, 1992. [Google Scholar]
- [7].Grill Warren M and Kirsch Robert F. “Neuroprosthetic applications of electrical stimulation”. In: Assistive Technology 12.1 (2000), pp. 6–20. [DOI] [PubMed] [Google Scholar]
- [8].Zhang Dingguo et al. “Quantifying different tactile sensations evoked by cutaneous electrical stimulation using electroencephalography features”. In: International journal of neural systems 26.02 (2016), p. 1650006. [DOI] [PubMed] [Google Scholar]
- [9].Rajdev Pooja, Ward Matthew, and Irazoqui Pedro. “Effect of stimulus parameters in the treatment of seizures by electrical stimulation in the kainate animal model”. In: International journal of neural systems 21.02 (2011), pp. 151–162. [DOI] [PubMed] [Google Scholar]
- [10].Pineau Joelle et al. “Treating epilepsy via adaptive neurostimulation: a reinforcement learning approach”. In: International journal of neural systems 19.04 (2009), pp. 227–240. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [11].Kimiskidis Vasilios K et al. “Transcranial magnetic stimulation (TMS) modulates epileptiform discharges in patients with frontal lobe epilepsy: a preliminary EEG-TMS study”. In: International journal of neural systems 23.01 (2013), p. 1250035. [DOI] [PubMed] [Google Scholar]
- [12].Yaqub M Atif et al. “Control of transcranial direct current stimulation duration by assessing functional connectivity of near-infrared spectroscopy signals”. In: International Journal of Neural Systems 32.01 (2022), p. 2150050. [DOI] [PubMed] [Google Scholar]
- [13].Shi Yuxi et al. “Galvanic Vestibular Stimulation-Based Prediction Error Decoding and Channel Optimization”. In: International Journal of Neural Systems 31.11 (2021), p. 2150034. [DOI] [PubMed] [Google Scholar]
- [14].Nayak Raghavendra and Banik Ratan K. “Current innovations in peripheral nerve stimulation”. In: Pain research and treatment 2018 (2018). [DOI] [PMC free article] [PubMed] [Google Scholar]
- [15].Chakravarthy Krishnan et al. “Review of recent advances in peripheral nerve stimulation (PNS)”. In: Current pain and headache reports 20 (2016), pp. 1–7. [DOI] [PubMed] [Google Scholar]
- [16].Slavin Konstantin V. “Peripheral nerve stimulation for neuropathic pain”. In: Neurotherapeutics 5.1 (2008), pp. 100–106. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [17].Eldabe Sam, Buchser Eric, and Duarte Rui V. “Complications of spinal cord stimulation and peripheral nerve stimulation techniques: a review of the literature”. In: Pain Medicine 17.2 (2016), pp. 325–336. [DOI] [PubMed] [Google Scholar]
- [18].Nashold BS Jr et al. “Long-term pain control by direct peripheral-nerve stimulation.” In: JBJS 64.1 (1982), pp. 1–10. [PubMed] [Google Scholar]
- [19].Mobbs RJ, Nair S, and Blum P. “Peripheral nerve stimulation for the treatment of chronic pain”. In: Journal of clinical neuroscience 14.3 (2007), pp. 216–221. [DOI] [PubMed] [Google Scholar]
- [20].Campbell James N and Long Donlin M. “Peripheral nerve stimulation in the treatment of intractable pain”. In: Journal of neurosurgery 45.6 (1976), pp. 692–699. [DOI] [PubMed] [Google Scholar]
- [21].Günter Clara, Delbeke Jean, and Ortiz-Catalan Max. “Safety of long-term electrical peripheral nerve stimulation: review of the state of the art”. In: Journal of neuroengineering and rehabilitation 16 (2019), pp. 1–16. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [22].Silberstein Stephen D et al. “Safety and efficacy of peripheral nerve stimulation of the occipital nerves for the management of chronic migraine: results from a randomized, multicenter, double-blinded, controlled study”. In: Cephalalgia 32.16 (2012), pp. 1165–1179. [DOI] [PubMed] [Google Scholar]
- [23].Helm Standiford et al. “Peripheral nerve stimulation for chronic pain: a systematic review of effectiveness and safety”. In: Pain and Therapy 10 (2021), pp. 985–1002. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [24].Shannon Robert V. “A model of safe levels for electrical stimulation”. In: IEEE Transactions on biomedical engineering 39.4 (1992), pp. 424–426. [DOI] [PubMed] [Google Scholar]
- [25].McCreery Douglas B et al. “Charge density and charge per phase as cofactors in neural injury induced by electrical stimulation”. In: IEEE Transactions on Biomedical Engineering 37.10 (1990), pp. 996–1001. [DOI] [PubMed] [Google Scholar]
- [26].Wei Xuefeng F and Grill Warren M. “Analysis of highperimeter planar electrodes for efficient neural stimulation”. In: Frontiers in neuroengineering (2009), p. 15. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [27].McIntyre Cameron C and Grill Warren M. “Finite element analysis of the current-density and electric field generated by metal microelectrodes”. In: Annals of biomedical engineering 29 (2001), pp. 227–235. [DOI] [PubMed] [Google Scholar]
- [28].Paknahad Javad et al. “Mechanisms underlying activation of retinal bipolar cells through targeted electrical stimulation: a computational study”. In: Journal of Neural Engineering (2021). [DOI] [PMC free article] [PubMed] [Google Scholar]
- [29].Kosta Pragya et al. “Model-based comparison of current flow in rod bipolar cells of healthy and early-stage degenerated retina”. In: Experimental Eye Research 207 (2021), p. 108554. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [30].Kosta Pragya, Loizos Kyle, and Lazzi Gianluca. “Stimulus waveform design for decreasing charge and increasing stimulation selectivity in retinal prostheses”. In: Healthcare Technology Letters 7.3 (2020), pp. 66–71. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [31].Stang John et al. “Recent advances in computational and experimental bioelectromagnetics for neuroprosthetics”. In: 2019 International Conference on Electromagnetics in Advanced Applications (ICEAA). IEEE. 2019, pp. 1382–1382. [Google Scholar]
- [32].Bingham Clayton S et al. “Admittance method for estimating local field potentials generated in a multi-scale neuron model of the hippocampus”. In: Frontiers in Computational Neuroscience 14 (2020), p. 72. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [33].Bingham Clayton S et al. “Model-based analysis of electrode placement and pulse amplitude for hippocampal stimulation”. In: IEEE Transactions on Biomedical Engineering 65.10 (2018), pp. 2278–2289. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [34].Cela Carlos J. “A multiresolution admittance method for large-scale bioelectromagnetic interactions”. PhD thesis. North Carolina State University, 2010. [Google Scholar]
- [35].Raspopovic S, Capogrosso M, and Micera S. “A computational model for the stimulation of rat sciatic nerve using a transverse intrafascicular multichannel electrode”. In: IEEE Transactions on Neural Systems and Rehabilitation Engineering 19.4 (2011), pp. 333–344. [DOI] [PubMed] [Google Scholar]
- [36].Kosta P et al. “Simulation-Based Optimization of Figure-ofEight Coil Designs and Orientations for Magnetic Stimulation of Peripheral Nerve”. In: IEEE Transactions on Neural Systems and Rehabilitation Engineering (2020). [DOI] [PubMed] [Google Scholar]
- [37].Kosta P, Warren DJ, and Lazzi G. “Selective stimulation of rat sciatic nerve using an array of mm-size magnetic coils: a simulation study”. In: Healthcare technology letters 6.3 (2019), pp. 70–75. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [38].RamRakhyani AK et al. “A μm-Scale Computational Model of Magnetic Neural Stimulation in Multifascicular Peripheral Nerves”. In: IEEE Transactions On Biomedical Engineering 62.12 (2015), pp. 2837–2849. [DOI] [PubMed] [Google Scholar]
- [39].Loizos Kyle et al. “A multi-scale computational model for the study of retinal prosthetic stimulation”. In: 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society. Chicago, IL, USA, 2014, pp. 6100–6103. DOI: 10.1109/EMBC.2014.6945021. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [40].Eberdt Michael, Brown Patrick K, and Lazzi Gianluca. “Twodimensional SPICE-linked multiresolution impedance method for low-frequency electromagnetic interactions”. In: IEEE transactions on biomedical engineering 50.7 (2003), pp. 881–889. [DOI] [PubMed] [Google Scholar]
- [41].Stang John et al. “Recent Advances in Computational and Experimental Bioelectromagnetics for Neuroprosthetics”. In: 2019 International Conference on Electromagnetics in Advanced Applications (ICEAA). 2019, pp. 1382–1382. [Google Scholar]
- [42].Paknahad Javad et al. “Responsiveness of Retinal Ganglion Cells Through Frequency Modulation of Electrical Stimulation: A Computational Modeling Study*”. In: 2020 42nd Annual International Conference of the IEEE Engineering in Medicine Biology Society (EMBC). 2020, pp. 3393–3398. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [43].Paknahad Javad et al. “Color and cellular selectivity of retinal ganglion cell subtypes through frequency modulation of electrical stimulation”. In: Scientific reports 11.1 (2021), pp. 1–13. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [44].Paknahad Javad et al. “Targeted Stimulation of Retinal Ganglion Cells in Epiretinal Prostheses: A Multiscale Computational Study”. In: IEEE Transactions on Neural Systems and Rehabilitation Engineering 28.11 (2020), pp. 2548–2556. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [45].Du Jinze et al. “Electrical Stimulation Induced Current Distribution in Peripheral Nerves Varies Significantly with the Extent of Nerve Damage: A Computational Study Utilizing Convolutional Neural Network and Realistic Nerve Models”. In: International Journal of Neural Systems 33.04 (2023). p. 2350022. DOI: 10.1142/S0129065723500223. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [46].Christensen Michael B and Tresco Patrick A. “Differences exist in the left and right sciatic nerves of naıve rats and cats”. In: The Anatomical Record 298.8 (2015), pp. 1492–1501. [DOI] [PubMed] [Google Scholar]
- [47].Comin César H et al. “Statistical physics approach to quantifying differences in myelinated nerve fibers”. In: Scientific reports 4.1 (2014), pp. 1–11. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [48].Sandell Julie H and Peters Alan. “Effects of age on nerve fibers in the rhesus monkey optic nerve”. In: Journal of Comparative Neurology 429.4 (2001), pp. 541–553. [DOI] [PubMed] [Google Scholar]
- [49].McNeal Donald R. “Analysis of a model for excitation of myelinated nerve”. In: IEEE Transactions on Biomedical Engineering 4 (1976), pp. 329–337. [DOI] [PubMed] [Google Scholar]
- [50].Butson Christopher R et al. “Selective neural activation in a histologically derived model of peripheral nerve”. In: Journal of neural engineering 8.3 (2011), p. 036009. [DOI] [PubMed] [Google Scholar]
- [51].Du Jinze et al. “Electrode Spacing and Current Distribution in Electrical Stimulation of Peripheral Nerve: A Computational Modeling Study using Realistic Nerve Models”. In: 2021 43rd Annual International Conference of the IEEE Engineering in Medicine Biology Society (EMBC). 2021, pp. 4416–4419. DOI: 10.1109/EMBC46164.2021.9631068. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [52].Du Jinze et al. “Electrical Stimulation Induced Current Distribution in Peripheral Nerves Varies Significantly with the Extent of Nerve Damage: A Computational Study Utilizing Convolutional Neural Network and Realistic Nerve Models”. In: Artificial Intelligence in Neuroscience: Affective Analysis and Health Applications. Ed. by Vicente José Manuel Ferrández et al. Cham: Springer International Publishing, 2022, pp. 526–535. ISBN: 978–3-031–06242-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [53].Ruck Dennis W, Rogers Steven K, and Kabrisky Matthew. “Feature selection using a multilayer perceptron”. In: Journal of neural network computing 2.2 (1990), pp. 40–48. [Google Scholar]







