Probabilistic and deep learning approaches for conductivity-driven nanocomposite classification

Abstract

To foster greater trust and adoption of machine learning models, particularly neural networks, it is essential to develop approaches that quantify and report epistemic uncertainties alongside random uncertainties, which often affect the accuracy of Recurrent Neural Networks (RNNs). Addressing these challenges, this study proposes a hybrid approach integrating Bayesian techniques and deep learning to improve the classification of nanocomposites with a focus on evaluating their conductivity properties. The proposed framework begins with a Bayesian Network (BN) model, which provides probabilistic insights into the conductive behavior of nanocomposites by analyzing the distribution and interaction of their constituent nanoparticles. This probabilistic foundation is complemented by a Recurrent Neural Network (RNN) based on the Transformer architecture, which enhances classification accuracy by capturing sequential dependencies and complex data patterns. The hybrid model combines the probabilistic reasoning capabilities of BNs with the deep learning strengths of RNNs, yielding a more robust and adaptable classification methodology. While this study primarily focuses on methodological advancements, experimental results demonstrate that the hybrid model significantly outperforms individual approaches in terms of key evaluation metrics. This integrated framework thus represents a promising step toward improving the predictive classification of nanocomposite conductivity, offering a balance between probabilistic interpretability and data-driven accuracy.

Introduction

Neural networks have revolutionized the field of machine learning, driving advancements in numerous domains such as computer vision, natural language processing, and healthcare¹. However, their broader adoption remains hindered by two critical challenges: the lack of interpretability and the absence of robust confidence estimates in predictions^2–4. Interpretability is essential for gaining insights into model decisions, particularly in high- stakes applications, while confidence estimates are crucial for assessing the reliability of predictions. Addressing these limitations is vital for fostering trust in machine learning systems and ensuring their safe deployment in critical scenarios such as autonomous vehicles⁵, medical diagnostics⁶, and risk-sensitive financial models.

Bayesian deep learning has emerged as a promising approach to overcoming these limitations by incorporating uncertainty estimation directly into neural network frameworks. By explicitly modeling uncertainties, Bayesian methods enhance the reliability of predictions and provide actionable insights, thereby making machine learning safer and more trustworthy for real-world applications^7,8. Despite notable progress, the application of deep neural networks in binary classification tasks faces inherent challenges, such as the unbounded nature of the logistic loss function, which complicates the analysis of model generalization and performance^2,9. This highlights the need for improved methodologies that can account for uncertainties while maintaining high accuracy.

In machine learning, uncertainty can be categorized into two types: epistemic uncertainty and aleatoric uncertainty. Epistemic uncertainty, which arises due to limited knowledge or data, can be reduced with additional data or improved model structures. In contrast, aleatoric uncertainty is inherent to the data and cannot be reduced¹⁰. While traditional neural networks primarily provide class-dependent likelihoods, they fall short of delivering meaningful confidence measures that are independent of the training classes. This gap underscores the importance of developing advanced techniques that not only quantify epistemic uncertainty but also integrate it seamlessly into predictive models.

Bayesian networks (BNs) provide a structured framework for capturing dependencies between variables and incorporating prior knowledge, making them an effective tool for prediction tasks in diverse domains¹¹. Recent advancements have extended Bayesian networks to sequence modeling, integrating them with techniques like Hidden Markov Models (HMMs) and Conditional Random Fields (CRFs) to better capture temporal dependencies^12,13.

Deep Bayesian networks further enhance this capability by incorporating multiple layers to handle high-dimensional and complex data, broadening their applicability in areas such as natural language processing and genomics^14,15.

Meanwhile, the evolution of neural network architectures has led to the development of Recurrent Neural Networks (RNNs) and the Transformer architecture, which excel in modeling sequential data. RNNs, particularly those enhanced with mechanisms like Long Short-Term Memory (LSTM) units, are well-suited for capturing temporal dependencies in data^16,17. However, they struggle with long-range dependencies due to issues like vanishing gradients. Transformers address these limitations by employing self-attention mechanisms, enabling them to effectively model complex dependencies over extended sequences¹⁸. Combining the strengths of RNNs with Transformers has proven to be a powerful approach to tackle sequential tasks in domains such as speech recognition, time series forecasting, and natural language processing¹⁹.

This study proposes a hybrid framework that integrates Bayesian networks with RNNs based on the Transformer architecture. The Bayesian component provides a probabilistic foundation for modeling dependencies and uncertainty, while the deep learning component leverages advanced sequence modeling capabilities to enhance classification accuracy. By combining probabilistic reasoning with data-driven pattern recognition, this hybrid approach aims to improve the classification of nanocomposites, particularly in evaluating their conductivity properties. This integration represents a significant step forward in developing robust, interpretable, and reliable machine learning methodologies for complex classification tasks. Nanocomposites represent a class of engineered materials in which nanoscale components, such as nanoparticles, nanofibers, or nanotubes, are embedded within a bulk matrix material. This combination is designed to leverage the unique properties of nanomaterials²⁰ such as their high surface area to volume ratio, quantum effects, and exceptional mechanical, electrical, and thermal characteristics to enhance or tailor the properties of the composite material, as research in nanotechnology advances, nanocomposites continue to evolve, offering new possibilities for high-performance materials in various industrial and technological domains²¹. The integration of machine learning into nanomaterial research is transforming the field by providing effective tools to predict and optimize material properties, improve synthesis techniques, and refine characterization methods²². Hong Wang et al. provide insights that enhance the understanding of the role of machine learning in materials science and pave the way for new interdisciplinary advances in this area²³.

A key feature of nanocomposites lies in the interaction between the matrix and the nanostructures. These interactions can lead to significant improvements in mechanical strength, conductivity, thermal stability, and other material properties. For example, conductive nanocomposites, which are the focus of this study, often rely on the uniform dispersion of conductive nanoparticles (such as carbon nanotubes, graphene, or metal oxides) within an insulating polymer matrix to achieve desired electrical properties. The quality of the dispersion and the percolation threshold the point at which a continuous conductive network is formed are critical factors influencing the conductivity of the nanocomposite. In addition to the physical and chemical properties of the nanoparticles themselves, the design and performance of nanocomposites depend heavily on factors such as particle size, shape, surface functionalization, and the strength of interfacial bonding between the nanostructures and the matrix. These parameters are often difficult to quantify and model due to their inherent complexity and variability. This study acknowledges the importance of these principles in the design of methodologies for nanocomposite classification. The Bayesian Network component of our hybrid framework provides a probabilistic approach to capture the influence of these material specific factors, while the Transformer based Recurrent Neural Network leverages the ability to identify complex patterns and sequential dependencies in experimental datasets. By combining these techniques, the hybrid approach is positioned to bridge computational advancements with a deeper understanding of the material science context, enabling robust and interpretable predictions of conductivity attributes. This integration of domain knowledge into the computational framework not only enhances the credibility and applicability of the proposed methodology but also underscores the interdisciplinary nature of this research, linking advanced machine learning with material science insights to address contemporary challenges in nanocomposite evaluation.

The model begins by processing the input sequence, which represents various features of nanocomposites. An RNN layer using LSTM initially handles this sequence, capturing temporal dependencies and generating hidden states. The Transformer component then applies self-attention to these hidden states, allowing the model to evaluate the significance of different parts of the sequence, thus capturing intricate patterns and interactions within the data.

The output from the Transformer is passed through dense layers, leading to a final classification layer that assigns the input sequence to a specific category, predicting properties such as the conductivity of nanocomposites.

This approach offers several advantages, including the ability to handle long-range dependencies. The self- attention mechanism in Transformers, for example, allows the model to capture these dependencies more effectively.

Furthermore, combining the sequential data handling of RNNs with the self-attention capabilities of Transformers enhances classification accuracy. This hybrid model is versatile, capable of adapting to different types of sequential data, and is thus suitable for a wide range of classification tasks.

The components that contribute to model validity provide valuable insights into the prediction’s reliability. For instance, the experimental results from Hossein et al.²⁴ show that when the model is trained on regular images and tested on negative images, its accuracy is significantly lower compared to testing on regular images. This suggests that current training methods do not effectively enable models to generalize concepts.

Additionally, this paper aims to explore a real-world application of this hybrid model for Nanocomposite Classification, particularly in assessing their conductivity, to overcome the limitations inherent in neural networks.

In natural language processing, recurrent neural networks (RNNs) and transformer models have demonstrated remarkable performance in tasks such as machine translation, sentiment analysis, and text generation²⁵. These models excel at learning contextual relationships among words in sentences, aiding in the generation of coherent and contextually accurate predictions. Thus, integrating RNNs with Transformer architecture for classification leverages the strengths of both methods, efficiently handling complex, sequential data²⁶. In this work, it is crucial to clearly distinguish between nanocomposites and nanoparticles to ensure terminological precision. Nanoparticles refer to nanoscale materials characterized by at least one dimension below 100 nanometers, often employed as individual entities with unique physicochemical properties. In contrast, nanocomposites are hybrid materials formed by embedding nanoparticles or other nanostructures within a matrix material to enhance or tailor specific properties, such as mechanical strength, conductivity, or thermal resistance. While nanoparticles may serve as components of nanocomposites, the latter represents a broader material system. Throughout this manuscript, the term nanocomposites is used exclusively to refer to such composite systems, while nanoparticles are mentioned only in the context of their role as reinforcing or functionalizing agents within these systems.

The advancements in machine learning algorithms have significantly contributed to the classification and study of nanoparticles and nanocomposites. Early methods primarily relied on statistical models and linear classifiers, which were limited in handling the complex and high-dimensional datasets characteristic of nanomaterials. With the evolution of machine learning, advanced algorithms such as support vector machines (SVMs), random forests, and, more recently, deep learning models like convolutional neural networks (CNNs) and recurrent neural networks (RNNs), have enabled more accurate and efficient analyses. These algorithms can capture intricate patterns and nonlinear relationships between structural, compositional, and property-related features of nanomaterials. Furthermore, the integration of transformer-based architectures and unsupervised learning techniques has enhanced feature extraction and classification, paving the way for deeper insights into nanocomposite behaviors, such as conductivity, mechanical strength, and thermal stability. These developments have empowered researchers to design and optimize nanomaterials with unprecedented precision and performance.In this study, we propose a hybrid approach that integrates Bayesian Networks (BNs) with Transformer-based Recurrent Neural Networks (RNNs) for the classification of nanocomposites based on their conductivity. This section will provide a detailed discussion of how the synergy between these two models enhances the overall performance, the key technical benefits of the hybrid approach, the methodological nuances and trade-offs involved, and the empirical evidence supporting its effectiveness. Bayesian Networks (BNs) contribute to improving model robustness by introducing probabilistic reasoning and uncertainty management. In this hybrid approach, the BN serves as an expert intervention agent, capable of handling missing or uncertain data and providing probabilistic insights into the relationships between input features and the target variable, which in this case is the conductivity of nanocomposites. The probabilistic nature of BNs allows the model to represent uncertainty in the predictions, offering more reliable and interpretable results, especially in cases where data may be incomplete or noisy. Transformer-based RNNs, on the other hand, are highly capable of modeling complex sequential data. Their ability to capture long-range dependencies and intricate patterns in large datasets makes them particularly useful in dealing with sequential data in nanocomposite classification tasks. By integrating these two approaches, we combine the strengths of BNs in uncertainty quantification with the deep learning capabilities of Transformer RNNs in handling sequential dependencies. This synergy enables the model to make more informed predictions while managing uncertainty effectively. The integration of Bayesian Networks (BNs) with Transformer-based Recurrent Neural Networks (RNNs) represents a significant advancement in the classification of nanocomposites, particularly in evaluating their conductivity. BNs enhance model robustness by introducing probabilistic reasoning and uncertainty quantification, acting as expert intervention agents that handle missing or noisy data while providing insights into the relationships between input features and target variables. This capability ensures more interpretable and reliable predictions, which is crucial in scientific and industrial applications. Meanwhile, Transformer RNNs excel at modeling.

complex sequential data, effectively capturing long-range dependencies and intricate patterns in datasets that describe nanocomposite behavior. Combining these approaches creates a synergistic framework where BNs manage uncertainty and improve interpretability, while Transformer RNNs leverage their deep learning strengths to handle sequential dependencies. This hybrid model offers several key benefits, including improved interpretability, robustness, and accuracy, enabling it to tackle challenging tasks like conductivity evaluation with superior precision, recall, and F1 scores compared to individual models. However, integrating these techniques introduces methodological challenges, such as increased computational complexity due to attention mechanisms and probabilistic inference, high data quality requirements, and risks of overfitting, especially with small or noisy datasets. Addressing these issues requires efficient optimization strategies, robust data preprocessing, and regularization techniques. Empirical evidence underscores the hybrid model’s efficacy, showcasing its ability to outperform standalone models while reducing uncertainty in predictions an invaluable trait for real-world applications where actionable insights are critical. Despite its current limitations, such as sensitivity to noisy data and computational demands, the hybrid model’s strengths in accuracy, interpretability, and robustness position it as a powerful tool for advancing nanocomposite classification and unlocking new possibilities in predictive analytics.

This paper is organized into several sections, each addressing a specific aspect of our research. It begins by outlining the methodology and data processing techniques used to prepare the dataset for analysis. We then delve into Bayesian networks, providing a comprehensive explanation of their probabilistic framework, followed by a detailed case study to illustrate their practical application.

In the subsequent section, we introduce the Recurrent Neural Network (RNN) model based on transformers, highlighting its architecture and key features. Building on this, we propose a novel hybrid approach that integrates Bayesian networks with the transformer-based RNN. This approach is designed to enhance predictive accuracy and optimize other significant performance metrics. To further validate the effectiveness of our model, we perform a comparative analysis with traditional machine learning methods, including Random Forest and K-Nearest Neighbors (KNN). This comparison underscores the advantages of our proposed approach over conventional techniques. Finally, the paper concludes with a thorough discussion of the results, focusing on their implications, limitations, and potential directions for future work.

Methodology

Problem formulation

Accurate determination of the conductive properties of nanocomposites is essential for their application across various technological fields^27,28. However, this task is complicated by the extremely small size of nanoparticles and the limitations of current technologies. Specifically, measuring and classifying their conductivity (Good, Medium, or Bad) presents significant challenges, especially when certain physical properties are not directly measurable. This research aims to address these challenges by developing innovative solutions for nanoparticle characterization.

Our challenge lies in the fact that neural Networks alone are insufficient due to their specific learning-based outcomes. Therefore, expert intervention is required, which, in our case, will be provided by a Bayesian network. By integrating these algorithms into a unified model, we aim to leverage the strengths of each method to enhance overall prediction performance. The goal is to develop a predictive model capable of delivering more accurate and reliable results by harnessing the synergy among different classification algorithms.

Data processing

We used seven different variables as inputs for our prediction model. Firstly, The size of nanomaterials (S)²⁹. Next, tensile strength (T), elastic modulus (M)³⁰,Flexural strength (F), molecular weight (W) Temperature (T)M³¹, it is typically measured in degrees Celsius and the concentration (CO)³².We employed electrical conductivity (C)³³ as output.

Figure 1 displays the histograms of the various input variables used in the model. Each histogram illustrates the distribution of values for a specific variable, providing a clear view of how the data is distributed across different intervals. These visual representations also help identify the ranges of variability, which are crucial for understanding the behavior of the data and their potential impact on the results.

Fig. 1.

Data distribution across intervals for different features.

Before applying the hybrid model, the nanocomposite data undergoes preprocessing to eliminate outliers, missing values, or errors. This process includes normalizing features, handling missing values, and encoding categorical data. Once the data is prepared, it is ready to be fed into both the RNN and BN.

The Table 1 presents the mean values (μ)³⁴ and standard deviations (σ)³⁵ of the different variables. The different (μ), (σ) values and the units for the input variables suggest that the prediction model must be able to handle variables with diverse ranges and variabilities.

Table 1.

Statistical distribution and unit of each physical parameter used for classification.

	Mu (μ)	Sigma (σ)	Units
S	4,99E-08	2,88E-08	Nm
T	42131807,78	21842552,22	MPa
M	1246,068561	144,1231986	Pa
F	946030205,9	626024130,9	MPa
W	5,02E-08	2,85E-08	N
TM	148,9624714	88,51007507	°C
CO	50,87826087	29,00588561	%

Flowchart

Flowchart of Fig. 2 delineates the systematic process for modeling and evaluating machine learning performance on nanomaterial data. The process starts with data collection from sources such as the Nanomaterial Registry, NPDatabase, and private databases. The collected data then undergoes comprehensive preprocessing steps, including understanding, cleaning, selection, visualization, extraction, and scaling. This preprocessed data is then used to develop two types of models: Bayesian networks and Transformer-based recurrent neural networks (RNN). Each model is evaluated individually through specific case studies. Following this, the models are combined to create a hybrid model, which undergoes rigorous evaluation to compare its performance with the individual models.

Fig. 2.

Methodology of conductivity classification.

A critical decision point follows the evaluation: if the hybrid model demonstrates good performance, the process proceeds to the final output. If the performance is unsatisfactory, the process loops back to reconfigure the modeling process. The evaluation metrics, including accuracy, mean squared error (MSE), and R² (coefficient of determination), guide this decision-making process, ultimately aiming to optimize the understanding and efficacy of machine learning models applied to nanomaterial data.

Integration of transformer RNNs and bayesian networks for nanocomposite classification

The integration of Transformer-based Recurrent Neural Networks (RNNs) with Bayesian Networks (BNs) for nanocomposite classification combines the strengths of both techniques to achieve more robust, interpretable, and accurate predictions, especially for complex tasks like determining nanocomposite conductivity.

Processing with the transformer-based RNN

To ensure a robust and reproducible classification of nanocomposites, our methodology integrates a Transformer- based Recurrent Neural Network (RNN) that processes the dataset through a structured pipeline, including data encoding, feature extraction, and sequential modeling. Below, we provide a comprehensive breakdown of these steps.

Data encoding

The raw data set consists of numerical and categorical features that represent the physical, chemical, and structural properties of nanocomposites. To prepare these inputs for the Transformer-based RNN, we apply a structured encoding process:

Categorical features encoding: discrete variables such as nanocomposite type, synthesis method, and doping material are converted using embedding layers rather than traditional hot encoding. This allows the model to capture semantic relationships between categories while avoiding the curse of dimensionality.

Embeddings are learned during training, where each categorical feature is mapped into a lower-dimensional continuous space. The embedding vector size is determined based on the rule:

1

which balances expressiveness and computational efficiency.

Numerical features normalization: Continuous attributes such as electrical conductivity, thermal resistance, and density are normalized using Min-Max scaling, ensuring all values fall within a [0, 1] range. This normalization helps stabilize the training process by preventing features with larger magnitudes from dominating the learning process.

To handle potential outliers, we also apply Robust Scaling, which is computed as:

2

Where IQR is the interquartile range.

Temporal or sequential encoding: If data is collected in sequential batches (e.g., time-dependent measurements of conductivity evolution), a positional encoding is applied.

We use sinusoidal positional encoding, defined as:

3

4

where pos is the position index in the sequence and d is the embedding dimension. This encoding preserves ordering information for sequence-based classification tasks.

Feature extraction with transformer-based RNN

Once the data is encoded, the Transformer-based RNN extracts relevant features by combining the self-attention mechanism of Transformers with the sequential processing power of RNNs.

Multi-head self-attention mechanism: The Transformer encoder computes relationships between input features using scaled dot-product attention, defined as:

5

Where Q,K, V are the query, key, and value matrices, and d_k is the key dimension.

By using multi-head attention, the model learns multiple feature dependencies simultaneously, enabling better gen- eralization for complex material properties. The output from self-attention is then passed through Layer Normalization and residual connections to improve training stability.

Recurrent processing with gated units: While the Transformer module effectively captures long-range dependencies, we incorporate an LSTM module based on RNN to refine sequential feature representations. The hidden state ℎ_t at time t is updated using.

6

Where is the update gate, ⊙ represents element-wise multiplication, and ℎt̃ is the candidate activation.

This hybrid approach benefits from both self-attention (global feature learning) and gated recurrence (temporal dependencies).

Feature selection and dimensionality reduction: To ensure that only the most relevant features are used for classification, we apply Principal Component Analysis (PCA) or an Attention-based feature selection mechanism before feeding the output into the Bayesian Network.

The Transformer output is pooled using global average pooling, reducing the feature space while preserving the most significant patterns.

Output layer and bayesian network integration

The extracted high-level features are finally passed to the Bayesian Network, which models probabilistic depen- dencies among different material properties and classifies nanocomposites based on learned distributions.

The Bayesian Network constructs a directed acyclic graph (DAG) where nodes represent features, and edges encode dependencies. Probabilities are computed using Bayes’ theorem:

7

Application of the bayesian network (BN)

When the RNN generates its initial prediction, the Bayesian Network comes into play. The BN takes this initial prediction, along with relevant input data, and uses it to establish a probability distribution that models the uncertainties associated with the prediction. The Bayesian Network (BN) is used to capture the uncertainties associated with the properties of the nanocomposites. It integrates prior knowledge about the relationships between the nanocomposite characteristics and their conductivity, adjusting the predictions made by the Recurrent Neural Network (RNN) in response to uncertainties observed in the data. For instance, if some properties exhibit significant variability or uncertainty, the BN may lower the confidence in the initial prediction. Furthermore, the BN enhances the outputs of the RNN through Bayesian inference, enabling specific features to be weighted more heavily according to their relevance for conductivity. This probabilistic method proves useful in managing cases where information is incomplete or uncertain.

RNN and BN integration

A crucial element of the integration between the RNN and the BN is the feedback loop. When the BN modifies the RNN’s initial predictions, these modifications are sent back to the RNN to update its weights. This process allows the system to improve progressively, enhancing the accuracy of its predictions. Once the BN refines the predictions, the model combines the probabilistic insights from the BN with the RNN’s feature extraction capabilities to finalize the classification of the nanocomposites based on their conductivity. This combined approach produces more robust and reliable results.

Prediction and evaluation

Once the RNN and BN have interacted, the model generates a final prediction for the conductivity of the nanocomposites. This output represents the combination of the probabilistic information from the BN and the complex feature extraction from the RNN. The hybrid model is then evaluated using appropriate performance metrics, such as accuracy, etc. Experimental results are compared to the performance of individual models (RNN or BN) to demonstrate the improvement achieved by the hybrid approach.

Bayesian networks

Explanation

A Bayesian network, also referred to as a Bayesian belief network or probabilistic graphical model, is a powerful tool in machine learning and artificial intelligence for probabilistic reasoning and decision-making³⁶. It represents a set of variables and their probabilistic dependencies using a directed acyclic graph (DAG). Below is an overview of Bayesian networks, their advantages, and their limitations in nanomaterial classification:

Bayesian networks offer several benefits for nanomaterial classification, including the ability to incorporate uncertainty, provide transparent representations, integrate prior knowledge, and effectively manage dependencies³⁷. These features make them particularly useful for modeling complex systems where uncertainty is inherent.

However, Bayesian networks also have some limitations. These include challenges with complexity and scalability, the assumption of conditional independence, the need for sufficient data, and difficulties in modeling causality³⁸.

In summary, Bayesian networks present a strong framework for classifying nanomaterials by accurately modeling probabilistic relationships and handling uncertainty. Their capacity to integrate prior knowledge and offer transparent representations makes them valuable for decision-making in nanomaterial applications. Nevertheless, practical applications should carefully consider the issues of scalability and the assumptions underlying these models. In this work, we leverage the strengths of Bayesian Networks (BNs) to address the classification of nanomaterials with a specific focus on their conductivity properties. Unlike traditional approaches that rely on generic statistical models or linear classifiers, we integrate BNs with a recurrent neural network (RNN) framework to effectively model the complex dependencies and nonlinear relationships inherent in nanomaterial datasets. This hybrid approach enables the probabilistic inference of conductivity classifications by capturing both structural and compositional factors. To enhance the framework’s robustness, we incorporate transformer-based architectures for feature extraction, which significantly improve the accuracy of BN predictions. Previous studies in the literature have highlighted the applicability of BNs in materials science for tasks involving uncertainty quantification and decision-making; however, their potential for conductivity-focused classification of nanocomposites remains underexplored. Our contributions lie in the novel application of this combined RNN-BN approach, which not only achieves high predictive performance but also provides interpretable probabilistic insights into the conductivity behavior of nanomaterials.

Causal relationships in a directed acyclic graph (DAG)

Pathways

Size (s) → Electrical Conductivity (C) The size of nanoparticles directly affects electrical conductivity, as quantum properties (such as quantum confinement effects) depend on particle size. A directional path → indicates that changes in size directly influence conductivity.

Weight (w) → Electrical Conductivity (C) The weight of nanomaterials may affect their density or overall structure, which in turn can influence conductivity. This relationship is often indirect, mediated by properties like porosity or concentration.

Concentration (CO) → Electrical Conductivity (C) The concentration of nanoparticles in a matrix directly impacts their ability to form continuous conductive paths. For instance, higher concentrations can increase the probability of percolation, enhancing conductivity. This is a direct and often dominant pathway in the model.

Temperature (T M) → Electrical Conductivity (C) Temperature influences electron movement within nanoma- terials, thereby affecting conductivity. In many materials, increased temperature reduces conductivity due to enhanced electron scattering, but this effect can vary depending on the material.

Flexural Strength (F) → Electrical Conductivity (C) Flexural strength reflects mechanical properties related to the microstructure of nanomaterials. A causal link → may exist if changes in flexural strength represent modifications in nanoparticle arrangement or connectivity, indirectly affecting conductivity.

Interactions between inputs

Certain inputs may interact, creating indirect pathways in the DAG:

• Concentration (CO) → Size (s) → Electrical Conductivity (C): A higher concentration could influence the average particle size during synthesis, thereby impacting conductivity.

• Temperature (T M) → Flexural Strength ( ) → Electrical Conductivity (C): Temperature may alter the mechanical structure of the material, indirectly affecting conductivity.

Directionality and causality

The directional pathways in the DAG must be justified by clear causal assumptions:

• Direct Pathway: A direct pathway (e.g., s→ C) implies that the input variable directly affects the output without intermediaries.

• Indirect Pathway: An indirect pathway (e.g., T M→ F→ C) suggests that one input influences another, which subsequently impacts the output.

• Acyclicity: Loops are not allowed in a DAG, ensuring the relationships remain acyclic.

These directional pathways guide the modeling of causal relationships and form the basis for calculating the Conditional Probability Tables (CPTs) in a Bayesian Network, enabling probabilistic and interpretable analyses of the effects of inputs on electrical conductivity.

Case study

We have defined specific conditions for each input to determine the probabilities linked to the output. Using the properties of the normal distribution, we divide the intervals based on the mean (μ) and standard deviation (σ). For instance, the interval for Weight is divided into three categories: “Low”, “Medium”, and “High”, leveraging the properties of the normal distribution.

• Medium : Values between.

• High : Values greater than + .

• Low : Values less than − .

We take another example of conditions, the intervals of Concentration will be defined as follows:

• Medium to high : Values between.

• Low to medium : − .

Various functions have been developed to assess and classify the values of different data columns, enabling a more precise analysis of the data in terms of probabilities associated with the expected outcomes.

We configured the conditional probability tables (CPTs) for the variables S, W, CO, TM, and F, and within the Bayesian network, with respect to the variable . These tables were then normalized and visualized in Figs. 3, 4, 5, 6 and 7.

Fig. 3.

Probability distribution of S conditional on C in the BN.

Fig. 4.

Probability distribution of W conditional on C in the BN.

Fig. 5.

Probability distribution of CO conditional on C in the BN.

Fig. 6.

Probability distribution of TM conditional on C in the BN.

Fig. 7.

Probability distribution of F conditional on C in the BN.

In these figures, the columns represent the different categories of S, while the rows correspond to the categories of C. Each cell contains a probability value and is color-coded to facilitate visual interpretation. The color coding helps us observe how the variable S changes in relation to C. The probabilities of S vary significantly depending on the values of C, highlighting a strong conditional dependence between these variables in the Bayesian network. This pattern holds true for the other variables as well.

After analyzing the simulation results based on Bayesian networks, we obtained an accuracy of 64.81% and a mean square error (MSE) of 0.52. These results indicate that the Bayesian networks approach exhibits relatively modest performance in predicting conductivity, highlighting the need to explore other methods to improve the accuracy and reliability of predictions (Table 2).

Table 2.

Performances generated by bayesian networks.

	Accuracy	MSE	RMSE	R
Bayesian networks	64.81%	0.52	0.72	–

Figure 8 illustrates the Bayesian network model, highlighting the conditional dependencies between the central variable and the dependent variables S, W, TM, and M. The variable C, which can take on the states good, Medium, and Bad, exerts a direct influence on the probability distributions of the states of the other variables. For example, being in the Good state with a probability of 98.04% has a substantial impact on the distributions of S, W, TM and M.

Fig. 8.

Visualization of inference results in the Bayesian network.

The directional arrows in the model represent the dependency relationships, while the accompanying conditional.

probability bars visualize the likely distributions of the dependent variables’ states. This Bayesian network provides a clear framework for analyzing how changes in C propagate through the system, affecting other variables. Furthermore, the model demonstrates impressive computational efficiency, achieving inference within 0.72 milliseconds, enabling real-time execution of complex probabilistic calculations.

These Bayesian network models reveal how changes in the probability distribution of the central variable C impact the dependent variables S, W, TM, and M. When is fixed at 100% Good, the dependent variables exhibit concentrated and predictable distributions; for instance, is entirely classified as Low with 100% probability. In contrast, when C has a mixed probability distribution across Good, Medium, and Bad (lower context), the dependent variables display more dispersed distributions, reflecting greater uncertainty and variability.

The following Fig. 9 illustrates that the model achieves inference times of 0.48 milliseconds and 0.58 milliseconds in two respective contexts, showcasing its efficiency in real-time probabilistic computations. These findings underscore the dynamic interactions between C and other variables, highlighting how the state of directly impacts the network’s behavior and variability³⁹.

Fig. 9.

Inference results of weight.

Transformer-based RNN

Explanation

Transformer-based architectures, originally developed for Natural Language Processing (NLP) tasks such as automatic translation and text classification, have demonstrated exceptional capabilities in handling sequence-related data due to their attention mechanism. Unlike traditional recurrent or convolutional networks, Transformers leverage multi-head attention to capture long-range dependencies within sequences, making them particularly effective for processing complex, high-dimensional datasets. These innovations have since extended beyond NLP to domains where sequential or structured data, like time series or molecular properties, play a critical role. In the context of nanomaterials classification based on conductivity, these Transformer-based methods provide a robust framework for analyzing intricate relationships between input variables such as size, weight, concentration, temperature, and flexural strength. The attention mechanism within Transformers enables the model to weigh the relative importance of each input feature dynamically, capturing nonlinear and nonlocal interactions that significantly influence the material’s electrical properties. For example, correlations between temperature and conductivity, or the combined effect of size and molecular weight, can be modeled more effectively compared to traditional machine learning approaches. Pre- trained Transformer models like BERT (Bidirectional Encoder Representations from Transformers) and its adaptations provide additional advantages through transfer learning⁴⁰. By pre-training on large datasets, these models can learn generalized representations, such as underlying patterns in structural or compositional features, which can then be fine-tuned for domain-specific tasks like nanomaterials classification. The bidirectional attention mechanism in BERT is particularly useful for identifying subtle interactions and dependencies among input variables, allowing for a more accurate prediction of conductivity. Furthermore, the adoption of Transformer-based architectures in this study allows the integration of sequence-based input data, such as experimental conditions or processing steps, into the classification framework. This integration improves the model’s ability to generalize across different datasets, aiding in the prediction of conductivity under varying experimental and material conditions. By leveraging the strengths of Transformers, this work addresses the challenges of analyzing complex nanomaterials datasets, demonstrating their applicability beyond NLP and their potential to advance material science research.

Case study

After obtaining simulation results with an RNN based on the Transformer model, we achieved an accuracy of 83.56%. Additionally, the mean square errors (MSE) and root mean square errors (RMSE) were 0.04 and 0.22, respectively. The correlation coefficient (R) also showed a strong relationship with a value of 0.94 (Table 3). These results demonstrate the effectiveness of our approach, with promising performance in terms of prediction accuracy and reliability.

Table 3.

Simulation results transformer-based RNN.

	Accuracy	MSE	RMSE	R
Transformer based RNN	83.56%	0.04	0.22	0.94

Transformer-based RNN and bayesian network

Explanation

The main challenge of our approach lies in the inherent limitations of using neural networks exclusively. While these models are effective, their results remain confined to specific learning schemes, limiting their ability to generalize in more uncertain or complex contexts. In this regard, the expertise of a Bayesian networks specialist becomes crucial. By integrating these two approaches into a unified model, we aim to strategically leverage their respective strengths. Neural networks, particularly those based on architectures such as Transformers or recurrent neural networks (RNNs), excel in processing sequential data and capturing long-term dependencies⁴¹. In contrast, Bayesian networks provide a structured modeling of probabilistic relationships and effectively manage uncertainty. This synergy between methodologies paves the way for significantly improved overall predictive performance. Our approach is grounded in an iterative process of rigorous testing and validation. By combining algorithms, we commit to systematically evaluating their effectiveness in varied environments and real-world scenarios⁴². The ultimate goal is to design a predictive model capable of delivering more accurate, reliable, and robust results while incorporating probabilistic dimensions often overlooked by traditional neural architectures.

Transformer-based RNNs represent a major breakthrough in tasks involving sequential data. Transformers’ ability to capture long-range dependencies, coupled with RNNs’ capability to maintain states across successive input sequences, makes these models particularly effective for applications such as machine translation or text generation⁴³.

Conversely, Bayesian networks stand out for their capacity to model conditional relationships and uncertainties in complex datasets. They are particularly suited to domains where understanding probabilistic dependencies between variables is essential, such as medical diagnostics or decision-making under uncertainty⁴⁴. In summary, while Transformer-based RNNs excel in tasks involving sequential dependencies and long-term relationships, Bayesian networks add significant value in domains requiring deep probabilistic modeling and precise uncertainty management. These two paradigms are not mutually exclusive but rather complementary, opening the door to significant advance- ments in artificial intelligence and machine learning applications³.

Case study

After combining the two approaches, we obtained the following results explained in Table 4. An accuracy rate of 95.88%, a root mean square error (MSE) of 0.0286, a root mean square error (RMSE) of 0.169 and a correlation coefficient (R) of 0.95. These results show a clear improvement over previous results, demonstrating the increased effectiveness of our combined approach.

Table 4.

Simulation results of bayesian networks & RNNT.

	Accuracy	MSE	RMSE	R
BN & RNNT	95.88%	0.0286	0.169	0.95

Scalability and robustness of the hybrid model

As the volume of data increases and the diversity of nanocomposites expands, maintaining high performance becomes a major challenge. However, the hybrid model is specifically designed to effectively address these challenges through several mechanisms. First, the scalability of the Transformer-based RNN is a key advantage. This type of model is particularly well suited for large datasets, leveraging the increased volume of information to enhance generalization and predictive accuracy. Additionally, techniques such as batch processing and parallelization optimize training on large datasets, ensuring efficient handling of diverse and high-volume inputs. Furthermore, the flexibility of the Bayesian component plays a crucial role in adapting to new categories of nanocomposites. By incorporating prior knowledge and probabilistic reasoning, the model can adjust to evolving data distributions without requiring extensive retraining, thereby improving generalization to previously unseen data. To further enhance this adaptability, transfer learning can be employed. This approach allows a model initially trained on one dataset to be fine-tuned with new nanocomposite categories, reducing the need for training from scratch while maintaining optimal performance. Moreover, the Bayesian approach also contributes to the model’s regularization and robustness. By continuously updating prior distributions, the model remains resilient to data variations and ensures reliable predictions, even when faced with diverse and noisy datasets. Complexity calculations further support the efficiency of the hybrid model, with a Bayesian Network complexity of 512 and a significantly higher Transformer-based RNN complexity of 3,663,249,408. Despite this complexity, the simulated training time, calculated as.

remains efficient at 0.3674 s, demonstrating the feasibility of the model for large-scale applications (Fig. 10). Finally, we plan to further improve the model by integrating techniques such as data augmentation to enhance generalization, as well as incorporating advanced neural architectures, including attention mechanisms, to extract the most relevant features regardless of the nanocomposite type or scale. Through these strategies, the hybrid model is well equipped to maintain high performance despite increasing data volume and complexity. Future improvements will further enhance its scalability and adaptability to the challenges posed by the growing diversity of nanocomposites.

Fig. 10.

Dataset complexity and training time.

Traditional methods

We have incorporated a comparative analysis of our approach with two well-established methods, the Random Forest method and the K-Nearest Neighbors (KNN) method. These models were selected due to their popularity and their proven performance in similar classification tasks.

The random forest method

Random Forest is an ensemble learning method that constructs multiple decision trees during training and combines their predictions to improve accuracy and reduce overfitting⁴⁵. Using this method, we obtained the following results, an accuracy of 85.62% and a mean squared error (MSE) of 0.3836, as shown in Fig. 11.

Fig. 11.

Evaluation of MSE and classification accuracy over epochs for the Random Forest algorithm.

The K-nearest neighbors (KNN) method

KNN algorithm is a simple, non-parametric method that classifies data points based on the majority class among their -nearest neighbors in the feature space⁴⁶. Using this method, we achieved an accuracy of 76.03% and a mean squared error (MSE) of 0.5890, as illustrated in Fig. 12.

Fig. 12.

Evaluation of MSE and classification accuracy over epochs for the KNN algorithm.

Discussions

Table 5 present the simulation results for accuracy comparing the three approaches. In fact, the Bayesian network reached an accuracy of 64.81%: Besides, the method of Transformer based RNN became 83.56%. To improve the limitation of neural networks, we focused on the combination and so that we obtained an accuracy value equal to 95.88%.

Table 5.

Simulation results of accuracy for three aproches.

	Accuracy
Bayesian network	64.81%
Transformer based RNN	83.56%
BN & RNNT	95.88%

The validation of our work must be approved by calculating some metrics such as MSE which is 0.52 for BN, 0.04 for RNNT and 0.0286 for the combination. Besides, we calculate the RMSE which is 0.22 for RNNT and 0.169 for the combiantion. The metric R for the correlation doesn´t change between RNNT and the combianation. It’s nearly to 0.95, the same thing for R² which is nearly to 0.80 for both of the RNNT and the combination and finally the MAE which is the same (Table 6).

Table 6.

Table of simulation results for metrics.

	BN	RNNT	BN & RNNT
MSE	0.52	0.04	0.0286
RMSE	0.72	0.22	0.169
R	–	0.94	0.95
R2	–	0.80	0.82
MAE	–	0.05	0.07

The most interesting metric is the MSE which has decreased to achieve our target the zero value in using the combination between the RNNT and the BN. Adding to this, the accuracy percentage is the most important when combining the Informatics side (neural networks) and the expertise based on the BN.

So, we have exceeded the limitation of the first method and validated our work by finding considerable results.

Figure 13 presents the histogram of Accuracy results for the three approches. In fact, it´s clearly that it increases from the use of Bayesian network to transformer based RNN and finally the combination between the first and the second approach. In our study, accuracy is defined as the classification performance evaluated exclusively on the test set. The test set was separated from the training set to ensure unbiased evaluation and to prevent overfitting. The split was done using stratified sampling to ensure that the class distribution in the test set reflects that of the entire dataset. For regression tasks, we employed two key metrics: Mean Squared Error (MSE) and Root Mean Squared Error (RMSE). Both metrics were calculated over the test set and were used to assess the model’s predictive accuracy for continuous outcomes. To ensure that MSE and RMSE calculations are fair and meaningful across different features with varying scales, data normalization was applied before model training. Continuous features were normalized using either Z-score normalization to bring them to a comparable scale. These preprocessing steps ensure that the performance metrics reflect the model’s ability to predict across all features without being unduly influenced by their original magnitude. We did not apply explicit data weights during the MSE and RMSE calculations, although the normalization process accounts for the disparity in feature scales. Future work may incorporate weighted losses if certain observations are deemed more significant based on domain knowledge or characteristics of the data set.

Fig. 13.

Accuracy results for three approaches.

Morever, in our work, we aimed to improve the classification of nanocomposites for conductivity testing using a hybrid approach that integrates Bayesian Networks (BNs) and Recurrent Neural Networks (RNNs) based on Transformer architecture.

We began by using Bayesian Networks to analyze and classify nanocomposites based on their conductivity characteristics. BNs provided a probabilistic framework to model the relationships and dependencies between nanoparticle distribution and their impact on conductivity. This approach offered a nuanced understanding of how varying nanoparticle distributions influenced conductivity outcomes, enabling informed decision-making through confidence intervals and probabilistic predictions rather than deterministic outputs.

After the BN analysis, we employed RNNs based on Transformer architecture to enhance classification accuracy. Transformers are renowned for capturing long-range dependencies and intricate sequential patterns in data. The RNN model excelled at learning from sequential data, such as temporal variations in nanoparticle arrangements that affected conductivity over time. By incorporating Transformer-based RNNs, our model effectively captured and utilized complex relationships within nanocomposite data, improving predictive accuracy compared to traditional sequential models.

To leverage the strengths of both BNs and Transformer-based RNNs, we proposed a hybrid model. This integrated approach combined the probabilistic reasoning of BNs with the deep learning capabilities of RNNs. The hybrid model synergistically utilized BNs to provide probabilistic insights and RNNs to leverage sequential dependencies, resulting in a more robust and accurate classification framework for nanocomposite conductivity. Experimental evaluations demonstrated that our hybrid model significantly outperformed standalone BNs and RNNs, validating its efficacy in predictive modeling and classification tasks. The results clearly demonstrate the significantly superior performance of our method, which combines the Bayesian Network (BN) with the Recurrent Neural Network Transducer (RNNT), compared to the Random Forest and K-Nearest Neighbors (KNN) approaches based on the metrics of Accuracy and Mean Squared Error (MSE). These findings, as summarized in Table 7, highlight the effectiveness of integrating BN and RNNT for achieving higher accuracy and lower MSE, thus validating the robustness of our proposed approach. However, while Random Forest and KNN are robust and widely used methods, their performance appears limited in this specific context due to their inability to effectively model complex relationships between variables.

Table 7.

Performance metrics comparison across different models.

	BN & RNNT	Random forest	KNN
Accuracy	95.88%	85.62%	76.03%
MSE	0.0286	0.3836	0.5890

The superior performance of the hybrid model compared to traditional models such as Random Forest and KNN can be attributed to several factors. First, the synergy between the Bayesian network and the Transformer-based RNN enables uncertainty management and the ability to capture complex and non-linear relationships in the data, leading to more accurate and robust predictions. While Random Forest and KNN are effective for simpler, less complex tasks, these models struggle to capture the intricate patterns present in high-dimensional or non-linear datasets, such as those used in nanocomposite classification. Furthermore, the Bayesian + Transformer-based RNN hybrid model excels in modeling complex relationships that may be overlooked by models relying solely on statistical or heuristic- based feature selection, such as Random Forest or KNN. This makes the hybrid model particularly well-suited for nanocomposite data, where relationships between features are often non-linear and intricate. Additionally, the Bayesian component allows the hybrid model to quantify uncertainty, enhancing its ability to provide reliable predictions even in the presence of noisy or incomplete data. Lastly, the Transformer-based RNN component of the hybrid model automatically learns relevant features from raw data, reducing the need for extensive manual feature engineering, which is essential in Random Forest or KNN models. Therefore, the performance improvement arises from the synergy between Bayesian reasoning and the Transformer-based RNN, rather than solely from the increased complexity of the model. This ability to capture complex relationships, model uncertainty, and learn directly relevant features from the data enables the hybrid model to outperform traditional methods in this context. Finally, our study advanced the application of deep learning techniques in nanocomposite classification and addressed the need for managing uncertainties through Bayesian methodologies. By combining BNs and Transformer-based RNNs in a hybrid model, we established a powerful tool for predictive modeling in conductivity testing, paving the way for more reliable and insightful decision-making in materials science and engineering applications.

The experimental results demonstrate the successful application of machine learning techniques to classify nanocomposites, but their practical significance and potential applications warrant further explanation. These findings could have substantial real-world benefits, such as improving nanocomposite design, optimizing manufacturing processes, or enhancing the performance of materials for specific applications. For example, accurate classification systems could save time, reduce costs, and improve product quality by enabling rapid prototyping and discovery of materials with desired properties, such as enhanced conductivity for advanced electronics or energy storage. Additionally, this approach could aid in quality control during industrial manufacturing, ensuring consistent material performance. Highlighting these practical implications will not only underscore the real-world value of this research but also make it more impactful for industries such as electronics, aerospace, and medical devices.

Limitations

The hybrid method combining Transformer-based RNNs and Bayesian Networks (BNs) for classifying nanocom- posites has several advantages, but it also comes with some significant limitations such as the high computational complexity. In fact, the integration of Transformer-based RNNs with Bayesian Networks increases the computational load significantly. Transformer RNNs require substantial computational resources to process long or complex sequences, while Bayesian Networks add additional complexity due to the need for managing uncertainties and probabilistic calculations. This can lead to longer processing times and a higher demand for hardware resources. Dependency on Input Data Quality: As with any machine learning method, the performance of the model heavily depends on the quality of the input data. Incomplete, noisy, or poorly labeled data can negatively impact model accuracy. Transformer-based RNNs are particularly sensitive to erroneous sequences, and Bayesian Networks may become less effective if uncertainties are not well represented. Then, the overfitting Risk for thhe hybrid model may suffer from overfitting if the amount of training data is insufficient relative to the model’s complexity. In particular, Transformer RNNs have a high capacity to learn fine details in the data, which can lead to excessive fitting to the training data, reducing its ability to generalize to new data. In addition, the interpretability Challenges of Bayesian Networks provide a useful probabilistic component to understand uncertainties, the interpretation of the hybrid model as a whole can be difficult. Complex models like Transformer-based RNNs are often considered “black boxes,” and even with the integration of BNs, it can be challenging to explain why certain decisions are made, which is crucial for scientific and industrial applications. While RNNs and BNs can model complex relationships and uncertainties, this method may struggle to capture all the intricate interactions in highly heterogeneous or nonlinear systems. In the case of some nanocomposites, very specific physical phenomena might not be well represented by the model, leading to inaccuracies in predicting properties like conductivity.Besides, implementing this method requires expertise in both materials science and machine learning. Properly integrating nanocomposite theory with advanced machine learning models requires a deep understanding of both fields, which can be a barrier for researchers or practitioners who may not have expertise in both domains.

Conclusion

To promote the widespread adoption of machine learning models, especially neural networks, it was essential to tackle the issue of quantifying and effectively conveying the uncertainties tied to their predictions. A key focus was on epistemic uncertainty, which stems from gaps in knowledge or data, rather than from intrinsic randomness.

Bayesian methods provided a systematic framework for addressing these uncertainties. By employing probability distributions, these techniques not only represented the model’s predictions but also captured the associated uncertainty. This dual capability proved indispensable in scenarios where decision-making critically depended on the accuracy and confidence levels of the model’s outputs.

In our study, we focused on advancing the classification of nanocomposites for conductivity testing using a hybrid approach that integrated Bayesian Networks (BNs) and Recurrent Neural Networks (RNNs) based on Transformer architecture.

At the outset, we utilized Bayesian Networks to analyze and classify nanocomposites based on their conductivity properties. BNs offered a probabilistic framework to model the relationships and dependencies between nanoparticle distribution and their effects on conductivity. By leveraging this approach, we were able to gain deeper insights into the probabilistic nature of nanocomposite behavior, revealing how different nanoparticle distributions influenced conductivity outcomes. This probabilistic method facilitated more informed decision-making by providing confidence intervals and probabilistic predictions, as opposed to relying solely on deterministic results.

After conducting an analysis using Bayesian networks (BN), we introduced Transformer-based recurrent neural networks (RNNs) to enhance classification accuracy. Transformers are well-known for their ability to capture long- range dependencies and intricate sequential patterns in data. The RNN model was particularly adept at learning from sequential data, such as temporal variations in nanoparticle arrangements that influenced conductivity over time. By incorporating Transformer-based RNNs, our model effectively captured and utilized complex relationships within nanocomposite data, improving predictive accuracy compared to traditional sequential models.

To leverage the strengths of both BNs and Transformer-based RNNs, we proposed a hybrid model. This integrated approach combined the probabilistic reasoning of BNs with the deep learning capabilities of RNNs. The hybrid model synergistically utilized BNs to provide probabilistic insights and RNNs to exploit sequential dependencies, resulting in a more robust and accurate classification framework for nanocomposite conductivity.

Although traditional methods were applied to verify the effectiveness of our hybrid model, the results showed that it was even more effective. Experimental evaluations based on comprehensive metric assessments demonstrated that our hybrid model significantly outperformed standalone BNs and RNNs, validating its efficacy in predictive modeling and classification tasks.

In conclusion, our study not only advanced the application of advanced deep learning techniques in nanocomposite classification but also addressed the critical need for managing uncertainties through Bayesian methodologies. By combining BNs and Transformer-based RNNs in a hybrid model, we established a powerful tool for predictive modeling in conductivity testing, paving the way for more reliable and insightful decision-making in materials science and engineering applications.

Author contributions

Wejden Gazehi: Conceptualization, formal analysis and and writing original draftRania Loukil: Investigation, methodology and writing original draft. Mongi Besbes: Visualisation, validation, supervising and writing-Review and editing.

Data availability

The dataset analyzed in the current study can be obtained by contacting the corresponding author. [Wejden gazehi, wejden.gazehi@istic.ucar.tn]

Declarations

Competing interests

The authors declare no competing interests.