Multi-level screening method for network security alarms based on DBSCAN algorithm and rete rule inference

Abstract

In response to the limitations of existing network security alert screening methods in handling high-noise and incomplete data, this paper proposes a multi-level alert screening framework based on DBSCAN density clustering and RETE rule reasoning. The proposed method achieves adaptive analysis and precise screening of alert data by constructing a multi-stage processing pipeline that integrates density clustering, fuzzy reasoning, and dynamic neural networks. Key innovations include: employing the DBSCAN algorithm to perform unsupervised clustering and noise identification of alert data; introducing an improved RETE rule reasoning mechanism that supports weighted fuzzy matching to enhance fault tolerance for incomplete alert streams; and designing a BP neural network with dynamically adjustable structure to achieve accurate alert classification. Experimental results demonstrate that the proposed method achieves significant performance advantages on multiple real-world and benchmark datasets, with a true positive rate of 96.6%, a noise rate controlled within 18.7%, and CPU utilization below 1%, substantially outperforming existing mainstream solutions and exhibiting high practical application value.

Introduction

With the rapid development and widespread application of Internet technologies, network security has become a critical issue in the digital era. Enterprises, government agencies, and individual users face increasingly complex network attack threats during their online activities - such as malware, phishing attacks, and Distributed Denial of Service (DDoS) attacks. These attacks can lead to serious consequences, including data breaches and system crashes, and may also pose significant risks to national security, social stability, and personal privacy. To address these network security challenges, various types of security devices and systems have been deployed, such as Intrusion Detection Systems (IDS) and Security Information and Event Management (SIEM) systems. These systems monitor network activities in real time and generate a large volume of security alerts^1,2. However, due to the expansion of networks and the continuous evolution of attack methods, these alerts are often characterized by noise, incompleteness, and heterogeneity, which presents considerable challenges for security analysts. Therefore, quickly and accurately identifying genuine threats from massive alert data has become an urgent issue. As a result, research on multi-level screening of network security alerts is of great importance.

Jin et al.³ investigated computer terminal code vulnerability mining and developed a tool called Cker. While their approach demonstrated high detection accuracy for specific vulnerabilities such as heap overflow (HOF) and use-after-free (UAF), its performance on other vulnerability types was considerably lower, with detection rates sometimes barely exceeding 20%. This significant disparity highlighted a critical lack of comprehensive coverage, limiting the method’s utility for broad-based security alert screening where diverse threat types were prevalent. Wei et al.⁴ addressed network intrusion detection by integrating an improved K-means clustering algorithm with artificial intelligence techniques. Although their model attempted to overcome traditional issues such as low efficiency and poor accuracy, it remained fundamentally constrained by the inherent limitations of the K-means paradigm - including sensitivity to initial centroid selection and the requirement to predefine the number of clusters (K-value). These constraints often led to suboptimal and unstable clustering performance when applied to the complex, non-uniform distributions characteristic of real-world network alert data. Zeng et al.⁵ focused on network anomaly detection using an Error Back Propagation (BP) neural network optimized with bio-inspired algorithms such as the tuna swarm and improved sparrow search algorithm. Their model achieved high convergence speed and an abnormal data type recognition accuracy exceeding 90%; however, its notably low average recall rate of 29.568% revealed a substantial weakness in identifying true positive security incidents, which was a paramount requirement for effective threat detection. Finally, Zuberi et al.⁶ proposed an intelligent alarm system for the Internet of Things (IoT) that leveraged machine learning to analyze sensor data and trigger alerts upon detecting suspicious activities, achieving an accuracy of 91.12%. Despite its high performance within the constrained and specific context of IoT environments, the system’s reliance on particular sensor data and IoT network characteristics raised questions about its generalizability and effectiveness when applied to conventional IT network infrastructures with different data patterns and attack vectors.

The core novelty of this study lies not in the invention of new base algorithms, but in their strategic integration into a coherent, multi-stage pipeline that systematically addresses the aforementioned limitations. We demonstrate that the synergy between these carefully selected and adapted components yields superior performance compared to what any single technique or simple ensemble could achieve.

Specifically, the proposed method is designed to leverage the unique strengths of each stage: First, the Density-Based Spatial Clustering of Applications with Noise (DBSCAN) algorithm is employed not merely for clustering, but as an adaptive segmentation tool that replaces the rigid, predefined structure of K-means⁴. It automatically identifies dense regions of alerts in the time domain, effectively filtering noise and creating coherent event windows without requiring a predetermined K-value. Second, the FP-growth and improved Im-Rete algorithms operate on these cleansed event windows. Their combination is crucial - FP-growth efficiently discovers frequent co-occurrence patterns, while the Im-Rete algorithm, enhanced with a fuzzy inference strategy, provides robust rule-based correlation that is tolerant to missing alerts often found in real network data, thereby addressing a key weakness in systems like^3,6. Finally, these correlated and enriched features are fed into a dynamically optimized BP neural network. This final stage overcomes the static model limitations of⁵ by adapting its structure to the data complexity, enabling it to learn the nuanced, non-linear relationships between correlated alerts that signify a true multi-stage attack, which consequently achieves higher recall and true positive rates.

The technical contributions of this paper are summarized as follows:

1. A novel multi-stage alert screening framework that intelligently integrates density-based clustering, frequent pattern mining, fuzzy rule inference, and neural network classification to address the full lifecycle of alert analysis, from noise reduction to root cause identification.

2. An improved Rete algorithm (Im-Rete) incorporating a fuzzy inference strategy and dynamic node pruning, which enhances tolerance to missing alert data and improves the efficiency of correlation analysis in noisy environments.

3. A time-series filtering mechanism for association rules that incorporates temporal attack logic, effectively pruning semantically incorrect rules generated by conventional unordered mining algorithms like FP-growth.

4. A dynamic optimization strategy for the BP neural network that automatically adjusts the number of hidden layer units during training, preventing overfitting and underfitting to achieve robust performance across diverse alert datasets.

5. Comprehensive experimental validation on multiple public and private datasets, demonstrating significant improvements in processing speed, noise reduction, true positive rate, and computational efficiency compared to state-of-the-art baseline methods.

A data mining framework for network security alarm correlation

Network security alarm data preprocessing

Network alarm data primarily originates from alarm databases, system logs, asynchronous alarms, and active polling. Although diverse sources provide basic information for fault detection⁷, preprocessing alarm data is crucial for enhancing data quality. Original data often contain noise, duplication, missing information, and inconsistencies, which impair analysis accuracy. Operations such as normalization, attribute selection, deduplication, redundancy removal, cleaning, and sorting can reduce errors, rendering the data more accurate, complete, and consistent. High-quality data underpins multi-level security alert screening by ensuring its accuracy and effectiveness. It also reduces storage and computing requirements, lowers algorithm complexity, and improves screening efficiency.

Figure 1 illustrates the specific workflow of the network security alarm information preprocessing module. The module comprises two components: (1) Formalized data processing, where the original alarm log - generated by multi-source devices - is formalized to produce alarm data in a unified format for subsequent analysis; (2) Data preprocessing steps, including attribute selection, deduplication, data cleaning, and sorting. The preprocessed alarm data serves as the input for subsequent association analysis.

Fig. 1.

Process of alarm information preprocessing module.

Formalized data processing

The original data collected by the computer situational awareness platform are alarm logs from various security devices. In theory, alarm correlations can be obtained from device manufacturers or domain experts. However, owing to the lack of uniform industry specifications, alarm information formats vary according to different manufacturers’ specifications. Therefore, device-specific alarm rules cannot satisfy all requirements⁸. To address these issues, combined with the business characteristics and specific requirements of the computer system, multi-source alarm logs are formalized according to a set of standards provided by a computer company. After standardization, the alarm logs maintain a unified format with parallel important attributes. The parsed alarm logs are then stored in the alarm database system, serving as the original data for subsequent analysis in this study.

Data preprocessing steps

The data preprocessing step comprises four components: (1) attribute selection, extracting relatively important alarm attributes; (2) deduplication, removing redundant and duplicate alarm information; (3) data cleaning, handling missing data values; (4) data sorting, arranging alarm data in chronological order.

Attribute selection

The alarm logs captured by the computer situational awareness platform retain detailed information about attack events. After formal processing, the alarm data still contain numerous attributes. Because large-scale alarm data require substantial storage space, and excessive fields hinder the mining of alarm association patterns, this study first selects features from the alarm data. Based on computer network characteristics and preliminary analysis of alarm data, this study selects a set of relatively important attribute features, Alarm={alm_id, alm_time, alm_name, attack_tgt, attack_src}, constituting the alarm 5-tuple. The alarm ID alm_id identifies an alarm attack event. The alarm time alm_time and attack event alm_name participate in each stage of the network security alarm information association analysis framework. The attack source attack_src and attack target attack_tgt are used for deduplication and redundancy removal of alarm information⁹.

Deduplication and redundancy removal

Duplicate alarms are classified into two types: (1) those generated by the same security device, and (2) those triggered by different security devices. Their common characteristic is that the repeated alarm information corresponds to the same attack event. According to prior knowledge and alarm information characteristics, repeated alarm information typically occurs within a very small time threshold . The duplicate alarms defined in this paper are shown in Eq. (1):

1

According to Eq. (1), the conditions for events i and j to be considered duplicate alarms are as follows: they share the same alarm name, attack source, and attack target; and the time difference between attack event i and attack event j falls within threshold ¹⁰.

Data cleaning

If the alarm data in the computer network situational awareness platform reaches a certain volume over a daily period, some missing fields are inevitable. In data analysis, three primary methods exist for handling missing values: simple deletion, imputation, and no processing. Given that the focus of this study is network security alarm data, and missing values are insufficient to constitute a complete attack event, the simple deletion method is adopted for missing values, which proves effective for this context.

Data sorting

After duplicate and redundant alarm data entries have been removed, the dataset is sorted using alarm time as the key. This sequential ordering is necessary because the construction of the subsequent alarm transaction set requires ensuring that alarm events maintain temporal sequence properties.

Clustering of network security alarm data

DBSCAN is a density-based clustering algorithm that defines clusters through density, unlike K-Means which requires a predefined cluster number. It clusters data based on local density distribution, automatically identifying clusters of arbitrary shapes while effectively handling noisy data. Network security alarm data often exhibits inherent uncertainty and density variations, to which DBSCAN adapts effectively by grouping similar alarms and filtering out noise. Using DBSCAN, flow alarm data is partitioned into multiple temporal segments based on preprocessing results and original data characteristics. Optimal DBSCAN parameters are selected through constraint optimization, and sliding time windows are employed to extract alarm transactions for clustering. This methodology enhances both the processing efficiency and accuracy of alarm data analysis, thereby supporting subsequent security analysis and decision-making processes.

When selecting parameters for the DBSCAN algorithm, the data distribution must be thoroughly considered. Through visualization and analysis of the raw alarm data, preliminary estimates can be made for the potential ranges of the neighborhood radius and the minimum number of points. A distance-based statistical method is adopted to calculate the average distance between data points, which serves as the initial estimate for the neighborhood radius. The minimum number of points is determined based on data dimensions and the expected minimum cluster size. To further optimize parameter selection, the dataset is divided into training and validation sets. Different parameter combinations are applied for clustering on the training set, and the clustering quality is evaluated on the validation set. The parameter combination that demonstrates the best performance on the validation set is selected as the final DBSCAN configuration.

The occurrence of computer alarms is stochastic; however, highly correlated alarm data tend to cluster densely within specific time periods, whereas weakly correlated or uncorrelated alarms typically occur at larger intervals. Under the influence of various factors, a small number of noise alarms are also generated. Using a fixed sliding time window may truncate highly relevant alarm data into different transactions during dense alarm periods, while generating numerous empty alarm transactions during sparse periods and failing to filter out noise alarms, thereby reducing the efficiency and accuracy of alarm association rule mining. The DBSCAN algorithm, as a density-based clustering analysis method, can identify clusters of arbitrary shapes without requiring a predefined number of clusters and can effectively handle dataset noise. Consequently, the DBSCAN algorithm is employed to partition alarm data into multiple intensive alarm periods, determine optimal input parameters based on constraints, and then apply a sliding time window to extract alarm transactions within each period. This approach not only mitigates the impact of noise alarms on transaction extraction but also identifies optimal input parameters, offering significant flexibility and practical utility¹¹.

The sliding time window primarily addresses the issues of network device time asynchrony and excessively small intervals between alarm events. Consequently, the sliding time window method is employed to extract transactions within each period after clustering. Since alarm events within the same time window belong to the same transaction, the valid range for the time window width W is , where represents the maximum interval between adjacent alarms within the time range, and denotes the time range width. To fully utilize alarm data and prevent nearly simultaneous alarms from being truncated into separate alarm transaction sets, the sliding step size should be selected to ensure sufficient overlap between consecutive time windows. A smaller sliding step results in greater overlap of alarm data between adjacent windows and generates more extracted transactions, whereas a larger sliding step produces less overlap and fewer extracted transactions. When the sliding step exceeds the time window width, some alarm information may be missed. Therefore, the valid range for the sliding step s is , where indicates the minimum interval between adjacent alarms, and W represents the time window width¹².

Implementation of association mining for network security alarm data

Network security alarm association mining builds upon the clustering of network security alarm data to further analyze correlations among alarms. Following data preprocessing, the alarm system must mine rules from the processed alarm information and generate corresponding association rules to address emerging network risks, thereby establishing a critical foundation for subsequent multi-level screening of network security alarms.

The FP-growth algorithm is employed for mining association rules from alarm information. During association rule mining, the FP-growth algorithm traverses the database twice without generating candidate item sets, and this streamlined process enhances algorithmic efficiency¹³. The key factor enabling the FP-growth algorithm to effectively improve computational efficiency lies in its compression of the dataset into a Frequent Pattern Tree (FP-tree) while preserving the original data structure. The specific computational procedure for the FP-growth algorithm is as follows:

Let represent the set of all distinct items in the dataset, T denote a non-empty subset of I constituting a transaction, D represent the transaction set (dataset), and N indicate the total number of transactions. The calculation proceeds as follows:

1. Perform the first scan of the dataset to identify the corresponding frequent itemsets and their support counts, where support count refers to the frequency of each item’s appearance in the dataset. Sort the frequent itemsets in descending order to obtain Table L. With the support threshold set to 2, the procedure continues as follows:

   2

2. Construct the FP-tree by initializing its root node. Perform the second database scan, sort the items according to Table L, and build the tree paths using the root node as the starting point.

After completing the path creation for all transactions in the dataset, the frequent item header table is constructed concurrently with the FP-tree. Finally, the suffix pattern is linked to the frequent pattern generated by the conditional FP-tree to produce the final frequent itemset.

Although the FP-growth algorithm addresses the computational intensity and inefficiency issues of the Apriori algorithm in data mining, it still exhibits limitations including low efficiency, computational complexity, and reduced detection rates when applied to network security alarm association rule mining. To overcome these limitations of the FP-growth algorithm, this paper proposes an enhanced approach incorporating dynamic node insertion. Specifically, the FP-tree generation process is improved and integrated with the Max-IFP maximum frequent pattern mining algorithm to perform data mining¹⁴, resulting in an improved Max-IFP association algorithm.

The dynamic node insertion method dynamically adjusts the node insertion sequence based on the current node’s support and item frequency, ensuring the compactness and balance of the FP-tree structure. During implementation, starting from the root node, the support or frequency of each potential child node is calculated in real-time, with items possessing higher support or frequency being prioritized for insertion as child nodes. As an optimization technique for mining maximum frequent itemsets, the Max-IFP algorithm effectively reduces the number of candidate itemsets by incorporating the concept of maximum frequent itemsets during the FP-tree construction phase. Specifically, it performs backtracking from the leaf nodes of the FP-tree to identify maximum frequent itemsets that are not contained within other frequent itemsets, subsequently generating association rules. These two enhancement methods collectively improve the efficiency of the FP-growth algorithm when processing large-scale or high-dimensional datasets. The pseudocode of the improved FP-growth algorithm, illustrating its main procedural steps, is presented below:

Algorithm.

Improved FP-Growth.

The specific improvement of the algorithm is as follows: assuming the new FP-tree is a directed tree, the construction procedure primarily consists of: (1) performing the first scan of the transaction database to obtain corresponding support counts while generating frequent 1-itemsets, which are sorted in descending order based on their occurrence frequency, denoted as L; (2) constructing the root node labeled “null,” then sorting the frequent items of each transaction in the selected database according to L; (3) representing the sorted frequent itemset as , where P denotes the first element and P represents the remaining list of elements.

Implementation of a multi-level screening method for network security alerts

Analysis of network security alarm correlation based on the rete algorithm

The Rete algorithm serves as an efficient pattern matching method for analyzing alert correlations. Through rule definitions, the system identifies logical relationships among alerts and executes corresponding actions. This approach enhances rule engine matching efficiency and has been applied to alert data correlation analysis to improve root cause identification¹⁵. The system matches real-time alerts with predefined rules, analyzes correlated alert patterns, and identifies root causes to assist network operators in rapid security vulnerability localization and diagnosis. However, as a pattern-based exact matching algorithm, the conventional Rete approach may miss critical alert data due to network impairments including link failures, congestion, latency, and packet loss, consequently compromising the accuracy of alert-rule matching and inference.

To address the aforementioned issues, an improved rule reasoning algorithm named Im_Rete is proposed. This algorithm incorporates a fuzzy inference strategy for handling missing alarms based on the characteristics of network security alert data, thereby adapting to the problem of alarm loss caused by network security vulnerabilities. It maintains a balance between reasoning speed and accuracy enhancement, enabling more effective analysis of alert correlations.

The rule representation in traditional alert correlation analysis is shown in Eq. (3), where represents the n-th pattern in the i-th rule and denotes the conclusion of the i-th rule.

3

The Rete algorithm is a pattern matching algorithm where, for Eq. (3), the rule can only be activated if all values of are satisfied. If any join operation fails during propagation through the inference network, rule matching fails. However, this precise matching approach is inadequate for handling the problem of missing alarms caused by network security vulnerabilities. To address this limitation, a fuzzy inference strategy oriented toward alarm missing is adopted, which assigns different weights to the rule antecedents. If a particular join operation fails and the corresponding pattern is not critical to the rule, the failure can be ignored, thereby enhancing the fault tolerance of the inference network. The improved rule can be expressed as follows:

4

In Eq. (4), denotes the weight of pattern , with and . Let represent the truth value of pattern : when facts match , ; otherwise, . For a chosen threshold , the rule can be activated as long as the weight sum of the facts matches is not less than , as expressed in Eq. (5).

5

The Im-Rete algorithm, as an enhancement to the traditional Rete algorithm, addresses the issues of alarm missing and alarm storms in network fault diagnosis. By introducing a fuzzy inference strategy that employs membership functions to quantify alarm uncertainty, it enables fuzzy matching for certain rules. Combined with a probability association model, it constructs an association matrix based on historical data and prioritizes matching high-probability rule branches to reduce invalid computations. Simultaneously, it utilizes a dynamic node pruning mechanism to remove long-term inactive nodes and subtrees within the Beta network, thereby reducing memory consumption. Through hierarchical matching optimization, rules are categorized according to conditional complexity, with simple conditions processed preferentially to rapidly screen candidate rules, consequently improving diagnostic efficiency and accuracy. The pseudocode of the Im-Rete algorithm, outlining its primary procedural steps, is presented below:

Algorithm.

Im-Rete.

The Im-Rete algorithm employs a fuzzy inference strategy designed to handle missing alarms, assigning a weight to each antecedent pattern in the rule based on Fuzzy C-Means (FCM) clustering, with a matching threshold . During network propagation, if a join operation fails, the failure is ignored provided the minimum among the rule patterns is less than . When propagation reaches a terminal (leaf) node, the rule is activated if is within the preset threshold ; otherwise, activation fails. To determine rule weights more accurately, the FCM algorithm computes the fuzzy membership degree of an alarm relative to the root alarm.

Alarm quantification

Each alarm attribute reflects the importance of an alarm from a different perspective. Therefore, fuzzy alarms are employed to comprehensively fuzzify all attribute values of alarms, with the quantification of alarm attributes serving as a prerequisite for this process. The following properties are selected:

① Alarm node: The number of links connected to an alarm node reflects its topological importance, where a higher number of connections indicates greater significance¹⁶.

② Alarm severity: Predefined by vendors and experts, alarm severity ranges from 1 to 5, with larger values representing more severe alarms.

③ Alarm type: According to the TCP/IP model, lower layers provide services to upper layers. Security vulnerabilities in lower layers can enable intrusion behaviors to propagate upward through these services, triggering secondary alarms in higher layers, making lower-layer alarms relatively more important.

Determination of fuzzy weights

Linguistic variables are defined to evaluate fuzzy set based on proximity to the root alarm, with proximity degrees decreasing sequentially. Thus, fuzzifying alarm J involves finding its fuzzy membership vector relative to F. The FCM implementation partitions n vectors into c fuzzy clusters (where c = 5 represents the number of alarm levels), with each alarm belonging to clusters with certain membership values within the interval [0,1].

Membership matrix:

6

Normalization processing of membership vectors:

7

Objective function:

8

In Eq. (8), represents the i-th cluster center, is the weighted index¹⁷, and denotes the Euclidean distance between data element j and cluster center . A new objective function is constructed as follows:

9

In Eq. (9), represents the Lagrange multiplier for the constraint. Differentiating Eq. (9) yields:

10

11

In Eqs. (10) and (11), the pair constitute the necessary condition for minimizing the objective function¹⁸.

Let the linguistic fuzzy evaluation set have corresponding weights , respectively. Then, for an alarm J with fuzzy membership vector , the weights are determined as follows:

12

Chronological-order-aware filtering of network security alarm rules

By incorporating chronological ordering, real-time monitoring and response to network security alerts can be effectively achieved. When a new alert is generated, the system can immediately perform filtering and judgment based on predefined rules, enabling timely implementation of corresponding security measures. This approach helps reduce alert processing latency and enhances the overall responsiveness of the network security system. Through chronological filtering, repeated alerts occurring within short time intervals that demonstrate no significant threat can be eliminated, thereby reducing the workload for subsequent multi-level screening. This optimization of the screening process contributes to improved efficiency throughout the entire system’s processing pipeline.

The Rete rule inference engine facilitates the filtering of chronological alarms by identifying temporally specific relationships. While the FP-Growth algorithm, as an unordered mining method, may discover both A→B and B→A rules, such bidirectional associations are often invalid in computer security alarm scenarios. For example, directory traversal (a reconnaissance attack) and injection attack (an execution phase) possess distinct temporal sequences where the reverse rule becomes meaningless. Empirical alarm data demonstrates that authentic associations follow temporal sequences, where earlier attacks trigger subsequent ones, consistent with typical network attack patterns (target selection, reconnaissance, execution)¹⁹. Therefore, after mining alarm rules using FP-Growth, this study introduces a time-sequence-based filtering algorithm (Algorithm 1) to incorporate temporal relationships among alarm events and reduce redundant rules. Figure 2 illustrates the process of obtaining valid computer security alarm rules.

Algorithm 1.

Filter alarm association rules.

Fig. 2.

Flowchart of generating alarm rules.

The core innovation of the flowchart in Fig. 2 lies in introducing a temporal filtering layer to address logically inconsistent rules generated by traditional association rule mining algorithms like FP-Growth in cybersecurity contexts. This additional step verifies whether the rule antecedent temporally precedes the consequent, ensuring that final rules such as “scan → exploit” align with realistic attack sequences. This enhancement significantly improves the logical coherence and practical utility of the generated rules, thereby establishing a solid foundation for subsequent precise screening stages.

The rule filtering algorithm processes Original_Rules, which constitute the initial rule set mined from alarm transactions using the FP-Growth algorithm, and produces Effective_Rules as output, representing the valid alarm rules satisfying temporal constraints. The algorithm employs Alarm_TimeSet as an auxiliary temporal filtering tool. This alarm time set is generated concurrently during transaction creation through a dynamic sliding time window approach. Each transaction window records both alarm event identifiers and their corresponding occurrence timestamps²⁰. Consequently, utilizing Alarm_TimeSet enables efficient filtering of alarm rules that violate chronological dependencies.

Multi-level screening of network security alerts using an improved BP neural network

The traditional BP neural network features a fixed structure where the number of hidden layer units is predetermined during the design phase. However, practical applications in network security alert multi-level filtering require different network architectures across various tasks and datasets to achieve optimal performance. The dynamic adjustment approach for hidden layer units enables the network to adapt its structure according to task requirements and data characteristics, thereby enhancing both flexibility and generalization capability. This methodology prevents training inefficiencies caused by excessively complex or simplistic network configurations: overly complex networks result in slow training processes and susceptibility to local optima, while overly simplistic networks fail to capture complex data patterns. The dynamic adjustment mechanism incrementally modifies the number of hidden layer units based on training feedback, maintaining network training efficiency. Applying the optimized BP neural network to multi-level alert screening represents an innovative approach, as such filtering typically involves substantial data volumes that demand efficient and accurate processing algorithms. The BP neural network’s powerful nonlinear mapping capacity and self-learning capability enable rapid classification and screening of alert information, consequently improving both the efficiency and accuracy of alert processing. The pseudocode for the enhanced dynamic BP network, outlining its primary procedural steps, is presented below:

Algorithm.

Dynamic BP network.

The output from chronological filtering serves as the input for multi-level filtering, contributing to further alarm reduction and improved screening accuracy. Since the alarm data in this study are represented in textual format and the nodes containing security vulnerabilities correspond to specific device endpoints, they cannot be directly utilized as inputs for multi-level alarm filtering. Consequently, preprocessing is required to transform this information into corresponding vector representations. This paper defines an m-dimensional alarm vector as follows:

13

In Eq. (13), represents a feature within the m-dimensional alarm vector. An n-dimensional network security vector is defined as:

14

In Eq. (14), indicates the presence of security vulnerabilities at device points. Additionally, the input and output vectors of hidden layer neurons are defined as and , respectively:

15

16

The output layer contains a number of neurons equal to the target dimension, with its input and output vectors defined relative to the hidden layer model as follows:

17

18

The connection weights and denote the weights between input neuron i and hidden neuron j, and between hidden neuron j and output neuron t, respectively. The terms and represent the thresholds of hidden neuron j and output neuron t, respectively, where . In designing the hidden layer, the primary challenge involves selecting the number of neurons, which is often determined empirically through extensive experimentation, as no exact analytical expression exists for this purpose. The number of hidden units depends strongly on problem complexity and the counts of input and output units. Excessively many hidden units prolong training time, may yield suboptimal errors, reduce fault tolerance, and impair generalization to unseen samples. Thus, an optimal number of hidden units must be established.

During the optimization of the BP neural network, a dynamic adjustment strategy for hidden layer units is implemented. Specifically, the model begins training with a minimal number of hidden units and gradually increases this number as training progresses. After each unit augmentation, model performance is evaluated on a validation set. The expansion ceases when validation performance no longer demonstrates significant improvement. Furthermore, pruning and growth strategies are incorporated by eliminating redundant connections and units while introducing new units based on actual requirements to optimize the network architecture. For convergence determination, a dual criterion combining loss function and performance metrics is established: training terminates when the loss function remains below a predefined threshold for multiple consecutive epochs while validation performance shows no substantial improvement. Based on these definitions, this study utilizes randomly selected groups of and as the BP network’s input and target vectors, employing the following algorithm for network training.

Algorithm 2.

BP network training.

Experimental design

Experimental setup

To validate the effectiveness of the multi-level network security screening method integrating the DBSCAN algorithm and Rete rule inference proposed in this paper, four representative state-of-the-art methods published in recent years are selected as performance comparison baselines. These baseline methods encompass multiple technical approaches for network security threat detection: the vulnerability mining-based method from reference³ represents technical pathways addressing attack root causes; the enhanced clustering approach in reference⁴ exemplifies unsupervised learning solutions for anomaly detection; the optimized neural network method in reference⁵ demonstrates supervised deep learning applications in anomaly detection and early warning; the IoT and machine learning integrated system in reference⁶ represents intelligent alert technology in specific application scenarios. Through systematic comparison with these diverse technical routes, the comprehensive performance of the proposed multi-level screening method can be thoroughly validated in terms of effectiveness, superiority, and generalizability.

The relevant parameters and advantages and disadvantages of different methods are shown in Table 1.

Table 1.

Comparison between existing methods and the proposed method.

Method	Core parameters	Strengths	Limitations
Methods of literature3	Vulnerability type features; Detection threshold;	High precision detection of HOF and UAF vulnerabilities; Suitable for small-scale datasets	Low detection rate for non-target vulnerability types; Limited applicability;
Methods of literature4	K value (number of clusters); Initial centroids;	Improved detection accuracy; Suitable for static network environments	Depends on pre-defined K values; Poor adaptability to dynamic network environments
Methods of literature5	Network layers; Number of neurons; Learning rate;	Fast convergence speed; Strong ability to recognize regular patterns;	Low accuracy in detecting specific types of abnormal data; Dependent on predefined parameters
Methods of literature6	Sensor types; Feature dimensions;	High accuracy detection of suspicious network activity	Lack of multi-level alarm screening and correlation analysis; Suitable for anomaly detection
Design method	DBSCAN: eps, min_samples; Im-Rete: Activation threshold θ; BP Network: Dynamic structure adjustment parameters;	Fuzzy reasoning with strong fault tolerance Dynamic network, self optimizing structure High accuracy, low noise Efficient computing	Multiple parameters need to be adjusted; -Long training time for large-scale datasets

This study conducted experimental validation using widely adopted public benchmark datasets and real industrial private datasets.

For public benchmarks, two recognized datasets in cybersecurity research are selected: CIC-IDS2017, which simulates realistic network environments containing various common attacks including brute-force, DoS, DDoS, web attacks, and infiltration, providing comprehensive network traffic and alert features - with the “Thursday” data containing complete attack cycles being utilized; and UNSW-NB15, which incorporates complex attack behaviors in modern hybrid networks and generates extensive alert logs from network traffic, using its standard partitioned training and testing sets.

For private datasets, to verify methodological practicality in specific industrial scenarios, alert logs are collected over seven consecutive days (May 2022) from a Chinese computer company’s network security situational awareness platform, comprising approximately 100,000 raw alerts. Through analysis of data characteristics and simulation based on its format and distribution patterns over two months, an expanded dataset of approximately 200,000 alerts is generated, containing 220 network nodes and 296 distinct alert types. The experimental environment utilizes Windows 10 operating system, AMD Ryzen 3700 × 8-core CPU, and 16GB RAM, with Python 3.7.3 as the software platform. Table 2 presents alarm data samples and analyzes their characteristics, while simulations conducted according to the alarm data format and distribution patterns over more than two months generate approximately 200,000 alerts including 220 network nodes and 296 different alarm types.

Table 2.

Sample alarm data.

Alarm serial number	Alarm ID	Alarm name	Alarm recovery type	Alarm severity	Event type	Alarm time	Alarm source
376,905	11,606	User resource congestion	Security risk recovery	Secondary	Service quality	2022/11/13 11:35:42	Node2
376,904	11,935	Process CPU overload	Security risk recovery	importance	Operating system	2022/11/13 11:35:36	Node3

Experimental result analysis

To evaluate the robustness of the proposed method and guide parameter configuration, sensitivity analysis is performed on key parameters of the three core algorithms. All experiments are conducted on the private dataset using a controlled variable approach, where all parameters except the target parameter remain fixed. The results are reported as average values of key performance indicators (accuracy rate, noise rate).

As shown in Table 3, the neighborhood radius eps of DBSCAN is crucial for clustering performance. When eps is too small (0.1), the algorithm fails to form effective clusters, causing numerous related alarms to be misclassified as noise, which sharply increases the noise rate and reduces the true positive rate. When eps is too large (1.0), clusters from different attack events are incorrectly merged, similarly increasing the noise rate and decreasing the true positive rate. When eps = 0.5, the algorithm best identifies natural clusters of alarm events, achieving optimal balance between true positive rate and noise rate.

Table 3.

Sensitivity analysis of DBSCAN parameters (min_samples = 5).

Neighborhood radius (eps)	Positive discriminant rate (%)	Noise level (%)	Remarks
0.1	89.5	28.4	Cluster too fragmented, too many noise points
0.3	95.8	19.2	Better performance
0.5	96.6	18.7	Recommended parameters for stable performance
0.7	95.1	22.5	Merge different clusters to increase noise
1.0	90.3	30.1	The cluster is overloaded, and performance is significantly reduced.

Table 4 illustrates the sensitivity of the fuzzy rule activation threshold θ in the Im-Rete algorithm. While a lower threshold (0.5) maintains high recall, it activates numerous low-confidence rules, introducing excessive false positives that decrease the true positive rate and increase noise rate. Conversely, an excessively high threshold (0.9) ensures high reliability (low noise rate) for activated rules but filters out many valid rules, causing false negatives and reducing the true positive rate. The value θ = 0.7 represents an optimal balance that effectively controls noise while maintaining high accuracy.

Table 4.

Sensitivity analysis of the rete algorithm’s rule activation threshold.

Threshold (θ)	Correct judgment rate (%)	Noise ratio (%)	Remarks
0.5	92.1	25.8	The rules are too lenient, resulting in many false positives
0.7	96.6	18.7	Recommended parameters, balance between accuracy and recall
0.8	97.5	16.1	The rules are stricter, and the risk of underreporting increases slightly
0.9	94.2	14.5	The rules are too strict, resulting in a large number of missed reports

From Table 5, the dynamic BP network demonstrates insensitivity to its initial hidden layer size, reflecting the effectiveness of its dynamic structure adjustment strategy. Even when initialized with a compact architecture (e.g., 5 neurons), the algorithm achieves near-optimal performance through its growth mechanism. However, selecting a moderate initial size (e.g., 10 neurons) yields optimal efficiency while avoiding unnecessary training time costs. Conversely, an excessively large initial network (e.g., 50 neurons) not only fails to improve performance but may also induce overfitting due to model complexity, significantly increasing training duration.

Table 5.

Sensitivity analysis of the initial hidden layer size in dynamic BP networks.

Size of initial hidden layer	Final hidden layer size	Correct judgment rate (%)	Training time (s)	Remarks
5	18	96.2	125	Start with a small network and grow it multiple times
10	15	96.6	98	Recommended parameters for fast convergence and good performance
20	22	96.5	115	The initial scale is slightly larger, and the training time is not advantageous
50	51	95.8	210	The network is too large and prone to overfitting, resulting in performance degradation.

Parameter sensitivity analysis demonstrates that the proposed method maintains stable and superior performance across reasonable ranges of key parameters (eps between 0.3 and 0.7, θ between 0.7 and 0.8, initial hidden layer size between 5 and 20), confirming the method’s robustness. Furthermore, the analytical results provide clear parameter configuration guidelines for practitioners, thereby enhancing the method’s practical applicability.

To ensure objective evaluation of the multi-stage network security alert screening approach integrating DBSCAN clustering and Rete rule inference, the time required for generating alert-associated transaction databases was compared between the proposed method and existing approaches. Since MATLAB operates as an interpreted language, program execution times exhibit inherent variability. This study therefore uses the average of three experimental trials as the final comparative metric. The comparative results are presented in Fig. 3.

Fig. 3.

Comparison of filtering time for the updated alarm database across different methods.

As shown in Fig. 3, the multi-stage network security alert screening method integrating DBSCAN clustering and Rete rule inference requires significantly shorter processing times for updated alert databases, consistently completing within 100 s. In contrast, existing methods documented in literature exceed 90 s for the same task. The proposed design demonstrates superior efficiency in mining updated alert databases. Furthermore, as incremental datasets are added more frequently, the temporal efficiency of the incremental mining algorithm increases correspondingly, thereby conserving computational resources. These results clearly demonstrate the procedural efficiency of the proposed methodology. Compared to approaches relying on single complex models (such as the optimized BP neural network in reference⁵ or specialized scenario models (such as the IoT system in reference⁶, the multi-level pipeline design achieves computational load distribution and optimization. Specifically, DBSCAN clustering efficiently performs noise filtering and data segmentation, substantially reducing both the volume and complexity of data for subsequent association rule mining and neural network filtering. Additionally, the enhanced FP-Growth and Im-Rete algorithms avoid substantial unnecessary computations through their efficient data structures and fuzzy matching strategies. Consequently, despite incorporating more processing stages, the overall processing time is markedly reduced, confirming the efficiency advantages of this integrated architecture.

To evaluate the advantages of the proposed method in analyzing and processing network security alert data, this study conducts comparative tests with existing literature methods. Thirty-two sets of computer network security alert data are randomly selected from the dataset. After parameter optimization, the designed method first performs K-means temporal clustering combined with DBSCAN keyword clustering on the alert data, generating initial clustering results. Subsequently, two temporal dimensions are incorporated into the DBSCAN clustering input while maintaining identical parameters to obtain secondary clustering results. The noise rate outcomes for both clustering approaches are presented in Table 6.

Table 6.

Comparison of noise ratio between the two algorithms.

Type	Design method	Methods of literature3	Methods of literature4	Methods of literature5	Methods of literature6
Noise rate/%	18.70	20.50	20.80	20.33	20.03
Number of noise points	6	10	11	12	13
95% confidence interval	[17.92,19.48]	[19.65,21.35]	[19.98,21.62]	[19.51,21.15]	[19.20,20.86]

The comparative results in Table 6 demonstrate that the proposed method achieves the lowest noise rate at only 18.70%, indicating superior noise reduction performance. The methods from references⁴ to⁶ yield noise rates of 20.5%, 20.8%, 20.33%, and 20.03% respectively, showing similar performance with minor variations. Although reference⁶ achieves the lowest noise rate (20.03%) among comparative methods, its advantage remains marginal. The proposed method also produces the fewest noise points (6), consistent with its optimal noise rate. According to the 95% confidence intervals in Table 2, the proposed method demonstrates higher statistical stability in noise suppression, with a confidence interval of 17.92%−19.48% and width of 1.56% points - significantly narrower than the 1.70–1.84% points observed for references^3–6. This indicates stable noise rates between 17.92% and 19.48% with minimal sensitivity to experimental conditions, confirming strong robustness. In contrast, the wider and overlapping confidence intervals of references^3–6 - such as the partial overlap between reference⁵ (19.51%−21.15%) and reference⁶ (19.20%−20.86%) - reflect greater performance fluctuation without clear statistical distinction, further emphasizing the stability advantage of the proposed method. In summary, the proposed method exhibits optimal noise reduction performance both in terms of noise rate magnitude and noise point quantification.

To evaluate the application performance of the proposed method, the true positive rate is measured using both existing literature methods and the proposed approach across ten subnet alarm datasets. To ensure statistical reliability in performance comparison, the following significance testing strategy is implemented: thirty independent experimental trials are conducted for all methods, with random stratified sampling employed for training-testing splits (70%/30%) in each trial. All performance metrics are recorded per experiment. Subsequently, paired sample t-tests are performed to compare the proposed method against each baseline across all metrics at a 95% confidence level (α = 0.05).

The comparative true positive rates of different methods are presented in Fig. 4.

Fig. 4.

Comparison of root-cause alarm detection rates among different methods.

The comparative results of root cause alarm true positive rates across different methods are presented in Fig. 4. The proposed method achieves an average true positive rate of 96.6% ± 1.2% across ten subnets, with the lower bound of its 95% confidence interval exceeding the upper bounds of all comparative methods. Paired t-test results demonstrate statistically significant differences between the proposed method and reference³ (76.5% ± 3.5%)⁴, (75.8% ± 4.1%)⁵, (78.3% ± 3.8%), and⁶ (61.2% ± 3.2%), with all p-values < 0.001. This provides strong statistical evidence for the superior and more stable detection accuracy of the proposed approach. The substantial accuracy improvement directly validates the synergistic benefits of multi-technique integration. Specifically, DBSCAN effectively eliminates interference terms, establishing a cleansed data foundation for subsequent analysis; the FP-Growth and Im-Rete algorithms further uncover temporal and logical correlations among alerts, elevating isolated alarm events to contextual attack scenarios; ultimately, the dynamic BP neural network, empowered by these associated features, can more effectively learn genuine attack patterns.

To validate the advantages of the proposed method, experiments utilize alert datasets generated by outside.tcpdump over two-week, three-week, and five-week periods, yielding 25,000, 138,000, and 162,000 alerts respectively. The proposed method is compared with existing literature methods, with comparative results of CPU utilization across different alert volumes presented in Fig. 5.

Fig. 5.

Comparison of CPU utilization across different methods.

In-depth analysis of Fig. 5 reveals that all four baseline methods exhibit significantly higher CPU utilization than the proposed method. Specifically, the proposed method maintains CPU usage below 1%, whereas reference³ demonstrates the highest consumption at 5%. This minimal CPU utilization confirms the proposed method’s substantial advantage in computational efficiency. The exceptionally low resource requirement indicates strong potential for real-time or near-real-time monitoring applications, consistent with the efficiency advantages shown in Fig. 3. This efficiency stems from three key factors: first, DBSCAN combined with sliding time windows eliminates continuous full-data computation; second, the dynamic node pruning mechanism in the Im-Rete algorithm reduces memory and computational overhead during rule matching; third, and most importantly, the cascaded filtering architecture progressively reduces data volume at each processing stage. Such low resource consumption makes the method particularly valuable for deployment in resource-constrained edge computing environments.

Conclusion

The multi-level network security alert screening framework integrating DBSCAN clustering and Rete rule inference combines the advantages of both approaches to effectively filter and analyze diverse security alerts, significantly improving processing accuracy and efficiency. However, practical implementation requires further investigation into the fusion mechanism between DBSCAN and Rete algorithms to achieve deeper integration and more precise alert localization. Meanwhile, continuous algorithm optimization remains essential to address evolving cybersecurity threats. Future research directions may include developing more effective algorithmic fusion techniques, further enhancing computational performance, and incorporating additional advanced technologies to strengthen security alert processing capabilities. This study demonstrates that comprehensive preprocessing, feature extraction, and classification of raw security alerts can effectively eliminate noise and irrelevant information, thereby improving alert accuracy. These capabilities enable security operators to rapidly identify and respond to genuine threats while reducing both false positives and false negatives. Furthermore, the methodology facilitates discovery of novel attack patterns and timely updates to defense strategies, ultimately enhancing overall network security posture against increasingly sophisticated threats. Research on multi-level alert screening represents a significant driver for advancing cybersecurity technology, and continued exploration of novel screening methodologies will foster further innovation and progress in the field.

The principal mathematical notations and their descriptions used throughout this paper are summarized in Table 7.

Table 7.

Summary of symbol definitions.

Parameter	Definition
	A collection of alerts, including the name, time, and source
	Alarm name, used to identify the alarm type
	Alarm timestamp
	Attack source, indicating the source that triggers the alert
	Absolute value function, used to calculate the difference between two time values
	Time threshold
	Represents the i-th and j-th alarms respectively
L	A frequent itemset table that contains each frequent item and its support count
I	The set of all distinct items in the data set
	The n-th pattern in rule i
	Conclusion of Rule i
	Mode weight
	The truth function of mode
	Rule activation threshold
U	Fuzzy membership matrix
	The j-th alarm has a membership degree in the i-th cluster.
c	Number of clusters (alert level)
n	Total alerts
	Objective function of the Fuzzy C-Means (FCM) algorithm
	The local error function of the i-th cluster
	The Euclidean distance from the j-th alarm to the i-th cluster center
m	weighted index number
	The constrained FCM objective function incorporates Lagrange multipliers to address the normalization constraints of membership degrees.
	lagrange multiplier
	Cluster center i
	The feature vector of the j-th alarm
	The Euclidean distance from the j-th alarm to the i-th cluster center
	The Euclidean distance from the j-th alarm to the k-th cluster center
	Alert J’s combined weight
	Weight of the i-th alert level
	The m-dimensional feature vector of the k-th alarm
	Alert M features
	n-dimensional network security vector
	Used to determine if the device point has a security vulnerability
	Input vector for hidden layer neurons
	Input to the p-th neuron in the hidden layer
	Hidden layer neuron output vector
	The output of the p-th neuron in the hidden layer
	Input vector for the output layer
	The n-th element in the input vector of the output layer
	Output vector of the output layer
	The n-th element in the output layer’s output vector

Author contributions

Lin Ni: Conceptualization, Methodology, Writing - Original Draft, Formal Analysis, Investigation, Supervision, Writing - Review & Editing Shuai Zhang: Methodology, Software, Validation, Formal Analysis, Data Curation, Project administration Kun Huang: Investigation, Resources, Visualization, Data Curation Yifan Wang: Validation, Resources, Writing - Review & Editing.

Data availability

Data is provided within the manuscript.

Declarations

Competing interests

The authors declare no competing interests.