Layered Conditional Random Fields for Network Intrusion Detection
4.7 Robustness of the System
The prime reason for better attack detection accuracy for conditional random fields is that they do not consider observation features to be independent. This results in capturing the correlation among different features in the observation resulting in higher accuracy. Considering both, the accuracy and the time required for testing, layered conditional random fields score better.
To determine the statistical significance of our results, we rank all the six systems in order of significance for detecting Probe, DoS, R2L and U2R attacks. We use the Wilcoxon test [128]
with 95% confidence interval to discriminate the performance of these methods. We compare the ranking for various methods in Table 4.16, where a system with rank ‘1’ represents the best system.
Table 4.16: Ranking Various Methods for Intrusion Detection Probe DoS R2L U2R
Layered Conditional Random Fields 1 1 1 1
Conditional Random Fields 4 4 3 2
Layered Decision Trees 1 1 4 3
Decision Trees 1 1 1 5
Layered Naive Bayes 6 5 5 3
Naive Bayes 5 5 5 6
The results of the test indicate that layered conditional random fields are significantly better (or equal) for detecting attacks when compared with other methods. Thus, layered conditional random fields are a strong candidate for building effective and efficient network intrusion detection systems.
4.7 Robustness of the System
In order to test the robustness of our system, it is important to perform similar experiments with a number of other data sets. However, given the domain of the problem, no other data sets are freely available which can be used for similar experimentation. To ameliorate this problem to some extent and to study the robustness of our system, we add substantial amount of noise in the training data and perform similar experiments.
4.7.1 Addition of Noise
We control the addition of noise in the data by two parameters, the probability of adding noise to a feature, ‘p’, and the scaling factor, ‘s’. We perform four set of experiments with noisy data, one for each layer. For every set of experiment, we vary the parameter ‘p’ from 0 and 1 (by keeping it at values 0.10, 0.20, 0.33, 0.50, 0.75, 0.90 and 0.95) and vary the parameter ‘s’ from -1000 and +1000. In case, when the original feature is ‘0’, we add noise to that feature by using an additive function (a random value between -1000 and +1000) instead of scaling. We represent the effect of noise for detecting Probe, DoS, R2L and U2R attacks separately in Figures 4.4, 4.5, 4.6 and 4.7 respectively. The figures clearly suggest that the layered conditional random fields are robust to noise in the training data and perform better than other methods.
4.7 Robustness of the System 75
30 40 50 60 70 80 90 100
0 10 20 30 40 50 60 70 80 90 100
F-Measure
Noise %
LCRF CRF DT NB
Figure 4.4: Effect of Noise on Probe Layer
89 90 91 92 93 94 95 96 97 98 99
0 10 20 30 40 50 60 70 80 90 100
F-Measure
Noise %
LCRF CRF DT NB
Figure 4.5: Effect of Noise on DoS Layer
0 5 10 15 20 25 30 35 40 45
0 10 20 30 40 50 60 70 80 90 100
F-Measure
Noise %
LCRF CRF DT NB
Figure 4.6: Effect of Noise on R2L Layer
0 10 20 30 40 50 60
0 10 20 30 40 50 60 70 80 90 100
F-Measure
Noise %
LCRF CRF DT NB
Figure 4.7: Effect of Noise on U2R Layer
4.8 Conclusions 77
4.8 Conclusions
In this chapter, we addressed the core issues concerning the anomaly and hybrid intrusion detec-tion systems at the network level; viz, the accuracy of attack detecdetec-tion, capability of detecting a wide variety of attacks and efficiency of operation. Our experimental results in Section 4.5.1 show that conditional random fields are very effective in improving the attack detection rate and decreasing the false alarm rate. Having a low false alarm rate is important for any intrusion detec-tion system. Further, experimental results presented in Secdetec-tion 4.5.2, show that feature selecdetec-tion and implementing the layered framework significantly reduces the time required to train and test the model. Experiments also suggest that conditional random fields can be very effective in re-ducing the false alarms, thereby improving the attack detection accuracy. Further, our system can be implemented to detect a variety of attacks including the DoS, Probe, R2L and the U2R. Other type of attacks can also be detected by adding new layers in the system, making our system highly scalable. We compared our approach with some well known methods for intrusion detection such as the decision trees and naive Bayes. These methods, however, cannot detect the R2L and the U2R attacks effectively, while our integrated system can effectively and efficiently detect such attacks giving an improvement of 34.5% for the R2L attacks and 34.8% for the U2R attacks. Our system also helps in identifying an attack once it is detected at a particular layer which expedites the intrusion response mechanism, thus minimizing the impact of an attack. We showed that our system is robust to noise in the training data and performs better than any other compared system.
Our system has all the advantages of the layered framework discussed in the previous chapter, and, in particular the number of layers in the system can be easily increased or decreased giving flexibility to network administrators.
Our system can clearly provide better intrusion detection capabilities at the network level.
However, as discussed earlier, to provide a higher level of security it is significant to detect in-trusions at the application level along with detecting inin-trusions at the periphery of the network.
Hence, in the following chapters, we focus on developing intrusion detection systems which can operate at the application level and which can be effective in detecting application level attacks.