Testing Of Network Intrusion Detection System
B.S.Chaitanya Vamsee Pavan
KL University,Vijayawada Andhara Pradesh,India [email protected] M.Nalini Sri KL University,Vijayawada Andhara Pradesh,India [email protected] Jagadeep Vegunta KL University,Vijayawada Andhara Pradesh,India [email protected]
ABSTRACT:
Network based intrusion detection system use the models of attacks to identify intrusive behavior ability of systems to detect attacks by quality of models which are called signatures. Some attacks exploits in different ways. For this reason we use testing tools that able to detect goodness of signatures. This technique describes test and evaluate misuse detection models in the case of network-based intrusion detection systems. we use Mutant Exploits are working against vulnerability applications. This mutant exploit is based on mechanism to generate large no. of exploit by applying mutant operators. The results of the systems in detecting these variations pro-vide a quantitative basis for the evaluation of the quality of the corresponding detection model. but here we are going to find defects of this testing and is this test will provide 100% security for this system (or) not. and also which technique gives much security among these techniques fuzzy logic, neural networks, hybrid fuzzy and neural networks, naïve bayes, genetic algorithms and data mining.
Keywords: mutant exploits, intrusion detection, Security testing
1.
Intrusion Detection System
:
An intrusion detection system (IDS) is a device or software application that monitors network and/or system activities for malicious activities or policy violations and produces reports to a Management Station. Some systems may attempt to stop an intrusion attempt
but this is neither required nor expected of a monitoring system. Intrusion detection and prevention systems (IDPS) are primarily focused on identifying possible incidents, logging information about them, and reporting attempts. In addition, organizations use IDPS for other purposes, such as identifying problems with security policies, documenting existing threats, and deterring individuals from violating security policies. IDPS have become a necessary addition to the security infrastructure of nearly every organization.
IDPS typically record information related to observed events, notify security administrators of important observed events, and produce reports. Many IDPS can also respond to a detected threat by attempting to prevent it from succeeding. Intrusion detection is a security technology that attempts to identify and isolate ``intrusions'' against computer systems. Different ID systems have differing classifications of ``intrusion''; a system attempting to detect attacks against web servers might consider only malicious HTTP requests, while a system intended to monitor dynamic routing protocols might only consider RIP spoofing. Regardless, all ID systems share a general definition of ``intrusion'' as an unauthorized usage of or misuse of a computer system.
Intrusion detection is an important component of a security system, and it complements other security technologies. By providing information to site administration, ID allows not only for the detection of attacks explicitly addressed by other security components (such as firewalls and service wrappers), but also attempts to provide notification of new attacks unforeseen by other components. Intrusion detection systems also provide forensic information that potentially allow organizations to discover the origins of an attack. In
this manner, ID systems attempt to make attackers more accountable for their actions, and, to some extent, act as a deterrent to future attacks.
2. Network Intrusion Detection System
:
NIDS is working based on the CIDF (common intrusion detection framework) model. here, NIDS contains four elements as shown in fig1. Those are event generator box, analysis box, storage box and counter measure machine. The purpose of an Event generator box is to provide information about events to the rest of the system. An ``event'' can be complex, or it can be a low-level network protocol occurrence. It need not be evidence of an intrusion in and of itself. E-boxes are the sensory organs of a complete IDS--- without Event generator box inputs , an intrusion detection system has no information from which to make conclusions about security events.Analysis boxes analyze input from event generators. A large portion of intrusion detection research goes into creating new ways to analyze event streams to extract relevant information, and a number of different approaches have been studied. Event analysis techniques based on statistical anomaly detection, graph analysis, and even biological immune system models have been proposed.
Event generator boxes and Analysis boxes can produce large quantities of data. This information must be made available to the system's operators if it is to be of any use. The Data storage box component of an IDS defines the means used to store security information and make it available at a later time.
Many ID systems are driven off of audit logs provided by the operating system, detecting attacks by watching for suspicious patterns of activity on a single computer system. This type of IDS is good at discerning attacks that are initiated by local users, and which involve misuse of the capabilities of one system. However, these ``host based'' intrusion detection systems have a major
shortcoming: they are insulated from network events that occur on a low level.
FIG1:CIDF(common intrusion detection frame works)
Network intrusion detection systems are driven off of interpretation of raw network traffic. They attempt to detect attacks by watching for patterns of suspicious activity in this traffic. Network ID systems are good at discerning attacks that involve low-level manipulation of the network, and can easily correlate attacks against multiple machines on a network.
It's important to understand that while network ID has advantages over host-based ID, it also has some distinct disadvantages. Network ID systems are bad at determining exactly what's occurring on a computer system; host based ID systems are kept informed by the operating system as to exactly what's happening.
3. Techniques used for NIDS
:Depending on the type of analysis carried out intrusion detection systems are classified as either signature-based or anomaly-based. Signature-based schemes seek defined patterns, or signatures, within the analyzed data. For this purpose, a signature database corresponding to known attacks is specified a priori. On the other hand, anomaly-based detectors attempt to estimate
the „„normal‟‟ behavior of the system to be protected, and generate an anomaly alarm whenever the deviation between a given observation at an instant and the normal behavior exceeds a predefined threshold. Another possibility is to model the „„abnormal‟‟ behavior of the system and to raise an alarm when the difference between the observed behavior and the expected one falls below a given limit.
Signature and anomaly-based systems are similar in terms of conceptual operation and composition. The main differences between these methodologies are inherent in the concepts of „„attack‟‟ and „„anomaly‟‟. An attack can be defined as „„a sequence of operations that puts the security of a system at risk‟‟. An anomaly is just „„an event that is suspicious from the perspective of security‟‟. Based on this distinction, the main advantages and disadvantages of each IDS type can be pointed out.
Signature-based schemes provide very good detection results for specified, well-known attacks. However, they are not capable of detecting new, unfamiliar intrusions, even if they are built as minimum variants of already known attacks. On the contrary, the main benefit of anomaly-based detection techniques is their potential to detect previously unseen intrusion events. However, and despite the likely inaccuracy in formal signature specifications, the rate of false positives in anomaly-based systems is usually higher than in signature based ones.
Given the promising capabilities of anomaly-based network intrusion detection systems (A-NIDS), this approach is currently a principal focus of research and development in the field of intrusion detection. Various systems with A-NIDS capabilities are becoming available, and many new schemes are being explored. However, the subject is far from mature and key issues remain to be solved before wide scale deployment of A-NIDS platforms can be practicable.
Machine learning based NIDS is one of the classification of anomaly based NIDS. Machine learning techniques are based on establishing an explicit or implicit model that enables the patterns analyzed to be categorized. A singular characteristic of these schemes is the need
for labeled data to train the behavioral model, a procedure that places severe demands on resources.
In many cases, the applicability of machine learning principles coincides with that for the statistical techniques, although the former is focused on building a model that improves its performance on the basis of previous results. Hence, a machine learning A-NIDS has the ability to change its execution strategy as it acquires new information. Although this feature could make it desirable to use such schemes for all situations, the major drawback is their resource expensive nature.
Several machine learning-based schemes have been applied to A-NIDS. Some of the most important are cited below, and their main advantages and drawbacks are identified.
3.1. Bayesian networks:
A Bayesian network is a model that encodes probabilistic relationships among variables of interest. This technique is generally used for intrusion detection in combination with statistical schemes, a procedure that yields several advantages including the capability of encoding interdependencies between variables and of predicting events, as well as the ability to incorporate both prior knowledge and data. However, as pointed out in a serious disadvantage of using Bayesian networks is that their results are similar to those derived from threshold-based systems, while considerably higher computational effort is required.
Although the use of Bayesian networks has proved to be effective in certain situations, the results obtained are highly dependent on the assumptions about the behaviour of the target system, and so a deviation in these hypotheses leads to detection errors, attributable to the model considered.
3.2Neural networks:
With the aim of simulating the operation of the human brain (featuring the existence of neurons and of synapses among them), neural networks have been adopted in the field of anomaly intrusion detection, mainly because of
their flexibility and adaptability to environmental changes. This detection approach has been employed to create user profiles to predict the next command from a sequence of previous ones to identify the intrusive behavior of traffic patterns etc.
However, a common characteristic in the proposed variants, from recurrent neural networks to self-organizing maps is that they do not provide a descriptive model that explains why a particular detection decision has been taken.
3.3.Fuzzy logic techniques:
Fuzzy logic is derived from fuzzy set theory under which reasoning is approximate rather than precisely deduced from classical predicate logic. Fuzzy techniques are thus used in the field of anomaly detection mainly because the features to be considered can be seen as fuzzy variables This kind of processing scheme considers an observation as normal if it lies within a given interval .
Although fuzzy logic has proved to be effective, especially against port scans and probes, its main disadvantage is the high resource consumption involved. On the other hand, it should also be noticed that fuzzy logic is controversial in some circles , and it has been rejected by some engineers and by most statisticians, who hold that probability is the only rigorous mathematical description of uncertainty.
3.4.Genetic algorithms:
Genetic algorithms are categorized as global search heuristics, and are a particular class of evolutionary algorithms (also known as evolutionary computation) that use techniques inspired by evolutionary biology such as inheritance, mutation, selection and recombination. Thus, genetic algorithms constitute another type of machine learning-based technique, capable of deriving classification rules and/or selecting appropriate features or optimal parameters for the detection process .
The main advantage of this subtype of machine learning ANIDS is the use of a flexible and robust global search method that converges to a solution from multiple directions, whilst no prior knowledge about the system
behaviour is assumed. Its main disadvantage is the high resource consumption involved.
3.5.Clustering and outlier detection:
Clustering techniques work by grouping the observed data into clusters, according to a given similarity or distance measure. The procedure most commonly used for this consists in selecting a representative point for each cluster. Then, each new data point is classified as belonging to a given cluster according to the proximity to the corresponding representative point. Some points may not belong to any cluster; these are named outliers and represent the anomalies in the detection process.
Clustering and outliers are used at present in the field of IDS, with several variants depending on how the question „„Is the isolated outlier an anomaly?‟‟ is answered. For example, the KNN (k-nearest neighbor) approach uses the Euclidean distance to define the membership of data Points to a given cluster, while other systems use the Mahalanobis distance. Some detection proposals associate a certain degree of being an outlier for each point.
Clustering techniques determine the occurrence of intrusion events only from the raw audit data, and so the effort required to tune the IDS is reduced.
3.6. Additional considerations on A-NIDS processing KDD and data mining:
In addition to the above described A-NIDS techniques, there are others that may help in the task of dealing with the amount of information contained within a dataset. Two of these techniques are principal component analysis (PCA) and association rule discovery.
PCA is a technique that is used to reduce the complexity of a dataset. It is not a detection scheme itself but an auxiliary one. A given data collection (or dataset), obtained by means of the different sensors in the target environment, becomes more and more extensive and complex as the number of different services and speed of the networks grow. To simplify the dataset,
PCA makes a translation on a basis by which n correlated variables are represented in order to reduce the number of variables to d < n, which will be both uncorrelated and linear combinations of the original ones. This makes it possible to express the data in a reduced form, thus facilitating the detection process .
On the other hand, the aim in association rules discovery is to obtain correlations between different features extracted from the training datasets. By means of these association rules it is possible, for example, to find internal relations between data corresponding to a specific connection. In some algorithms for association rules and frequent episodes are contributed.
To conclude the present section, let us present an important discussion of A-NIDS techniques. During recent decades several scientific communities have contributed to analyzing information from high volume databases. However, in the 1990s, KDD („„Knowledge Discovery in Databases‟‟) burst onto the scene, to „„identify new, valid, potentially useful and comprehensible patterns for data‟‟. Data mining techniques appeared as a particular case of KDD these consisted of „„learning algorithms to large data repositories with the purpose of automatically discovering useful information‟‟.
As a specific use case, KDD and data mining have been widely applied in the last few years to correlate traffic instances in network related databases. It is now commonplace to categorize and refer to different IDS processing approaches using the term „„data mining‟‟, as a generic wildcard analysis-related concept. In this line, almost every processing scheme (statistical algorithms, neural networks, fuzzy methods, instance-based learning procedures, and so on) is now considered a data mining technique.
3.7. MUTANT EXPLOITS:
The testing technique is based on an automated mechanism to generate a large number of variations of an exploit by applying mutant operators to an exploit template. The mutant exploits are then run against a victim system where the vulnerable applications and/or operating
systems are installed. The attacks are analyzed by a network- based intrusion detection system. The intrusion alerts produced by the NIDS are then correlated with the execution of the mutant exploits. By evaluating the number of successful attacks that were correctly detected, it is possible to get a better understanding of the effectiveness of the models used for detection.
Obviously, this technique does not provide a formal evaluation of the “goodness” of an attack model. Nonetheless, claim that this is a valid way to improve one‟s confidence in the generality of a detection model. Note that the technique could be easily extended to host-based intrusion detection systems and to systems that use anomaly detection approaches. Nonetheless, hereinafter we will limit the scope of our analysis to network-based misuse detection systems.
The mutation process is deterministic and guided by a seed value, which makes the mutations reproducible. The mutant operators are supposed to preserve the “effectiveness” of the attack, that is, all the generated mutants are supposed to be functional exploits. Unfortunately, both the exploits and the attack targets may be very complex. Therefore, it is possible that a variant of an exploit becomes ineffective because of some condition that may be difficult (or impossible) to model.
To address this issue, the technique relies on an oracle to determine if an attack has been successful or not. In most cases, the oracle mechanism can be embedded in the exploit itself, for example by crafting an exploit so that it will generate side effects that can be used to determine if the exploit was successful. However, in some cases it is not possible to generate evidence of the effectiveness of an attack as part of its execution, and, for those cases, an external oracle that reports on the outcome of specific attacks has to be developed.
4. HOW MUTANT EXPLOITS DEFEAT NIDS:
Here testing IDS‟s are NIDS. example of network misuse could be running are exploit against server, scanning the entire network hosts, which results in Denial of service attack. they
perform rigorous tests ISS real secure, snort. They chosen because they are leading products .
It is difficult because “attacks that exploit certain vulnerability may do so in completely different ways”. it is easy to write IDS signatures for public known attacks. Realistically, not all exploits are going to be released. IDS systems typically have signatures for thousands of exploits. these are very static, effectively searching for specific packet (or) set of packets across network. here problem is when an exploit is mutated that allows it to still functioning in compromising a host.
Application layer mutations include protocol round, FTP, HTTP evasion techniques .these change the data sent to application by exploit, in a way which application still understand attacker‟s message. However message does not match IDS signatures.
Exploit layer mutation are newest to group. they include polymorphic shell code and alternate encoding. This shell code part is added to IDS signature . IDS check for shell code regardless exploit some shell code usually pushed onto compromised host in order to run some command for attacker. Encode shell code in different formats by “insertion instructions “. In this way functional shell code does not match IDS signature.
Each exploit was run through mutation engine . to generate mutant exploit combination once a particular exploit was found. Which evade IDS system yet still functioned, authors moved to next exploit. By applying above tests snort detected 4 out of 10 exploits. And IIS real secure detect only one exploit.
Signature IDS system are only one layer another type is based on cisco s net flow technology. Another layer of IDS that should not over locked is host based IDS. Often exploits create anomalies in log file which can watched for and reported. even before attackers explores enough of compromised systems to discover alarm message sent.
5.RESULTS:
We have mentioned some techniques above. If apply those techniques to NIDS then results will be as follows:
5.1. Mutant Exploits:
The first represents the ability of the intrusion detection system to correctly detect the baseline attack when the exploit was not subjected to any mutation technique. The second reports whether the IDS was able to detect all of the mutations of the same attack attempted during the experiment. In the last column we summarize the key techniques that enabled the mutated exploits to evade detection, when applicable.
The total number of possible mutants that the engine can generate is a key value that must carefully be tuned for each exploit. This number depends on how many mutation techniques are applied to the exploit and on the way in which each technique is configured. For instance, an application- level transformation that consists of modifying the number of space characters between the HTTP method (e.g., GET or POST) and the requested URL can generate a large number of mutants, one for each number of space characters selected. When composed with other techniques, this operator may lead to an unmanageable number of mutant exploits.
Snort correctly detected all instances of the baseline attacks. The exploit mutation engine, however, was able to automatically generate mutated exploits that evaded Snort‟s detection engine for 6 of the 10 attacks. In this case, however, the exploit mutation engine was able to generate mutant exploits that evaded detection by Real Secure in 9 out of 10 cases. Even though it is tempting to make relative comparisons between the two systems, strong conclusions cannot be drawn due to the non-exhaustive nature of the exploration of the detection space. Nonetheless, it can be concluded that both sensors proved to be surprisingly vulnerable to the generated mutant exploits.
5.2. Fuzzy Logic:
In this analysis, we have two types of data sets which are training and testing data sets. Each data set contains 34 attributes. In the testing phase, the testing dataset is given to the proposed system, which classifies the input as a normal or
attack. The obtained result is then used to compute overall accuracy of the proposed system. The overall accuracy of the proposed system is computed based on the definitions, namely precision, recall and F-measure which are normally used to estimate the rare class prediction. It is advantageous to accomplish a high recall devoid of loss of precision. F-measure is a weighted harmonic
mean which evaluates the trade-off between them. PRECISION=TP/(TP+FP) RECALL=TP/(TP+FN) OVERALL ACCURACY=(TP+TN)/(TP+TN+FP+FN)
By analyzing the result, the overall performance of the proposed System is improved significantly and it achieves more than 90% accuracy for all types of attacks.
5.3. Uusupervised Outlier:
In this technique we detect intrusion by using outliers. This outliers are found by applying random forest algorithms. This algorithm uses proximities to find outliers. With respect to random forests algorithm, outliers can be defined as the cases whose proximities to other cases in the dataset are generally small [15]. Outlier-ness indicates a degree of being an outlier. It can be calculated over proximities. class(k) = j
denotes that k belongs to class j. prox(n,k) denotes the proximity between cases n and k. here we take data sets which include attacks by giving percentages. for example 1% data set means that data set contains 1 % of attacks.
We evaluate the performance of our system by the detection rate and the false positive rate. The detection rate is the number of attacks detected by the system divided by the number of attacks in the dataset. The false positive rate is the number of normal connections that are misclassified as attacks divided by the number of normal connections in the dataset. We can evaluate the performance by varying the threshold of outlier-ness. result indicates high detection rate by having low false positive rate. For example, the detection rate is 95% when the false positive rate is 1%. When the false positive rate is reduced to 0.1%, the detection rate is still over 60%.
5.4. Neural Networks:
Our SOM contains a grid of neurons each possessing a weight vector of the length of |FINALSET|. After training, the weight vectors essentially reflect the number of hits to a particular port in a certain time interval dt. When determining the best match between an input vector x = [x1,x2,…,xn] and the weight vectors mi = [mi1,mi2,…,min], we use a simple Euclidean distance formula. Once the best match is found, the weights of the neurons are updated through standard SOM formulas with linearly decreasing learning and neighborhood functions (Kohonen 1995). To develop the clusters from the SOM, we compute a frequency value to count how many times a particular neuron and members of its neighborhood were chosen as the Best Matching Unit (BMU) during training. The neurons with the highest frequency value are selected to be centroids, the centers of clusters.
We had a great deal of trouble trying to get our neural network to detect all types of attacks simultaneously. However, the neural network performed well when tested on individual types of attacks one at a time. Our training and testing in this area was limited however, because our dataset did not contain many instances of the same attack. Table 1 shows some of our results where sshprocesstable is the name of a particular type of denial of service attack. In Table 1, columns 2, 3, 4, and 5 respectively refer to the correct prediction of normal traffic intensity, the incorrect prediction of normal traffic intensity, the correct prediction of attack traffic intensity, and the incorrect. Corre ct Norm al Predic tions False Nega tives Corre ct Attack Predic tions False Posit ives Union of All Attacks 100% 0% 24% 76% Sshproce sstable 100% 0% 100% 0%
5.5. Hybrid Fuzzy & Neural Networks:
In this technique we use both Fuzzy and neural network techniques. Initially, system takes input from KDD data set and then applies FCM clustering to that data set. This FCM is used for separation of normal from attacks. Then we apply MLP algorithm for classification of attacks. During testing phase, the accuracy classification of each attack types was calculated.
Attack name Inpu t 1 out put Accur acy Inpu t2 out put Accur acay Dos 230 88 230 89 99.99 % 204 63 204 63 100% U2R 7 7 100% 2 2 100% U2L 608 608 100% 5 2 40% Prob 130 1 130 1 100% 665 666 99.8% UNKN OWN 18 17 94.4% 114 166 68.6% Time(se c) 5.82 92 4.67 66 5.6. NAÏVE BAYES:
We first describe the data set used in this experimentand then discuss the results obtained. Finally, we evaluate our approach and compare the results with the results obtained by other researchers using BPN algorithms and with the best result of the KDD‟99 contest.
For our experiments, we choose the naïve Bayes Classifier in WEKA (Waikato Environment forKnowledge Analysis) [19]: with full training set and 10- fold cross validation for the testing purposes. In 10-fold cross-validation, the available data is randomly divided into 10 disjoint subsets of approximately equal size.
One of the subsets is then used as the test set and the remaining 9 sets are used for building the classifier. The test set is then used to estimate the accuracy. This is done repeatedly 10 times so that each subset is used as a test subset once. The accuracy estimates is then the mean of the estimates for each of the classifiers. Cross-validation has been tested extensively and has been found to generally work well when sufficient data is available. A value of 10 for this has been found to be adequate and accurate. Finally, the ROC (Receiver Operating
Characteristic) curve is obtained as a measure of performance analysis of our approach, using MATLAB7.0. The experiment is carried out using a machine with Intel Pentium4 processor, 2.8GHz speed, and 512MB RAM. In our case, the detection rate is 95%, with an error rate of 5%. Moreover, it performs faster which takes only 1.89 seconds to build the model. However, in comparison to BPN, our approach generates more false positives, but, it is efficient, cost effective and takes less time.
6. CONCLUSION:
Network based intrusion detection systems rely on signatures to recognize malicious traffic. The quality of a signature is directly correlated to the IDS‟s ability to identify all instances of the attack without mistakes. Unfortunately, closed-source systems provide little or no information about both the signatures and the analysis process. Therefore, it is not possible to easily assess the quality of a signature and de-termine if there exist one or more “blind spots” in the attack model.
Writing good signatures is hard and resource-intensive. When a new attack becomes publicly known, NIDS vendors have to provide a signature for the attack in the shortest time possible. In some cases, the pressure for providing a signature may bring the signature developer to write a model tailored to a specific well-known exploit, which does not provide comprehensive coverage of the possible ways in which the corresponding vulnerability can be exploited.
This paper presents six techniques for testing of Network intrusion detection system. For each technique how much accuracy it gives when testing of NIDS. In those six techniques hybrid fuzzy and neural network technique is best technique which gives accuracy 99.99%. if we compare it with mutant exploits also this was the best technique. Because , this hybrid fuzzy and neural network is anomaly based intrusion detection system has ability to detect they are capable of detecting new, unfamiliar intrusions, even if they are built as minimum variants of already known attacks.
7. References:
[1] Giovanni Vigna, William Robertson, Davide Balzarotti-Testing Network Intrusion detection System by using Mutant exploits
[2] HONG HAN,XIAN LIANG LU,LI-YOUNG REN, Using data mining to discover signatures in NIDS.
[3] R. Shanmugavadivu, Dr.N.Nagarajan ,NIDS using Fuzzy Logic.
[4] Muna Mhammad T. Jawhar, Monica Mehrotra, Design NIDS using hybrid fuzzy and neural networks.
[5] Wei Li, Using Genetic Algorithm for Network Intrusion Detection.
[6] Mrutyunjaya Panda and Manas Ranjan Patra, Network Intrusion Detection Using Naive Bayes. [7] jiong zhong and mohammed zulkarnine, anomaly based NIDS with unsupervised outlier detection.
[8] Simon Edwards - Network intrusion detection system-Important IDS network security and
Vulnerabilities.
[9] ]. P. Garcıa-Teodoroa,, J. Dıaz-Verdejo, G.
Macia-Ferna´ndez, E. Vazquez-Anomaly based Network intrusion system – techniques, systems, challenges.
