ANALYSIS OF SCAN ATTACK THROUGH THRESHOLD BASED BEHAVIOR MODEL

ANALYSIS OF SCAN ATTACK

THROUGH THRESHOLD BASED

BEHAVIOR MODEL

S.M.NANDHAGOPAL, M.E., (Ph.D).,

RESEARCH SCHOLAR, PRIST UNIVERSITY, VALLAM, TANJAVUR, TAMILNADU, INDIA. Mail ID: [email protected]

Dr.P.THANGARAJ.,M.E.,Ph.D.,

PROFESSOR AND HEAD, DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING, BANNARI AMMAN INSTITUTE OF TECHNOLOGY, SATHYAMANGALAM,

ERODE, TAMILNADU, INDIA.

Abstract Network anomaly detection is getting most significant in research area. Behavior tracking of traffic is a process by which the ongoing observed behavior of a host is tracked and compared with the legitimate model. Several methods for behavior recognition exist. But usage of threshold value for detecting scan attack is a novel approach. This paper aims at classifying the network DNS traffic as an abnormal or normal by tracking the flow of traffic. The paper's main objective is to create a geometric anomaly detection system, a predictive model capable of perception of normal and abnormal behavior of network DNS traffic. The proposed method is unique in this aspect that by applying a modified threshold based scan attack detection that detects attacks with very high accuracy.

Keywords: DNS, DLL, Address Resolution Protocol, PRNG, PNP.

Introduction

Probing the traffic flow of a network can detect the hosts, known as bots, performing any malicious activity. Each bot may perform different kind of malicious activities that can be categorized into Scanning, Spamming, Binary downloading and exploit attempts which are the most common and efficient malicious activities a bots master may command their bots to do. In this work we focus on scanning activities.

Network scanning detection

Network scanning detection techniques are implemented in number of network-based intrusion detection systems. A number of definitions found that attempt to define network-based intrusion detection systems based on the classification of their underlying detection principles, however, they can be broadly classified into three types: (1) specification-based detection (2) anomaly detection and (3) misuse detection. We present a novel technique to detect the scanning systems within a local network based on anomalous behaviors they parade when utilizing Domain Name Service (DNS)). Detecting what intruders are scanning for can alert a security analyst or system manager to what services or types of computers are being battered. Gathering the services being targeted before an attack permits the administrator to take measures to protect the resources e.g. installing patches, removing services on machines which do not need to be running them or firewalling services from the outside.

Considering its importance, the problem of scan detection has been given a lot of significance by a large number of analysts in the network security community. Initial problem solving techniques simply counted the number of IPs that a source IP made connection attempts to on each destination port and concluded every source IP a scanner whose count surpassed a threshold. We concentrated on detecting horizontal scans from a single source that are trying to discover the active hosts on a network or a particular service across a network. This approach can be used to detect scanning botnets as each contributing source can be noticed individually. We are not considering vertical scans which attempt to count the number of services on a single host. A most efficient and widely quoted method for detecting scanners is the threshold based methods. It can detect single source scanning after 4 to 5 connection attempts. Though they are effective, these approaches suffer from two restrictions: they can be evaded by mixing probe attempts with accesses to familiar hosts, and they cannot handle scan issues of stateless protocols.

DNS BASED DETECTION:

DNS provides a detailed mapping of IP address with its domain names. Address resolution can be stated as the process of determining the hardware address of a host in its network. We noted that scanning activity in a network can be detected by noting the suspicious scanning systems which fails to utilize the features of DNS protocol. As this is the most popular way of attack. We see how this technique can be utilized to know the remote scanning attacks. A technical call random scanning approach has been utilized in

Whyte et al. it states that some worm system utilizes IP address instead of domain name in order to avoid the DNS query. So such types of agents who create new connection from network and not related to any DNS activity can be easily defined as anomalous. If we observer for such an agent, we can detect L2R or L2L inter-cell network scanning activity. But this technique fails to detect intra-cell or R2L network scanning activity.

Literature survey

Pratik at el presented an intrusion detection technique based on traffic anomaly to protect the operation of DNS from cyber-attacks. It instruments machine learning approach to record the normal behavior of the protocol and then compare it with the attack for detection. It can be used for detecting any existing as well as new attacks. Even modified attacks over DNS can be detected with less false positive rates.

In this approach, it is assumed that the DNS always have a stateful approach in its queries and replies from any source are dependent on each other and the protocol goes through a well-established state machine in making their functionalities.

The DNS messages can be used to create DNS state machine. The transition of DNS from state to state will be depend on the R(TYP) and Q(TYP) of the DNS packets sent or received. Type (TYP) field of DNS have 70 different types. Since this number of transitions available, every transaction can accept maximum 70 different values for it. So it is practically difficult to go through all the combinations. Successively, n gram approach is used to model the DNS state transitions.

Yousof et al had designed a system to detect the botnets by monitoring and finding the relationship between the log files of various hosts for the controversies in their behavior. It utilizes file sizes for this operation.

This goal is attained by API socket function calls which are intercepted in communication application to generate the data. A technique

The captured API socket function calls and their relevant arguments are stored in a separate log files. It utilized a system-wide intercepting scheme which observed all threads running on the system to intercept API socket function calls. One of the best way to intercept an API socket function calls is to use a Dynamic Link Library (DLL) file which replaces the target function to be intercepted and then insert the DLL file into the address space of target process. They intercepted API socket function calls utilized by communication programs, and store them with their arguments into log file. During this period, another program is introduced to record the change of log file size. This record is created every second for a particular period of time say t. It is considered that the log files are safe and attacker cannot erase the log files. After the time t, the recorded data is sent to the analyzer. It reads the recorded data of each host and check to see is there any changes from current state with previous state for all recorded data from different hosts. If changes occurs then a value of one is produced, otherwise zero is returned.

This work presented two different methods that are healthy to time-varying traffic behavior: a strong model-free and a strong model-based method that can be considered as generalizations of the corresponding methods.

SCANNING ACTIVITY DETECTION

David Whyte presented two new techniques to detect scanning systems within a local network based on the anomalous behaviors they exhibit when using the address resolution protocols (i.e. Address Resolution Protocol (ARP) [48], and Domain Name Service (DNS) [37]). Based on our evaluation, using datasets obtained from a small university network, these techniques offer a significant improvement over existing scanning detection techniques. Specifically, in certain network environments, these techniques can identify scanning hosts within an enterprise network after the observation of only a single scanning attempt with a low false positive and negative rate.

ARP-based detection: This behavioral signature is based on the observation that a scanning host targeting systems within its own network exhibits anomalous behavior distinct from normal ARP activity; specifically, scanning hosts exhibit discernible behavioral changes in the amount and pattern of ARP request activity of the scanning host, because a scanning host targeting local network hosts triggers the broadcast of anomalous ARP requests.

DNS-based detection: It exploit the observation that the vast majority of publicly available services are accessed through the use of DNS as this protocol provides the mapping between numeric IP addresses and the corresponding alphanumeric names. Many fast scanning worms and automated scanning tools use a pseudo random number generator (PRNG) to generate 32-bit random numbers that correspond to an IPv4 address. The use of a numeric IP address, instead of the qualified domain name of the system, obviates the need for a DNS query. If we do not observe DNS activity before a new connection is initiated, we consider this connection anomalous and potential scanning activity.

Katerina et al discussed a technique which used 3 different features. The behaviors of the web traffic is considered and clustered in order to segregate the attack sessions from the attack detection sessions.

These types of clustering operation by monitoring the behavior of the network detects the anti-virus software, intrusion detection systems which are primarily focused on content based signatures.

These signatures are inherently susceptible to inaccuracies due to polymorphic and metamorphic nature of malicious code. (2) We use four data sets, each in duration of 4-5 months, which allows us to compare the performance of clustering achieved in classifying malicious activities aimed at different system configurations and/or different time periods, and thus, to some extent, to generalize our observations. (3) We use several feature selection methods in combination with K-means clustering to explore whether attacks differ from vulnerability scans only in a small subset of features. The results show that, depending on the feature selection algorithm and the dataset, anywhere from two to ten features can be used to perform clustering, typically with better performance than when all 43 features are used. The results further show that the best features differ across data sets collected by different system configurations.

Methodology

As stated earlier, the DNS traffic of an existing host in network have been monitored for its anomaly behavior. The regular traffic of the network shall be named as “normal model”.

Normal Traffic Modeling Engine is used to model the traffic of the network in order to extract the statistical parameters of the normal traffic.

Anomaly Score (AS) of traffic is noted once it seems to be deviating its flow from the “normal model”. By knowing such a type of traffic, their score of deviation will be calculated which is discussed later in this section.

Matching Engine is utilized to perform this calculation of AS. But in order to assess the exact score we need to concentrate on the threshold value of the deviation so the lesser the deviation from normal traffic can be left without disturbing. In order to make an exact threshold value we need to use normality during training called Normal Score (NS).

Various parameters need to be calculated in our algorithm. They are

Pij: it is used to assess the conditional probability of transition from state i to state j caused by kij number of ARP requests, given the sequence of observed transitions in matching process. It can be described as follow:

1 2.. 1

(S | S

)

k

ij ij I I I

P



P

Where, Tij is transition from state i to state j,

P

(S

S

S

S )

1 2 1 2

1 2 1 2

. 3.... ij

. 3....

(S

S

S

S )

(S

S

S

)

i

i I I I I IK

I I I I IK

ij

P

P

P







Partial Anomaly Point (PAP)

PAP can be derived from assessing the deviation of states during steady growth of the traffic.

2

2

(

t )

, if t

j jk

jk j











0 , if

t





where tjk is interval of time between kth and (k+1)th ARP requests so that the node will be in state j after kth ARP request. Steady state duration values, greater than average value for each state, indicate normal behaviors.

Normal Point (NP)

NS shows the degree of normality of the flow and is a function of partial NP's (PNP). We can define PNS as normality score in ith observation interval in training period. NS calculates as following equation:

NP = MAX{PNP}

Where MAX gets the maximum value among PNP’s. For calculating PNP, we use the same method as used for calculation of AS, but in this case for normal ARP traffic. We get this normal traffic from training traffic in different time intervals and calculate PNP for each duration. It is obvious that to obtain a right value for NP from PNP's, these time intervals should be of the same length.

Accuracy in Threshold Value

Threshold based problem solving techniques needs to make some tradeoff between the parameters independent of the type of problems it addresses. Here we uses the threshold value and the count of false alarm as the key factors in improving the algorithm efficiency.

If we assign lower threshold value, then many normal network behaviors would be considered as abnormal whereas if a large threshold value is assigned, then even abnormal behavior would be considered as normal. So in order to make right decisions on the threshold value we need to estimate the right traffic normality value. So the NS gets it exacts value based on monitoring the normal traffic of network during the training period.

The AS value we calculated should satisfy the below criteria.

AP Thi > NPi is the threshold value for node i.

In order to set the threshold value, we will assign a maximum threshold value and then we measure the false alarm during the simulation process. Then we adjust the threshold value so that the false alarms don’t get decrease in a particular value. This could be fixed as the right threshold value.

We can judge the correctness of threshold value from the point where reducing the value of threshold does not allow false alarm to reduce.

The simpler way to assess the threshold value if to set it k times more than NP. Experimental results

The first phase of the proposed technique is modelling normal behaviour of DNS traffic in training period with clean conditions for the enterprise network. An important factor affecting the results of our technique is purring this traffic from anomalous DNS requests. This work is performed in DNS Traffic Purring Engine. The output of this block is almost a normal DNS traffic. The hops number grows with extending training time duration. The process of extending training time may result in decreased growth rate of hop count in normal model. This process continues until the model reaches to some fixed state's number reaching fixed hops for majority of hosts. Although, maximum number of nodes have behaviours similar to nodes but a few nodes show a dissimilar behaviour. The models corresponding to these nodes continue to improve, even within a big training duration. These nodes are classically servers that interact with the whole network. Some infected nodes reveal such behaviour too.

Performance of Anomaly Nodes

0 10 20 30 40 50 60 70 80 90

1 2 3 4 5 6 7 8 9

Training Period(In Days)

Nu

m

b

e

r o

f Ho

p

s

8.15.12.028 8.15.1.02 8.15.1.20 8.50.10.035

Fig 1: The traffic analysis of DNS

0 10 20 30 40 50 60 70 80 90

1 2 3 4 5 6 7 8 9

Time Period (In Days)

Nu

m

b

e

r o

f

Ho

p

s 8.5.1.028

8.5.1.022

8.5.1.210

8.5.1.030

28.15.11.088

36.18.34.045

Fig 2: A close observation of attacker’s traffic

Conclusion

This paper presented an anomaly detection mechanism for detecting abnormal ARP traffic. The approach can be used to detect nodes with abnormal ARP traffic within an enterprise network. As described above, a modified threshold based model has been trained for each node. Network ARP traffic to be tested is fed to the system then detecting phase of algorithm starts. In this phase, a comparison is performed between the normal model and the traffic fed to the system. The results of implementing the methodology show that our proposed system is a very powerful and accurate host-based approach which can distinguish between normal and abnormal ARP traffic, making the approach unique. There are few factors affecting the results of the algorithm, to a large extent which includes threshold value, the length of sequences of transitions used and so on. Therefore these values should be set exactly. Our decisions were based on the trial results and statistical characteristics of the real network traffic under the analysis. Taking these issues into account, we attained an anomaly detection system with 90% and above accuracy. Adjustment of the exact parameters can definitley gets the detection accuracy of above 99%.

References

[1] M. Roesch, “Snort-lightweight intrusion detection for networks,” in Proceedings of the 13th USENIX conference on System administration, 1999, pp. 229–238.

[2] V. Paxson, “Bro: a system for detecting network intruders in real-time,” Computer networks, vol. 31, no. 23, pp. 2435–2463, 1999. [3] P. Barford, J. Kline, D. Plonka, and A. Ron, “A signal analysis of network traffic anomalies,” in Proceedings of the 2nd ACM

SIGCOMM Workshop on Internet measurement, 2002, pp. 71–82.

[4] W. Lu and A. A. Ghorbani, “Network anomaly detection based on wavelet analysis,” EURASIP Journal on Advances in Signal Processing, vol. 2009:837601, 2009.

[7] R. P. Lippmann, D. J. Fried, I. Graf, J. W. Haines, K. R. Kendall, D. McClung, D. Weber, S. E. Webster, D. Wyschogrod, R. K. Cunningham, and M. A. Zissman, “Evaluating intrusion detection systems: The 1998 DARPA off-line intrusion detection evaluation,” in Proceedings of 2000 DARPA Information Survivability Conference and Exposition (DISCEX’00), vol. 2, 2000, pp. 12– 26. [8] A. Dembo and O. Zeitouni, Large Deviations Techniques and Applications, 2nd ed. NY: Springer-Verlag, 1998.

[9] N. Leavitt, “Network-usage changes push Internet traffic to the edge,” Computer, pp. 13–15, 2010.

[10] K. Thompson, G. J. Miller, and R. Wilder, “Wide-area Internet traffic patterns and characteristics,” Network, IEEE, vol. 11, no. 6, pp. 10–23, 1997.

[11] A. King, B. Huffaker, A. Dainotti, and K. C. Claffy, “A coordinated view of the temporal evolution of large-scale Internet events,” Computing, pp. 53–65, January 2013.

[12] Sandvine, “Global Internet phenomena report,”

https://www.sandvine.com/downloads/general/global-internetphenomena/2013/sandvine-global-internet-phenomena-report-1h- 2013.pdf, 2013.

[13] P. J. Huber, “A robust version of the probability ratio test,” The Annals of Mathematical Statistics, vol. 36, no. 6, pp. 1753–1758, 1965.

[14] W. Hoeffding, “Asymptotically optimal tests for multinomial distributions,” Ann. Math. Statist., vol. 36, pp. 369–401, 1965. [15] I. C. Paschalidis and D. Guo, “Robust and distributed stochastic localization in sensor networks: Theory and experimental results,”

ACM Transactions on Sensor Networks (TOSN), vol. 5, no. 4, pp. 34:1–34:22, 2009.

AUTHORS BIOGRAPHY:

S.M.Nandhagopal completed his B.E. (Computer Science and Engineering) in 2004 from Dr.Pauls Engineering College under Madras University, Tamilnadu, India and M.E. (Computer Science and Engineering) in 2009 from Vellalar College of Engineering & Technology under Anna University, Coimbatore, Tamilnadu, India, Ph.D pursuing in (Computer Science and Engineering) PRIST University at Thanjavur, Tamilnadu, India. He is working as an Associate Professor, Department of Computer Science and Engineering at ASL Paul’s College of Engineering & Technology, Coimbatore, and Tamilnadu, India. He is a member of various professional bodies like ISTE, IEEE and IAENG. His research areas include networks, and Wireless Sensor Networks and having 10 years of teaching experience in Engineering Colleges.