UNSW-NB15 Dataset

The dataset from the Cyber Range Lab of the Australian Centre for Cyber Security (ACCS) is used [38], [108]. In this dataset, a hybrid of real up-to-date normal and ab- normal/malicious network traffic activities were generated in a synthetic environment. The UNSW-NB15 dataset is regarded as complex due to it matching the behaviours seen amongst some types of attacks and normal traffic. This dataset contains over 2 million sample data elements from two different simulation periods. In UNSW-NB15, there are nine categories of attacks [38]:

1. Fuzzers: a technique where the attacker tries to uncover new and unknown vul- nerabilities in a program, operating system, or network by feeding it with the widest possible range of unexpected input of random data to make it crash. 2. Analysis: Variety of intrusions which aim to penetrate internet applications using

ports (e.g., port scans), emails (e.g., spam), and web scripts (e.g., HTML files). 3. Backdoor: an intruder attempts to gain remote access to a device through by-

passing authentication methods. As a result, he/she will be able to alter files, steal sensitive data and/or install malicious software.

4. Denial of Service (DoS): an intrusion that causes computer resources to be so heavily used as to prevent the authorised requests from accessing a device. 5. Exploit: a sequence of instructions that make use of a glitch, bug or vulnerability

and generates an unintentional or unsuspected behaviour on a host or network. 6. Generic: an attack that is executed against all block-cipher to produce a collision

with no consideration of the details of how that block-cipher is implemented. 7. Reconnaissance: can be defined as a probe; an attack that gathers information

about a computer network to evade its security controls.

8. Shellcode: an attack in which the attacker penetrates a slight piece of code start- ing from a shell to control the compromised machine.

9. Worm: a type of computer malware which once it is installed by unaware users it can spread itself on the targeted system. It causes damage to the host network by exhausting its bandwidth and/or modifying and deleting system files.

There were 47 different reliable features that were extracted from the raw network packets. The final features that we adopted in our experiments are listed in Table 4.3 alongside their descriptions. The UNSW-NB15 dataset features are generated utilising tools such as Argus and Bro-IDS. In addition, authors wrote procedures to create new features based on relations among extracted features (e.g., is_sm_ips_ports and ct_state_ttl). They included a variety of packet-based features and flow-based features. These features are grouped into five sets:

(a) Flow features: involve the identifier attributes between hosts (i.e. client or server) for instance IP address, port number and protocol type.

(b) Basic features: include protocol connection properties.

(c) Content features: contain the attributes of TCP/IP; additionally they include some properties of http services.

(d) Time features: involve the attributes time, for instance, arrival time between packets, start/end packet time, and round-trip time of TCP protocol.

(e) Additional generated features: This category can be further divided into two groups: general-purpose features, whereby each feature has its own purpose, in order to protect the service of protocols, and connection features that are built from the flow of 100 recorded connections based on the sequential order of the last time feature.

Table 4.2 compares the experimental data of UNSW-NB15 dataset alongside its distri- bution of each category between training and testing datasets.

Category Training Set Testing Set

Normal 56,000 37,000 Analysis 2,000 677 Backdoor 1,746 583 DoS 12,264 4089 Exploits 33,393 11,132 Fuzzers 18,184 6,062 Generic 40,000 18,871 Reconnaissance 10,491 3,496 Shellcode 1,133 378 Worms 130 44 Total Records 175,341 82,332

Table 4.2: UNSW-NB15 Dataset Distribution [38]

The UNSW-NB15 dataset was split with an approximately 60%:40% proportion of the training and testing datasets sequentially with no redundant instances amongst the training and testing dataset. 68.06% of training instances are attacks whereas 55.06% of testing instances are attacks.

Id Feature Name Description

1 dur Record total duration 2 sbytes Source to destination bytes 3 dbytes Destination to source bytes 4 rate Number of packets per second 5 sttl Source to destination time to live 6 dttl Destination to source time to live 7 sloss Source packets retransmitted or dropped 8 dloss Destination packets retransmitted or dropped 9 sload Source bits per second

10 dload Destination bits per second 11 spkts Source to destination packet count 12 dpkts Destination to source packet count 13 swin Source TCP window advertisement value 14 dwin Destination TCP window advertisement value 15 Stcpb Source TCP base sequence number

16 dtcpb Destination TCP base sequence number

17 smeansz Mean of the packet size transmitted by the srcip 18 dmeansz Mean of the packet size transmitted by the dstip 19 trans_depth The connection of http request/response transaction 20 response_body_len The content size of the data transferred from http 21 sjit Source jitter (mSec)

22 djit Destination jitter (mSec) 23 sinpkt Source inter-packet arrival time 24 dinpkt Destination inter-packet arrival time

25 tcprtt Setup round-trip time, the sum of ’synack’ and ’ackdat’ 26 synack The time between the SYN and the SYN_ACK packets 27 ackdat The time between the SYN_ACK and the ACK packets 28 is_sm_ips_ports If srcip = dstip and sport = dsport, assign 1 else 0 29 ct_state_ttl No. of each state according to values of sttl and dttl 30 ct_flw_http_mthd No. of methods such as Get and Post in http service

31 is_ftp_login If the ftp session is accessed by user and password then 1 else 0 32 ct_ftp_cmd No of flows that has a command in ftp session

33 ct_srv_src No. of rows of the same service and srcip in 100 rows 34 ct_srv_dst No. of rows of the same service and dstip in 100 rows 35 ct_dst_ltm No. of rows of the same dstip in 100 rows

36 ct_src_ltm No. of rows of the srcip in 100 rows

37 ct_src_dport_ltm No of rows of the same srcip and the dsport in 100 rows 38 ct_dst_sport_ltm No of rows of the same dstip and the sport in 100 rows 39 ct_dst_src_ltm No of rows of the same srcip and the dstip in 100 records