Multiple Step Anomalies for Strong Normal Behaviour

If we need to raise an alarm at certain step of the multiple steps specification, then formula 8.4 can be applied up to that step. For example, if we have four specification steps and the requirement is to have an alarm when the third and the fifth steps failed, then we use formula 8.4 twice. Once to represent the first three steps of the specification and again to represent the full four steps of the specification. If the requirements is to raise at each step, then we would need to use formula 8.4 three times, that is, for the first two, first three, and all the four steps. This means we need to write three formulae that will have syntax similar to the three formulae in 8.5. Also, the negation of these formulae will have similar syntax to the formulae in 8.6. It would be more convenient if this requirement of raising alarm at each step can be entered in one formula. For this, we suggest the second syntactical formula:

(ϕ→ψ) (8.7)

where:

• ϕis a predicate (representing a packet).

• ψ is either:

– Formula of the form ♦[t1,t2]Pi , wherePi is first order predicate; – Pi∧ψ

Chapter 8. Potential Use of The new System in Anomaly Based IDS 119

Using the above syntax, we can write formulae for two or more steps as follows (Pi

denotes first order predicate):

(P1 →♦[t1,t2]P2) (P1 →♦[t1,t2](P2∧♦[t3,t4]P3)) (P1 →♦[t1,t2](P2∧♦[t3,t4](P3∧♦[t5,t6]P4))) . . . (8.8)

If we take the last formula above to represent four normal steps specification we can conclude the following about when it will not hold in one of the following cases:

• ifP1 holds but not P2;

• ifP1 and P2 holding and notP3;

• ifP1,P2, and P3 holding and notP4.

We can argue that this way of writing stronger requirements makes it easier in case we need to raise an alarm at each consecutive steps. In the grammar file, the parse and lexical rules for these syntax are shown in Figures 8.2, 8.3, and 8.4.

Example 7.3: In this example we use the TCP simultaneous connection synchronization specification as described in RFC793 [40]. There are two ways for establishing connec- tions in TCP as in the RFC, the three-way handshake and the simultaneous connection synchronization. In the simultaneous connection specification the normal steps between clients A and B for the process as in the RFC is shown in Figure 8.5. In the following we specify the equivalent to observable steps 2,3,5,6 in Figure 8.5:

• Client A sends a packet to B with source IP (x1), source port (x2), destination IP

(x3), destination port (x4), initial sequence number =SA, and sets thesyn flag.

• Client B just after the time that A is sending the above request, sends a packet to A with source IP (x3), source port (x4), destination IP (x1), destination port

(x2), initial sequence number =SB, and sets thesyn flag.

• Client A receives the synchronization request from B and responds with a packet that has the acknowledge number =SB+ 1 and both the syn and ack flags set.

• Client B receives the synchronization request from A and responds with a packet that has the acknowledge number =SA+ 1 and both thesyn and ack flags set.

Chapter 8. Potential Use of The new System in Anomaly Based IDS 120 Figure 8.2: P arser Ru le s (1 of 3)

Chapter 8. Potential Use of The new System in Anomaly Based IDS 121 Figure 8.3: P arser Ru le s (2 of 3) for The Anomaly Based Detection

Chapter 8. Potential Use of The new System in Anomaly Based IDS 122 Figure 8.4: P arser Ru le s (3 of 3) for The Anomaly Based Detection

Chapter 8. Potential Use of The new System in Anomaly Based IDS 123

Figure 8.5: Simultaneous Connection Synchronization (RFC793, September 1981)

((∀x1, x2, x3, x4, SA, SB)

((∃y6, y7, y9, y10, y11, y12)P(x1, x2, x3, x4, SA, y6, y7,1, y9, y10, y11, y12)→

[0,1]((∃w6, w7, w9, w10, w11, w12)P(x3, x4, x1, x2, SB, w6, w7,1, w9, w10, w11, w12)→

[0,1]((∃z9, z10, z11, z12)P(x1, x2, x3, x4, SA, SB+ 1,1,1, z9, z10, z11, z12)→

♦[0,1](∃k9, k10, k11, k12)P(x3, x4, x1, x2, SB, SA+ 1,1,1, k9, k10, k11, k12)))))

The above formula is the formula of the specification entered by the user. In the mapping the above formula is translated into the negative form (formula 8.6 as following:

((∃x1, x2, x3, x4, SA, SB)

♦[0,1]((∃y6, y7, y9, y10, y11, y12)P(x1, x2, x3, x4, SA, y6, y7,1, y9, y10, y11, y12)∧

♦[0,1]((∃w6, w7, w9, w10, w11, w12)P(x3, x4, x1, x2, SB, w6, w7,1, w9, w10, w11, w12)∧

♦[0,1]((∃z9, z10, z11, z12)P(x1, x2, x3, x4, SA, SB+ 1,1,1, z9, z10, z11, z12)∧

¬_♦[0,1](∃k9, k10, k11, k12)P(x3, x4, x1, x2, SB, SA+ 1,1,1, k9, k10, k11, k12)))))

The above formula is for weak normal requirements specification, which means that it will raise an alert only when the fourth packet or the fourth step of the simultaneous handshake is missing. This formula was translated intoSSQLcode and tested using the custom data file prepared in section 7.1.2. No alert or alarm was raised. In fact, the custom data file has a total of 88458 TCP connection records, but the simultaneous way of TCP connection is used to handle simultaneous requests for a connection which occur rarely.

The interesting finding was when we implemented this example (7.3) as strong re- quirements, which means that an alert will be raised whenever a step (2,3, or 4) is not completed. The result was a total of 88458 alerts. This is because all the connections in the test data file used the three handshake way of tcp connection (explained earlier in Section 5.2). The three way TCP handshake and the simultaneous TCP handshake share the first step but they differ in the second step. This take us back to the reporting

Chapter 8. Potential Use of The new System in Anomaly Based IDS 124

issue of anomaly behaviour in multiple step specifications that we discussed earlier in this chapter.