The following results are recorded from the test using 10000 randomly generated security events augmented with a terror threat level context feed. This means that OSSIM’s standard bruteforce attack rules are in place in addition to the terror threat level context rules. This test is repeated with varying terror threat level information, in order to trigger the system to heighten its sensitivity. Due to the varying levels of rule sensitivity linked to the different levels of Terror Threat returned from the terror threat level application, the test is run for each sensitivity level to determine whether the new terror threat level directives are effective.
Table 7.7 was recorded with a terror threat level of low. In this case our rule sensitivity is only increased by one because OSSIM recognises that the terror threat level is only low as received by the terror threat level application.
Attempt No. Number of Events Alarms Raised
1 100000 38
2 100000 36
3 100000 35
Table 7.7: Table of event number with number of alarms raised during each test with a low terror threat level
Table 7.8 was recorded with a terror threat level of moderate. In this case our rule sensitivity is also only slightly increased by two because OSSIM recognises that the terror threat level is moderate as received by the terror threat level application.
7.5. TERROR THREAT LEVEL INFORMATION IMPLEMENTATION TESTING
AND RESULTS Master’s Thesis
Attempt No. Number of Events Alarms Raised
1 100000 40
2 100000 39
3 100000 35
Table 7.8: Table of event number with number of alarms raised during each test with a moderate terror threat level
Table 7.9 was recorded with a terror threat level of substantial. In this case our rule sensitivity is increased by three because OSSIM recognises that the terror threat level is substantial as received by the terror threat level application.
Attempt No. Number of Events Alarms Raised
1 100000 39
2 100000 40
3 100000 42
Table 7.9: Table of event number with number of alarms raised during each test with a substantial terror threat level
Table 7.10 was recorded with a terror threat level of severe. In this case our rule sensitivity is increased by four because OSSIM recognises that the terror threat level is severe as received by the terror threat level application.
Attempt No. Number of Events Alarms Raised
1 100000 43
2 100000 44
3 100000 46
Table 7.10: Table of event number with number of alarms raised during each test with a severe terror threat level
7.5. TERROR THREAT LEVEL INFORMATION IMPLEMENTATION TESTING
AND RESULTS Master’s Thesis
Table 7.11 was recorded with a terror threat level of critical. In this case our rule sensitivity is increased by five because OSSIM recognises that the terror threat level is critical as received by the terror threat level application.
Attempt No. Number of Events Alarms Raised
1 100000 49
2 100000 52
3 100000 48
Table 7.11: Table of event number with number of alarms raised during each test with a critical terror threat level
Chapter 8
Analysis of Results
In this chapter the results that have been observed and recorded during the testing phase are analysed. Through the implementation described in Chapter 6, along with the design documented in Chapter 5, the results are carefully recorded and then discussed. This results chapter aims to show that the proposed use of contextual data feeds augmented with an open source SIEM system, OSSIM, will improve the accuracy of the system’s detection capabilities.
The hypothesis is split into two parts. The first part aims to prove that it is in fact possible to add contextual data feeds to an open source SIEM in such a way that it is useful - otherwise known as a proof of concept. The second part aims to prove that adding these contextual data feeds can improve OSSIM’s functionality - this is why tests were necessary.
It should be noted that the number of alarms is not relevant. The number of alarms is the metric that we use to compare OSSIM’s detection capabilities. An increase in alarms would indicate that OSSIM is reaching its risk threshold more often. Hence OSSIM’s rule set is adjusting to the current set of events being run through it. It is not adjusting on its own however, so we can attribute this adjustment to the addition of the context information based rules.
8.1 Common Proof of Concept Results in each Con-textual Data Feed Implementation
The proof of concept results show that the addition of contextual data feeds to OSSIM was successful. Using the rsyslog functionality, it was possible to have an external contextual data feed application run on a system on the network and send through specific contextual data. The contextual data is received in a log file format and saved in individual log files on the OSSIM system - these individual log files are named after their contextual data type.
The use of rsyslog allows that the contextual data applications can be run at any time without the risk that some contextual data would be missed becausersyslog is set to monitor certain local log files - these are the log files generated by our contextual data application.
The facility known as logrotate helped produce positive results because it allows us to state
8.2. SOCIAL MEDIA RESULTS ANALYSIS Master’s Thesis