The previous section described how probabilistic signature activation decreases false nega- tive error rates under certain conditions. However, most uses of anticipatory approaches (such as when a system is not under heavy loads) will generally result in additional er- rors. These errors can be both false positives and false negatives. In general, anticipatory optimizations result in false negatives due to packets being processed against subsets of the original signature-sets. However, some types of signature chaining and anticipatory mechanisms can also result in false positives even though the signature-set is only ever being decreased in size.
3.3.1 Signature Chaining & Flow Tracking
Signature chaining is most often used to activate a signature which depends upon prior signatures, as in: if fire(sa) then activate(sb).
Two signatures which have a negated relationship can result in a false positive if sais
absent from the secondary signature or if packet forwarding to secondary IDS instances have disabled session tracking. For example, a false positive will occur within a secondary IDS instance if sais not part of the secondary IDS signature-set and there is a signature
If flow-tracking is disabled entirely in a secondary signature-set can also result in error related to the direction of an established connection or the firing of alerts which relate to closing sessions which are not known to be established. To alleviate issues in signature- chaining caused by signature-set partitioning we must both a) keep chained signatures together when signature-sets are partitioned; and b) redirect packets to secondary sets which require as preconditions a particular flow state. For example, if the signatures used for prediction in the primary signature-set, si 2 S, properly characterize flow, then we
can infer that the flow is established and in which direction for any secondary signature- set (s0
i 2 S0). It is relatively straightforward to partition secondary signature-sets into
“to server” and “to client” sets and craft predictor rules to forward to the correct secondary instance. However, if the event that initiated packet forwarding did not track the session state for the connection then flow state of future packets will be unknown. Similarly, if any of the signatures in S0 signature-sets has flow-tracking with negated preconditions,
applying packets for connections with unknown flow state can result in false positives.
3.3.2 Systematic Errors
It is also possible to introduce systematic increases in false-negatives. This is particularly relevant when non-probabilistic or rule-based forwarding choices are made. If an attacker were to know the rule-based forwarding choices being made then it would be an easy target for circumventing detection. Rule-based predictors should be used only in instances where there is high confidence that the information being used to initiate packet forwarding is trustworthy. Alternatively, random testing of rule-based predictor assumptions (by disabling predictive forwarding) can help determine if an attacker is circumventing detection through misuse of predictor choices.
For the Bayesian predictor discussed in Sect. 5.1.3 a fixed confidence threshold can also incorrectly classify traffic due to short-term biases in traffic and sequence distributions.
Increasing the window-size over which statistics are collected can help to minimize this issue, but would likely result in poorer performance. The problem with fixed thresholds and short time windows is that statistics are unlikely to be correct outside of the time window in which they were calculated. A possible solution is to perform online learning of attack and event distributions and of appropriate decision boundaries. Unfortunately, online learning can also lead to to systematic errors, both unintentional and attacker-directed. Another approach may be characterizing various event distributions and using distribution change detection methods between sets of known distributions. This may result in less susceptibility to attacker-directed biases and represents a problem has been studied heavily in the statistics literature[74, 50].
3.3.3 Error Detection
Error detection is a non-trivial task for detection systems. Several interesting approaches have been proposed and implemented by others[56], though many of these are only rele- vant when system loads are high enough for the IDS to drop incoming packets. Similar approaches can be taken to estimate errors for IDS event predictors. Lee et al. describe an approach which injects traffic into the system to determine the rate at which packets are being dropped[56]. This approach could be easily extended to inject packets which are intended to create an alert. Missed alerts can be used as a proxy to estimate IDS misclassification rates and to adjust predictor threshold accordingly.
If a fixed threshold is used then traffic which meets the criteria will always be forwarded and errors cannot be detected. If a probabilistic choice is made that is based on the current threshold, then packets which would have failed the criteria can be tagged in order to measure whether an error would have been made had they passed the criteria.