MCOD versus AnyOut Base Learner for E-RAIDS in

In the following, we analyse the performance of E-RAIDS with MCOD base learner versus AnyOut base learner in terms of the pre-defined evaluation measures: T PT out ofPT; FPAlarm; and F1 measure.

Tables 7.2 and 7.3 present the maximum T PT and the minimum FPAlarm at- tained by E-RAIDS-MCOD and E-RAIDS-AnyOut over the communities. The re- sults are reported in terms of the parameter values in the given sequencek, r, w for E-RAIDS-MCOD andτ, oscAgr, wfor E-RAIDS-AnyOut respectively.

TABLE7.3: Minimum FPAlarm over communities associated with (1) the number of neighboursk, the distance parameterr, and the window sizewwhich achieved the minimum FPAlarm for E-RAIDS-MCOD, and (2) the outlier score thresholdτ, the size of outlier score aggregateoscAgr, and the window sizewwhich achieved the mini- mum FPAlarm for E-RAIDS-AnyOut. Note that the parameter combination(s) which achieved the minimum FPAlarm are listed. The results are displayed ashk, r, wifor

E-RAIDS-MCOD andhτ, oscAgr, wifor E-RAIDS-AnyOut. community E-RAIDS-MCOD Parametersk, r, w

com-P 0 50,0.4,200 60,{0.3,0.5,0.6},200 com-S 0 50,0.4,200 50,0.7,100 60,0.3,200 60,0.5-0.7,100 70,0.6-0.7,150 com-I 0 50,0.3,200 60,0.5,200 70,0.5-0.7,200

community E-RAIDS-AnyOut Parametersτ, oscAgr, w

com-P 2 ∀τ,∀oscAgr,200

com-S 2 ∀τ,∀oscAgr,150-200 com-I 1 0.1,{2,4},200 0.3,2-6,200

7.4. Experiments 137

E-RAIDS-MCOD Fig. 7.3 presents the variation of F1measure as a function of window sizew for E-RAIDS with MCOD base learner over the communities. The results are reported with respect tokandrparameter values.

A preliminary analysis of the F1 measure shows no evident pattern in terms of any of the parametersk,r, or w. Over the community com-P, E-RAIDS-MCOD achieves the maximum F1=0.9411;k=60, r=0.7, w=150. It detects the maximum TPT=16 out of PT=17, thus missing one malicious insider threat. However, it flags only one false positive alarm (FPAlarm=1). Furthermore, Table 7.3 shows that E-RAIDS- MCOD reports the minimum FPAlarm=0atw=200for different values ofkandr.

Over the community com-S, E-RAIDS-MCOD achieves the maximum F1=0.9523;

k=70, r={0.6,0.7}, w=150. It detects a TPT=20out of PT=22, while flagging no false positive alarms (FPAlarm=0). The maximum TPT=21is attained atk=70, r=0.4, w=150, however, flagging FPAlarm=2.

Over the community com-I, E-RAIDS-MCOD achieves the maximum F1=0.6451;

k=70, r=0.3, w=100. It detects a TPT=10 out of PT=12, thus missing two malicious insider threats, while flagging FPAlarm=9. Nevertheless, Table 7.3 shows that E- RAIDS-MCOD reports the minimum FPAlarm=0atw=200for different values ofk andr.

We can deduce that, in these experiments, the window size w=150,200give the best performance for E-RAIDS-MCOD in terms of the evaluation measures.

E-RAIDS-AnyOut Fig. 7.4 presents the variation of F1 measure as a function of window sizewfor E-RAIDS with AnyOut base learner over the communities. The results are reported with respect toτ andoscAgrparameter values.

It is evident that F1 measure is inversely proportional to the parameteroscAgr for E-RAIDS-AnyOut. The values of F1 measure atoscAgr=2 is the highest with respect to all the window sizesw=50to200over all communities. Moreover, Figure 7.4 reveals that F1 measure is directly proportional to the parameterw. The value of F1 measure increases as the window sizewincreases.

Over the community com-P, E-RAIDS-AnyOut achieves the maximum F1=0.9142;

138 Chapter 7. Streaming Anomaly Detection for Insider Threat Detection

(A) com-P.

(B) com-S.

(C) com-I.

FIGURE7.3: The variation of F1 measure as a function of window sizewfor E-RAIDS with MCOD base learner over the communities. The x-axis represents the window size

w={50,100,150,200}, the y-axis represents the values of F1 measure, and the legend represents the values of the radius parameterr={0.3,0.4,0.5,0.6,0.7}. The graphs in

Fig. 7.3a, Fig. 7.3b, and Fig. 7.3c represent the results fork={50,60,70}.

the false positive alarms to FPAlarm=2. Table 7.3 shows that E-RAIDS-AnyOut re- ports the minimum FPAlarm=2∀τ,∀oscAgratw=200. Thus, the higher the window size, the lower the FPAlarm. Table 7.2 shows that E-RAIDS-AnyOut reports the max- imum TPT=16atoscAgr=2in general terms∀τ,∀w. Thus, the lower theoscAgr, the higher the TPT detected.

Over the community com-S, E-RAIDS-AnyOut achieves the maximum F1=0.9090;

τ=0.7, oscAgr=2, w=150. It detects a TPT=20out of PT=22, while flagging two false positive alarms (FPAlarm=2).

7.4. Experiments 139

(A) com-P.

(B) com-S.

(C) com-I.

FIGURE7.4: The variation of F1 measure as a function of window sizewfor E-RAIDS with AnyOut base learner over the communities. The legend represents theoscAgr

parameter values. The x-axis represents the window sizew={50,100,150,200}, the y-axis represents the values of F1 measure, and the legend represents the values of

oscAgr={2,4,6,8}. The graphs in Fig. 7.4a, Fig. 7.4b, and Fig. 7.4c represent the results forτ={0.1,0.4,0.7}.

∀τ, oscAgr=2, w=200. It detects a TPT=12out of PT=12, while flagging one false pos- itive alarm (FPAlarm=1).

In terms of the evaluation measures, E-RAIDS-MCOD outperforms E-RAIDS- AnyOut over the communities, where E-RAIDS-MCOD achieves a higher F1 mea- sure over com-P and com-S, a higher TPT over com-S, and a lower FPAlarm=0over all communities.

140 Chapter 7. Streaming Anomaly Detection for Insider Threat Detection