robustness of activity hotspots detection

the exception of Cut Ratio, which outperformed Conductance in terms of Separability in the dy-namic scenario. One minor change observed is that the FOMD scoring function is now slightly preferred over TPR in terms of better Clustering Coefficient.

Overall, the goodness experiments in this section suggest that, to identify more clustered, dense and cohesive communities in Twitter in a time-aware context, FOMD and TPR are the better choices for structural scoring functions. If dense but more separated communities are desired by the analyst, then Conductance or Cut Ratio should be considered.

5.6 robustness of activity hotspots detection

The robustness of the scoring functions in the context of the temporal sub-communities gener-ated using user the proposed activity hotspots H(C) is now investiggener-ated. Good scoring functions should be stable under small perturbations and reduce their performance under strong distur-bance. In Section4.4, a set of community perturbation strategies were proposed for studying the robustness of the structural scoring functions: Node Swap, Random, Expand and Shrink. In this section, these strategies are now applied to the temporal sub-communities H(C) generated using activity hotspots and compared to the static scenario.

The perturbation experiment for the dynamic scenario is now defined as follows. Similar as in Section4.4, the perturbation intensity is varied in the range p ∈ [0.01, 0.60], e.g. in the Node Swap strategy this means exchanging between 1 and 60 % of the members of a community, and observe the averaged Z-score across all ground-truth temporal sub-communities H(C) in all community type and datasets. Figures5.12,5.13,5.14and5.15respectively show the averaged Z-score results for the Node Swap, Random, Shrink and Expand perturbation strategies under the proposed intensities for each ground-truth dataset, including a plot with all the data combined.

5.6.1 Node Swap

For the Node Swap perturbation in Figure5.12, similar to the static case the TPR and FOMD scores perform the best in all the long timespan datasets (WorldCup2014, RTE2015 and Ire-land2017), followed by Conductance and Flake ODF. In the case of the Pope Event datasets, Conductance and Flake ODF instead are observed as more robust scores. In contrast, Modularity

0.0 0.1 0.2 0.3 0.4 0.5 0.6

Figure 5.12: Z-scores of intensities for the Node Swap perturbation strategy applied to all temporal sub-communities H(C) based on user activity hotspots generated from sub-communities C for all community types in each Twitter ground-truth dataset. A combined plot is also presented.

and Cut Ratio do not degrade as gracefully – particularly Modularity – when the perturbation is increased, revealing their inability to handle noisy data in Twitter also for the dynamic scenario.

5.6.2 Random

For the Random perturbation in Figure4.7, again the results are very similar as in static case.

The internal and mixed connectivity families of scores – FOMD, TPR, Conductance and Flake ODF – consistently perform the best, with the internal family being very robust (for example in the Ireland2017 dataset). Cut Ratio and Modularity still perform poorly in in presence of strong noise in the dynamic scenario, with their Z-scores having very small variation under higher levels of perturbation in contrast to the other scoring functions.

5.6.3 Expand and Shrink

Lastly, the Expand and Shrink perturbations results seen in Figure 5.14 and Figure 5.15 also confirm TPR and FOMD as generally robust scores for Twitter functional communities in the dy-namic scenario using user activity hotspots, specially the Shrink perturbation. Likewise the static

5.6 robustness of activity hotspots detection 105

Figure 5.13: Z-scores of intensities for the Random perturbation strategy applied to all temporal sub-communities H(C) based on user activity hotspots generated from sub-communities C for all community types in each Twitter ground-truth dataset. A combined plot is also presented.

0.0 0.1 0.2 0.3 0.4 0.5 0.6

Figure 5.14: Z-scores of intensities for the Expand perturbation strategy applied to all temporal sub-communities H(C) based on user activity hotspots generated from sub-communities C for all community types in each Twitter ground-truth dataset. A combined plot is also presented.

0.0 0.1 0.2 0.3 0.4 0.5 0.6

Figure 5.15: Z-scores of intensities for the Shrink perturbation strategy applied to all temporal sub-communities H(C) based on user activity hotspots generated from sub-communities C for all community types in each Twitter ground-truth dataset. A combined plot is also presented.

scenario, The Cut Ratio score is unable to handle communities that get smaller in Twitter, how-ever its robustness improves when communities expand. The Modularity score performs more robustly than in the static case in the larger WorldCup2014, RTE2015 and Ireland2017 datasets for the Shrink strategy. However, for the other datasets it remains consistently well performing in small intensities for Expand and large for Shrink, but again degrades with larger expansions and smaller reductions, similarly to the static case. The resolution limit of Modularity [FB07]

still produces a negative effect in the temporal sub-communities H(C).

In this new experiment, TPR and FOMD from the internal connectivity family again proved to be robust community scoring functions for Twitter interaction streams in a dynamic setting, while Modularity and Cut Ratio proved weaker in the same context. Modularity, however, demonstrated to be more robust in the temporal sub-communities H(C) due to their smaller sizes compared to the original communities C. Alternatively, Flake ODF and Conductance – in a lesser degree – from the mixed connectivity family are also reasonably robust choices for microblogging data in the dynamic scenario using user activity hotspots.

5.6 robustness of activity hotspots detection 107

Table 5.8: Average absolute increment of Z-score between small (p = 0.04) and large (p = 0.20) community perturbations for the dynamic scenario. Largest differences (most robust and sensitive scores) are in bold. The previous best scores in the static scenario (Table4.3) are marked with .

Family Score N.Swap Random Expand Shrink

External Cut Ratio 0.1032 0.0639 0.1072 0.0007 Internal FOMD  0.6098  1.0362  0.0361  0.5902

Internal TPR 0.4790 0.8623 0.0005 0.4919

Mixed Conductance 0.3549 0.6627 0.0732 0.2298

Mixed Flake ODF 0.2738 0.7174 0.1127 0.3099

Net-Model Modularity 0.0591 0.0540 0.0525 0.0930

5.6.4 Detection Sensitivity

Finally, the sensitivity of the scoring functions in terms of small and large perturbations is re-investigated for the dynamic scenario presented in this chapter. For this experiment, again the change of Z-score between a small (p = 0.04) and a large (p = 0.20) perturbation is mea-sured, giving preference to scoring functions that quickly degrade in presence of strong pertur-bations. The difference Z(f, h, 0.20) − Z(f, h, 0.04) is averaged across all ground-truth temporal sub-communities H(C) in all community type and datasets, and the results can be seen in Ta-ble5.8. In these results, large differences indicate that the community scoring function is both robust and sensitive, and the previous bests from Section4.3are marked with .

In this new experiment, FOMD again remains as the most robust and sensitive scoring func-tion for the dynamic scenario. Nevertheless, important differences can be highlighted with respect to the static case in Table4.3. First, for the Node Swap perturbation FOMD has a pos-itive difference (+0.1298) in the dynamic case than in the static case, however in the Random and Shrink strategies it has a negative difference (−0.1289 and −0.1104 respectively). This result indicates that, despite FOMD still being the preferred scoring function, it is also slightly less robust and sensitive in comparison to the static scenario. Moreover, Flake ODF and Cut Ratio surpass FOMD for the Expand perturbation in comparison with the static case, suggesting that in expanding communities the external connectivity is also necessary to be considered.

In general, the FOMD and TPR scores (internal connectivity family) still stand as the most robust and sensitive scores in this experiment for all the perturbation strategies but Expand – where the mixed and external connectivity scores are better – in the dynamic scenario. The Modularity score performs poorly under every perturbation strategy except Shrink, where only Cut Ratio is worse for microblogging data using user activity hotspots.