Temporal Analysis

We next study the temporal characteristics of network heartbeats. Each heartbeat has two salient attributes: period and lifespan. The period is the amount of time between successive connections.

The lifespan is the elapsed time between the first and last connections.

Figure 5.3 shows CDFs of the period and the lifespan for the network heartbeats observed.

Figure 5.3(a) includes all heartbeats, while Figure 5.3(b) excludes P2P heartbeats. This division was made because P2P applications generate many more heartbeats than regular applications do.

This causes spikes in the frequency distribution, which are removed by excising all P2P heartbeats.

The upper line (in blue) is for the heartbeat period. The lower line (in green) is for the lifespans.

Both are measured in seconds. Note that the horizontal axis is log scale, and that the vertical dashed lines denote one minute, one hour, and one day periods.

(a) All Heartbeat Traffic

(b) Non-P2P Heartbeat Traffic

Figure 5.3: CDFs of heartbeat periods/lifespans.

Heartbeat periods tend to be short. Even so, heartbeats also have a wide range of periods. The shortest period was 10 seconds, and although our method could detect a period as long as 12 days, the longest observed period was 8.8 days². It should be noted that 10 seconds is the smallest period we can detect using the default parameters presented in Table 3.3 from Chapter 3. This means that

2This was a Web proxy validating a cached object.

there are likely heartbeats with even shorter periods, however our method cannot detect them using our default parameters. Despite this large range, heartbeat periods are concentrated at the lower end of the distribution, with periods up to 1 minute representing 40% of all heartbeat periods, and periods under one hour comprising 97%. One-minute periods were common, especially for P2P applications. The most frequent longer periods were 15, 30, 50, and 60 minutes. Periods longer than one day accounted for less than 0.1% of all observed periods even so, it is interesting to know what they are.

Heartbeat lifespans also tend to be short. Lifespans have a broad range, with the shortest being only 30 seconds, and the longest being 47.9 days. However, 18% have a lifespan under 1 minute, and 71% have a lifespan under one hour. This is skewed to be shorter than the actual heartbeat lifespan due to the irregularity issues discussed in Chapter 4. Lifespans vary more widely than periods, but there is a significant peak around 10 minutes for P2P. About 30% of the lifespans exceed one hour.

To better grasp the effects of irregularity on temporal characteristics we have repeated our analysis on regular and irregular heartbeats.

Figure 5.4: CDF of periods and lifespans of regular heartbeats.

Figure 5.4 is generated in the same manner as Figure 5.3. Figure 5.4 shows the CDF for period

and lifespan of regular heartbeats. Irregular heartbeats make up over 99% of all heartbeats, so the CDFs presented in Figure 5.3 suffice for comparing regular and irregular heartbeats.

Regular heartbeats tended to have both long lifespans and long periods. Lifespans ranged from 35 to 47.9 days. This is to be expected due to the definition of a regular heartbeat. However, regular heartbeats also tended to have long periods. These periods often conform to well-defined, intuitive values. The periods were heavily concentrated into daily (49%) and weekly (41%) patterns. The remaining 10% fall into patterns that are divisible by an hour, specifically 1, 6, 8, 33, 48, 72, 96, 120, 140, and 210 hours. The shortest of these belonged to an NTP application. The longest was a Web proxy validating a cached object every 8.8 days.

As explained above, irregular heartbeats have almost identical temporal characteristics to all heartbeats. However, there are two notable differences: the longest irregular lifespan and longest irregular period. The longest irregular life span of 45 days was a daily NetBIOS scan. This heartbeat was irregular because it did not start until 3 days after we began observation. The longest irregular period was 5 days, which was related to two heartbeats: a NetBIOS scan (not the same one mentioned above) and an HTTP request checking for database updates. As discussed in the analysis above, there were some more popular lifespans and periods, but they were considerably more varied than in regular heartbeats.

This additional analysis reveals that regular heartbeats are more consistent than irregular heart-beats. Regular heartbeats have periods that fall mainly into specific intuitive values. Though irregular heartbeats demonstrate this too, there is significantly more variation. Likewise, when accounting for the first and last period, regular heartbeats all last the entire duration of the log.

However, the shortest and longest irregular lifespans are 45 days apart. Thus, we can infer that we are mostly observing fragments of regular heartbeats that are non-deterministically interrupted by the causes of irregularity.

From our analysis, we can conclude that “typical” heartbeats tend to have both short periods and lifespans. If one were to monitor on the end-devices, by using the MAC address, or observing

on the host machine itself, the lifespans would likely be much longer (i.e., as long as the application in question is installed/running on that device). However, from the perspective of the network operator, the lifespans appear shorter.

Analysis of the relationship between period and lifespan reveals a moderate positive correla-tion, with a correlation coefficient of +0.73. This correlation is clearly demonstrated in Figure 5.5.

In this figure, each heartbeat’s lifespan has been plotted against its period in the same manner as Figure 5.2 from the previous section. Figure 5.5 shows a clear trend of heartbeats with longer pe-riods having longer lifespans. Intuitively, it makes sense that heartbeats with longer pepe-riods have longer lifespans, since they require more time to be noticed.

Figure 5.5: Scatterplot showing the correlation between heartbeat period and duration.

This observation implies that while there do exist long continuous heartbeats, most heartbeats tend to be produced in short, discontinuous bursts. Short lifespans can be due to the applications,

or reflect normal end-user behavior.

Applications are sometimes designed to generate heartbeats with short lifespans. This observa-tion is supported by manual inspecobserva-tion of heartbeats produced by specific applicaobserva-tions. End-user applications such as the Steam video game client, or P2P applications, produce lots of heartbeats with short periods and lifespans. These applications are popular with end-users, and are known to produce heartbeats [6, 37]. Services such as NTP produce heartbeats with long periods and long lifespans. These heartbeats are not user-triggered, however, so they are less prevalent.

End-user behaviour produces short lifespans by introducing irregularity. End-users tend to start and stop applications, relocate, or shut down their devices, particularly in a BYOD environment.

This behaviour causes irregularity, thus cutting heartbeats short. A natural consequence of this is shorter lifespans.