Buffer management

The main purpose of the buffer management system (BMS) is to capture all packets that are marked as interesting by a low-level processing task (e.g., a kernel FPL filter) and hand a reference to these packets to the appropriate user applications.

Note that in our packet processing framework, FFPF, there is always one producer (packet source) and many consumers (filters), which allows us to implement lock-free circular buffers to store the packets.

As also illustrated in Figure 4.1, Circular buffers in FFPF have two indices to indicate the current read and write positions. These are known asRandW, respectively. Whenever

W catches up with R, the buffer is full. The way in which the system handles this case is defined by the buffer management system (BMS). The modular design of FFPF allows different BMSs to be used. The administrator chooses the BMS at startup time. The optimal choice depends on the mix of applications that will be executed and their relative importance.

− filter 1

− filter n

incoming packet

1

R

n

R

W

Figure 4.1: Circular buffer in FFPF.

At the time of writing, two BMSs have been defined. The first is known as ‘slow reader preference’ (SRP) and is commonly used in existing systems. The second is known as ‘fast reader preference’ (FRP) and is a novel way of ensuring that fast readers are not slowed down by tardy applications.

Buffer management is concerned with how readers and writers synchronise and share their buffers. SRP corresponds to ‘traditional’ buffer management in which new packets are dropped if the buffer is full. While convenient, the disadvantage is that one tardy application that fails to read the (shared)PBufat a sufficiently high rate, causes packet loss for the entire group. FRP, in contrast, simply keeps writing regardless of whether the buffer is full. The trick is that it enables applications to checka posterioriwhether the packets they just pro- cessed have been overwritten. For efficiency, applications in both SRP and FRP may process their packets inbatches(e.g. 1000 packets at a time) in order to minimise context switches. Details about SRP and FRP are provided in [14].

A high-level overview of the FFPF implementation on commodity PCs is shown in Fig- ure 4.2. It shows that the central component of FFPF is the Buffer Management System (BMS). BMS consists of a main buffer (PBuf) shared by all applications that may access it. BMS also includes several secondary buffers and pointer lists needed to assure a good system management. For instance, assuming the system hasmfilters (F1, F2, ...Fm) registered in the

filter listg_listFilters. Each filter may access a packet in the sharedPBufby one of the following two ways: (1) directly getting the next available packet inPBuf by reading X up to the globalW position, increasing its localRposition inPBuf, and eventually marking, into a local index buffer (IBuf), the packets found interesting for other filters such as higher level applications, or (2) by getting the next packet pointed by the localIBufbeing marked as interested beforehand by other filters. The latter option is useful when using a chain of filters. The main purpose of the BMS is to capture all packets that are considered ‘interesting’ and hand a reference to these packets to the appropriate applications. The idea behind FFPF is simple. Users startapplicationsin user-space (see App.1, App.n in Figure 4.2) that load filters. A filter is an application oriented on packet processing only and it runs at low levels such as OS kernels, or even in hardware. An application may use the processing results of one or more loaded filters by way of a shared memory that is calledMBufand is located inside each filter. Moreover, an application may use a filter as a ‘pre-processor’ so as to offload part of its heavy packet processing job onto lower processing levels. In this case, the application retrieves from its loaded filter indexes to the packets needed for a more exhaustive processing than the filter was able to perform.

4.1 FFPF on commodity PCs 57 Fm F2 F1 00000000 00000000 00000000 00000000 11111111 11111111 11111111 11111111 F2 local pkts index App 1 local pkts index App 2 local pkts index App n F1 R IBuf Fm R IBuf mmap g_listFilters network devices userspace kernel packet W PBuf

Figure 4.2: The FFPF implementation on commodity PCs.

pushes it into the main shared packet bufferPBufand advances the globalW pointer with one slot. When a filter classifies a packet as ‘interesting’ for other filters or user applications, it returns a non-zero value, a pointer to the packet is placed in the filter’s index bufferIBuf, and the localRpointer advances one slot. An application may use the filter’s index buffer to find the packets in which it is interested. In addition, a filter has its own chunk of memory (known as‘MEM’) that is shared with the application and which it may use to store results to be read by the application or temporary values that do not disappear between invocations (persistent state). All buffers are memory mapped, so no copying between kernel and user-space is necessary.