3.3
NETWORK STATE STORAGE AND MANAGEMENT
Network state is stored in tables, which contain tuples with expiry times and size constraints that are declaratively specified at table creation time as described in Chapter2. Duplicate entries (tuples) are allowed in tables, and the mechanisms for maintaining these duplicates differ based on whether they are hard-state or soft-state tables as defined in Chapter2. In hard-state tables, a derivation count is maintained for each unique tuple, and each tuple is deleted when its count reaches zero. In soft-state tables, each unique tuple has an associated lifetime that is set based on the specified expiration of its table during creation time. Duplicates result in extension of tuple lifetime, and each tuple is deleted upon expiration based on its lifetime. The lifetimes of soft-state tuples are enforced by purging the soft-state tables of any expired tuples whenever they are accessed. Tables are named using unique IDs, and consequently can be shared between different queries and/or dataflow elements.
As basic data types, DN uses Values, and Tuples. A Value is a reference-counted object used to pass around any scalar item in the system; Value types include strings, integers, timestamps, and large unique identifiers. The Value class, together with the rules for converting between the various value types, constitute the concrete type system of DN. A Tuple is a vector of Values, and is the basic unit of data transfer in DN. Dataflow elements, described below, pass tuples between them, and tables hold sets of tuples.
Queries over tables can be specified by filters, providing an expressivity roughly equivalent to a traditional database query over a single table. In-memory indices (implemented using standard hash tables) can be attached to attributes of tables to enable quick equality lookups. Note that the table implementation—including associated indices—is a node-local construct.
3.3.1 RAPIDNET DECLARATIVE NETWORKING ENGINE
The RapidNet [RapidNet] declarative networking engine compile the NDlog programs into appli- cations (with an execution model similar to Click [Kohler et al.,2000]) executable in ns-3 [ns-3], an emerging discrete event-driven simulator similar to the popular ns-2. Like its predecessor, ns-3 emulates all layers of the network stack, supporting configurable loss, packet queuing, and network topology models.
RapidNet has been used as an experimental platform [Liu et al.,2012,Muthukumar et al.,
2009a] for a variety of declarative wireless routing protocols on the ORBIT [ORBIT] wireless
testbed. The ability to run the same application in these two modes enables us to execute each NDlog program at scale in simulation and in an actual implementation running on a testbed.
In the initial design phase of RapidNet, a network protocol design is used as the basis for specifying the network protocol using the NDlog declarative networking language. In the simulation
mode, the RapidNet compilation process generates ns-3 code from the NDlog protocol specifications.
The generated code either runs as an ns-3 application, or replaces routing protocol implementations at the network layer. In the implementation mode, declarative networking specifications are directly executed and deployed either by using actual network sockets implemented on a real network across multiple machines. Simulation mode enables a comprehensive examination under various network
topologies and conditions, while the implementation mode allows different hosts in a testbed envi- ronment to execute the deployed system over a real network.
Since declarative networks share common functionalities such as the network stack, multiplex- ing tuple messages entering and leaving the dataflow, and database functionalities, all these utilities are defined in a shared RapidNet library. This enables one to simplify the compilation process to only the relevant database operations to implement the distributed dataflows for the corresponding declarative network specification. This also enables one to easily incorporate multi-query optimiza-
tions to share computations across declarative networks in future.
Figure 3.4: RapidNet demonstration [Muthukumar et al.,2009a,b] of a link-state routing protocol.
The RapidNet system takes input declarative network specifications which are automatically compiled to ns-3 code for execution in the ns-3 simulation and emulation modes. Network traces are directed to a ns-3 visualizer that will display the actual movement of nodes during the simulation, and side-by-side actual performance statistics of the protocol obtained from the ns-3 network statistics package.
Figure3.4shows an example execution of RapidNet system, based on a declarative mobile ad-hoc network (MANET) routing protocol [Liu et al.,2009a,2011a]. The system (as demon- strated in [Muthukumar et al.,2009b]) visually shows the actual movements of nodes communicat- ing via 802.11b ad-hoc mode using one of ns-3’s supported mobility models (e.g., random waypoint,
3.4. SUMMARY 27 Brownian motion, hierarchical mobility, etc.). Each RapidNet node incrementally updates its routes using a variety of declarative MANET routing protocols such as hazy-sighted link-state rout- ing (HSLS) [Santivanez et al.,2001], optimized link-state routing (OLSR) [Clausen and Jacquet,
2003], dynamic source routing (DSR) [Johnson and Maltz,1996], and summary-vector based epi- demic routing [Vahdat and Becker, 2000]. Actual performance statistics of the protocol are also displayed side-by-side the actual protocol itself.
3.4
SUMMARY
This chapter presents an overview of the DN declarative networking engine, with an emphasis on its architecture, various components (planner, dataflow installer, dataflow engine), dataflow framework and network state management. The next chapter describes the planner component in greater detail, and shows how NDlog programs can be compiled into dataflow-based execution plans to implement the network protocols using the DN dataflow engine.
29
C H A P T E R 4
Distributed Recursive Query
Processing
One of the main challenges of using a declarative language is to ensure that the declarative specifica- tions, when compiled and executed, result in correct and efficient implementations that are faithful to the program specifications. This is particularly challenging in a distributed context, where asyn- chronous messaging and the unannounced failure of participants make it hard to reason about the flow of data and events in the system as a whole. This chapter addresses this challenge by describing the steps required for the DN planner to automatically and correctly generate execution plans from the NDlog rules.