Stream Programming Languages

As noted in [123]a number of programming languages have integrated the concept of streams. An exhaustive listing of the features of all the streaming languages can be very long. Bearing that in mind, we will give a brief overview of some of the notable languages here.

Each language introduces its own terminology to describe components of the language, some of the examples are:

The graph representing the application contains the vertices representing compu- tation and edges representing connection between them, also denoted as the networkor an execution plan

Vertices representing computation are also denoted as operator node, operator, node, function, kernel, filter, component, or box

Edges representing connection are also referred to as channel, data path, stream, or data arc

The action performed by a node is denoted by terms invocation, execution, firing or activation

A single unit of data travelling on a stream can be identified as envelope, message, or record

mented and compiled natively in the underlying systems, e.g. in C++. The streams are uni-directional, while the program structure is dynamic. The data is processed by operators. The operators are made up of two parts: The first part contains strictly serial computation called sub-operators, i.e. it should not be possible to decompose it any further. The second part specifies how all the sub-operators of an operator are connected. Apart from user-specified sub-operators, there are two implicit special sub-operators that define the input and the output for the operator. Each operator may have multiple inputs or outputs, which in turn allow operations like gather and scatter. The technical documentation presents a detailed description of the language syntax[12].

The application development environment called Auto-Pipe[27]is geared towards simplifying the development of complex streaming applications for hybrid architec- tures. In order to achieve its goal, Auto-Pipe uses a coordination language called X[40]

to connect kernels written in traditional languages in a data-streaming style, i.e. X can express the nodes and edges of a task-processing graph, as well as the configuration, platform definition, and device bindings. X allows representation of a program in terms of coordination and communication of kernels written in a different computation language. Since Auto-Pipe is targeting hybrid architecture, each kernel may have several platform-specific implementations, e.g. American National Standards Institute (ANSI) C, Compute Unified Device Architecture (CUDA), and VHSIC Hardware Description Language (VHDL). All platform-specific implementations of a kernel are required to provide the same interface and streaming data semantics, e.g. input/output ports and data types. This in turn guarantees the correctness of a program, regardless of where each kernel is mapped. The runtime of X is responsible for making sure that the data arrives at the correct kernel, even if it is executed on a different platform. In addition to language X, the Auto-Pipe environment has X-Com, a compiler that takes in the application’s specification of computation, resources, and a topology of these resources to generate a set of source files that can be compiled for each device in the system. The X-Dep, an application deployment tool, is used to deploy the application on the target hardware.

StreamIt [127] from Massachusetts Institute of Technology (MIT) is believed to be the most popular streaming language based on the ideas of the SDF in the research community. StreamIt is accompanied by an optimising compiler and a RTS. Since StreamIt employs the SDF model, it requires the kernel to have static input and output rates. That is, the number of items consumed on the input stream and produced on the output stream must be constant, from one invocation to the next. Every StreamIt program is a hierarchical composition of three basic stream structures, called

Pipeline, SplitJoin, and FeedbackLoop. Although StreamIt is based on SDF, which requires synchronous communication between kernels, it also has a distinct feature called teleport messaging the ability to send control messages to other kernels as an asynchronous method call. These control messages are used to change the behaviour of the receiving kernel. In addition, as these control messages do not use the same infrastructure as data items, they can be communicated to the kernels independently of the data-flow graph of an application. To support node synchronisation, StreamIt introduces the conception of information flow, in which messages can carry timing information when transferred over streams[128]. This is required as all the kernels in StreamIt execute independently, without any notion of global time. Since StreamIt is based on the SDF model, it only supports a static schedule. This in turn allows the compiler to perform enough analysis and optimisation to produce an efficient implementation.

S-Net is a declarative coordination language that aims to support the transition from sequential code to parallel code, where the concurrency handling is completely managed by S-Net[49]. The kernels are implemented in an independent computational language, e.g. ANSI C. The streams in S-Net are uni-directional. In contrast to StreamIt, the program structure in S-Net can be changed dynamically at runtime and S-Net provides asynchronous communication over streams. Furthermore, there is no restriction on the rate of consumption/production of messages on the streams. Compared to StreamIt, S-Net is more general and closer to KPNs. We will cover S-Net with a more detailed description in Section2.2.