Software Ecosystem

One of our primary goals with Charj is to create a tool that is actually useful in practice for creating programs based on the Charm++ runtime system. In order to accomplish this goal, we must allow the programmer to make use of the preexisting suite of tools that exist to support Charm++ programs, while adding new Charj-specific tools that interact well with the existing codebase. As discussed in chapter 3, Charm++ programs are largely composed of

C++ code, with an accompanying interface (.ci) file that specifies informa- tion about parallel-specific features of the code. The Charm++ software distribution includes a translator, charmxi, which can read an interface file and produce stub code that ties the programmer’s application code to the runtime system. The translator produces two output files: the declarations file (.decl.h), which contains forward declarations for all Charm++-specific functions and variables, and the definitions file (.def.h), which contains their implementations. These generated files are then included in the user’s C++ implementation, along with any needed C++ headers, and from that point on the process of producing a functioning application binary is identical to that of standard C++, with the caveat that the binary must be linked against the Charm++ runtime libraries.

The process of creating a binary from the C++ source code that results from the combination of the user’s own code and the output of the charmxi translator is not specific to Charm++. However, this process can become quite involved, given that the user must specify the include path for the Charm++ system headers, the path to the Charm++ libraries and any li- braries that they depend upon, and provide the appropriate flags to the linker. These flags may vary significantly depending on the particular com- piler and compiler version being used and the location of system libraries on the machine where compilation occurs. To mitigate this problem and simplify the toolchain needed by Charm++ programmers, the standard distribution of Charm++ includes a wrapper script, charmc, which handles many of the details of the translation, compilation, and linking process.

One advantage of using Charm++ as the basis for Charj is the ability to make use of the significant institutional support for Charm++. Default Charm++ installations are commonly provided on supercomputers, and the engineering effort required to make the Charm++ software environment work effectively across a wide variety of hardware and software configurations has already been done. By piggybacking on the existing Charm++ infrastruc- ture, we avoid a substantial effort that is not directly tied to our research goals.

In order to effectively integrate with Charm++, we provide tools to aid the programmer in going from a Charj program (possibly interacting with or partially composed from Charm++ code) to a functional application, with- out giving up access to the features provided by charmc. The core Charj

.ci .C .h decl def Charm++ App

(a) Compilation process for a Charm++ application. .ci .C .h decl def Charm++ App .cj

(b) Compilation process for a Charj ap- plication.

Figure 4.1: In a Charm++ application, the programmer specifies an inter- face (.ci) file that accompanies the C++ code that forms the bulk of their application and specifies type signatures and visibility information about re- motely invocable functions. A corresponding Charj program integrates this information directly into the application, and the Charj compiler generates code targeting the Charm runtime.

compiler is a Java application described in detail in the following sections. It takes Charj source files as input and outputs C++ code and Charm interface definitions suitable for compilation by charmc, as shown in figure 4.1. The Charj compiler accepts a number of optional command-line arguments that control features such as the verbosity of its diagnostic output and the level of warning and error messages produced.

In order to simplify the compilation process for end users, we provide a wrapper script called charjc. This wrapper accepts as arguments the union of legal arguments to the Charj compiler and legal arguments to charmc. It invokes the Charj compiler on the input source files, passing all Charj op- tions through. It then takes the Charm++ interface and C++ code output of the Charj compiler and invokes charmc on them, applying the remaining charmc command-line flags. The output of this process includes both the source code output of the Charj compiler and the binary output obtained from charmc. This output can be linked directly with the Charm libraries and Charj runtime. By making the output of the Charj compiler as close as possible to a normal Charm program, we make it easier to integrate Charj code into existing Charm code and vice-versa, while also giving the program- mer an easy way of inspecting the outcome of the Charj compilation process. Although the Charj compiler and the charjc wrapper script that handles ar- gument passing and invoking charmc on the output of the Charj compiler

are distinct entities, for brevity we use the name of the wrapper script which invokes the Charj compiler, charjc, synonymously with the Charj compiler itself in places where this distinction is not important.