Regression
Monitoring
with Gauger
Introducing Gauger, a lightweight tool for visualizing
performance changes that occur as software evolves.
platform-specific. This can make it necessary to perform performance evaluations on a range of systems.
The Gauger package described in this article provides developers with a sim-ple, free software tool to track system performance over time. Gauger is
lightweight, language-independent and portable. Gauger collects any number of performance values from multiple hosts and visualizes their development over time (Figure 1). In order to use Gauger, developers need to add the necessary instrumentation to their code to obtain a performance measurement and then submit it to Gauger with the gauger function call. The gauger function argu-ments are the description of the value, a category, the value itself and a unit of measurement. Gauger’s Web interface then allows visitors to group metrics by
category or by execution host and adjust the visualized revision range or the size of the plot. Gauger is ready to be used with many programming languages and revision control systems, and it is easily expandable to accommodate new ones.
Gauger’s Web interface can be used to analyze the collected performance data in various ways. It can combine different metrics in a single plot and offers a color-coded guide to help visitors select only unit-wise compatible metrics.
Gauger also allows users to normalize the data in order to mask differences in absolute performance between different execution hosts. If multiple measure-ments were taken for the same revision, Gauger will show the average and standard deviation as long as only a single metric is plotted. For larger projects with many metrics or execution
WWW.LINUXJOURNAL.COM SEPTEMBER 2011 | 69
Figure 1. Gauger in action: this screenshot shows performance measurements obtained and visualized by Gauger for the GNUnet Project over the course of a few revisions.
hosts, Gauger offers a search feature to locate the desired plots. An additional instant search keeps the menus free of irrelevant items.
Finally, should further fine-tuning be needed (for example, for use in presentations), Gauger can be used to retrieve the gnuplot source of any plot. The generated gnuplot source includes the plotted data.
Gauger’s Architecture
Gauger uses a traditional client-server architecture, where the clients report performance measurements to a central server. This architecture allows machines behind NAT or with otherwise restricted Internet access to provide performance measurements to Gauger.
All of the performance-monitoring machines to be used with Gauger should install the Gauger client, and the software to be tested should be
integrated with the appropriate lan-guage bindings. Lanlan-guage bindings are designed to be transparent and (except for a few extra system calls) have no negative effects in case the Gauger client is not installed on the machine. Thus, it is safe to commit the language bindings to a project repository. As long as the (tiny) lan-guage bindings are included, integrat-ing Gauger will not disrupt operations on systems where the Gauger client is not installed.
The Gauger server runs the data collection and visualization part. Data is logged through a RESTful API and saved in human-readable plain-text files. The primary job of the server is to provide a dynamic Web interface to visualize and analyze the collected data. All the communication between Gauger clients and the Gauger server is done in standard HTTP(S) requests FEATURE Performance Regression Monitoring with Gauger
Figure 2. Gauger architecture: the Gauger server is responsible for authentication and receives performance data from the clients. The results are stored in a simple text format in a local directory. PHP is used to generate the Web site.
so that only port 80 (or 443) needs to be open (Figure 2).
Installation
Each Gauger client installation requires a local Python (> or = than 2.6) interpreter. For the Gauger server, a Web server installation with PHP and gnuplot is required.
The provided install.sh script can be used to install the client, install the server code into an appropriate loca-tion and generate an updated Apache configuration. The script prompts for key configuration options, such as the installation path and the desired URL at which the Gauger server should run. Installations that do not use Apache currently require manually configuring the Web server.
Configuration
Each part of Gauger uses a simple configu-ration file. The Gauger client configuconfigu-ration file contains the remote server URL, followed by the user name and password.
Here’s a sample configuration:
https://gnunet.org/gauger/ username password
The Gauger server configuration file
contains the directory where data and authentication information are stored.
Listing 1 shows a sample server con-figuration. When the auto-add feature is enabled, new hosts can be added by logging in to the Web site using a fresh hostname and password.
Integrating Gauger
Adding a single simple call at the places where performance measure-ments are obtained typically is all that’s required to integrate Gauger with existing projects. This call then starts the Gauger client process, which, if installed and configured correctly, submits the performance
WWW.LINUXJOURNAL.COM SEPTEMBER 2011 | 71
Listing 1. A few basic configuration options and a list of clients with their passwords’ hashes are needed for the Gauger server configuration file.
[config]
data = "/var/lib/gauger"
salt = "makemyhashesunique"
auto_add = 1 [hosts]
debian-amd64-grothoff="da39a..."