class jolt.tasks.Test(*args, **kwargs) A test task.
The test task combines a regular Jolt task with a Python unittest.TestCase. As such, a test task is a collection of similar test-cases where each test-case is implemented as an instancemethod named with a test_
prefix. When executed, the task runs all test-case methods and summarizes the result.
All regular unittest assertions and decorators can be used in the test methods. For details about inherited task attributes, seejolt.tasks.Taskand Python unittest.TestCase.
Example:
class OperatorTest(Test):
def test_add(self):
self.assertEqual(1+1, 2)
def test_sub(self):
self.assertEqual(2-1, 1)
cleanup()
Implement this method to clean up after a test setup(deps, tools)
Implement this method to make preparations before a test
7.13 Tools
class jolt.Tools(task=None, cwd=None, env=None) A collection of useful tools.
Any {keyword} arguments, or macros, found in strings passed to tool functions are automatically expanded to the value of the associated task’s parameters and properties. Relative paths are made absolute by prepending the current working directory.
append_file(pathname, content) Appends data at the end of a file.
Parameters
• pathname (str) – Path to file. The file must exist.
• content (str) – Data to be appended at the end of the file.
archive(pathname, filename) Creates a (compressed) archive.
The type of archive to create is determined by the filename extension. Supported formats are:
• tar
• tar.bz2
• tar.gz
• tar.xz
• zip
Parameters
• pathname (str) – Directory path of files to be archived.
• filename (str) – Name/path of created archive.
autotools(deps=None)
Creates an AutoTools invokation helper
builddir(name=None, incremental=False, unique=True) Creates a temporary build directory.
The build directory will persist for the duration of a task’s execution. It is automatically removed after-wards.
Parameters
• name (str) – Name prefix for the directory. A unique autogenerated suffix will also be appended to the final name.
• incremental (boolean) – If false, the created directory is deleted upon completion of the task.
Returns Path to the created directory.
buildroot
Return the root path of all build directories
checksum_file(filelist, concat=False, hashfn=<built-in function openssl_sha1>, filterfn=None) Calculate a checksum of one or multiple files.
Parameters
• filelist (str,list) – One or multiple files.
• concat (boolean) – Concatenate files and return a single digest. If False, a list with one digest for each file is returned. Default: False.
• hashfn – The hash algorithm used. Any type which provides an update() and hexdigest() method is accepted. Default: hashlib.sha1
• filterfn – An optional data filter function. It is called repeatedly with each block of data read from files as its only argument. It should return the data to be included in the checksum. Default: None
Returns A list of checksum digests, or a single digest if files where concatenated.
chmod(pathname, mode)
Changes permissions of files and directories.
Parameters
• pathname (str) – Path to a file or directory to change permissions for.
• mode (int) – Requested permission bits.
cmake(deps=None)
Creates a CMake invokation helper compress(src, dst)
Compress a file.
Supported formats are:
• .bz2
• .gz
• .xz
Parameters
• src (str) – Source file to be compressed.
• dst (str) – Destination path for compressed file. The filename extension determines the compression algorithm used.
copy(src, dst, symlinks=False)
Copies file and directories (recursively).
The directory tree structure is retained when copying directories.
Parameters
• src (str) – Path to a file or directory to be copied.
• dest (str) – Destination path. If the string ends with a path separator a new direc-tory will be created and source files/directories will be copied into the new direcdirec-tory. A destination without trailing path separator can be used to rename single files, one at a time.
• symlinks (boolean, optional) – If True, symlinks are copied. The default is False, i.e. the symlink target is copied.
cpu_count()
The number of CPUs on the host.
Returns The number of CPUs on the host.
Return type int cwd(pathname, *args)
Change the current working directory to the specified path.
This function doesn’t change the working directory of the Jolt process. It only changes the working directory for tools within the tools object.
Parameters pathname (str) – Path to change to.
Example
with tools.cwd("subdir") as cwd:
print(cwd)
download(url, pathname, exceptions=False, **kwargs) Downloads a file using HTTP.
Parameters
• url (str) – URL to the file to be downloaded.
• pathname (str) – Name/path of destination file.
• exceptions (boolean) – Raise exception if connection fails. Default: false.
• kwargs (optional) – Addidional keyword arguments passed on directly requests.
get().
environ(**kwargs)
Set/get environment variables.
Only child processes spawned by the same tools object will be affected by the changed environment.
The changed environment is only valid within a context and it is restored immediately upon leaving the context.
Parameters kwargs (optinal) – A list of keyword arguments assigning values to environ-ment variable.
Example
with tools.environ(CC="clang"):
tools.run("make all")
expand(string, *args, **kwargs)
Expands keyword arguments/macros in a format string.
This function is identical to str.format() but it automatically collects keyword arguments from a task’s parameters and properties.
Parameters
• string (str) – The string to be expanded.
• args (str, optional) – Additional positional values required by the format string.
• kwargs (str, optional) – Additional keyword values required by the format string.
Returns Expanded string.
Return type str
Example
target = Parameter(default="all") verbose = "yes"
def run(self, deps, tools):
print(tools.expand("make {target} VERBOSE={verbose}")) # "make all
˓→VERBOSE=yes"
expand_path(pathname, *args, **kwargs)
Expands keyword arguments/macros in a pathname format string.
This function is identical to str.format() but it automatically collects keyword arguments from a task’s parameters and properties.
The function also makes relative paths absolute by prepending the current working directory.
Parameters
• pathname (str) – The pathname to be expanded.
• args (str, optional) – Additional positional values required by the format path-name.
• kwargs (str, optional) – Additional keyword values required by the format path-name.
Return str: Expanded string.
expand_relpath(pathname, relpath=None, *args, **kwargs) Expands keyword arguments/macros in a pathname format string.
This function is identical to str.format() but it automatically collects keyword arguments from a task’s parameters and properties.
The function also makes absolute paths relative to a specified directory.
Parameters
• pathname (str) – The pathname to be expanded.
• relpath (str, optional) – Directory to which the returned path will be relative. If not provided, the joltdir attribute is used.
• args (str, optional) – Additional positional values required by the format path-name.
• kwargs (str, optional) – Additional keyword values required by the format path-name.
Return str: Expanded string.
extract(filename, pathname, files=None) Extracts files in an archive.
Supported formats are:
• tar
• tar.bz2
• tar.gz
• tar.xz
• zip
Parameters
• filename (str) – Name/path of archive file to be extracted.
• pathname (str) – Destination path for extracted files.
• files (list, optional) – List of files the be extracted from the archive. If not provided, all files are extracted.
file_size(pathname) Determines the size of a file.
Parameters pathname (str) – Name/path of file for which the size is requested.
Returns The size of the file in bytes.
Return type int getcwd()
Returns the current working directory.
getenv(key, default=”)
Returns the value of an environment variable.
Only child processes spawned by the same tools object can see the environment variables and their values returned by this method. Don’t assume the same variables are set in the Jolt process’ environment.
glob(pathname, expand=False) Enumerates files and directories.
Parameters
• pathname (str) – A pathname pattern used to match files to be included in the returned list of files and directories. The pattern may contain simple shell-style wildcards such as
‘*’ and ‘?’. Note: files starting with a dot are not matched by these wildcards.
• expand (boolean) – Expand matches to absolute paths. Default: false.
Returns A list of file and directory pathnames. The pathnames are relative to the current working directory unless the pathname argument was absolute.
Example
textfiles = tools.glob("*.txt")
map_concurrent(callable, iterable, max_workers=None) Concurrent ~map().
Parameters
• callable – A callable object to be executed for each item in the collection.
• iterable – An iterable collection of items.
• max_workers (optional) – The maximum number of worker threads allowed to be spawned when performing the work. The default is the value returned byjolt.Tools.
cpu_count().
Returns List of return values.
Return type list
Example
def compile(self, srcfile):
objfile = srcfile + ".o"
tools.run("gcc -c {0} -o {1}", srcfile, objfile) return objfile
srcfiles = ["test.c", "main.c"]
objfiles = tools.map_concurrent(compile, srcfiles)
map_consecutive(callable, iterable) Same as map().
meson(deps=None)
Creates a Meson invokation helper read_file(pathname, binary=False)
Reads a file.
replace_in_file(pathname, search, replace) Replaces all occurrences of a substring in a file.
Parameters
• pathname (str) – Name/path of file to modify.
• search (str) – Substring to be replaced.
• replace (str) – Replacement substring.
Example
version = Parameter(default="1.0")
def run(self, deps, tools):
tools.replace_in_file("Makefile", "VERSION := 0.9", "VERSION := {version}
˓→")
rmtree(pathname, *args, **kwargs) Removes a directory tree from disk.
Parameters
• pathname (str) – Path to the file or directory to be removed.
• ignore_errors (boolean, optional) – Ignore files that can’t be deleted. The default is False.
rsync(srcpath, dstpath, *args, **kwargs)
Synchronizes files from one directory to another.
The function performs a smart copy of files from the srcpath directory to the dstpath directory in such a way that dstpath will mirror srcpath.
If dstpath is empty, the files are copied normally.
If dstpath already contains a sub or superset of the files in srcpath, files are either copied or deleted depending on their presence in the source directory. Common files are only copied if the file content differs, thereby retaining metadata (such as timestamps) of identical files already present in dstpath.
Parameters
• srcpath (str) – Path to source directory. The directory must exist.
• dstpath (str) – Path to destination directory.
run(cmd, *args, **kwargs)
Runs a command in a shell interpreter.
Parameters
• cmd (str) – Command format string.
• args – Positional arguments for the command format string.
• kwargs – Keyword arguments for the command format string.
• output (boolean, optional) – By default, the executed command’s output will be written to the console. Set to False to disable all output.
• output_on_error (boolean, optional) – If True, no output is written to the console unless the command fails. The default is False.
• output_rstrip (boolean, optional) – By default, output written to stdout is stripped from whitespace at the end of the string. This can be disabled by setting this argument to False.
• shell (boolean, optional) – Use a shell to run the command. Default: True.
Example
target = Parameter(default="all") verbose = "yes"
def run(self, deps, tools):
tools.run("make {target} VERBOSE={verbose} JOBS={0}", tools.cpu_count())
sandbox(artifact, incremental=False, reflect=False)
Creates a temporary build directory populated with the contents of an artifact.
Files are copied using rsync.
Parameters
• artifact (cache.Artifact) – A task artifact to be copied into the sandbox.
• incremental (boolean) – If false, the created directory is deleted upon completion of the task.
• reflect (boolean) – If true, a virtual sandbox is constructed from artifact metadata only. Files are not copied, but instead symlinks are created pointing at the origin of each file contained within the artifact. The sandbox reflects the artifact with a live view of the the current workspace.
Returns Path to the build directory..
Return type str
Example
def run(self, deps, tools):
sandbox = tools.sandbox(deps["boost"], incremental=True)
setenv(key, value=None)
Sets or unset an environment variable.
Only child processes spawned by the same tools object can see the set environment variable and its value.
Don’t assume the same variable is set in the Jolt process’ environment.
Parameters
• key (str) – Name of variable to set.
• value (str) – Value of the variable or None to unset it.
symlink(src, dst, replace=True, relative=True) Creates a symbolic link.
Parameters
• src (str) – Path to target file or directory.
• dst (str) – Name/path of symbolic link.
• replace (boolean) – Replace existing file or link. Default: false.
• relative (boolean) – Create link using relative path to target. Default: false (absolute path).
thread_count()
Number of threads to use for a task.
Returns number of threads to use.
Return type int tmpdir(name)
Creates a temporary directory.
The directory is only valid within a context and it is removed immediately upon leaving the context.
Parameters name (str) – Name prefix for the directory. A unique autogenerated suffix will also be appended to the final name.
Example
with tools.tmpdir("temp") as tmp, tools.cwd(tmp.path):
tools.write_file("tempfile", "tempdata")
unlink(pathname, *args, **kwargs) Removes a file from disk.
To remove directories, usermtree().
Parameters pathname (str) – Path to the file to be removed.
upload(pathname, url, exceptions=False, auth=None, **kwargs) Uploads a file using HTTP (PUT).
Parameters
• pathname (str) – Name/path of file to be uploaded.
• url (str) – Destination URL.
• exceptions (boolean) – Raise exception if connection fails. Default: false.
• auth (requests.auth.AuthBase, optional) – Authentication helper. See re-quests.auth for details.
• kwargs (optional) – Addidional keyword arguments passed on directly to ~re-quests.put().
which(executable)
Find executable in PATH.
Parameters executable (str) – Name of executable to be found.
Returns Full path to the executable.
Return type str
write_file(pathname, content=None, expand=True) Creates a file.
Note: Existing files are overwritten.
Parameters
• pathname (str) – Name/path of file to be created.
• content (str, optional) – Data to be written to the file.
• expand (boolean, optional) – Expand macros in file content. Default: True.