This section will discuss challenges implementing Huffman decoding on GPUs. Only a simple Huffman decoder on the GPU has been implemented. It is able to decode Huffman encoded data from libhuffman. Given the sequen- tial nature of Huffman coding the performance is limited to that of a single thread. A parallel approach has been investigated in [12], more details about this in Section 5.3.
The implementation of the Huffman decoder consists of two different log- ical parts, one builds the Huffman tree another decodes a Huffman encoded stream using the tree.
Table 4.1 gives the file format of data Huffman encoded with libhuffman. The number of codes, that is, leaf nodes in a Huffman tree, is given by the first four bytes in the encoded Huffman stream. The following four bytes gives the decoded size, which is the same at the original size. Then after these two values follows the actual definitions of the codes. The format of a code entry is: One byte gives the encoded symbol, followed by a byte giving the bit length. After these two bytes follows the necessary number of bytes to hold the bit string, the last byte might have additional unused bits. After all the code entries the actual encoded data is found.
The tree generation uses an array of a struct to represent nodes. This way pointers are not needed and a node can be referenced through an index
4.3. HUFFMAN DECODING IMPLEMENTATION 45
Offset Length (in bytes) Description
0 4 Number of codes (in big-endian)
4 4 Original size
8 variable Code 1
..
. ... ...
varies variable Code N
varies variable Encoded data
Table 4.1: File format of Huffman encoded data from libhuffman.
into the array. This technique is described in the book Introduction to Al- gorithms, [13]. The reason this is done is because one early implementation using pointers resulted in incorrect use of shared memory, where global mem- ory was accessed instead of the shared memory. This was found by analyzing the parallel thread execution (PTX) assembly code generated by the CUDA compiler. Where access to memory was done through the the instruction st.global for storing and ld.global for loading, that is, from global mem- ory instead of shared memory, where the respective instructions should have been used: st.shared and ld.shared, see [14] for details. It might have been an erroneous implementation, but it is safer to use array indices when pointers are not needed. The NVIDIA CUDA programming guide also men- tions the restrictions when using pointers with respect to addressing global and shared memory spaces:
“Pointers in code that is executed on the device are supported as long as the compiler is able to resolve whether they point to either the shared memory space or the global memory space, otherwise they are restricted to only point to memory allocated or declared in the global memory space”, [11].
The node structure consists of three members all of the same type, un- signed int, as observed in Table 4.2. The members zero and one is given the index of its child. As the tree is never traversed from a node up to the root, there is no need to keep track of a node’s parent. Although the symbol can only have 256 different values, an unsigned int (32-bits) is used to represent the symbol, because a compaction of the data structure is not possible. The reason for this is because of a restriction in the instruction set architecture (ISA) of the PTX virtual machine used to generate device code for the GPU. The the document describing the PTX ISA mentions its alignment requirement, stating that all PTX instructions that access mem- ory requires the address to be aligned to a multiple of the transfer size. It is
Type Member Name Description
unsigned int zero Index of the child at the edge marked 0 unsigned int one Index of the child at the edge marked 1 unsigned int symbol Value of the symbol
Table 4.2: Node structure
therefore little to gain by having the last member as an 8-bit type, even if it could result in less used storage, the cost would be lower performance. This is because data has to be transferred with lower transfer size, which result in lower bandwidth. Furthermore, to be able to transfer a 32-bit value it has to be aligned at an address multiple of 4. That is why the structure is organized as it is.
The nodes of the tree is allocated consecutively, starting with index 0 for the root of the tree. Then nodes are added to the tree referencing other nodes through indices. Index 0 is reserved as a special marker for child nodes, indicating that there is no child. This value was chosen since no node can have the root as a child. A node with value zero in both member zero and one is a leaf node.
Decoding a Huffman encoded stream on the GPU requires that the Huff- man tree is built before the decoding can start. Then the encoded stream is parsed starting at its very beginning. The decoding is as described in Section 2.1.3. For each decoded symbol one starts at the root, the node at index 0, then follows the indices found in members zero and one depending on the current bit value found in the bit string. If a bit is zero then the index found in member zero is followed, and the same applies for bits with value one, except then the index found in member one is followed. When both the values found in member zero and member one are 0 then the node is a leaf node. At the point of reaching a leaf node the value in the member symbol is emitted (a byte) to the output stream.