The Copying Collector

Short lived objects reside in the young generation, thus, many dead object are likely to fill the young generation. When an allocation request fails due to the lack of free memory, the memory manager triggers a minor collection operation to collect the Eden space and evacuates surviving objects to the survivor space or to the old generation. The space left in

3.6. JAVA VIRTUAL MACHINE AND GARBAGE COLLECTION

Figure 3.11: A diagram of the GCTask and thread-local data structure.

the Eden space would be a large contiguous memory for future allocations.

Data Structures

The copying collector uses various data structures to manage the collection operations. First, GCTaskQueue is a shared deque, which uses the Arora queue design [Arora et al., 2001] Arora queue implements single producer/multiple consumers paradigm. From one end, only one thread can enqueue, whereas, multiple threads can dequeue from the other end using atomic operations. This queue implements the Arora design [Arora et al., 2001].

At one end, a thread called the Virtual Machine (VM) thread populates the queue with GCTasksduring the sequential phase. GCTasks are tasks with different functionalities and the three main types are: root scanning, work stealing and finalizing tasks; these tasks are explained later.

When the parallel phase begins execution, the collector threads may push GCTasks of other types to manage the parallelism. Therefore, this queue’s end must be wrapped with a lock to synchronize thread access. The other queue’s end is lock-free and the collector threads use atomic operations to pop GCTasks.

Second, every collector thread creates a private deque to iterate over root closures. The Arora queue enables the queue to push and pop from one end. The other end is left for starving threads that run out of work to steal references and balance the load across participating threads. To avoid space overflow, the Hotspot JVM creates a stack accessed by the owner thread only to push references when the Arora queue is full.

3.6. JAVA VIRTUAL MACHINE AND GARBAGE COLLECTION

Third, the VM thread pushes different types of GCTasks into the GCTaskQueue. There are three main GC task types that manage the parallel collector thread execution.

Root Scanning Task: A root scanning task directs a garbage collector thread to a memory area, where potential root references can be found. For example, root ref- erences may reside in TLABs, Java Native Interface (JNI) handles, or card tables; therefore, the VM thread creates many root scanning tasks and pushes them into the GCTaskQueue. The card table refers to a table that records inter-generational refer- ences. References from the old generation to the young generation is recoded in the card table.

Work Stealing Task: The collector threads process references by the means of clo- sures in which a root reference and its sub-graph are copied to other heap spaces. However, the depth of root closures is not equal; deep closures require additional time for processing. Consequently, the VM thread creates work stealing tasks to maintain the load balance across the collector threads. The number of work stealing tasks is equal to the number of the collector threads.

Finalizing Task: At the top of the queue, a unique task is used by the termination protocol to synchronize the collector threads and hand over the control to the VM thread.

Algorithms

The garbage collection begins with a sequential code phase executed by the VM thread for preparing, bookkeeping and managing the collection operations. The collector threads park at the GCTaskQueue monitor, waiting for GCTasks to process. As described earlier, the GCTaskQueue is prepared in this phase and the VM thread hands in the control to the parallel phase and sleeps.

The parallel phase starts by waking up the collector threads, notifying them that the GC- TaskQueue has GCTasks ready for processing. The collector threads compete on the queue lock and the successful thread pops a GCTask, which should be a root scanning task. Other threads spin on the lock, trying to lock the queue and pop a GCTask. Once root scanning tasks have been processed, every collector thread pops a work stealing task and attempts to get work from busy workers. The remainder of this section focusses on two parallel tech- niques: work processing and work stealing.

Work processing begins by popping a root scanning task from the GCTaskQueue. This task directs the collector thread to a memory area and scans resident objects to find ones that have a reference to the heap. These objects are considered as roots and are copied to the thread’s

3.6. JAVA VIRTUAL MACHINE AND GARBAGE COLLECTION

local Arora queue. When the collector thread completes scanning the memory area, the local queue begins to be ready for processing the root references and their attached sub-graphs. The collector thread pops a reference, copies it to the target heap space, and scans it if it has a reference. References that are discovered while processing the root closure, are pushed into the thread’s local Arora queue. The traversal order is depth-first, thus children and parents are co-located together. If a collector thread comes across a referenced object that already has been copied, it skips that object and pops another reference. Once all references in the local queue have been processed, the collector thread pops another root scanning task, if any, or a work stealing task.

A work stealing task enables the collector thread that has finished root scanning tasks to peek on other threads’ queues. The existing stealing algorithm selects two queues randomly and compares their sizes. The larger queue size is chosen and the stealing thread atomically pops a reference from the end. The stolen reference is processed in the same way described for root scanning task. In the case when a stealing thread fails to obtain a reference, it repeats the process many times until it reaches a threshold. Hotspot sets the threshold equal to double the number of participating threads.

While the collector threads work on root scanning and work stealing tasks, one thread pops the finalizing task and gets promoted as a leader to terminate the parallel phase. Every collector thread increases a global counter atomically to indicate that reference processing work has been completed. The finalizer thread checks the counter and if all threads have finished their work, it wakes up the VM thread and hands over control to it. All garbage collector threads return and park at the GCTaskQueue monitor. The sequential code executes the remaining collection operations and resumes application execution.