4.1
case-study 1: image processing
In this section, we answer the question what are the characteristics of traditional image processing applications implemented using the dynamic dataflow model of computation in the context of a parallel shared memory cluster with limited memory capacity. We use our time-approximate platform simulator to evaluate different CVE con- figuration options.
In general, computer vision algorithms implement two types of processing, the feature detection and the descriptor generation. The feature detectors examine every pixel in an image in order to deter- mine if it meets the criteria of a feature. A good example is the Fea- tures from Accelerated Segment Test (FAST) feature detector [215] used with the ORB application. FAST determine keypoints of interest by finding rapid changes in direction on image edges. Feature detectors are highly data parallel but this parallelism is often unbalanced; the image pixels that satisfy the feature criteria require more processing than “uninteresting” pixels.
Feature descriptor generation requires computation across inde- pendent image patches. For example, with binary descriptors, such as the BRIEF [216], a patch centered around a detected keypoint needs to be described as a binary string. Binary detectors use a sampling pattern, pick N pairs of points on the pattern and determine whether the first element or the second element of the pair is greater than the other and define the pair as binary 1 or 0 correspondingly. The result- ing N-bit vector is the feature descriptor for the patch to be used for feature matching. This requires computations with higher complex- ity and less data parallelism compared to the feature detection, with irregular memory access patterns to non-contiguous image patches. On the other hand, the processing is usually well balanced across different patches of interest.
Given above general observations, an efficient implementation of a computer vision algorithm requires pipelining unbalanced tasks of the feature detection, pipelining the feature detection with feature descriptors generation, and pipelining different pyramid levels (avoid a barrier between the pyramid levels). Our performance evaluation shows that the dataflow execution model has significant benefits:
• The dynamic dataflow implementation leads to efficient pipelin- ing of application tasks, efficiently hides off-cluster access la- tency, and achieves near optimal load balancing between paral- lel processing elements.
• We were able to perform extensive application exploration with respect to parallelism, its granularity, pipelining, etc., in a very short period of time.
4.1 case-study 1: image processing 112
• The dynamic dataflow implementation is scalable with respect to the number of processing elements, TCDMmemory capacity, and off-cluster memory access latency.
We proceed with detailed performance characterization of two typ- ical image processing applications: the Oriented FAST and Rotated Brief (ORB) [25] already introduced in section2.4, and the Face Detec- tion (FD) [26]. Since our first implementation targets low-resolution applications, we perform the performance characterization on a 640 × 480 VGA image containing about 12.000 FAST keypoints for the
ORB, and a 320 × 240 QVGA image containing 10 human faces for the Face Detection. These two images are representative of the most demanding processing requirements for both applications.
Using the StreamDrive framework, we parallelized the reference sequential implementation, optimized the processing pipeline, and performed extensive performance exploration of the two applications. Each application dataflow graph has been parameterized with respect to the target platform configuration. Thus there is a dataflow graph tuned for each platform configuration tuple (P : M):
1. P is the number of available processing elements, which affects how many dataflow actors are instantiated.
2. M is the capacity of clusterTCDM memory, which determines the buffering size of dataflow channels.
Using the StreamDrive successive refinement development flow, in less than 6 weeks, we have parallelized both applications and ex- plored different parallelization strategies and dataflow graph param- eters for tuples with P from 1 to 16 processing elements and M in 64KB, 128KB, 256KB, and 512KB.
4.1.1 ORB
The ORBalgorithm tries to identify a set of objects inside an image and computes their BRIEF descriptors. Figure 4.1 shows the func- tional block diagram of the referenceORBalgorithm.
The processing is applied to an image pyramid of 8 scaled down images, generated by Scale. At each pyramid level, the objects are identified by first detecting the keypoints of interest via the FAST algo- rithm [217]. Irrelevant FAST keypoints are dropped in Nonmax, and the remaining keypoints are sorted by Cull function based on Harris scores [218]. For each “good” keypoint, the algorithm computes ori- entation of the object around the keypoint, Angle, and the object’s BRIEF descriptor. The Angle computation inspects a N × N patch around the keypoint from the scaled image. The BRIEF computation requires a M × M patch from the Gaussian blurred image produced by the Gauss function.
4.1 case-study 1: image processing 113
Figure 4.1: Functional blocks from reference ORB algorithm.
As explained earlier, the image pyramid scaling is done outside of the CVE cluster by a specialized scaler hardware block. Therefore, the input to ourORBimplementation is a sequence of scaled images. Along with several obvious parallelization choices, eg. FAST , Harris, Angle, and BRIEF computations are naturally data-parallel,ORBalso puts to evidence several parallelization difficulties:
• ORB computation is largely spatially unbalanced - some parts of the image may not have any keypoints, while others contain many; this is challenging for the runtime load balancing strat- egy.
• The Nonmax and the Cull computations are sequential and se- rialize the processing; they account for non-parallelizable part from the Amdahl’s law standpoint and limit performance scal- ing.
• The Cull that performs sorting of keypoints creates a synchro- nization barrier - Angle and Brief computations cannot start until all keypoints in an input image have been detected and sorted; efficient pipelining of consecutive image pyramid im- ages is necessary in order to fill processing elements with work while sorting is performed.
• The access pattern to image patches from the Angle and the Brieffunctions is irregular and cannot be predicted in advance; this requires efficient dynamic pipelining in order to hide mem- ory transfers latency with computations.
• Smaller scaled-down images require much less work compared to larger images at lower scaling factors; this requires low- overhead runtime implementation and scheduling in order to efficiently handle actors with small workloads.
Figure 4.2 shows theORBStreamDrive dataflow graph. The FAST , Harris, Angle, and Brief actors are data-parallel such that several
4.1 case-study 1: image processing 114 SRC Input image via DMA To Angle Angle B BRIEF Descriptors From Cull Harris B C FAST « B FAST Harris non max Cull « Angle Input patch via DMA « BRIEF Blurr patch via DMA Gauss Blurr image via DMA DST
Figure 4.2: ORB StreamDrive dataflow graph.
instances of each can be created depending on number of available processing elements. The Gaussian blur is performed using theHWC
convolution block. The DMA is used to perform several memory transfers: (1) reading input scaled images from off-cluster memory, the DMA is managed by a special SRC actor; (2) writing the Gaussian blur image to off-cluster memory, the DMA is managed by a special DST actor; (3) reading input scaled image patches used by the Angle actor, the DMA is managed by the actor itself; and (4) reading blur image patches used by the Brief actor, the DMA is also managed by the actor itself.
Table 4.1 reproduces the Table2.3 from chapter2 showingORBac- tors token granularities. Thus, the FAST actor accepts a window of 7 input image lines and generates a number of keypoints correspond- ing to that window. All remaining actors’s inputs are keypoints gen- erated by previous actors. In addition to keypoints, the Harris, the Angleand Brief actors require small image patches of 9 × 9, 31 × 31 and 41 × 41 pixels, respectively. In our implementation, the Harris is processing keypoint line-by-line in raster scan order, and therefore accepts a window of 9 lines of input image in addition to the FAST keypoints. The Angle and Brief actors process keypoints in order sorted by the Cull, they take small input image patches as additional input.
The StreamDrive ORB implementation relies on the HWC convolu- tion block for computing the Gaussian blurred image. The Gaussian function implements a 7 × 7 filter over a 640 × 480 image. At 16 MAC/cycle that the HWC is able to perform, the Gauss processing can be done in a single HWC block, in parallel to the rest of theORB
4.1 case-study 1: image processing 115
# Actor Port Token size
FAST IN One image line
OUT One keypoint NONMAX IN One keypoint OUT One keypoint HARRIS IN One keypoint REF One image line OUT One keypoint
CULL IN One keypoint
OUT One keypoint
ANGLE IN One keypoint
REF One image patch OUT One keypoint GAUSS IN One image line
OUT One image line
BRIEF IN One keypoint
BLUR One image patch OUT One descriptor
Table 4.1: Granularity of actors in ORB dataflow graph.
Altogether, our ORBimplementation scales up to 54 actors, includ- ing up to 16 FAST , 16 Angle, and 16 Brief actors, as well as 2 Harris actors.
4.1.2 Face Detection
The Face Detection application detects faces in an input image.
scale integral cascade detection Scaled images Input image face-list face- list faces Integral data
Figure 4.3: Functional blocks from reference Face Detection algorithm.
As the Figure 4.3 shows, reference FD algorithm describes a pipeline of four main functions: the Scale that creates the image pyra- mid with 16 scaling levels; the Intg generating an integral image and a square integral image for each scaled input; the Cascade function that scans the integral image patch-by-patch (a small rectangle), analyzing these patches through a cascade database of features; when a candi- date face is found, the patch is sent to the List function, which checks and sorts all candidate faces in order to produce a list of found faces. The Cascade performs the pattern-matching and is by far the most time-consuming of all functions. The main difficulty in efficiently im- plementing theFDis that the Cascade processing is very unbalanced:
4.1 case-study 1: image processing 116 INTG « B C INTG SRC Input image via DMA To CASCADE CASC B CSV « « CASC CSV LIST Results via DMA From INTG
Figure 4.4: Face Detection StreamDrive dataflow graph.
age patch to other. Thus, while one of the actors blocks the process- ing pipeline with a heavy workload patch, other actors risk to remain waiting for the processing pipeline to unblock. We have implemented a special Cascade Slow Vehicle (CSV) actor, which handles the heavy workload patches while more numerous low-workload patches con- tinue to be processed by the regular Cascade actor. The CSV actors allow the out-of-order patch processing enabling the regular patches that follow a heavy workload patch to be completed before the heavy workload patch completes.
Figure 4.4 shows the Face Detection StreamDrive graph. Similar to ORB, the image pyramid is built in a pre-processing step outside of the FDprocessing. The three main actors, the Intg, the Cascade, and the CSV are implemented as data-parallel actors. There are no application-specific hardware elements used by the FD implementa- tion, this would require very narrowly specialized hardware. There- fore, all processing is done is software. The DMA is used to read the input scaled images from off-cluster memory, and to write the resulting face list to the off-cluster memory at the end of processing. The DMA transfers for the input images are managed by special SRC actor, while writing back of results is handled by the List actor itself.
Table 4.2showsFDactors token granularities:
The FD actors token granularity is straightforward. The Intg ac- cepts the input image one line at a time and generates integral image patches. These patches are then processed by the Cascade and by the Csv actors that generate face descriptors for those patches that contain faces. The List takes the face descriptors, matches them with the face database and outputs the matching faces as result.
The FD implementation is a full software implementation (except for the image pyramid scaling) with no application-specific hardware
4.1 case-study 1: image processing 117
# Actor Port Token size INTG IN One image line
OUT One integral image patch CASCADE IN One integral image patch
OUT One face descriptor CSV IN One integral image patch
OUT One face descriptor LIST IN One face descriptor
OUT One face
Table 4.2: Granularity of actors in FD dataflow graph.
elements used. Our Face Detection implementation scales up to 42 actors, including up to 8 Intg, and up to 16 Cascade and CSV actor instances.
4.1.3 StreamDrive Parallelization Overhead
The parallelization overhead is a penalty paid for parallelizing an application. The StreamDrive parallelization overhead results from (1) the communication overhead including the reserve, push, pop, and release functions; (2) the DMA management for moving the data be- tween the off-chip memory and the clusterTCDM; and (3) the runtime scheduler overhead including the broadcast and collect synchroniza- tion. The communication and the DMA management overhead is scalable, i.e. from Amdahl’s law perspective it contributes to the paral- lelizable part of the application. The runtime scheduler overhead, on the other hand, grows with the number of actors and communication channels. It is important that the scheduler has as low-overhead as possible because from Amdahl’s law standpoint, it contributes to the non-parallelizable part of the application. In order to characterize the StreamDrive parallelization overhead, we measure the performance of theORBandFDdataflow implementations configured for 1PEand 512KB of TCDMwith off-cluster latency of 1 processor cycle. In this configuration the off-cluster data transfers are completely hidden and do not affect measured application cycle count.
Figure 4.5shows the breakdown of ORBactors execution time into times spent in computation, in the communicationAPI, and perform- ing the DMA management tasks. The overheads from the FAST , the Angle and the Brief actors are small compared to the computation part and, most importantly, these actors are parallelizable including this overhead. The Harris actor performs relatively little computation per input and suffers from higher communication overhead, 24.2%. However, this overhead is also parallelizable. The Nonmax and Cull actors also have heavy communication overheads of 35.0% and 18.7% respectively. These actors, however, cannot be parallelized and there-
4.1 case-study 1: image processing 118 0 0,2 0,4 0,6 0,8 1 1,2
Computation Runtime Data Transfer
Figure 4.5:StreamDrive parallelization overhead: ratio of time spent in com-
putation vs. data transfer vs communication API, for the ORB application. 1 0 0,2 0,4 0,6 0,8 1 1,2
INTG CASCADE CSV LIST
Computation Runtime Data Transfer
Figure 4.6:StreamDrive parallelization overhead: ratio of time spent in com-
putation vs. data transfer vs communication API, for the FD application.
fore contribute to performance scaling degradation (explained later). Altogether, the three actors, the Nonmax, the Harris and the Cull, perform each relatively little computation per input. One possibility that we explored is to merge these three actors into a single bigger actor. However, this only works well when the total number of actors is low because the Nonmax and the Cull cannot be parallelized and the resulting merged actor is difficult to load balance with the rest of the application. The better performance is achieved by not combining these actors in favor of a better load balancing. We have observed that even though the relative overhead penalty seems high, the combined total overhead of the three actors in terms of application processing time is less than 0.5% (not counting the Gaussian filter). The Angle and the Brief actors overhead count includes both, the communica- tion and the DMA management overhead, because they manage the DMA for transferring reference image patches around each keypoint from off-cluster memory to the TCDM. Their data transfer manage- ment overhead is 6.2% and 6.7%, respectively. This overhead corre-
4.1 case-study 1: image processing 119 0,00 0,20 0,40 0,60 0,80 1,00 1,20 1 2 4 6 8 10 12 14 16
Actors Scheduler Idle
(a) 512KB 0,00 0,20 0,40 0,60 0,80 1,00 1,20 1 2 4 6 8 10 12 14 16
Actors Scheduler Idle
(b) 256KB 0,00 0,20 0,40 0,60 0,80 1,00 1,20 1 2 4 6 8
Actors Scheduler Idle
(c) 128KB 0,00 0,20 0,40 0,60 0,80 1,00 1,20 1 2 4
Actors Scheduler Idle
(d) 64KB
Figure 4.7:Ratio of time spent, on average, by each PE in actor computation,
runtime scheduler, and the idle, for the ORB application.
sponds to many relatively small DMA transfer requests for reference image patches.
Figure 4.6 shows similar breakdown of FD actors execution time. The StreamDrive communication overhead is only noticeable with the List actor which performs very little work per input face. There is no need to further optimize this since the List only account for 0.000332% of theFDtotal execution time.
It is interesting to consider our results in a context of existing state- of-the-art runtime environments. Compared to theKPNimplementa- tion by Haid, the StreamDrive synchronization is faster: less than 40 processor cycles per blocking access (a reserve or a pop) on average versus 150 reported in [23].
The scheduling overhead is affected by the number of actors and the number of communication channels in the application (including the number and size of the broadcast and the collect connections).
Figures 4.7 and 4.8 show the percentage of time spent on average by each processing element in actor computation, runtime scheduler, and the idle time, for different dataflow graph configurations. The Y-axis show the percentage of time that eachPEspends on average in different parts of the processing; the X-axis plots the number of PEs active in various dataflow graph configurations. There are plots for dataflow graph configurations with 64KB8
, 128KB, 256KB, and 512KB of available TCDM memory. For each TCDM capacity, the number of
8 The smallestFDconfiguration requires 128KB, therefore there is no 64KB plot for the
4.1 case-study 1: image processing 120 0,00 0,20 0,40 0,60 0,80 1,00 1,20 1 2 4 6 8 10 12 14 16
Actors Scheduler Idle
(a) 512KB 0 0,2 0,4 0,6 0,8 1 1,2 1 2 4 6 8 10 12 14 16
Actors Scheduler Idle
(b) 256KB 0 0,2 0,4 0,6 0,8 1 1,2 1 2 4
Actors Scheduler Idle
(c) 128KB
Figure 4.8:Ratio of time spent, on average, by each PE in actor computation,
runtime scheduler, and the idle, for the FD application.
dataflow actors increases with the number ofPEs. For example, a 1PE ORBconfiguration contains 8 actors, while a 16PE configuration con-
tains 54 actors. Similarly, the 1PE FD configuration contains 5 actors, while the 16PEconfiguration has 42 actors. The StreamDrive runtime scheduler is very efficient: the time spent in the scheduler remains