Properties of Canopy Graphs

Every canopy graph shares three basic properties. All indicate that the throughput of a pipeline depends on its occupancy. All are bounded by a set of line segments that can be defined in terms of the type of interaction within the pipeline that limits the throughput, and all are convex polygons.

!

%

!

!

!

%

1223$)425%

6

7

'#

3

8

7

$

3

"%

Figure 3.28: Canopy graphC with four limiting segments Effects of Occupancy on Throughput

The throughput of a pipeline is affected by the total number of data items and also by the total number of stages without a data item, or “holes”. The effect of occupancy on throughput is described in Section2.2.2, which gave background on the use of canopy graphs. This section aims to generalize and give intuition about the relationship of occupancy and throughput in pipelined systems. For reference, Figure3.28shows a generic canopy graph.

The throughput and occupancy of a pipeline are highly dependent on each other. In par- ticular, the throughput may be limited by the lack of data within the system; this occurs when the environment does not provide data as quickly as the pipeline can use it. Consider the case of only one data item flowing through the system. The frequency with which the pipeline can produce data when it is limited to contain only one data item is one over the total latency of the pipeline. Adding a second data item will double the total number of data items and therefore tend to double the throughput. This linear relationship between the data items and throughput holds within the data limited region. In Figure3.28, this relationship gives the shape of the line segment labeledbf orward.

quickly yet not removing completed data items as fast as the system produces them. Holes, in some ways, are the dual of data within a system. There are some key differences, however, that lead the hole limited region to differ from the data limited region. For example, if a circuit contains a parallel construct, the two branches of the construct will always contain the same number of data items. The number of holes, however, is determined by the number of stages minus the number of data items. Two parallel branches, therefore, can have adifferentnumber of holes, even though they must always have thesamenumber of data items. As a result, the hole-limited region of the canopy graph can be more complex than the data-limited region as different portions of the circuit begin to dominate at different occupancies.

Finally, the worst cycle time in the within the system system can also limit the through- put; this the speed limit set by the pipeline itself that cannot be exceeded regardless of the environment behavior. This occurs when neither data nor holes are lacking, but some stage or set of stages create a cycle that limits throughput. This limit to the throughput is analogous to previous methods that characterize circuit performance with single performance number [29, 36, 71, 65]. When applied to a canopy graph, however, this shows not only a single throughput number but also a range of occupancies that will allow the pipeline to reach that maximum throughput. This occupancy or range of occupancies is known as theideal occu-

pancyof the pipeline.

Bounded by a Set of Line Segments

Every canopy graph is bounded by a set of boundary line segments,B, which can be defined based on the type of interaction within the pipeline that is limiting the throughput. The set of boundary segments forms a function of throughput vs. occupancy, and the canopy graph is the hypograph of this function, excluding points that have negative values. C_{B ={(k, tpt) :}

k _∈Rn

∧k _≥0, tpt _∈R_{∧tpt≥0, tpt≤B(k)} ⊆}Rn_[41]

Figure3.28shows an example canopy graph. The boundaries of this canopy graph consist of the following types of limiting segments:

1. forward segmentsbound the operating region in the data-limited region of the canopy graph and are determined by the total forward latency of the system. Because the throughput increases as a new data item enters, the slope of the forward line segments is always positive. The line segment labeledbf orward in Figure3.28is a typical forward

boundary segment.

2. top segmentsbound the operating region in the cycle limited region of the canopy graph

and are determined by the longest effective cycle time. Because the limiting cycle in the pipeline does not depend on occupancy, the top segment always has a slope of zero. The line segment labeledbtop in Figure3.28is a typical top boundary segment.

3. reverse segments bound the hole-limited region and are determined by the maximum

occupancy and reverse delay of the system. Because adding an additional hole (i.e. re- ducing the occupancy) will increase the throughput when the system is hole limited, the slopes of the reverse segments are always negative. The line segments labeledbreverse,0

andbreverse,1 in Figure 3.28represent the kind of line segments that can define the op-

erating region in the hole-limited region.

In a system that is initialized as empty, the forward segment of a canopy graph will cross through the point at the origin. As a result, such systems will always have exactly one forward segment and one or more reverse segments. In some cases, the top segment may be absent; this is a degenerate case in which the top segment is a single point.

Convex Set of Points

This section states without proof that all canopy graphs for systems that are composed of the hierarchical components that Section3.3 describes will be convex. The conventional defini- tion of convexis used here: for all points x and y that are in the set C_{, all points on a line}

segment connectingxandyare also withinC_{. [41]}

canopy graph composition maintains this property. Note that this thesis does not claim to prove that all pipeline operating regions can be defined by a canopy graph. In particular, no general proof of operating region convexity is provided.

Intuitively, since the throughput first increases with increased occupancy, then levels off with the maximum throughput, and then decreases with a decreased number of holes, this will always define a canopy-like shape that is monotonically increasing and then monotonically decreasing. Since both throughput and occupancy are also limited to be positive numbers, this bounds the entire canopy graph to a convex shape.