Interconnection Networks
• The note is mainly based on Chapters 1, 2, and 4 of Interconnection Networks, An Engi-neering Approach by J. Duato, S. Yalamanchili, and L. Ni and Section 2.3.2.1 of Multi-wavelength Optical Networks, A Layered Approach, by T.E. Stern and K. Bala.
• Interconnection for digital systems
Interconnection networks provide the interconnection for digital systems. Examples of interconnection networks include the internal buses in VLSI circuits, telephone switches and networks, networks for parallel/distributed computing systems (including vector su-percomputers, multicomputers, multiprocessors, cluster/network of workstations), LAN, MAN, WAN, and networks for industrial applications and electronic devices.
• Parallel computing and networks
Parallel computing systems have been developed to meet the increasing demands on com-puting powers. A bottleneck in parallel computing systems is the the communication between processors. Therefore, the performance of interconnection networks is a critical issue in parallel computing. This has been a major driving force for the research of inter-connection networks. The study of interinter-connection networks in parallel computing system includes the performance and cost issues.
• Parallel computer architecture
Distributed memory multiprocessors (multicomputers)
A multicomputer consists of a set of processors, each processor has its own memory, in-terconnected by a network. Communications between processors are realized by message passing on the interconnection network. It is easy to make a multicomputer with a large number of processors (and thus a large computing power in theory) but it is difficult to make programs on multicomputers because data and tasks need to be distributed to processors in an efficient way.
Shared memory mutliprocessors
In this model, all processors in a system share a common memory space. The commu-nication between processors are realized by read/write the shared memory cells through interconnection networks. This simplifies the data exchange between processors. When the system is small, the access time to the memory of every processor can be considered
uniform. However, this is not true if the system is large. Distributed shared-memory multiprocessors
This model combined the previous two models, each processor has a local cache memory, and all processors share a common main memory.
Multicomputers, shared memory multiprocessors, and distributed shared-memory multi-processors are known asfine grainedparallel computing systems, because the computation and communication between processors can be performed in a highly synchronized way.
Network of workstations
This model refers to a set of workstations/PCs connected by a network such as a LAN. The model can be further classified into two categories: NOW (network of workstations) and COW (cluster of workstations). NOW is a system dedicated to the parallel computing. Usually, the performance issue of the network in NOW is considered when the system is built. COW refers to a set of workstations/PCs connected by a network that the system may work for other purposes but its extra computing power is used for parallel computing. This model is known as coarse grained parallel computing systems. The computation on this model can be bulk synchronized.
• Classification of Interconnection Networks
Interconnection networks provide the interconnection among end systems and can be classi-fied into shared-medium networks, direct networks, indirect networks, and hybrid networks.
• Shared-medium networks
In shared-medium networks, processors are connected by a common transmission medium such as a bus. All processors share the medium which does not generate message. To send a message to a destination, a source broadcasts the message on the medium and the destination picks up the message. Because processors may send messages to the medium simultaneously, the resolution of network access conflicts is needed. The nature of the shared medium also limits the bandwidth of the network and the number of end systems in the network. Examples of the shared medium networks include the Ethernet. The protocol of for the medium access control used in the Ethernet is known as CSMA/CD (carrier sense multiple access with collision detection).
• Point-to-point networks
In point-to-point networks, end systems are connected by point-to-point communication links. The networks can be further classified into two categories: direct networks and indirect networks.
In direct networks, point-to-point links directly connect each end system to some other end systems.
In indirect networks, end systems are connected via one or more switches, switches are connected via point-to-point links.
• Hybrid networks
Some networks may have more complicated structures such as hierarchical structures or hypergraph topologies. Such networks are classified as hybrid networks.
• Direct Networks
A direct network consists of a set of nodes and a set of point-to-point links. Each node is directly connected to a small subset of nodes by links. Each node performs both routing and computing. A direct network is usually modeled as a graph, with vertices and edges
of the graph for the nodes and links in the network, respectively. A direct network is characterized by its topology and the routing/switching technologies used in the network. Important topology properties of the network include node degree (the number of links connected to the node), diameter (the maximum distance between two nodes in the net-work), regularity (a network is regular when all nodes have the same degree), symmetry (a network is symmetric when it looks alike from every node), and orthogonal property (a network is orthogonal if its nodes and links can be arranged inndimensions such that each link is placed in exactly one dimension).
In direct networks, the paths for message transmission are selected by routing algorithms. The switching mechanisms determine how inputs are connected to outputs in a node. All the switching techniques can be used in direct networks.
• Popular topologies for direct networks
r-dimensional mesh
The r-dimensional mesh consists of N =k1×k2×...×kr nodes, ki nodes along dimension
i,ki ≥2. Each node is identified by a label (a1, ..., ar), 0≤ai ≤ki−1 and 1≤i≤r. Two
nodes u = (a1, ..., ar) and v = (b1, ..., br) are connected by a link iff there is exactly one j
such that aj =bj±1 and ai =bi for all i6=j.
The most important mesh networks in practice are the 2-D mesh (r = 2, k0 = k1 = n,
N =n2) and the 3-D mesh (r= 3, k
0 =k1 =k2 =n,N =n3).
r-dimensional torus
Similar to ther-dimensional mesh, ther-dimensional torus consists ofN =k1×k2×...×kr nodes, ki nodes along dimension i, ki ≥ 2. Each node is identified by a label (a1, ..., ar),
0≤ai ≤ki−1 and 1≤i≤r. Two nodesu= (a1, ..., ar) and v = (b1, ..., br) are connected
by a link iff there is exactly one j such that aj =bj±1 modkj and ai =bi for all i6=j. The torus can be considered as the mesh with wrap around connections. When k1 =k2 =
...=kr =k, the r-dimensional torus is called the k-ary r-cube. When r= 1, the network
is the ring. The 2-D torus (r = 2, k0 = k1 = n, N = n2) and the 3-D torus (r = 3,
k0 =k1 =k2 =n, N =n3) are important networks. Hypercube
The n-dimensional hypercube (n-cube) consists of 2n nodes. Each node is identified by
(a1, ..., an), ai ∈ {0,1},1≤i≤n. There is a link betweenu= (a1, ..., an) andv = (b1, ..., bn)
iff u and v differ in exactly one bit position. The n-cube has degree n and diameter n. Tree
The k-ary tree is a tree in which every node except leaves has exactly k children. When
k = 2, thek-ary tree is known as the binary tree. Cube-connected cycles
The cube-connected cycle network can be considered as an n-dimensional hypercube of virtual nodes, each virtual node is a ring of n nodes (n2n nodes in total). The
Shuffle-Net
The (δ, k)-ShuffleNet is a regular digraph of in-/out-degree δ, N = kδk nodes, and kδk+1 arcs. The nodes are arranged in k columns, each column has δk nodes. The nodes in each column are connected to the next column via δk+1 arcs in a generalization of the perfect shuffle pattern. The (δ, k)-ShuffleNet has diameter d= 2k−1 and N = d+12 δ(d+1)/2 nodes. deBruijn digraphs
The deBruijn digraph B(δ, d) has in-/out-degree δ, diameter d, and N =δd nodes. Each
node has a label (a1, ..., ad),ai ∈ {0,1, ..., δ−1}. There are arcs from nodev = (a1, a2, ..., ad)
to nodes with labels (a2, ..., ad, α), α∈ {0,1, ..., δ−1}.
Star graph
The n-dimensional star graph has n! nodes, each node is identified by a permutation of (1,2, ..., n). Nodes u and v are connected iff v can be obtained by exchanging the 1st symbol with the ith symbol, 2 ≤ i≤ n, in the permutation of u. The n-dimensional star graph has node-degree n−1 and diameter d3(n−1)/2e.
• Indirect Networks
Nodes are connected by network of switches which can be set dynamically in different topologies. Only nodes can be end systems (sources and destinations). A network can be modeled by a graph, with vertices for switches and edges for links. End systems are not shown usually. The source systems are connected to the inputs and the destination systems are connected to the outputs of the network. Typical indirect networks include the crossbar network and the multistage interconnection networks (MIN). Main factors for the networks include the topology, routing, and switching.
• Cross bar network
An r ×n crossbar network consists of r input lines, n output lines, and rn cross-points located at the intersections of the lines. At each cross point, the input line and output line are connected by a binary switch which has two states, connected and disconnected. Any set of point-to-point connections (permutations) can be realized on an n×n crossbar by closing one cross point in each row and each column. A problem with a crossbar switch is the large number of crosspoints in the switch.
• Multistage interconnection networks (MIN)
In MIN, the inputs are connected to the outputs through a number of switch stages. Key factors for the MIN include the number of stages and the connection pattern between stages.
• Three stage Clos networks
For a network with n inputs and n outputs, there are k p×m cross bar networks in the 1st stage, m k×k cross bar networks in the 2nd stage, and k m×pcross bar networks in the 3rd stage, where n =kp.
• Generalized Clos networks
For n a power of 2, taking p=m = 2, there aren/2 2×2 switches in each of the 1st and 3rd stages, and 2n/2×n/2 networks in the 2nd stage. Recursively realizing the networks in the 2nd stage, we get a 2 log2n−1 stages network (Beneˇs network, hasO(nlog2n) 2×2 switches).
A network withS connection states requires at least log2S binary switching elements. For
n×n network for realizing any permutations, there areS =n! connection states and needs at least nlogn−1.44n binary switches.
• Generalized MIN model
Assume that each stage has the same number of inputs and outputs. The connection patterns between stages can be defined by permutations.
• Basic Permutations
Assumptions: Each stage has kn−1 k ×k switches. kn inputs/outputs are identified by xn−1...x0, 0≤xi ≤k−1 for 0≤i≤n−1.
– Perfectk-shuffle
σk(x
n−1...x0) = xn−2...x1x0xn−1. σ2 perfectly shuffles N cards. – Inverse perfect shuffle connection
σk−1(xn−1...x0) = x0xn−1...x1. – Digit reversal connection
ρk(x
n−1...x0) =x0x1...xn−1
– The ithk-ary butterfly permutation, 0≤i≤n−1,
βk
i(xn−1...xi+1xixi−1...x1x0) =xn−1...xi+1x0xi−1...x1xi
– The ith cube connection Ei, 0≤i≤n−1, k = 2 Ei(xn−1...xi+1xixi−1...x0) = xn−1...xi+1x¯ixi−1...x0. – The ithk-ary baseline permutation, 0≤i≤n−1,
δk
i(xn−1...xi+1xixi−1...x1x0) = xn−1...xi+1x0xixi−1...x1. • Classification of MINs
Blocking network: A path from a free input to a free output is not always possible because of the conflicts with existing connection paths.
Nonblocking network: A path from a free input to a free output is always possible without affecting the existing connection path.
Rearrangeable networks: A path from a free input to a free output is always possible with possible rearrangements of the paths for existing connections.
Unidirectional MINs: Links and switches are unidirectional. Bidirectional MINs: Links and switches are bidirectional.
• Unidirectional MINs
An MIN of N inputs/outputs and k×k switches needs at least logkN stages to allow a connection between any input/output pair.
For the MINs of n stages, we number the stages 0,1, ..., n−1 from left (input) to right (output). Let Ci (1≤i≤n−1) denote the connection pattern between (i−1)st stage and
ith stage, C0 be the connection pattern between the sources and inputs of stage 0, and Cn
be the connection pattern between outputs of stage n−1 and the destinations. Baseline network: C0,σk;Ci (1≤i≤n),δnk−i.
Butterfly network: C0, β0k; Ci (1≤i≤n−1), βnk−i; Cn, β0k. Cube MINs: C0, σk; Ci (1≤i≤n), βnk−i.
Omega network: Ci (0≤i≤n−1), σk; Cn,β0k.
• Bidirectional MINs (BMINs)
The BMIN consists of bidirectional switches/links. Bidirectional switches support three types of connections: forward, backward, and turnaround.
End systems are connected to one side (e.g., left) of the network. Routing paths are estab-lished by crossing stages in forward direction, a turnaround connection, and in backward direction.
Butterfly BMINs: Can be viewed as a folded Beneˇs network
Inverse butterfly BMINs: Can be viewed as fat-tree (used in CM-5)
• Hybrid Networks
Hierarchical Networks: Example, hierarchical buses. Cluster based network.
Hypergraph topology.
• Blocking properties of indirect networks
Accessibility: A network is fully accessible, if there is a path from any input to any output in the network.
Non-blocking property: A set of one-to-one connection requests on anN ×N network can be defined by a permutation. A network is non-blocking, if any permutation can be realized by edge-disjoint paths in the network. Non-blocking properties can be further classified into rearrangeable blocking, wide-sense blocking, and strict-sense non-blocking, depending on if the permutation is realized statically or dynamically.
– Rearrangeable non-blocking
A network is rearrangeable non-blocking if any permutation can be realized by edge-disjoint paths when the entire permutation is known. In other words, any permutation can be statically realized. The word rearrangeable refers to that if the connection
requests in a permutation arrive dynamically, the permutation can be realized with possible rearranging active connections. This is equivalent to realize a permutation statically.
– Wide-sense nonblocking
When connection requests in a permutation arrive dynamically in sequence, the per-mutation can be realized by edge-disjoint paths without rearranging active connections subject to the condition that a selected path is used for each new connection request. In other words, any permutation can be dynamically realized with the help of a wise algorithm.
– Strict-sense nonblocking
When connection requests in a permutation arrive dynamically in sequence, the per-mutation can be realized by edge-disjoint paths without rearranging active connec-tions, any idle path can be used for each new connection request. In other words, any permutation can be dynamically realized.
Obviously, the strict-sense non-blocking implies the wise-sense non-blocking which implies the rearrangeable non-blocking.
• Networks by 2×2 Switches
A 2×2 switch has two input links and two output links and has a through state and a cross state for one-to-one connection from inputs to outputs. An r×n cross bar network can be constructed byr×nswitches at the r×n cross points ofrinput lines andn output lines. An n×n crossbar network is strict-sense nonblocking
MINs: Ann-dimensional MIN hasN = 2ninputs/outputs and multiple stages of switches, with N/2 switches in each stage. A k-stage network has k ×N/2 switches. A necessary condition for the full accessibility is that the network has at least n stages. Sufficient conditions for the full accessibility depend on connection patterns between stages.
Ak-stage MIN has 2k×N/2 distinct states. To realize all permutations on anN×N network, at leastN! distinct states of the network is needed. A necessary condition on the number of stages for rearrangeable non-blocking networks is k≥2 logN−O(1). Sufficient conditions on the number of stages for rearrangeable non-blocking networks, depend on connection patterns between stages. The n-dimensional Beneˇs network has 2 logN −1 stages and is rearrangeable non-blocking.
• Blocking MINs
There is a class of well known n-dimensional full accessible MINs with n stages of 2×2 switches. There is a unique path from any input to any output in a network of this class. Examples of the networks include baseline networks, omega networks, butterfly networks, indirect binary n-cube networks, and the inverse networks of the above. These networks are blocking networks. The networks in the class have similar structures. Especially, many of them are topologically or functionally equivalent.
• Routing on n-dimensional MINs
Routing on the n-dimensional Omega network Ωn
An inputu= (un−1...u1u0) and an outputv = (vn−1...v1v0) can be connected by the unique path (un−1un−2...u0) → (un−2...u0vn−1) → (un−3...u0vn−1vn−2) ... → (un−i...u0vn−1...vn−i+1) ... → (vn−1...v0). Collisions
If two paths share an edge then a collision occurs. Paths (un−1...u0) → (vn−1...v0) and (sn−1...s0)→(tn−1...t0) are edge-disjoint iff∀i,0≤i≤n−1,
un−i...u0vn−1...vn−i+1 6=sn−i...s0tn−1...tn−i+1
For two paths u→ v and s→ t, let α(u, s) be the largest l such that the rightmost (least significant) l bits of u and s are the same, and let β(v, t) be the largest l such that the leftmost (most significant) l bits of v and t are the same. Paths u → v and s → t are edge-disjoint if α(u, s) +β(v, t)< n.
Routing on other networks like butterfly networks is similar.
• Non-blocking networks
N ×N cross bar network
This network is strict-sense non-blocking. A problem with the cross bar network is the large number O(N2) of switches in the network. How to construct a non-blocking network with as few switches as possible has been a major research topic in circuit-switched networks. The number of switches can be reduced by increasing the number of stages in the network. Clos network
Three stage N ×N Clos network (N =k×p),
– k p×m cross bar networks in the 1st stage – m k×k cross bar networks in the 2nd stage – k m×pcross bar networks in the 3rd stage
– The outputs of one stage are connected to the inputs of the next stage by a shuffle pattern. More precisely, let thek networks in the 1st stage be identified by an integer
x0 ∈ {0,1, .., k−1} and let the m outputs of each switch be identified by an integer
x1 ∈ {0,1, .., m−1} and let the k outputs of each switch be identified by integer
x0 ∈ {0,1, .., k−1}. Then output (x1x0) of the 1st stage is connected to input (x0x1) of the 2nd stage. Similarly, output (x0x1) of the 2nd stage is connected to input (x1x0) of the 3rd stage.
A necessary and sufficient condition to make the three stage Clos network strict-sense non-blocking is m= 2p−1. An outline for proving this is given below.
For any idle input/output pair (u, v), where u is an input of network x0 in the 1st stage and v is an output of network x00 in the 3rd stage, at mostm−1 networks of the 2nd stage have been used for the connections from the inputs other than u of network x0, and at most m−1 networks of the 2nd stage have been used for the connections to the outputs other than v of network x00. Since there are 2m−1 networks at the 2nd stage, there is at least one network which is not used by any of the previous connections from network x0 or to network of x00. Therefore, (u, v) can be connected via that network of the 2nd stage. For N =p2, taking m = 2p−1, a non-blocking Clos-network can be realized by O(N3/2) switches.
Reduce switches by increasing stages
The number of switches can be further reduced by increasing the number of stages with a recursive construction. Let N = pr+1 (r ≥ 1) and m = 2p−1. The construction starts from a 3-stage network
– k=pr p×m cross bars in the 1st stage
– m pr×pr cross bars in the 2nd stage – pr m×pcross bars in the 3rd stage
Recursively realize the middle stage cross-bars until each of the cross-bars in the 2nd stage becomes ap×pnetwork. The number of switches in the above construction isO(N1+1/(r+1)). Beneˇs Networks
For N = 2n, taking p=m= 2,
– N/2 2×2 switches in the 1st stage – 2N/2×N/2 networks in the 2nd stage – N/2 2×2 switches in the 3rd stage
Recursively realizing the networks in the 2nd stage, we get a 2 log2N −1 stages network which is known as the n-dimensional Beneˇs network and has O(Nlog2N) 2×2 switches. The Beneˇs network is rearrangeable non-blocking.
A strict-sense non-blocking network can be constructed byn = log2N copies of the Beneˇs networks (known as Cantor network) as follows: At the input stage, there are N 1×n
switches. In the middle stage, there are n copies of the Beneˇs networks. At the output stage, there are N n×1 switches.
• Network equivalence
Topological equivalence: Two networks are topologically equivalent if one can be ob-tained from the other by relabeling switches and/or inputs/outputs of switches.
Functional equivalence: Two networks are functional equivalent if they realize the same set of permutations.
• Routing on direct networks
A direct network consists of a set of nodes, each is connected to some other nodes (neigh-bors) by point-to-point links. The network is also known as the point-to-point network or message-passing network. The term is usually used for parallel/distributed computing systems. The communication in the network is realized by message passing. Usually a mes-sage is partitioned into packets. A packet consists of a header and a data area. A header has routing and sequencing information. Usually each node in the network performs both computing and routing. In most systems, each node has a dedicated router to free the CPU from routing. Each router usually can realize non-blocking routing from its inputs to its outputs.
• Communication time
A common metric for communication time is communication latency which has three parts:
– Start-up latency
It is the time to handle packet at source and destination nodes and depends on the protocols and internal architectures of the nodes.
– Network latency
It is the time between a packet leaving the source and arriving at the destination as-suming that there is no contention during the transmission. It depends on the topology of the network, the channel capacity, switching techniques, routing algorithms, and structures of routers.
– Blocking time
It is the time delay caused by contentions. The contention can happen on links and routers. The blocking time depends on dynamic behaviors of the network. Queueing theory is a main methodology for analyzing the average blocking time.
Main factors on the communication latency include the topology, routing, flow control, and switching.
• Topology
The topology refers to how nodes are interconnected by communication links (channels). It is ideal that any two nodes is directly connected by a link. The corresponding graphs to such networks are complete. However, due to the hardware constraints and cost, it is not feasible to have large complete networks. Much research work has been done on network topologies. Some basic preferable properties of topologies include small diameter,
small node degree, multiple disjoint paths between nodes, symmetry, and regularity. Some properties contradict with each other, like diameter and node degree. Additional properties related to performances of the network include the following.
Orthogonal property, provides the base for dimension based routing which is easy to realize.
Bisection width, the minimum number of links to partition the network into two parts. This property is important to the fault tolerance of the network.
Channel bandwidth, the data rate of a link. It is the product of channel width and the channel rate, where the channel width is the number of bits can be transmitted in parallel on a link (the number of lines) and the channel rate is the peak transfer rate of bits on a single line. The channel bandwidth determines the performance of the link.
• Routing
Routing is to select a path for delivering messages from source to destination. The following strategies have been used for routing.
Source routing
The source node determines the routing path which is fixed. The packet must carry the information on the routing path.
Distributed routing
Each router decides the neighbor node to which it sends the message without the global information of the network. The algorithm used by routers should be fast and easy to implement.
Deterministic routing
A unique routing path is used for a source and destination pair. The path does not depend on the dynamic states of the network. This is also known as oblivious routing.
Adaptive routing
The routing path for a source and destination pair may not be unique. The path depends on the dynamic states of the network.
Minimal routing
The routing path is always a shortest path. We will study routing in more details later.
• Switching in direct networks
Both circuit-switching and packet-switching have been used in direct networks.
• Circuit switching
In circuit-switching networks, a dedicated path is set-up to connect a pair of source and destination. The message is transfered without buffering on the path. The communication
latency consists of the time for path set-up and the delay in transferring the message. In most of the circuit-switching parallel computing systems, the paths are set-up by passing a prob (control) packet from the source to the destination.
Let T be the communication latency,
Lc be the length of the control packet, Lbe the length of the message,
B be the channel bandwidth, and
dbe the length (the number of channels) in the path. Then
T =dLc
B +
L B.
If Lc << Lthen the latency can be considered independent of d. The circuit-switching is
not efficient for busty traffics. For short messages if a circuit is released each time, then the time for path set-up is excessive. If a circuit is not released then the channels in the circuit are idle, resulting in low utilization. Another problem is that one busy channel can block a whole circuit.
• Packet switching
In packet-switching networks, the path for transmitting messages from a source to a destina-tion may not be dedicated or unique and messages are transmitted with possible buffering at the intermediate routers on the path from the source to the destination. This over-comes some problems in circuit-switching networks. We discuss a number of variants of circuit-switching techniques.
Message-switching
In message-switching networks, messages are routed dynamically using routing algorithms. Usually the algorithms are distributed and adaptive ones. Messages are transmitted in a store-and-forward way: messages are stored in buffers at intermediate nodes and the whole message is stored before it is forwarded to the next node. Each message has a header which contains the routing information like the source/destination addresses. A link (channel) can be shared by messages for multiple connections. Letβbe the start-up time for transmitting a message at a node. The communication latency for message-switching networks is
T =d(β+ L
B).
For a network with diameter D the worst case lower bound for the latency is
D(β+ L
B).
Problems with the above networks include large buffers required at routers (because a buffer should be able to store messages of different sizes) and latency proportional to the distance from source to destination.
Packet-switching
In packet-switching networks, a message is partitioned into packets (usually of fixed size), each packet has a header containing routing information. Each packet is routed in a store-and-forward way independently. This allows two types of parallelism in routing. One is that several links in a same path are simultaneously used for transferring packets (packets pipelined) and the other is that packets can be routed on multiple paths from the source to the destination. Because the packets have the same size, it is easier to control the buffer size at routers. Packet-switching networks have better resource utilization than message-switching networks but have duplicated headers and more overhead at routers. Let P be the size of the packets. Then the latency for the first packet is
d(β+ P
B).
If packets are pipelined on the links of the path, the communication latency for the message is
T = (d+ L
P −1)(β+ P B).
For a network with diameter D and cut-width C (the number of edges that separate the network), the worst case lower bound for the latency is
max{Dβ, L BC}.
• Virtual cut-through
The virtual cut-through switching can be considered as a mixture of circuit-switching and packet-switching. The idea here is that when the next channel is available at an interme-diate router, the router sends the received part of a packet to the channel before the entire packet is stored. A router only buffers an entire packet when the output channel is busy. In the worst case, the latency of the network is similar to that of a packet-switched network. In the best case, the latency is similar to that of a switched network. In the ideal case, the latency is
T =dβ+ L
B.
Wormhole routing
In wormhole-switching networks, a packet is further partitioned into smaller units (flits). The first flit(s) contains the header and the other flits contain data. The routing decision is based on header flits and data flits follow the route for the header in a pipelined fashion (the router does not buffer data flits). Since data flits do not contain routing information, a contiguous path of channels is needed. When a header is blocked, all flits stop advancing and remain in channels. This is different from the virtual-cut-through switching where the data are moved from channels to buffers of the router. The latency of the network is
T =dβ+ T
if there is no contention. The advantages of the networks is small latency (almost inde-pendent to d if no contention) and very small buffers (just FIFO flit buffers). Problems include large blocking time if network congested and deadlocks in routing.
An example of a deadlock may like this: packet A holds some resources while requesting others held by packetB which is demanding the resources held byA. Resources are buffers in store-and-forward and virtual-cut-through networks and channels in circuit-switching and wormhole networks. Much work has been done on preventing and avoiding deadlocks. The details are omitted here.
Virtual channels
In the VC switching, a message/packet is transmitted through a virtual path. A virtual path is not a dedicated one for a connection but can be shared by multiple connections. Similar to the circuit-switching, there are three phases in the VC switching, connection establishment, transmission, and connection termination. The advantages of the virtual channel switching include the similar latency to and better resource utilization than those of circuit-switching networks. Disadvantages include the increased scheduling complexity and the physical channel shared by multiple virtual connections can be a bottleneck which increases the latency.
• Routing Strategies Deterministic routing
For orthogonal networks, an efficient approach is dimension-ordered routing. The routing path consists of a sequence of links with a specific dimension order. The intermediate routers can easily compute the next output link from the source and destination addresses in the message. An example of dimension-ordered routing is the E-cube routing on the hypercube network: A packet contains the destination address d ∈ {0,1}n. When the
packet arrives at a node with label v ∈ {0,1}n, the node computes d⊕v, where ⊕ is the
bitwise binary sum, and uses the link on the dimension corresponding to the rightmost (least significant) 1 in d⊕v as the output link (if d⊕v = 0, the packet has arrived at the destination d). Another example is mesh-XY-routing. In this approach, a packet is first routed in the X direction to the correct row and then in the Y direction to the correct column.
Adaptive routing
This routing strategy can choose routing paths based on the conditions of the network. There are a number of approaches. Details are omitted.
Table look-up routing
This is a distributed routing strategy. Each node keeps a table to indicate which outgoing link to use for a destination. The Internet uses this approach. Key points for this approach include getting information for updating the routing table and reducing the table size.
