4.3.1 2D Full Mesh
The simplest model with higher redundancy, serving also as the worst case estimate in terms of scalability, is a complete graph in which each AE communicates directly with all the remaining AEs, as shown in Figure 4.2. This model was studied and described in detail in [33].
FIGURE4.2: 2D full mesh.
Definition 4.2 (2D full-mesh network) Let’s have a network with AEs and clients with each AE populated with at least one client. 2D full-mesh network of AEs is a network, in which each AE sends data from each client to other client connected to the same AE and
4.3. DISTRIBUTION MODELS 20 it sends the data also to all other AEs in the network. Each AE thus receives the data from all other AEs and sends them to all the clients connected to that AE. 2
Let’s assume network of mtotAEs with full-mesh communication. n clients connect to
the AEs in such way that each AE has either nror nr− 1clients.
nr= n mtot (4.1) m1= nrmtot− n (4.2) m = mtot− m1 (4.3)
Definition 4.3 (Evenly populated AE network) A AE network with each AE having ei- ther nror nr− 1clients connected is called evenly populated AE network. All client are
active, i. e. both sending and receiving. 2
Theorem 4.1 In an evenly populated 2D full mesh of AEs, the inbound traffic is in = n streams. PROOF When full mesh operates in N:N way, the inbound traffic for AEs with nrclients
will be in = nr |{z} 1 + (m − 1)nr | {z } 2 + m1(nr− 1) | {z } 3 (4.4) (1 . . . directly connected clients, 2 . . . streams from m − 1 other AEs with nrclients, 3 . . .
streams from all m1clients with nr− 1clients) and for AEs with nr− 1clients
in1= nr− 1 | {z } 4 + mnr |{z} 5 + (m1− 1)(nr− 1) | {z } 6 (4.5) (4 . . . directly connected clients, 5 . . . streams from all m AEs with nr− 1clients, 6 . . . from
other m1− 1AEs with nr− 1clients).
It can be easily shown that in = in1and after some simplification the in formula can
be written as
in = nrm + m1nr− m1 (4.6)
Further substituting m and m1we get
in = n (4.7)
Theorem 4.2 In an evenly populated 2D full mesh of AEs, the limiting traffic in this mesh is the outbound traffic on the AE which is out = n2
rmtot+ nr(mtot− 2)streams.
PROOF Outbound traffic for AE with nrclients will be
out = (nr− 1)nr | {z } 7 + (m − 1)n2 r | {z } 8 + m1nr(nr− 1) | {z } 9 + nr(m + m1− 1) | {z } 2×10 (4.8) (7 . . . from directly connected clients to directly connected clients (the AE doesn’t send data to the client which sent them!), 8 . . . data from m − 1 AEs with nrclients to all own nr
clients, 9 . . . data from all m1AEs with nr− 1client to all own nrclients, 10 . . . data sent
to other m + m1− 1AEs) and for AE with nr− 1clients
out1= (nr− 2)(nr− 1) | {z } 11 + mnr(nr− 1) | {z } 12 + (m1− 1)(nr− 1)2 | {z } 13 + (nr− 1)(m + m1− 1) | {z } 2×14 (4.9)
13 9 8 7 1 nr nr− 1 4 nr− 1 nr 2 10 14 6 14 3 10 5 11 12
FIGURE4.3: Flow analysis in full 2D mesh of AEs. Bottom and right AEs are
populated with nr− 1clients, while top and left AEs are populated with nr
clients.
(11 . . . from directly connected clients to directly connected clients (the AE doesn’t send data to the client which sent them!), 12 . . . data from m AEs with nrclients to all own nr−1
clients, 13 . . . data from other m1− 1AEs with nr− 1client to all own nr− 1clients, 14 . . .
data sent to other m + m1− 1AEs). The numbers in equations correspond to numbers in
Figure 4.3 on page 21.
It can be easily shown that
out1
out = nr− 1
nr .
and we can also use use just simplified out formula
out = nr(nrm + m1nr+ m − 2) (4.10)
as out > out1for nr ≥ 1and m + m1 ≥ 2. For m + m1 < 2the full mesh loses sense and
thus out is the limiting value for outbound traffic. Further substituting m and m1we get
out = nr(mtot+ n − 2) = n2rmtot+ (m − 2)nr (4.11)
If we substitute the other way round the nrfrom (4.1) (which is not precise due to ceil
function) we get
out = n(mtot+ n − 2)
mtot (4.12)
and the ratio between out for full mesh of AEs and single AE out = n(n − 1) is ratio =mtot+ n − 2
4.3. DISTRIBUTION MODELS 22 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 # Clients 0 100 200 300 400 500 600 700 800 900 1000 Sent Bandwidth [Mbps] m=1 m=2 m=4 m=8 m=12 saturation of GE
FIGURE4.4: Behavior of 2D full mesh for DV clients. Dependence of limiting
outbound traffic on the number of 30 Mbps clients and the number of AEs in the mesh.
Fail-Over Operation
When a link or whole AE drops out in the full mesh, the accident only influences data distribution from/to the clients connected to that AE. In case of link failure inside the AE mesh, the client is requested to migrate to an alternative AE. In case that AE itself fails the client initiates migration on its own. Alternative AEs should be selected randomly to distribute load increase more evenly and the load increase will be at most d nr
m−1e. When
even this migration delay is not acceptable, it is possible for a client to be permanently con- nected to an alternative AE and just switch the communication. For even more demanding applications, the client can use more than one AE for sending in parallel.
Although this model seems to be fairly trivial and not that interesting, it has two basic advantages: first, the model is robust and failure of one node influences only data from/to the clients connected to that AE. Second, it introduces only minimal latency because the data flows over two AEs at most. Next we will examine another model that has the same latency and robustness properties but that scales better.
4.3.2 3D Layered-Mesh Network
The layered mesh model creates k layers, in which data from a single AE are only dis- tributed as shown in Figure 4.5. One layer is thus similar to 2D full mesh network except for that only one AE is both sending and receiving in one layer. Thus each client is con- nected to one layer for both sending and receiving (sending only if nr= 1; to receive data
from clients sending data via other layers, the client needs to receive data from remaining nr− 1clients of the AE used for sending) and to all other layers for receiving only. Each
layer comprises m AEs. For the sake of simplicity, we first assume that k = m and each AE has nrclients, thus nr=mn =nk.
Definition 4.4 Active AE is an AE with clients that are both sending and receiving. Non-
FIGURE4.5: 3D layered mesh.
Definition 4.5 (3D layered-mesh network) A network of AEs organized into 3D layered- mesh comprising k layers and m AEs in each layer. Each layer is used to distribute data from clients connected to one active AE. Thus each client is connected to one layer for
sending and receiving and all other layers for receiving only. 2
Theorem 4.3 In 3D layered-mesh network with each AE having nrclients, each AE has in = nr
inbound streams.
PROOF The active AE has nrclients and thus it receives in = nrstreams. Each non-active
AE receives in = nrstreams to distribute to its clients from active AE.
Theorem 4.4 In 3D layered mesh network with each AE having nrclients, the active AEs have
outs/r = n2
r+ nr(m − 2)outbound streams, and the non-active AEs have outr = n2routbound
streams.
PROOF Number of output streams for active AE with both sending and receiving (s/r)
clients is outs/r= nr(nr− 1) | {z } 1 + nr(m − 1) | {z } 2 = nr(nr+ m − 2) = n2r+ nr(m − 2) (4.14)
where part 1 is for directly connected clients and part 2 is for all the remaining m − 1 non-active AEs in the same layer.
For non-active AE that has only receiving (r) clients connected outr= n2
r. (4.15)
because the non-active AE distributes nrstreams (from nrclients of active AE) to its own
nrclients.
It is obvious that for nr ≥ 1and m > 2, it always hold outs/r > outr, for m = 2,
outs/r = outrand for m < 2 the distribution network doesn’t have sense. Thus the outs/r
4.3. DISTRIBUTION MODELS 24 The limiting throughput is the one which sending clients are connected to. Thus the ratio between such mesh (with total number of clients n = nrmand single AE is
ratio= n + m(m − 2)
m2(n − 1) (4.16)
while using total of km = m2AEs.
This model is problematic because of quadratic increase with respect to number of AEs used. However it seems to be the last model that doesn’t induce intermediate hops and thus keeps latency at the minimum.
Transition from 3D to 2D mesh
It is possible to perform vertical aggregation of AEs across 3D layers to get the 2D full mesh model as proved in Theorem 4.5.
Theorem 4.5 The 3D layered mesh model is extension of 2D full-mesh model and the latter model can be obtained by aggregating AEs of the first model.
PROOF To do so, we merge AEs that are positioned above each other (“flatten the layers”).
In such case, the the AE is once used as sending/receiving AE and m−1 times as receiving only AE.
Thus the number of input streams is m times nr(since once it gets nras both sending
and receiving AE and m − 1 times it gets nras receiving one)
in= mnr= n (4.17)
This relation is the same as (4.7). For number of output streams, it follows out = n2 r+ nr(m − 2) | {z } 1 + (m − 1)n2r | {z } 2 = n2r+ (m − 2)nr (4.18)
Part 1 is one occurrence of AE in sending/receiving role and part 2 is m−1 time occurrence of AE in receiving only role. The number of outbound streams is obviously equal to (4.11). Thus we have proved that 2D full-mesh model is just special variant of 3D layered-mesh
model.
Fail-Over Operation
Each of the mesh layers monitors its connectivity. When some layer disintegrates and becomes discontinuous, the information is broadcasted throughout the layer and to its clients. The clients that used that layer for sending are requested to migrate to randomly chosen layer from the remaining k − 1 layers and the listening-only clients simply discon- nect from this layer. Such behavior increases load on the remaining k − 1 layers but as the clients choose the new layer randomly, the load increases in roughly uniform way by at most d nr
k−1e.
4.3.3 3D Layered Mesh of AEs with Intermediate AEs
Definition 4.6 (q-nary distribution tree) The q-nary distribution tree is a directed acyclic graph, in which each parent node has q child nodes. Data in the distribution tree are
distributed according to orientation of edges. 2
Definition 4.7 (3D layered mesh with intermediate AEs) The 3D layered mesh with in- termediate AEs is layered structure where each layer is organized as follows: each layer has one active AE that is the root of the q-nary distribution tree in each layer. Receiving- only clients are connected to m − 1 leaf AEs of the distribution tree. 2
Definition 4.8 (Intermediate AE) Intermediate AE is an AE that doesn’t have any clients directly connected. Alternatively, it is any AE that is neither active nor non-active. 2
Let’s create q-nary tree used for distributing data from AE with sending clients to m−1 AEs with listening clients. When building q-nary tree with λ intermediate layers
λ = logq(m − 1) − 1, (4.19)
the total number of intermediate AEs is L = λ X p=1 qp=qλ+1− q q − 1 = qlogq(m−1)− q q − 1 = m − 1 − q q − 1 . (4.20)
Theorem 4.6 Numbers of flows in the 3D layered mesh with intermediate AEs and nr clients
per active and non-active reflectors are summarized in Table 4.1. The limiting traffic is number of outbound streams on active AE.
in out
s/r nr nr(nr− 1) + qnr
r nr n2r
int nr qnr
TABLE4.1: Flows in 3D network with intermediate AEs. “s/r” means outer AE
with sending clients connected, ”r” means outer AE with only receiving clients, “int” is intermediate AE.
PROOF Number of inbound streams is again in = nr for all types of AEs because the
network distributes nrstreams from nrclients of the active reflector.
Number of outbound streams on active AE is outs/r = nr(nr− 1) | {z } 1 + qnr |{z} 2 , (4.21)
where part 1 is distribution to locally connected clients and part 2 is for distribution of nr
data streams to first layer of intermediate reflector comprising q AEs.
Each intermediate AE distributes nrstreams further on to its q child AEs and thus
outint= qnr (4.22)
The non-active AE distributes nrdata streams to nrlocally connected clients and thus
outr= n2
r (4.23)
Since outs/r > outr ≥ outintfor q ≥ 2 and nr > 1, the limiting traffic is outs/r. For
q ≥ 2and nr= 1, outs/r = outint > outr. If q < 2, the q-nary distribution wouldn’t have
sense. Thus outs/rcan be understood as limiting outbound traffic.
There are however two disadvantages of this model:
• The number of hops inside the mesh increases by λ compared to simple 3D mesh model. This will increase latency but it is impossible to enumerate the latency in- crease in general as it depends on the underlying network topology and one-way delays of distribution between hop pairs.
4.3. DISTRIBUTION MODELS 26 • Compared to plain 3D model, the number of intermediate AEs further increases to
mtot= mk + Lk (4.24)
For m = k it is
mtot= m(m + L) (4.25)
This relation is illustrated in Figure 4.6 demonstrating quite extreme growth of total number of reflectors when having only 2 clients per one AE, while having reasonable properties as the number of clients per AE increases.
0 20 40 60 80 100 120 Number of clients (n) 100 200 300 400 500 600 700 800 900 1000
Number of AEs (m_tot)
n_r = 2, q = 2 n_r = 5, q = 2 n_r = 5, q = 5 n_r = 10, q = 2 n_r = 10, q = 5 n_r = 10, q = 10 n_r = 20, q = 10
FIGURE4.6: Number of AEs needed for 3D mesh with intermediate hops in which
k = mand q = nr.
Aggregation of inner AEs
To limit number of inner AEs, we can take into account the linear increase of limiting outbound flow on each inner AE qnras shown in Table 4.1.
Theorem 4.7 When only L AEs are used for all k layers, resulting numbers of data streams on internal AEs is inint= knrand outint= kqnr.
PROOF Because the number of streams distributed by intermediate AEs is knr, because
each of k layers distributes nrstreams. Thus number of input streams for each intermedi-
ate AE is inint= knr.
Since each of knr streams needs to be distributed to q child AEs, the number of out-
bound streams on intermediate AE is outint= kqnr.
When we use k = m (meaning there is the same number of outer AEs and number of layers), then inint= nand outint= qn, where n = mnris total number of clients.
In this section, we have shown trivial way to aggregate inner AEs in the 3D mesh of AEs. However, searching for optimum general aggregation of nodes in such network
(not only the inner ones) leads to creating optimal AE-based application-level multicast network.
This model is problematic because of increasing the number of AEs used. However it seems to be the last model that doesn’t introduce intermediate hops and thus keeps hop- count at minimum.
4.3.4 Multicast Schemes
In an ideal case, the multicast organization of the data distribution is the most efficient scheme to distribute data to multiple clients. However, it is very difficult for a user to place AEs into the physical network topology in such a way that no data will pass through any physical link twice. The only exception may be when AE network is implemented as a network of active routers, but this goes against the user-empowered approach we sup- port. Thus the multicast paradigm is only an upper-limit on efficiency of the distribution.
There are two basic approaches to build multicast distribution tree: source-based tree also known as shortest path tree (SPT) and shared tree. Regarding the synchronous char- acter of multimedia data distribution, the SPT with reverse path forwarding (RPT) has two major advantages: it minimizes latency compared to shared tree where the data is sent through rendezvous point and it provides shortest paths between the source and the receivers (advantage for large volume of data transmission).
To build SPTs, it is necessary to have underlying unicast routing information. This information can be maintained very efficiently by RON [2]. As an addition to fast conver- gence in case of network link failure, it is possible to define policy to select the shortest path not based on hop count, but based on path round trip time or even one way propagation delay if available.
Fail-Over Operation
Standard operation when the link failure occurs is to build a new SPT as described above. If even the convergence speed of RON is not acceptable, there is another possible strategy to minimize delay due to SPT reconstruction. It is possible to compute multiple SPTs at the same time, choose single SPT for data distribution and keep the remaining SPTs for fail-over operation. For permanent redundancy scenario, more than one SPT can be used simultaneously and duplicate data will be discarded by client applications. In full graph, there are n2− nlinks between the AEs. For a small number of AEs, alternative SPTs can
be computed that don’t use one selected link at a time. If that particular link fails, the al-