Important problems of graph theory and the Internet

(1)

Important problems of graph theory and the

Internet

Andrei Raigorodskii

Lomonosov Moscow State University, Moscow Institute of Physics and Technology, Yandex Division of Theoretical and Applied Research,

Moscow, Russia

(2)

Real-world web-graph

G= (V;E), where V —

(3)

The main objects

(4)

set of web-pages, set of web-sites,

(5)

The main objects

Real-world web-graph G= (V;E), where V — set of web-pages, set of web-sites, set of web-hosts,

(6)

set of web-pages, set of web-sites, set of web-hosts,

andE — the set of all hyperlinks between the vertices (nodes).

(7)

The main objects

Sometimes multiple edges are identified. Sometimes multiple edges and even loops are allowed.

(8)

How can we use this graph for practical purposes?

(9)

The main objects

(10)

Use its structure to find features improving learning to rank; Adjust algorithms including, say, those for crawling;

(11)

The main objects

Use its structure to find features improving learning to rank; Adjust algorithms including, say, those for crawling;

(12)

(13)

Some important properties/features, which must be

fastly checked on real data

Barab´asi–Albert, Watts–Strogatz, Newman, and many others in 90s–00s.

Web-graphs aresparse, i.e., their numbers of edges (links) are proportional to

(14)

Web-graphs aresparse, i.e., their numbers of edges (links) are proportional to their numbers of vertices.

Web-graphs have a unique “giant” connected component.

(15)

Some important properties/features, which must be

fastly checked on real data

their numbers of vertices.

Every two vertices in the giant component are connected by a path of short

length (5–6, 15–20 depending on what we mean by web-graph): diamG≈6

(16)

(the rule of 6 handshakes).

Web-graphs are robust when random vertices are destroyed (a giant component survives).

(17)

Some important properties/features, which must be

fastly checked on real data

their numbers of vertices.

Web-graphs are vulnerable to attacks onto hubs (many small components appear after a threshold is surpassed).

(18)

Web-graphs are vulnerable to attacks onto hubs (many small components appear after a threshold is surpassed).

The degree distribution is close to a power-law:

|{v∈V : degv=d}| n ∼ const d ;

where∈(2;3) depends on what we mean by web-graph.

(19)

More of important properties/features: global

clustering

(20)

LetG= (V;E) be a simple graph,|V|=n. Letv∈V. Denote byN v the set of neighbours ofv inG. Setn v=| N v|, i.e., n v = deg v. Ifn v >2, then the

clustering coefficientof the vertexv is

C v = |{{x;y} ∈E: x;y∈N v}| C 2 nv :

(21)

More of important properties/features: global

clustering

LetG= (V;E) be a simple graph,|V|=n. Letv∈V. Denote byN

v the set of neighbours ofv inG. Setn v=| N v|, i.e., n v = deg v. Ifn v >2, then the

clustering coefficientof the vertexv is

C v = |{{x;y} ∈E: x;y∈N v}| C 2 nv : Clustering coefficient 1

The global clustering coefficient ofGis

T(G) = P v∈V C 2 n v C v P v∈V C 2 n v :

(22)

Let](H;G) be the number of copies of a graphH in a graphG.

(23)

More on global clustering

Clearly T(G) = P v∈V C 2 nv C v P v∈V C 2 nv =3](K3;G) ](P2;G) ;

(24)

Let](H;G) be the number of copies of a graphH in a graphG. Clearly T(G) = P v∈V C 2 nv C v P v∈V C 2 nv =3](K3;G) ](P2;G) ;

whereK3is a triangle andP2 is a 2-path.

Thus, roughly speaking,T(G) is the probability that two neighbours of a vertex of

Gare themselves joined by an edge.

(25)

More on global clustering

Clearly T(G) = P v∈V C 2 nv C v P v∈V C 2 nv =3](K3;G) ](P2;G) ;

whereK3is a triangle andP2 is a 2-path.

Thus, roughly speaking,T(G) is the probability that two neighbours of a vertex of

Gare themselves joined by an edge.

Clearly the last formula forT(G) may be used for any graph, not only a simple

(26)

(27)

More of important properties/features: local

clustering

C v = |{{x;y} ∈E: x;y∈N v}| C 2 n v :

(28)

C v = |{{x;y} ∈E: x;y∈N v}| C 2 n v : Clustering coefficient 2

The local clustering coefficient ofGis

C(G) = 1 n X v∈V C v :

(29)

More of important properties/features: local

clustering

C(G) = 1 n X v∈V C v :

(30)

C(G) = 1 n X v∈V C v :

The valuesT(G) andC(G) are rather different.

As a rule,T(G)<C(G).

(31)

More of important properties/features: local

clustering

C(G) = 1 n X v∈V C v :

(32)

C(G) = 1 n X v∈V C v :

Both values are constant in real-world graphs (high clustering).

C(G) is much easier to calculate.

(33)

More of important properties/features: local

clustering

C(G) = 1 n X v∈V C v :

(34)

C(G) = 1 n X v∈V C v :

C(G) cannot be naturally defined for graphs with multiple edges and loops.

C(G) is more complicated for theoretical study (in random graphs).

(35)

(36)

Assortativity

For a simple graph, let

d nn( d) = 1 d· |{i: degi=d}| X i: degi=d X j: {i;j}∈E degj:

(37)

Degree correlations

Assortativity

d nn( d) = 1 d· |{i: degi=d}| X i: degi=d X j: {i;j}∈E degj: As a rule,d nn( d)∼d Æ .

(38)

Assortativity

For web-graphs,Æ<0 (calleddissassortative network).

(39)

Degree correlations

Assortativity

For web-graphs,Æ<0 (calleddissassortative network).

(40)

(41)

Page Ranks

Classical PageRank LetG n= ( V n ;E n) be a web-graph. Denote by PR(v) a PageRank of a vertexv. DefinePR(i), wherei∈ {1;:::;|V

n|}, as the solution of the following system of

linear equations: PR(i) =c X j→i PR(j) outdegj + c |V n| X j∈D PR(j) + 1−c |V n| ; i= 1;:::;|V n| ;

(42)

wherec∈(0;1) is a constant andDis the set of vertices having zero outdegrees.

Very complicated calculations and many algorithms for approximation.

(43)

Page Ranks

Very complicated calculations and many algorithms for approximation. However, now Classical PR is not a strong feature!

(44)

Very complicated calculations and many algorithms for approximation. However, now Classical PR is not a strong feature!

So many even more sophisticated algorithms.