Iterated Learning in Dynamic Social Networks
Bernard Chazelle [email protected]
Department of Computer Science, Princeton University, 35 Olden Street, Princeton, NJ 08544
Chu Wang∗ [email protected]
Amazon Inc.
500 Boren Avenue, Seattle, WA 98109
Editor:Mehryar Mohri
Abstract
A classic finding by (Kalish et al., 2007) shows that no language can be learned iteratively by rational agents in a self-sustained manner. In other words, ifAteaches a foreign language toB, who then teaches what she learned toC, and so on, the language will quickly get lost and agents will wind up teaching their own common native language. If so, how can linguistic novelty ever be sustained? We address this apparent paradox by considering the case of iterated learning in a social network: we show that by varying the lengths of the learning sessions over time or by keeping the networks dynamic, it is possible for iterated learning to endure forever with arbitrarily small loss.
1. Introduction
People typically form opinions by updating their current beliefs and reasons in response to new signals from other sources (friends, colleagues, social media, newspapers, etc.) (Tahbaz-Salehi et al., 2009; Acemoglu and Ozdaglar, 2011; Golub and Jackson, 2010). Suppose there were an information source that made a noisy version of the “truth” available to agents connected through a social network. Under which conditions would the agents reach consensus about their beliefs? What would ensure truthful consensus (meaning that the consensus coincided with the truth)? How long would it take for the process to converge? Addressing these questions requires agreeing on a formal model of distributed learning. Fully rational agents update their beliefs by assuming a prior and using Bayes’ rule to integrate all past information available to them (Acemoglu et al., 2011; Mueller-Frank, 2013; Lobel and Sadler, 2015; Mossel et al., 2011; Banerjee, 1992; Bala and Goyal, 1998). Full rationality is intractable in practice (Molavi et al., 2015; Rahimian and Jadbabaie, 2016a), so much effort has been devoted to developing computationally effective mechanisms, including non- (or partially) Bayesian methods (Jadbabaie et al., 2012; Molavi et al., 2015; Golub and Jackson, 2010, 2012; Jadbabaie et al., 2013). Much of this line of work can be traced back to the seminal work of (DeGroot, 1974) on linear opinion pooling.
As the simplest example of iterated learning in a social network, consider a system consisting of one teacher and one learner. The teacher samples data from a distribution and sends it to the learner; the learner updates her belief via Bayes rule repeatedly in order to learn that distribution. This system is equivalent to the usual Bayesian inference scenario. Under mild assumptions, the learner will eventually learn the ground truth asymptotically (Gelman et al., 2013).
∗. Chu Wang did this work prior to joining Amazon
c
When the network structure comes into play, the dynamics of the learning process becomes more complicated. If the social network forms a chainX, Y, Z, . . .such that agents teach each other in sequence: X teaches Y, who then teaches Z, and so on, by a classic result of Griffiths and Kalish (Griffiths and Kalish, 2005), the information from the source will vanish after a finite number of iterations. At that point, the agents, assumed to be rational, will be “teaching” each other nothing they don’t already know: iterated learning is not self-sustaining (Beppu and Griffiths, 2009; Griffiths and Kalish, 2007, 2005; Rafferty et al., 2009; Kirby et al., 2014; Griffiths et al., 2008; Perfors and Navarro, 2011; Rafferty et al., 2014; Smith, 2009; Kalish et al., 2007). Such findings are hard to validate empirically but variants of it are within the reach of experimental psychology (Kalish et al., 2007). Similar laboratory experiments with human subjects have confirmed the unstainability of iterated learning (Kalish et al., 2007; Beppu and Griffiths, 2009; Tamariz and Kirby, 2015; Bartlett; Griffiths et al., 2008). Similar results of unstainability are found in computational linguistics where, instead of agents sending information, the scenario is about language evolution through generations (Rafferty et al., 2009).
If agents interact in a more complicated and dynamic network, more possibilities of the learning dynamics will emerge. Dynamic networks are common occurrences in opinion dynamics (Hegsel-mann and Krause, 2002; Mohajer and Touri, 2013; Chazelle and Wang, 2016; Chazelle, 2015), but, to our knowledge, somewhat new in the context of social learning. Following the Bayesian-Without-Recall(BWR) model proposed by Rahimian and Jadbabaie in (Rahimian and Jadbabaie, 2016a), we assume the agents to be memoryless and rational: this means that they use Bayesian updates based on current beliefs and signals with no other information from the past, see also (Rahimian et al., 2015a,b; Rahimian and Jadbabaie, 2016b). The BWR model seeks to capture the benefits of rational behavior while keeping both the computation and the information stored to a minimum (Rahimian and Jadbabaie, 2016b).
We focus in this paper on sustainable learning: the conditions to ensure arbitrarily small informa-tion loss in truthful consensus (formal definiinforma-tion in the following secinforma-tions). For chained learning, we show how keeping the length of the training sessions (number of samples transferred) growing slightly allows iterated learning to be sustained in perpetuity. This resolves the paradox raised from language evolution models (Kalish et al., 2007). We further analyze the case when the learner has the ability to reach back to her early ancestors for “fresher” data instead of listening to her direct ancestor, and show how this “hopped” learning mechanism further helps prevent information decay. For the case when the learning network changes over time, we show that under the assumption that each agent hears a noisy signal from the truth at a frequency bounded away from zero, the system reaches truthful consensus almost surely with a convergence rate polynomial in expectation. The relation between the convergence rate and the graph structure is also revealed with a seemingly counter-intuitive finding that agents in a better connected network learn more slowly.
We first introduce the iterated learning model framework with the notation, definitions, and basic properties in Section 2. Then we exam the scenarios of chained learning, hopped learning, and networked learning in Section 3, Section 4, and Section 5, respectively. We show our main results in each section, followed by details of the proofs and discussions.
2. Models, Preliminaries, and Notation
over a state spaceΘ, which is denoted byµt,i. The interactions between agents are modeled by an
infinite sequence(Gt)t≥0, where eachGtis a directed graph over the node set{1, . . . , n}. An edge
pointing fromitojinGtindicates thatireceives data fromjat timet. Intuitively, the direction
of the edge has the same meaning as the “listen-to” activity. Typically, the sequence of graphs is specified ahead of time or is chosen randomly: the only condition that matters is that it should be independent of the randomness used in the data generating and learning process; specifically, taking expectations and variances of the random variables that govern the dynamics will assume a fixed graph sequence (possibly random). The adjacency matrix ofGtis denoted byAt: it is ann×n0/1
matrix.
2.1. The existence of an information source
When an information source exists whose belief is fixed, we label it agent0and refer to it as the truth. In such a case, the graphGtis over the node set{0,1, . . . , n}. Because agent0(if it exists)
holds the truth, no edge out of it points to another node. The adjacency matrix then becomes an (n+ 1)×(n+ 1)matrix whose first row is(1,0, . . . ,0), with a self-loop at agent 0 for simplicity.
2.2. Data generation
At timet≥0, each agenti >0samples a stateθt,i ∈Rconsistent with her own belief:θt,i∼µt,i.
A noisy measurementat,i =θt,i+εt,iis then sent to each agentjsuch that(At)ji= 1. All the noise
termsεt,i are samplediidfromN(0, σ2). An equivalent formulation is to say that the likelihood
functionl(a|θ) is drawn fromN(θ, σ2). In our setting, agentisends the same data to all of her neighbors; this is done for notational convenience and the same results would still hold if we were to resample independently for each neighbor. Except for the omission of explicit utilities and actions, our setting is easily identified as a variant of the BWR model of (Rahimian and Jadbabaie, 2016a).
2.3. Beliefs update
A single-step update for agent i > 0 consists of setting µt+1,i as the posterior P[µt+1,i|d] ∝
P[d|µt,i]P[µt,i], wheredis the data from the neighbors ofireceived at timet. For the case when the
beliefs are Gaussian, we get the classical update rules from Bayesian inference by plugging in the corresponding normal distribution (Box and Tiao, 2011). Updated beliefs remain Gaussian so we can use the notationµt,i ∼ N(xt,i, τt,i−1), whereτt,i denotes the precision (inverse variance)σt,i−2.
Writingτ =σ−2and lettingdt,idenote the outdegree ofiinGt, for anyi >0andt≥0, we have
(
xt+1,i= (τt,ixt,i+τ at,1+· · ·+τ at,dt,i)/(τt,i+dt,iτ);
τt+1,i=τt,i+dt,iτ,
(1)
whereat,1, . . . , at,dt,iare the signals received by agentifrom its neighbors at timet.
2.4. The influence of the graph sequence
The graph sequenceGtplays a crucial role in the dynamics of the system. A simple starting example
scenario. Instead of exhausting all possible graph sequences, in this paper, we focus on three representative types, namely thechained learning,hopped learning, andnetworked learningmodels. The fundamental problem we would like to solve is whether the system is able to converge to the truth; in other words, whether the truthful information is able to propagate uncorruptedly across the entire system.
3. Chained Learning
Following (Beppu and Griffiths, 2009; Griffiths and Kalish, 2007, 2005; Rafferty et al., 2009; Kirby et al., 2014; Griffiths et al., 2008; Perfors and Navarro, 2011; Rafferty et al., 2014; Smith, 2009; Kalish et al., 2007), we begin withchained iterated learning: a learner’s state of belief is modeled by a distribution over a hypothesis spaceH, which is itself equipped with a likelihood function:
P[d|h]indicates the probability of generating datad∈ Dgiven the hypothesish∈ H. A learner’s
state of belief may change after a learning process, and we naturally call her belief before and after the learning prior and posterior. Notice that the hypothesis spaceHis the same as the state space Θ, but we will useHto emphasize the application background and avoid confusion. The initial hypothesishinitgeneratesm1itemsiidfor the first learner. These items provide the training data d1 = (d1,1, . . . , d1,m1)with which the first learner Bayes-updates its prior. Its posterior is given by
settingt= 1in this formula:
P[h|dt] =P[dt|h]P[h]/P[dt], withP[dt] =
X
h∈H
P[dt|h]P[h]. (2)
From that point on, each successive learner updates its prior from their predecessor. For anyt >1, learnertreceivesmtitems sampled by the posterior of agentt−1to form the training setdt. To do
that, she picks a random hypothesishfromHwith probabilityP[h|dt−1](the posterior of learner t−1) and then samplesmtitemsiidfromhto formdt∈ Dmt. The posteriorP[h|dt]is derived
according to (2). Note that learnerthas no direct access to the posterior of learnert−1but only to data drawn from a hypothesis sampled from the posterior. Our formulation assumes a discrete space
Hbut extends to continuous settings, as we show in §3.5.
In the case of linguistic transmission, each hypothesish∈ His a “knob” whose setting is given by a number between 0 and 1, specifically the prior probabilityP[h]. All learners share the same
prior. Picking somehfrom that prior specifies alanguage(also denotedhfor convenience). In this case, a language is defined as a probability distribution overD, interpreted here as a set ofsentences. In this way, the prior can be viewed as a mixture overH: by abuse of terminology, we call it amixed hypothesis, which we distinguish from apurehypothesis of the formh∈ H(corresponding to a single-point distribution). Access to languagehis achieved by random sampling: the sentenced∈ D
is picked with probabilityP[d|h].
Iterated learning proceeds as follows. After selecting languagehwith probabilityP[h|dt−1], learner t collects mt independent samples from h. Thus, given a tuple dt = (d1, . . . , dmt) of
sentences fromD, the likelihoodP[dt|h]is equal toQ1≤k≤mtP[dk|h]. The learner is now ready
Suppose thatD={d1, . . . , ds}andH={h1, . . . , hn}are both finite. After observing the data
generated by the posterior of learnert−1, if learnertwinds up choosinghi then, by Bayesian
updating, the probabilityPij|tthat its posterior pickshjis given by:
Pij|t= X
d∈Dmt
P[hj|d]P[d|hi] =
X
d∈Dmt
P[d|hi]P[d|hj]P[hj]
Pn
k=1P[d|hk]P[hk]
. (3)
To our knowledge, the entire literature on the topic assumes a common, fixed sample size for all the learners:mt=m. Equation (3) can be then interpreted as marginalizing a Gibbs sampler over
the data space, which creates a Markov chain over the hypothesis spaceH: ifhtdenotes the row
vector formed by thenprobabilitiesP[hk|dt], thenht=ht−1Pt, whereh0 =hinit. Assuming
ergodicity (in this case, a fairly inconsequential technical assumption), the chain can be shown to converge to a unique stationary distributionh. It can be easily checked that it coincides with the prior:h= (P[h1], . . . ,P[hn])(Griffiths and Kalish, 2005; Norris, 1998); see (Rafferty et al., 2009,
2014) for an analysis of the mixing time in specific linguistic scenarios. This convergence reveals the long-term unsustainability of iterated learning. We show how diversifying the sample sizesmt,
hence making the Markov chain time-inhomogeneous, can overcome this weakness. In particular, we prove that it is sufficient formtto increase logarithmically with respect totin order to achieve
sustainability.
3.1. Self-Sustainability
We show how to make iterated learning self-sustaining in the presence of a finite hypothesis space
H = {h1, . . . , hn}. This involves specifying a sequence of training session lengthsm1, m2, . . . so that the posterior of any learner ends up differing fromhinitby an arbitrarily small amount. Formally, given anyδ, ε≥0, we say that iterated learning is(δ, ε)-self-sustainingif, with probability at least1−ε, a randomh∈ Hpicked from any learner’s posterior distribution differs fromhinitin total variation by at mostδ. We recall a few facts: the hypothesishdenotes a language modeled as a probability distribution overD; the total variation distance is half the`1-norm; and the posterior of learnertafter thet-th iteration is defined by marginalizingP[h|dt]over all samplesdtdrawn from a
randomhpicked from the posterior of learnert−1(orhinitift= 1). As a shorthand, we speak of ε-self-sustainability to refer to the caseδ = 0.
The parametersδandεallow us to distinguish between two metrics: the distance between two languages overDand the distance between two mixtures over H. The two notions could differ widely. For example, if all ofHcorresponds to languages very close tohinit, to achieve(δ, ε)-self-sustainability might be easy for a tinyδ >0but hopelessly difficult forδ = 0. The complexity of iterated learning depends on the geometry of the languages formed by the pure hypotheses. This is best captured by introducing a metric that, though more specialized than the total variation (it works only on the simplex of probability vectors), brings all sorts of technical benefits: theroot-sine distancebetween two probability distributionsu = (u1, . . . , us)andv = (v1, . . . , vs)overDis
defined as
dRS(u,v) =
v u u t 1 2
s
X
i,j=1
√
uivj−
√
ujvi
2 =
v u u t1−
Xs
i=1
√
uivi
2
Note that theroot-sine distancewill be used to measure similarities between two likelihoods, and we will continue the analysis of sustainability defined based on the total variation distance.
It would be surprising if this distance had not been used before, but we could not find a reference. We prove that it is indeed a metric in the Appendix and also explain its name. We show that it is related to the Hellinger, Bhattacharyya and total variation distances,dH,dB,dT V by the following
relations:
dH =
r
1−q1−d2
RS;
dB=−12ln(1−d2RS) ;
dT V ≤
√
2s dRS.
(5)
3.2. The results
We focus on the “pure” casehinit∈ H, and later briefly discuss how to generalize the method to mixed hypotheses. Using the shorthanddij fordRS(P[·|hi],P[·|hj]), we definedi := minj:j6=idij.
Letp = (p1, p2, . . . , pn)be the prior distribution overH, wherepi := P[hi]. We can obviously
assume that eachpiis positive and that all the pure hypotheses are distinct, hencedi >0. The two
theorems below assume thathinit=h1.
Theorem 1. For any positiveε <1, the following sample size sequence makes iterated learning ε-self-sustaining:
mt=
4 d21ln
nt ε p1
= 4 d21
logt
ε+C
,
for someC >0independent oft, ε,d1.
The factor4can be reduced to21+o(1)if we adjust the constantC. It is to be expected that the lengths of the training sessions should grow to infinity asp1tends to zero, as the vanishing prior makes it increasingly difficult for the posteriors to “attach” toh1. The session lengths are sensitive to the minimum distance between the languages specified byHand the target languageh1. Settling for (δ, ε)-self-sustainability allows us to remove this dependency.
Theorem 2. For any positiveδ, ε <1, the following sample size sequence makes iterated learning (δ, ε)-self-sustaining:
mt=
8sn2 δ2
lnt
ε+C
.
3.3. The proofs
To establish Theorem 1, we estimate the probabilityP∗that each leaner ends up pickingh1. Recall thathtis the posterior distribution of learnert, by the Markovian property of the system,
P∗=P[h0 =h1] Y
t≥0
P[ht+1 =h1|ht=h1] = Y
t≥1
P11|t. (6)
Since the matrixP|tis the transition matrix of a Markov chain, we proceed by bounding its off-diagonal elementsPij|tfori6=j. We have
Pij|t≤ X d∈Dmt
P[d|hi]P[d|hj]pj
P[d|hi]pi+P[d|hj]pj
= pj pi
X
d∈Dmt
pi
pj
P[d|hi]P[d|hj] pi
pj
P[d|hi] +P[d|hj]
≤ 1 2 rp j pi X
d∈Dmt
q
P[d|hi]P[d|hj] =
1 2 rp j pi X d∈D q
P[d|hi]P[d|hj]
mt ≤ 1 2 rp j pi exp mt 2 X d∈D q
P[d|hi]P[d|hj]
2
−1
,
where the two equalities are simple deformations, the first inequality is achieved by dropping some non-negative terms from the definition ofPij|tin (3), the second inequality is obtained via Young’s inequality, and the last inequality is from Taylor expansion of the natural logarithm function at 1. By definition of the root-sine distance, we have
Pij|t≤ 1
2 rp
j
pi
e−12d 2
ijmt (i6=j). (7)
Settingi= 1in (7) and summing over2≤j≤n, it follows by Cauchy-Schwarz that
n
X
j=2
P1|tj ≤ 1
2 s
n(1−p1) p1
e−12d 2
1mt. (8)
Combining (6) and (8) yields
P∗≥Y
t≥1
1−1
2 s
n(1−p1) p1
e−12d 2 1mt
!
≥1−1
2 s
n(1−p1) p1
X
t≥1 e−12d
2
1mt. (9)
Given0< ε <1, we constrain the sequence(mt)to satisfy:
X
t≥1 e−12d
2 1mt < ε
s 4p1 n(1−p1)
. (10)
For example, we can pick the sequence
mt=
1 d21 ln
n(1−p1)t4 ε2p
which completes the proof. A closer look at the calculation shows that the factort4can be reduced toCαt2+αfor any smallα >0and a suitable constantCα >0, which makes the dependency ont
arbitrarily close to(2/d21) lnt.
To prove Theorem 2, we set a target distanceρ:=δ/(n√2s)and find a subsetA⊆ Hsuch that (i)d1j ≤ρnforj ∈Aand (ii)dij ≥ρfori∈Aandj6∈A. To see why such a subset must exist,
consider spheres centered athinit=h1of radiuskρ, fork= 1, . . . , n+ 1(with respect todRS).
These definen+ 1disjoint (open) regions and, by the pigeonhole principle, at least one of them must be empty. We setAto include all the points in the regions preceding the empty one; note thath1 ∈A. The claim follows from the triangular inequality. We begin with a straightforward generalization of (8): for anyi∈A,
X
j6∈A
Pij|t≤ 1
2 s
n(1−pA)
pA
e−12ρ 2m
t, (11)
wherepA:= mini∈Api. Now letP∗be the probability thatht∈Afor eacht, then (6) and (9) are
generalized to
P∗≥Y
t≥1
1−max
i∈A
X
j /∈A
Pij|t
≥1− 1 2
s
n(1−pA)
pA
X
t≥1 e−12ρ
2m
t. (12)
Setting
mt=
1 ρ2ln
n(1−pA)t4
ε2p
A
(13)
ensures thatP∗ >1−ε. The root-sine distance between the languages denoted byh1and anyh∈A is at mostρn, so that, by (5), the total variation distance is bounded by√2sρn=δ, which concludes the proof of Theorem 2.
So far, we have analyzed only the “pure” casehinit∈ H. The idea of the training is to prevent the prior to “drag” the posterior mixture all acrossH. It should be clear that a similar result obtains if hinit∈∆His concentrated on a subsetAofH. The proof follows the path charted in Theorem 2 and need not be repeated here. It is crucial to note, however, that this result is to be understood in a coarse-graining sense: iterated learning cannot ensure that the original weights in the mixturehinit are retained but only thatAcontributes most of the mass in the posteriors. To retain the weights would require changing the stationary distribution to conform withhinit, as the process unfolds, something that straightforward Bayesian learning seems unable to do. Learning pure hypotheses bypasses that difficulty.
3.4. Applications
We briefly discuss a direct application of our results to a well-known model of language acquisition via iterated learning and we mention some natural extensions of the techniques.
language whose sentences are words in{0,1,?}kwith exactlymquestion marks and0,1matching
the binary decomposition ofi−1outside the question marks. For example, ifk = 4andm = 2, thenh3denotes the language
{00??,0?1?,?01?,0??0,?0?0,??10}.
We can assume thatmis much smaller thank. Each language has the same length mk
and the total number of sentences iss= mk
2k−m. The prior is given byP[hi] =pi= 1/n. Given a hypothesis
hi,P[d|hi] = 1/ mk
ifdhasmquestion marks and match the bits ofi−1elsewhere; else it is 0 (andd, hare called incompatible). Givenh∈ H,
P[d] =Ph∈HP[d|h]P[h] = 2m−k/ mk
;
P[h|d] =P[d|h]P[h]/P[d] = 2−m (or0ifd, hare incompatible).
We easily check thatd21 = 1− Ps i=1
√
aibi
2
≥1− m
k
2
> 12; hence, by Theorem 1, session lengthsmtno larger thanO(logtε)are sufficient to maintainε-self-sustainability.
Meanings and utterances. In the use of iterated learning for studying language evolution (Griffiths and Kalish, 2005; Perfors and Navarro, 2011), it is common to model the datadas a joint distribution (x,y)over a product spaceXmt × Ymt. The idea is to distinguish between “meanings” xand
“utterances”y. In this setting,P[d|h] =P[y|x, h]µ(x), whereµ(x)is the probability of generating
x. The transition matrix of the Markov chain thus becomes
Pij|t= X
x∈Xmt
X
y∈Ymt
P[hj|x,y]P[y|x, hi]µ(x)
= X
x∈Xmt
X
y∈Ymt
P[y|x, hi]P[y|x, hj]P[hj]
Pm
k=1P[y|x, hk]P[hk]
µ(x).
(14)
Since the outputynow depends on both the hypothesis and the input data, we redefinedij as the
root-sine distance between the two distributionsP[y|x, hi]µ(x)andP[y|x, hj]µ(x):
d0ij
2
:= 1−
X x∈X X y∈Y q
P[y|x, hi]P[y|x, hj]µ(x)
2
(15)
and we defined0i:= minj:j6=id0ij. Given anyi6=j,
Pij|t≤ X
x∈Xmt
X
y∈Ymt
P[y|x, hi]P[y|x, hj]pj
P[y|x, hi]pi+P[y|x, hj]pj
µ(x)
≤ 1 2 r pj pi X x∈Xmt X y∈Ymt q
P[y|x, hi]P[y|x, hj]µ(x)
≤ 1 2 r pj pi X x∈X X y∈Y q
P[y|hi]P[y|hj]µ(x)
mt ≤ 1 2 rp j pi exp mt 2 X x∈X X y∈Y q
P[y|x, hi]P[y|x, hj]µ(x)
2
−1
This gives us this new version of inequality (7), which we can use as the basis for a repeat of the argument of the previous section:
Pij|t≤ 1
2 r
pj
pi
e−12d
02
ijmt (i6=j). (16)
3.5. Gaussian chained learning
When iterated learning operates over a hypothesis spaceHparametrized continuously, say, inR, the
minimum root-sine distance usually vanishes and the previous arguments run into singularities and collapse. A new approach is needed. To make our discussion concrete, we assume that the prior distribution of each learner is a GaussianP[h]∼N(¯µ,σ¯2)and that the likelihood of producing data
dgiven hypothesishis also normal: P[d|h] =N(h, σ2). The likelihood can also be understood as
a noisy measurement ofh: d= h+φ, where the noiseφ ∼N(0, σ2). We assume that the data received by the first learner comes fromN(µ0, σ02). This is the simplest instance of a continuous setting in which the root-sine distance argument fails. We discuss it in some detail, considering both chained learning and its generalizations; and then we use the results to treat the case of iterated Bayesian linear regression.
During its training session, thet-th learner receives datadt= (dt,1, . . . , dt,mt)from its
prede-cessor: it is obtained by first picking a random hypothesishfrom the posterior of learnert−1and then collectingmtindependent random samples fromN(h, σ2). For the caset= 1, we can treat
the original teacher as learner0with its posterior equal toN(µ0, σ20). LearnertBayes-updates its posterior as follows:
P[h|dt]∝P[dt|h]P[h]∝exp
− 1
2σ2
mt
X
i=1
(dt,i−h)2
exp
− 1
2¯σ2(h−µ)¯ 2,
which is still Gaussian, with mean and variance denoted byµtandσt2, respectively. Carrying out the
usual square completion gives up these update rules: fort >0,
µt= τ¯+1mtτ (¯τµ¯+τ(dt,1+dt,2+· · ·+dt,mt))
τt = ¯τ +mtτ,
(17)
where we define the precisionsτ = 1/σ2,τ¯= 1/¯σ2, andτt= 1/σt2. We say that iterated learning is
ε-self-sustainingif|Eµt−µ0| ≤εandσt2+ varµtremains bounded for allt. Ifσt2+ varµt→0as
t→ ∞, we say that iterated learning isstronglyε-self-sustaining. We consider successively the case of chained iterated learning and the more challenging “hopping” scenario in which a new learner picks a random teacher from the past (instead of the previous one).
In chained iterated learning, the datadt,i is a noisy message drawn from the posterior of the
(t−1)-th learner; hencedt,i ∼N(µt−1, σ2t−1+σ2). In view of (17),µtis itself Gaussian. By taking
the expectation and variance of equation (17), we find the following recursive relations forEµtand
varµt: fort >0,
Eµt= τ¯+1mtτ ¯τµ¯+mtτEµt−1
;
varµt= mtτ 2
(¯τ+mtτ)2 var µt−1+σ
2
t−1+σ2
.
If we defineβt:=mtτ /(¯τ +mtτ), then (18) becomesEµt =βtEµt−1+ (1−βt)¯µ. Ifmt=m
is a constant, then so isβt, and the recursive relation (18) becomes
Eµt−µ¯=β1t(µ0−µ),¯
which shows thatEµtconverges toµ¯exponentially fast. As in the discrete case, iterated learning is
not self-sustainable with constant-length training sessions. By lettingmtincrease ast1+o(1) order,
however, we can achieve self-sustainability:
Theorem 3 For any0< ε <1, the following sample size sequence makes chained iterated learning stronglyε-self-sustaining:
mt=
|µ0−µ¯| ε
1 +1
c σ
¯ σ
2 t1+c,
for an arbitrarily small constantc >0.
Proof We observe thatEµtis a convex combination ofµ¯andEµs(s < t); specifically,
Eµt= t
Y
s=1
βsµ0+
1−
t
Y
s=1 βs
¯
µ. (19)
BecauseP
s>0(1/s)1+c<1 + R∞
1 x
−1−cdx= 1 + 1/c, we have
1 ≥
t
Y
s=1 βs=
t
Y
s=1
1− τ¯
msτ + ¯τ
≥1−
t
X
s=1 ¯ τ msτ + ¯τ
≥ 1− ε
|µ0−µ¯| c
c+ 1 X∞
s=1 1
s1+c >1−
ε
|µ0−µ¯| .
This shows that
|Eµt−µ0|=
1−
t
Y
s=1 βs
|µ¯−µ0| ≤ε.
By (17),σ2t = 1/τt<1/mtτ →0. Sinceσ2t−1 ≤σ¯2fort >1, it follows from (18) thatvarµt≤
(varµt−1+σ2+ ¯σ2)/mtfort >1, andvarµ1 ≤(σ02+σ2)/m1. WritingMt:=mtmt−1. . . m1, we have
Mtvarµt≤Mt−1varµt−1+Mt−1(σ2+ ¯σ2)
≤tMt−1(σ20+σ2+ ¯σ2),
and thusvarµt≤(σ20+σ2+ ¯σ2)t/mt→0sincemt= Ω(t1+c).
3.6. Iterated Bayesian Linear Regression
the way of theoretical analysis. This section is a first step toward filling this gap. The task at hand is to estimate a hypothesis h ∈ H := Rdgiven a noisy measurements on the hyperplane
y =hTx, wherex ∈Rd. In the Bayesian setting, we assume a Gaussian prior on the hypothesis
space: P[h]∼ N(¯µ,¯σ2Id). The data is given by(x, y), wherex ∼ N(0, Id)andy = hTx+φ,
for φ ∼ N(0, σ2) (withx, φ independent). Since we typically make several measurements, we write this (likelihood) relation in matrix form: y =Xh+φ, wherey ∈Rm (withmthe number
of measurements);φ ∼ N(0, σ2Im); andX is anm-by-dmatrix each of whose rows denotes a
random vectorx∼N(0, Id). This means that the matrixXis random (a fact of key importance in
our discussion below). We have:
P[φ]∼ exp−2σ12kφk22 (noise)
P[h]∼ exp−2¯σ12kh−µ¯k22 (prior)
P[y|X, h]∼ exp−2σ12ky−Xhk22 (likelihood)
In iterated Bayesian linear regression, the t-th learner receives her data from learnert−1. Here, learner0is treated just like any other agent, except that his priorP[h]∼N(µ0,σ¯2Id)is the
distribution to be learned iteratively. Since sampling from the prior is independent ofX, Bayesian updating gives the posteriorN(µt,Σt), where
P[h|X, y] =P[h]P[y|X, h]/P[y|X]∼exp
n
− 1
2¯σ2kh−µ¯k 2 2−
1
2σ2ky−Xhk 2 2 o
.
Completing the square in the usual fashion shows that the posterior of learnertis given by:
Σt= ¯σ−2Id+σ−2XtTXt
−1 ;
µt= Σt σ¯−2µ¯+σ−2XtTyt
,
(20)
where(Xt, yt)is the data gathered by learnertfrom her predecessor: specifically,yt=Xth+φt,
wherehis collected from the(t−1)-th learner by sampling his posterior distributionN(µt−1,Σt−1).
Theorem 4 Given any small enoughδ, ε > 0, the following sample size sequence for iterated Bayesian linear regression ensures thatkEµt−µ0k2≤δwith probability greater than1−ε:
mt=Dc
kµ0−µ¯k2 δ
σ ¯ σ
2
t1+c+Dcdlog
t+ 1 ε , for an arbitrarily smallc >0and a constantDcthat depends only on c.
Proof We proceed in two steps: first, we show that to keepEµtarbitrarily close toµ0for allthinges on spectral properties of certain random matrices; second, we call on known facts about the singular values of random Gaussian matrices to translate the spectral condition into a high-probability event. The proof unfolds as a series of simple relations, which we state first and then demonstrate. The first one follows directly from (20):
Eµt= (Id+Mt)−1(¯µ+MtEµt−1), where Mt:=
σ¯ σ
2
Note that (21) is a randomized recursive relation since the data pointsX1, X2, . . . are themselves random. We note that all the matrices whose inverses are taken are positive definite, hence nonsingular. To move on to our second relation, we define the matrix
Qt:= (Id+Mt)−1Mt(Id+Mt−1)−1Mt−1· · ·(Id+M1)−1M1,
fort >0, withQ0 =Id, and prove by induction that
Eµt=Qtµ0+ (Id−Qt)¯µ. (22)
The base case is obvious so we assume thatt >0: by (21),
Eµt= (Id+Mt)−1(¯µ+MtEµt−1)
= (Id+Mt)−1(¯µ+MtQt−1µ0+Mt(Id−Qt−1)¯µ)
= (Id+Mt)−1MtQt−1µ0+ (Id+Mt)−1(Id+Mt(Id−Qt−1))¯µ =Qtµ0+ (Id−(Id+Mt)−1MtQt−1)¯µ,
which proves (22). Our next goal is to bound the information decaykEµt−µ0k2. To do that, we investigate the spectral norm of the matrixId−Qt, which leads to our third relation. We prove by
induction that, fort >0,
kId−Qtk2 ≤
t
X
s=1
kAsk2, (23)
whereAs:= (Id+Ms)−1. Fort= 1,Q1= (Id+M1)−1M1 =Id−(Id+M1)−1and the claim follows. Ift >1, then
kId−Qtk2 =k(Id−Qt−1) + (Qt−1−Qt)k2
≤ kId−Qt−1k2+kQt−Qt−1k2≤
t−1 X
s=1
kAsk2+kΨk2,
whereΨ := (AtMt−Id)Qt−1. SinceAt(Id+Mt) =Id, we haveΨ =−AtQt−1. Each matrixMs
is positive semidefinite, so the eigenvalues of(Id+Ms)−1Msare of the formλ/(1 +λ), where
λ≥0. This shows that all the eigenvalues ofQsare between 0 and 1; thereforekQsk2 ≤1. The eigenvalues of Id−AtMt are the same as those of At; hence, by submultiplicativity, kΨk2 ≤
kAtk2kQt−1k2 ≤ kAtk2, which establishes (23).
We are now ready to express the information decay in spectral terms. Pick an arbitrarily small constantc >0and assume that
kAsk2 ≤ δ
kµ¯−µ0k2 c
1 +c 1
s 1+c
. (24)
By (22),Eµt−µ0 = (Id−Qt)(¯µ−µ0); therefore, by (23),
kEµt−µ0k2 ≤ kµ¯−µ0k2
t
X
s=1
kAsk2 ≤ δc 1 +c
t
X
s=1 s−1−c
≤ δc
1 +c
1 + Z ∞
1
x−1−cdx=δ,
The relation says that, on average, the means of any of the agents’ posteriors can be brought as close to the original mean to be learned as we want. We can turn this into a high-probability event by using some basic random matrix theory. Recall that Eµt is itself a random variable whose
stochasticity comes from the matricesXs, which are all drawn from Gaussians. BecauseMs is
positive semidefinite,
kAsk2 ≤ kMs−1k2 ≤
(σ/¯σ)2 λmin(XtTXt)
≤ σ/¯σ
σ1(Xt)
2
, (26)
which gives us a relation between the spectral norm of(Is+Ms)−1and the smallest singular value
σ1(Xt)of anmt-by-dmatrixXtwhose elements are drawniidfromN(0,1). The asymptotic
behav-ior ofσ1(Xt)for large values ofmthas been extensively studied within the field of random matrix
theory (Davidson and Szarek, 2001; Edelman, 1988; Rudelson and Vershynin, 2009). Following Theorem II.13 in (Davidson & Szarek (Davidson and Szarek, 2001)), for anyγt>0,
P[σ1(Xt)<
√
mt−
√
d−γt]≤e−γ 2
t/2.
We useCbelow as a generic constant large enough to satisfy the inequalities where it appears. Setting γt=C
p
log((t+ 1)/ε)ensures thatP
t>0e−γ
2
t/2 < ε, hence thatσ1(Xt)<√mt−
√
d−γtholds
for alltwith probability less thanε. With our setting ofmt, this means that, for allt >0,
P
h
σ1(Xt)≥
√
mt
2 i
>1−ε. (27)
Assuming the event in (27), it follows from (26) and our setting ofmtthat
kAtk2 ≤ 4 mt
σ ¯ σ
2
≤ δ
kµ¯−µ0k2 4
Dc
1 t
1+c ;
hence (24) holds forDclarge enough. By (25, 27), this proves that, with probability greater than
1−ε,kEµt−µ0k2 ≤δfor allt >0, which completes the proof.
4. Hopped Learning
In this section, we consider the “hopped learning” scenario in which learnerthops back to pick a teacher from{0,1, . . . , t−1}at random, and then takesmtsamples from her posterior. To model
the data generating process, we continue to adopt the Gaussian setting from Sections 3.5 and 3.6. Note that the graph sequenceGtbecomes random under the constraint that a learner can only get
data from the learners before her. Since multiple samples can be sent from a teacher to a learner, we usedt,s,ito denote thei-th sample generated by agentsat timet. Note that though a learner only
receives data from one teacher, without loss of generality, we assume all her predecessors generate samples but she only listens to one of them. The recursive relation forµtbecomes
µt=
βt
mt t−1 X
s=0 χt,s
mt
X
i=1
dt,s,i+ (1−βt)¯µ, (28)
where, givent, the random variableχt,sis 1 for a value ofspicked at random between0ands−1,
to earlier data, so one would expect the lengths of the training sessions to grow more slowly than in chained learning. The change is indeed quite dramatic:
Theorem 5 For any positive ε < |µ0 −µ¯|, the following sample size sequence makes hopped
iterating learningε-self-sustaining:
mt=Bc
|µ0−µ¯| ε
σ ¯ σ
2
(1 + logt)1+c,
for an arbitrarily smallc >0and a constantBcthat depends only onc.
Proof By taking expectation on both sides of (28), for anyt >0,
Eµt=
βt
t
t−1 X
s=0
Eµs+ (1−βt)¯µ,
We defineγ1 =β1 and, fort >1,
γt:= (1 +β1)
1 +β2 2
· · ·
1 + βt−1 t−1
βt
t .
We verify easily thatEµt = γtµ0+ (1−γt)¯µ, fort > 0; therefore, the first part in establishing
ε-self-sustainability consists of proving that
1≥γt≥1−
ε
|µ0−µ¯|
, (29)
which will show that|Eµt−µ0| ≤ε. Note that
γt≤
1 t t−1 Y s=1
1 +1 s
= 1.
Now define
αs =
ε
Bc|µ0−µ¯|s(1 + logs)1+c .
fors >0. We pick a constantBclarge enough so thatαsis small enough to carry out first-order
Taylor approximations around1 +αs. We find that
1 +βs s = 1 +
1 s
1− 1
1 +msτ /¯τ
≥1 +1 s
1− 1
(s+ 1)msτ /¯τ
≥1 +1 s
1− sαs
s+ 1
≥1 +1 s
(1−αs)≥
1 +1
s
e−2αs.
Thus,
γt≥
βt t t−1 Y s=1
1 +1 s
e−2Pts−=11αs =β
te−2 Pt−1
s=1αs ≥1− ε
|µ0−µ¯| ,
which establishes (29). Our derivation relies on the fact that
βt≥1−
ε
Bc|µ0−µ¯|(1 + logt)1+c
≥1− ε
and
t−1 X
s=1
1
s(1 + logs)1+c ≤1 +
1 (loge)1+c
Z t−1
2
1
x(lnx)1+cdx=O
1 c
;
hence,
e−2Pts−=11αs ≥e−O(ε/(cBc|µ0−µ|¯))≥1− ε
2|µ0−µ¯| .
Having shown that|Eµt−µ0| ≤εfor allt, it now suffices to prove thatσ2t + varµtremains
bounded. We note thatτt> mtτ → ∞, henceσt2 = 1/τt→0, so the remainder of the proof needs
to establish that the variance ofµtstays bounded. WritingDt,s :=dt,s,1+· · ·+dt,s,mt, we have
varDt,s=mtvardt,s,1=mt(σ2s+σ2+ varµs); hence
ED2t,s= varDt,s+ (EDt,s)2 =mt(σs2+σ2+ varµs) +m2t(Eµs)2.
In (28), the variablesχt,s andDt,s are independent, for0 ≤ s ≤ t−1; furthermore, Eχt,s =
Eχ2t,s= 1/t, andEχt,s1χt,s2 = 0ifs1 6=s2; therefore,
var [χt,sDt,s] =Eχ2t,sED2t,s−(Eχt,s)2(EDt,s)2 =
EDt,s2
t −
(EDt,s)2
t2 =mt
t
σ2s+σ2+ varµs+mt(Eµs)2
−mt
t 2
(Eµs)2
(30)
and, fors16=s2,
cov [χt,s1Dt,s1, χt,s2Dt,s2] = E[χt,s1χt,s2Dt,s1, Dt,s2]−E[χt,s1Dt,s1]E[χt,s2Dt,s2]
= E[χt,s1χt,s2]E[Dt,s1Dt,s2]−Eχt,s1EDt,s1Eχt,s2EDt,s2
= −1
t2 EDt,s1EDt,s2 =− mt
t 2
Eµs1Eµs2.
(31)
Then, by taking the variance on both sides of (28), we have
varµt=
βt mt
2 var
t−1 X
s=0
χt,sDt,s
=βt mt
2Xt−1
s=0
var [χt,sDt,s] +
X
0≤s16=s2≤t−1
cov [χt,s1Dt,s1, χt,s2Dt,s2]
= βt
mt
2Xt−1
s=0 mt
t
σ2s+σ2+ varµs+mt(Eµs)2
−mt
t
2Xt−1
s=0
Eµs
2
≤ 1
tmt t−1 X
s=0
σs2+σ2+ varµs+mt(Eµs)2
.
Notice thatσ2s →0and(Eµs)2 is bounded since|Eµt−µ0| ≤ ε. We conclude thatσ2t + varµt
5. Networked Learning
In this section, we study the information transfer and iterated learning with general graph sequence Gt. We assume that the initial beliefµ0,iof agentiis Gaussian: µ0,i∼ N(x0,i, σ02,i). Without loss
of generality, the truth is assumed to be a constant (single-point distribution: µt,0 = 0;σt,0 = 0 for allt) and the standard deviation is the same for all other agents, i.e.,σ0,i =σ0 >0fori >0. Because agent0holds the truth, no edge points out of it. The adjacency matrix ofGtis denoted by
At: it is an(n+ 1)×(n+ 1)matrix whose first row is(1,0, . . . ,0). Note thatnis the number of
learners and should not be confused with the number of hypotheses in Section 3.
5.1. The dynamics in matrix form
LetDtandPtdenote the(n+ 1)-by-(n+ 1)diagonal matrices diag(ηt,i)and(τ0/τ)I+Pt−k=01 Dk,
respectively, whereηt,i is the out-degree of agentiat timet,I is the identity matrix and the sum
is0fort= 0. It follows from (1) thatµt,i ∼ N(xt,i,(τ Pt)−ii1)fori >0. Regrouping the means in
vector form,xt:= (xt,0, . . . , xt,n)T, wherext,0 = 0andx0,1, . . . , x0,nare given as inputs, we have
xt+1= (Pt+Dt)−1(Ptxt+At(xt+ut+εt)), (32)
whereutis such thatut,0 ∼ N(0,0)and, fori >0,ut,i ∼ N(0,(τ(Pt)ii)−1); andεtis such that
εt,0 ∼ N(0,0)and, fori >0,εt,i ∼ N(0,1/τ). We refer to the vectorsxtandyt:=Extas the
mean processand theexpected mean process, respectively. Taking expectations on both sides of (32) with respect to the random vectorsutandεtyields the update rule for the expected mean process:
y0=x0 and, fort >0,
yt+1= (Pt+Dt)−1(Pt+At)yt. (33)
A key observation is that (Pt+Dt)−1(Pt+At) is a stochastic matrix, so the expected mean
processytforms a diffusive influence system (Chazelle, 2015): the vector evolves by taking convex
combinations of its own coordinates. What makes the analysis different from standard multiagent agreement systems is that the weights vary over time. In fact, some weights typically tend to 0, which violates one of the cardinal assumptions used in the analysis of averaging systems (Chazelle, 2015; Moreau, 2005). This leads us to the use of arguments, such as fourth-order moment bounds, that are not commonly encountered in this area.
5.2. The results
The belief vectorµtis Gaussian with meanxtand covariance matrixΣtformed by zeroing out the
top-left element of(τ Pt)−1. We say that the system reachestruthful consensusif both the mean
processxtand the covariance matrix tend to zero astgoes to infinity. This indicates that all the
agents’ beliefs share a common mean equal to the truth and the “error bars” vanish over time. In view of (1), the covariance matrix indeed tends to0as long as the degrees are nonzero infinitely often, a trivial condition. To establish truthful consensus, therefore, boils down to studying the mean processxt. We do this in two parts: first, we show that the expected mean process converges to the
truth; then we prove that fluctuations around it eventually vanish almost surely.1
Truth-hearing assumption: Given any interval of lengthκ:=b1/γc, every agenti >0has an edge (i,0)inGtfor at least one value oftin that interval.
Theorem 6 Under the truth-hearing assumption, the system reaches truthful consensus with a convergence rate bounded byO(t−γ/2η), whereηis the maximum outdegree over all the networks.
We prove the theorem in the next two sections. It will follow directly from Lemmas 7 and 8 below. The convergence rate can be improved to the order oft−(1−ε)γ/η, for arbitrarily smallε >0. The inverse dependency onγis not surprising: the more access to the truth the stronger the attraction to it. On the other hand, it might seem counterintuitive that a larger outdegree should slow down convergence. This illustrates the risk of groupthink. It pays to follow the crowds when the crowds are right. When they are not, however, this distracts from the lonely voice that happens to be right.
How essential is the truth-hearing assumption? We show that it is necessary. Simply having access to the truth infinitely often is not enough to achieve truthful consensus.
5.3. The proofs
In this subsection, we demonstrate technical details for the proof of the results. We begin with some repeated used inequalities.
5.3.1. USEFUL MATRIX INEQUALITIES
We highlight certain matrix inequalities to be used throughout. We use the standard element-wise notationR ≤S to indicate thatRij ≤ Sij for alli, j. The infinity normkRk∞= maxiPj|rij|
is submultiplicative:kRSk∞≤ kRk∞kSk∞, for any matching rectangular matrices. On the other
hand, the max-normkRkmax:= maxi,j|rij|is not, but it is transpose-invariant and also satisfies:
kRSkmax≤ kRk∞kSkmax. It follows that
kRSRTkmax≤ kRk∞kSRTkmax=kRk∞kRSTkmax
≤ kRk2∞kSTkmax=kRk2∞kSkmax. (34)
5.3.2. THE EXPECTED MEAN PROCESS DYNAMICS
We analyze the convergence of the mean process in expectation. The expected meanyt = Ext
evolves through an averaging process entirely determined by the initial valuey0 = (0, x0,1, . . . , x0,n)T
and the graph sequenceGt. Intuitively, if an agent communicates repeatedly with a holder of the
truth, the weight of the latter should accumulate and increasingy influence the belief of the agent in question. Our goal in this section is to prove the following result:
Lemma 7 Under the truth-hearing assumption, the expected mean processytconverges to the truth
Proof We defineBtas the matrix formed by removing the first row and the first column from the
stochasticPt−+11 (Pt+At). If we writeytas(0,zt)then, by (33),
0 zt+1
=
1 0 αt Bt
0 zt
, (35)
where αt,i = (Pt−+11)ii if there is an edge (i,0)at time tand αt,i = 0 otherwise. This further
simplifies to
zt+1 =Btzt. (36)
Let1be the all-one column vector of lengthn. SincePt−+11 (Pt+At)is stochastic,
αt+Bt1=1 (37)
In matrix terms, the truth-hearing assumption means that, for anyt≥0,
αt+αt+1+· · ·+αt+κ−1 ≥Q−t+1κ1, (38)
whereQtis the matrix derived fromPtby removing the first row and the last column; the inequality
relies on the fact thatPtis monotonically nondecreasing. For anyt > s≥0, we define the product
matrixBt:sdefined as
Bt:s:=Bt−1Bt−2. . . Bs, (39)
withBt:t=I. By (36), for anyt > s≥0,
zt=Bt:szs. (40)
To bound the infinity norm ofBt:0, we observe that, for any0≤l < κ−1, thei-th diagonal element ofBs+κ:s+l+1is lower-bounded by
κ−1 Y
j=l+1
(Bs+j)ii= κ−1 Y
j=l+1
(Ps+j+As+j)ii
(Ps+j+1)ii
(41)
≥
κ−1 Y
j=l+1
(Ps+j)ii
(Ps+j+1)ii
= (Ps+l+1)ii (Ps+κ)ii
≥ (Ps)ii
(Ps+κ)ii
.
The inequalities follow from the nonnegativity of the entries and the monotonicity of(Pt)ii. Note
that (41) also holds forl=κ−1since(Bs+κ:s+κ)ii= 1.
Since Pt−+11 (Pt+At) is stochastic, the row-sum of Bt does not exceed 1; therefore, by
pre-multiplyingBs+1, Bs+2, . . . on both sides of (37), we obtain:
Bs+κ:s1≤1− κ−1 X
l=0
Bs+κ:s+l+1αs+l. (42)
Noting thatkBtk∞ = kBt1k∞for anyt, asBtis non-negative, we combine (38), (41), and (42)
together to derive:
kBs+κ:sk∞≤1−min i>0
(Ps)ii
(Ps+κ)2ii
Let η:= maxt≥0max1≤i≤nηt,i denote the maximum outdegree in all the networks, and define
δ= min{τ0/τ,1}. For anyi >0ands≥κ, sδ
κ ≤(Ps)ii≤ηs+ τ0
τ ; (44) hence,
max
i (Ps+κ)ii≤η(s+κ) +
τ0
τ . (45) It follows that
(Ps+κ)ii−(Ps)ii
(Ps+κ)ii
= Pκ−1
l=0 ηs+l,i (Ps+κ)ii
≤ ηκ
2δ−1
s+κ . (46) Thus, we have
min
i>0
(Ps)ii
(Ps+κ)ii
= 1−max
i>0
(Ps+κ)ii−(Ps)ii
(Ps+κ)ii
≥1− ηκ
2δ−1 s+κ .
(47)
We can replace the upper bound of (43) by
1− 1
maxi>0(Ps+κ)ii
min
i>0
(Ps)ii
(Ps+κ)2ii
,
which, together with (45) and (47) gives us
kBs+κ:sk∞≤1−
1 η(s+κ) +τ0/τ
1−ηκ
2δ−1 s+κ
≤1− 1
2ηκ(m+ 2).
(48)
The latter inequality holds as long ass=mκ >0and
m≥m∗:= 2ηκ δ +
τ0 ηκτ. It follows that, form0 ≥m∗,
kB(m0+m)κ:m0κk∞≤ m+1
Y
j=2
1− 1
2ηκ(m0+j)
≤exp
− 1
2ηκ
m+1 X
j=2 1 m0+j
.
(49)
The matricesBtare sub-stochastic so that
kBtzk∞≤ kBtk∞kzk∞≤ kzk∞.
By (40), for anyt≥(m0+m)κ,
so that, by using standard bounds for the harmonic series,ln(k+ 1)<1 +12 +· · ·+1k ≤1 + lnk, we find that
kztk∞≤ kB(m0+m)κ:m0κzm0k∞
≤ kB(m0+m)κ:m0κk∞kz0k∞
≤Ct−1/(2ηκ),
whereC >0depends onz0, κ, η, τ0/τ. We note that the convergence rate can be improved to the order oft−(1−ε)γ/η, for arbitrarily smallε >0, by working a little harder with (48).
5.3.3. THE MEAN PROCESS DYNAMICS
Recall thatµt,i ∼ N(xt,i, τt,i−1), whereτt,i denotes the precisionσt,i−2. A key observation about
the updating rule in (1) is that the precisionτt,i is entirely determined by the graph sequenceGt
and is independent of the actual dynamics. Adding to this the connectivity property implied by the truth-hearing assumption, we find immediately thatτt,i → ∞for any agenti. This ensures that the
covariance matrixΣttends to0astgoes to infinity, which satisfies the second criterion for truthful
consensus. The first criterion requires that the mean processxtshould converge to the truth0. Take
the vectorxt−ytand remove the first coordinate(xt−yt)0to form the vector∆t∈Rn. Under the
truth-hearing assumption, we have seen thatyt→0(Lemma 7), so it suffices to prove the following:
Lemma 8 Under the truth-hearing assumption, the deviation∆tvanishes almost surely.
Proof We use a fourth-moment argument. The justification for the high order is technical: it is necessary to make a certain “deviation power” series converge. By (32),xtis a linear combination of
independent Gaussian random vectorsusandεsfor0≤s≤t−1, and thusxtitself is a Gaussian
random vector. Therefore∆tis also Gaussian and its mean is zero. From Markov’s inequality, for
anyc >0,
X
t≥0
P[|∆t,i| ≥c]≤
X
t≥0
E∆4t,i
c4 . (50)
If we are able to show the right hand side of (50) is finite for anyc >0, then, by the Borel-Cantelli lemma, with probability one, the event|∆t,i| ≥coccurs only a finite number of times, and so∆t,i
goes to zero almost surely. Therefore, we only need to analyze the order of the fourth momentE∆4t,i.
By subtracting (33) from (32), we have:
∆t+1 =Bt∆t+Mtvt, (51)
wherevt := ut+εt andMt := Pt−+11At; actually, for dimensions to match, we remove the top
coordinate ofvtand the first row and first column ofMt(see previous section for definition ofBt).
Transforming the previous identity into a telescoping sum, it follows from∆0 =x0−y0 =0and the definitionBt:s =Bt−1Bt−2. . . Bsthat
∆t= t−1 X
s=0
Bt:s+1Msvs= t−1 X
s=0
whereRt,s :=Bt:s+1Ms. We denote byC1, C2, . . . suitably large constants (possibly depending on κ, η, n, τ, τ0). By (44),kMsk∞≤C1/(s+ 1)and, by (49), for sufficiently larges,
kBt:s+1k∞≤C2(s+ 1)β(t+ 1)−β,
whereβ = 1/2ηκ <1. Combining the above inequalities, we obtain the following estimate ofRt,s
as
kRt,sk∞≤C3(s+ 1)−1+β(t+ 1)−β. (53)
In the remainder of the proof, the power of a vector is understood element-wise. We use the fact thatvsandvs0 are independent ifs6=s0 and that the expectation of an odd power of an unbiased Gaussian is always zero. By Cauchy-Schwarz and Jensen’s inequalities,
E∆4t = t−1 X
s=0 Rt,svs
!4
=
t−1 X
s=0
E(Rt,svs)4+
X
0≤s6=s0<t
3E(Rt,svs)2E(Rt,s0vs0)2
≤
t−1 X
s=0
E(Rt,svs)4+ 3 t−1 X
s=0
E(Rt,svs)2
!2
≤
t−1 X
s=0
E(Rt,svs)4+ 3t t−1 X
s=0
E2(Rt,svs)2
≤ (3t+ 1)
t−1 X
s=0
E(Rt,svs)4. (54)
Notice that since the variance ofvt= (vt,1, . . . , vt,n)T is nonincreasing, there exists a constantC4 such thatEvt,i4 ≤C4. By Jensen’s inequality and the fact that the variablesvt,i are independent for
different values ofi, we have, for anyi, j, k, l,
|Evt,ivt,jvt,kvt,l| ≤max k Ev
4
t,k.
By direct calculation, it then follows that
max
i E(Rt,svs)
4
i = max i E
Xn
j=1
(Rt,s)ijvs,j
4
≤ max
i
Xn
j=1
(Rt,s)i,j
4 max
k Ev
4
s,k
= kRt,sk4∞max k Ev
4
s,k
≤ C5(s+ 1)−4+4β(t+ 1)−4β. (55)
Summing (55) over0≤s≤t−1, we conclude from (54) thatE∆4t ≤C6t−2, and thus X
t≥0
E∆4t ≤C6 X
t≥1
By the Borel-Cantelli lemma, it follows that∆tvanishes almost surely.
Theorem 6 follows directly from Lemmas 7 and 8.
Why the truth-hearing assumption is necessary. We describe a sequence of graphs Gt that
allows every agent infinite access to the truth and yet does not lead to truthful consensus. For this, it suffices to ensure that the expected mean processytdoes not converge. Consider a system
with two learning agents with priors µ0,1 andµ0,2 from the same distributionN(2,1). We have x0,1 = x0,2 = y0,1 = y0,2 = 2and, as usual, the truth is assumed to be 0; the noise variance is σ2 = 1. The graph sequence is defined as follows: sett
1= 0; fork= 1,2, . . ., agent 1 links to the truth agent at timetkand to agent 2 at timestk+ 1, . . . , sk−1; then at timesk, agent 2 links to the
truth agent, and then to agent 1 at timessk+ 1, . . . , tk+1−1. Other than the links mentioned, we assume no additional link exists. The time pointsskandtkare defined recursively to ensure that
ysk,1 ≥1 + 2
−2k+1 and y
tk,2≥1 + 2
−2k. (57)
In this way, the expected mean processes of the two agents alternate while possibly sliding down toward 1 but never lower. The existence of these time points can be proved by induction. Since y0,2 = 2, the inequalityytk,2≥1 + 2
−2kholds fork= 1, so let’s assume it holds up tok >0. The
key to the proof is that, by (33), as agent 1 repeatedly links to agent 2, she is pulled arbitrarily close to it. Indeed, the transition rule gives us
yt+1,1=
(Pt)11 (Pt+1)11
yt,1+ 1 (Pt+1)11
yt,2,
where(Pt+1)11= (Pt)11+ 1, which implies thatyt,1can be brought arbitrarily close toyt,2while the latter does not move: this follows from the fact that any product of the formQtb
t>ta
t t+1 tends to0astb grows.2 Thus a suitably increasing sequence ofsk, tk ensures the two conditions (57).
The beliefs of the two agents do not converge to the truth even though they link to the truth agent infinitely often.
Acknowledgments
We wish to thank the anonymous referees for their useful comments and suggestions. Some results of this work were previously published in the conferences, Innovations in Theoretical Computer Science (ITCS 2017), and American Control Conference (ACC 2017) by the same authors. Chu Wang did this work prior to joining Amazon. The research of Bernard Chazelle was sponsored by the Army Research Office and the Defense Advanced Research Projects Agency and was accomplished under Grant Number W911NF-17-1-0078. The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the Army Research Office, the Defense Advanced Research Projects Agency, or the U.S. Government. The U.S. Government is authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation herein.
References
Daron Acemoglu and Asuman Ozdaglar. Opinion dynamics and learning in social networks.Dynamic Games and Applications, 1(1):3–49, 2011.
Daron Acemoglu, Munther A Dahleh, Ilan Lobel, and Asuman Ozdaglar. Bayesian learning in social networks. The Review of Economic Studies, 78(4):1201–1236, 2011.
Venkatesh Bala and Sanjeev Goyal. Learning from neighbours. The review of economic studies, 65 (3):595–621, 1998.
Abhijit V Banerjee. A simple model of herd behavior. The quarterly journal of economics, 107(3): 797–817, 1992.
FC Bartlett. Remembering: a study in experimental and social psychology (1932).
Aaron Beppu and Thomas L Griffiths. Iterated learning and the cultural ratchet. InProceedings of the 31st annual conference of the cognitive science society, pages 2089–2094, 2009.
A Bhattachayya. On a measure of divergence between two statistical population defined by their population distributions. Bulletin Calcutta Mathematical Society, 35:99–109, 1943.
George E.P. Box and George C. Tiao. Bayesian inference in statistical analysis, volume 40. John Wiley & Sons, 2011.
Bernard Chazelle. Diffusive influence systems. SIAM Journal on Computing, 44(5):1403–1442, 2015.
Bernard Chazelle and Chu Wang. Inertial Hegselmann-Krause systems. InProceedings of the IEEE American Control Conference (ACC), pages 1936–1941, 2016.
Dorin Comaniciu, Visvanathan Ramesh, and Peter Meer. Real-time tracking of non-rigid objects using mean shift. InComputer Vision and Pattern Recognition, 2000. Proceedings. IEEE Conference on, volume 2, pages 142–149. IEEE, 2000.
Kenneth R Davidson and Stanislaw J Szarek. Local operator theory, random matrices and banach spaces. Handbook of the geometry of Banach spaces, 1(317-366):131, 2001.
Morris H. DeGroot. Reaching a consensus. Journal of the American Statistical Association, 69(345): 118–121, 1974.
Alan Edelman. Eigenvalues and condition numbers of random matrices. SIAM Journal on Matrix Analysis and Applications, 9(4):543–560, 1988.
Andrew Gelman, Hal S Stern, John B Carlin, David B Dunson, Aki Vehtari, and Donald B Rubin. Bayesian data analysis. Chapman and Hall/CRC, 2013.
Benjamin Golub and Matthew O. Jackson. Naive learning in social networks and the wisdom of crowds. American Economic Journal: Microeconomics, 2(1):112–149, 2010.
Thomas L. Griffiths and Michael L. Kalish. A bayesian view of language evolution by iterated learning. InProceedings of the 27th annual conference of the cognitive science society, pages 827–832, 2005.
Thomas L. Griffiths and Michael L. Kalish. Language evolution by iterated learning with bayesian agents. Cognitive Science, 31(3):441–480, 2007.
Thomas L Griffiths, Michael L Kalish, and Stephan Lewandowsky. Theoretical and empirical evidence for the impact of inductive biases on cultural evolution. Philosophical Transactions of the Royal Society of London B: Biological Sciences, 363(1509):3503–3514, 2008.
Michiel Hazewinkel. Encyclopaedia of Mathematics. Springer Science & Business Media, 2013.
Rainer Hegselmann and Ulrich Krause. Opinion dynamics and bounded confidence models, analysis, and simulation. Journal of Artificial Societies and Social Simulation, 5(3), 2002.
Ali Jadbabaie, Pooya Molavi, Alvaro Sandroni, and Alireza Tahbaz-Salehi. Non-bayesian social learning. Games and Economic Behavior, 76(1):210–225, 2012.
Ali Jadbabaie, Pooya Molavi, and Alireza Tahbaz-Salehi. Information heterogeneity and the speed of learning in social networks. Columbia Business School Research Paper, (13-28), 2013.
Michael L Kalish, Thomas L Griffiths, and Stephan Lewandowsky. Iterated learning: Intergenera-tional knowledge transmission reveals inductive biases. Psychonomic Bulletin & Review, 14(2): 288–294, 2007.
Simon Kirby, Tom Griffiths, and Kenny Smith. Iterated learning and the evolution of language. Current opinion in neurobiology, 28:108–114, 2014.
Ilan Lobel and Evan Sadler. Preferences, homophily, and social learning.Operations Research, 64 (3):564–584, 2015.
Soheil Mohajer and Behrouz Touri. On convergence rate of scalar hegselmann-krause dynamics. In American Control Conference (ACC), 2013, pages 206–210. IEEE, 2013.
Pooya Molavi, Alireza Tahbaz-Salehi, and Ali Jadbabaie. Foundations of non-bayesian social learning. Columbia Business School Research Paper, 2015.
Luc Moreau. Stability of multiagent systems with time-dependent communication links. IEEE Transactions on Automatic Control, 50:169–182, 2005.
Elchanan Mossel, Allan Sly, and Omer Tamuz. From agreement to asymptotic learning. Arxiv preprint arXiv, 1105, 2011.
Manuel Mueller-Frank. A general framework for rational learning in social networks. Theoretical Economics, 8(1):1–40, 2013.
James R Norris. Markov chains. Cambridge university press, 1998.
Anna N Rafferty, Thomas L Griffiths, and Dan Klein. Convergence bounds for language evolution by iterated learning. InProceedings of the Thirty-First Annual Conference of the Cognitive Science Society, 2009.
Anna N Rafferty, Thomas L Griffiths, and Dan Klein. Analyzing the rate at which languages lose the influence of a common ancestor. Cognitive science, 38(7):1406–1431, 2014.
Mohammad Amin Rahimian and Ali Jadbabaie. Learning without recall from actions of neighbors. In2016 American Control Conference (ACC), pages 1060–1065. IEEE, 2016a.
Mohammad Amin Rahimian and Ali Jadbabaie. Naive social learning in ising networks. In2016 American Control Conference (ACC), pages 1088–1093. IEEE, 2016b.
Mohammad Amin Rahimian, Shahin Shahrampour, and Ali Jadbabaie. Learning without recall by random walks on directed graphs. In2015 54th IEEE Conference on Decision and Control (CDC), pages 5538–5543. IEEE, 2015a.
Mohammad Amin Rahimian et al. Learning without recall: A case for log-linear learning. IFAC-PapersOnLine, 48(22):46–51, 2015b.
Mark Rudelson and Roman Vershynin. Smallest singular value of a random rectangular matrix. Communications on Pure and Applied Mathematics, 62(12):1707–1739, 2009.
Kenny Smith. Iterated learning in populations of bayesian agents. InProceedings of the 31st annual conference of the cognitive science society, pages 697–702, 2009.
Alireza Tahbaz-Salehi, Alvaro Sandroni, and Ali Jadbabaie. Learning under social influence. In Decision and Control, 2009 held jointly with the 2009 28th Chinese Control Conference. CDC/CCC 2009. Proceedings of the 48th IEEE Conference on, pages 1513–1519. IEEE, 2009.
Appendix A. Analysis of the Root-Sine-Distance
In this appendix, we provide detailed discussion of the root-sine-distance. We will first show that it is a metric, followed by the proof of its equivalency to Euclidean distance and the discussion of its relation to other similar metrics. The notation in the appendix is local and should not be confused with the one used in the main body of this paper.
The root-sine-distance is a metric. The two forms of the functiondRSin (4) make it clear that
0 ≤dRS(a,b) ≤1anddRS(a,b) = 0if and only ifaandbare identical. We easily check that
dRSmakes the simplexSof distributions overDinto a metric space. Indeed,dRS(·,·)is obviously
symmetric, anddRS(a,b) = 0implies thata=b. To check the triangular inequality, notice that
dRS(a,b) =
v u u t1−
Xs
i=1 p
aibi
2
= sinh√a,√bi, (58)
where h√a,√bi is the angle between the unit vectors √a and √b, using the notation √v = (√v1, . . . ,
√
vs). To prove thatdRS(a,b) +dRS(b,c)≥dRS(a,c)for anya,b,c∈ S, we denote
byα, β, γthe corresponding angles in that order, ie,α=h√a,√bi, etc. The coordinates ina,b,c are nonnegative; therefore 0 ≤ α, β, γ ≤ π/2. These form the three angles at the origin of a tetrahedron with a vertex at the origin; therefore, by the triangular inequality in spherical geometry, α+β ≥γ. Ifα+β≤ π
2, thensinα+ sinβ≥sinαcosβ+ cosαsinβ= sin(α+β)≥sinγ. On the other hand, ifα+β > π/2, thensinα+sinβ= 2 sinα+2βcosα−β2 ≥2 sinπ4cosπ4 = 1≥sinγ, which establishes the triangular inequality.
Relation to the Euclidean distance. Shrinking the simplex S by a tiny amount, we define
Sε:={a∈ S :ε≤ai ≤1−ε}and note that
dE(a,b) :=ka−bk2 = v u u t
s
X
i=1
(√ai−
p bi)2(
√
ai+
p bi)2.
It follows that, fora,b∈ Sε,
1
2dE(a,b)≤dE(
√
a,
√
b)≤ 1
2√εdE(a,b). (59) On the other hand,k√ak2 = k
√
bk2 = 1, so the vectors
√
aand√bform an isosceles triangle; hence
dE(
√
a,√b) = 2 sin1 2h
√
a,√bi= sinh
√
a,√bi
cos12h√a,√bi =
dRS(a,b)
cos12h√a,√bi.
Since0≤ h√a,√bi ≤ π
2,
dRS(a,b)≤dE(
√
a,
√
b)≤√2dRS(a,b).
Together with (59) this shows that, for anya,b∈ Sε,
1
2√2dE(a,b)≤dRS(a,b)≤ 1
Relation to other distances. The metricdRS is related to the Hellinger and Bhattacharyya
dis-tances. WritingC(a,b) =Ps
i=1
√
aibi(Comaniciu et al., 2000), thendRS(a,b) =
p
1−C(a,b)2. The Hellinger distance is defined as dH(a,b) =
p
1−C(a,b) (Hazewinkel, 2013), while the Bhattacharyya distance is defined asdB(a,b) = −lnC(a,b) (Bhattachayya, 1943). The total
variation distancedT V is half the`1-norm; thereforedT V(a,b)≤ 12
√
s dE(a,b). Combining these
observations with (60) establishes (5):
dH =
r
1−q1−d2
RS;
dB=−12ln(1−d2RS) ;
dT V ≤
√
