A fundamental problem of network inference, which has been mentioned earlier, is that different causal models can generate the same or similar correlated out- put. In such cases it is impossible to reliably infer the causal network from mere stochastic relations. Unfortunately, correlated activity is the only information available to a network learner in order to infer an information flow network; revealed structures may therefore not match the causal connections. Since such discrepancy is not a failure of the inference method, it must be accounted for when evaluating results. In the presented framework (Fig. 6.4) this is done based on the concept of link plausibility, which allows sensibly tackling the problems of
q spike train spont. activity generator leakage factor synaptic efficiency spike train spike train metric distance LIF simulator correlation analysis network search heuristic/ enumerator score value SSS shift constant activity decay rate length gold spike train LIF simulator golden network
Figure 6.5: Alternative network inference assessment framework based on sim- ulation of learned networks in order to compare their dynamics to those they represent (work-flow from left to right). Processing stages represented by rect- angles; elements with edges rounded off indicate objects and parameters. A golden network defines excitatory synaptic connections in a neural network, which is simulated by imposing stimulation on model neurons. Potential net- works are simulated and scored in parallel in order to yield their dynamics as a neural network and their score value. The dynamics of the potential network and those of the golden network are compared using a spike train metric. If low distances correlate with high network scores, i.e. networks that replicate the dynamics of the data used for learning, the scoring function imposes a sensible ranking on network structures.
network assessment caused by observational equivalence and partial observabil- ity. These issues can also be addressed differently, and one alternative approach is presented next.
Instead of inferring reasonable links from network connectivity, a more prag- maticly seeming idea is to test whether a learned network generates appropriate dynamics (Fig. 6.5): Does a simulation of the inferred network result in spike
trains that are similar to the data it was learned from? In detail, like the golden
network, the learned network can be simulated using neuron models in order to generate spike train data. The spike trains of both networks can then be com- pared to each other, by quantifying the matching of the dynamics with spike train metrics (Section A.2). If the dynamics of the inferred network are similar to those it was learned from, the structure correctly reflects functional relations in the data and is thus a good network. Finally, the similarity of dynamics is related to the score value of the learned network; high scores would optimally correlate with low distances, i.e. similar dynamics. If this is the case, the score rates networks suitably.
stimulation spike train leakage factor synaptic efficiency network to assess LIF simulator q spike train gold spike train plausibility analysis golden network observable nodes plausible lags network to assess correlation analysis metric distance assessed network's score a b
Figure 6.6: Comparison of parameters and processing steps needed in order to assess a network (work-flow from top to bottom). Processing stages represented by rectangles; elements with edges rounded off indicate objects and parameters.
a Metric approach: The network to assess need to be simulated first in order to calculate the distance between its spike train and that of the golden net- work. The metric itself and the simulation must be parameterised. Note that additional parameters are needed in order to determine the stimulation activity (not shown). bPlausibility concept: Plausible lags must be specified in order to analyse the golden network. The learned network is then compared against the resulting reference network, which can be re-used for comparisons with further learned networks.
Like the plausibility concept, the alternative spike train metric based ap- proach can be used under conditions of partial observability. Therefore channels of the golden network’s spike trains would have to be filtered according to their observability (not shown in Fig. 6.5) before passing them on to the spike train metric and the scoring function. Either of the two presented concepts could thus be used to assess the SSS, but the plausibility concept is favoured because it is simpler. In fact, the two approaches differ widely in complexity: The metric based concept involves far more parameters and processing steps than that us- ing link-plausibility (Fig. 6.6). Additionally, there is also a significant difference in the computational costs of the two approaches. With the alternative metric based approach every network is simulated and afterwards the distance between spike trains is calculated. The computational costs for this are determined by the size of the network and, even more critically, the length of the simulated spike train. Increasing data length results in non-critical linear growth of costs for the neural simulation, but computational demands for applying the spike train metric rise over-proportionally. Practical application is thereby limited to
relative short data segments.7 In contrast, the plausibility analysis of a golden network does not depend on the length of the data but only on network size. The associated costs are comparatively low anyway and additionally, the golden network needs to be evaluated only once and can then serve as a reference for all networks that are learned from its dynamics. For these reasons the SSS has been assessed using the plausibility concept. Corresponding results are presented in the next chapter.
7Note that long data-sets should actually be favoured for assessments: Short data segments
might match well or bad due to random effects. In contrast, observing (dis)similar dynamics over a long period yields higher confidence that artefacts are smoothed out; the measured distance is then a reflection to what degree the learned network can replicate features of the data is has been inferred from.
Chapter 7
Application of the
Assessment Framework
The previous chapter motivated and described an assessment framework for in- formation flow network inference techniques. Results from applying the frame- work to the SSS are presented in this chapter. Details on the chosen parameters are given first in order to complete the description of the set-up. Next, the out- comes of different parameter series are presented and thereafter, these will be reviewed critically. Based on the outcome of the discussion, further evaluation series are performed in order to address identified issues.
7.1
Set-up and Parameters
The first step in the work-flow of the assessment framework is to specify a net- work, which defines the number of neural entities and their interaction (Fig. 6.4 on page 100). For the following simulation series, a feed-forward type structure has been chosen whose nodes corresponds to one neuron each (Fig. 7.1a). For the neural simulation, the dynamics of each neuron are generated according to an integrate and fire model (Section A.1.1). As neurons according to this model do not show intrinsic spiking, baseline activity in the network must be induced by stimulation. This is done with Poisson processes, which are used to generate spontaneous spikes for each unit independently. The level of baseline activity is controlled by a constant parameterλ= 10−1,15−1,25−1,30−1,40−1, or 50−1; the rate parameterλis negatively correlated with the level ofsponta-
neous spiking activity. Any spike propagates according to network connectiv-
ity to directly linked units, where it results in an excitatory potential. Such transmission occurs with a delay of one time-bin (1 msec). Each neuron inte-
3 7 t=1 t=2 t=3 t=4 t=5 t=6 t=7 t=0 13 18 23 28 33 2 6 12 17 22 26 31 5 11 21 10 16 30 1 4 8 14 19 24 29 32 27 25 9 15 20 38 36 35 34 37 signal progapagation t=1 t=2 t=3 t=4 t=5 t=6 t=7 t=0 3 13 23 6 11 21 8 29 32 27 25 20 38 35 signal progapagation a b
Figure 7.1: aSimulated feed-forward network (38 nodes, 47 links) with observ- able units (14 nodes≈36.8%) shaded in grey and hidden units unshaded. bOb- servable nodes from network shown in (a) with all 34 plausible links (≈18.7% out of 182 possible links) for plausible lags lmin = 1 andlmax = 3. Examples
of plausible links can be those which exist in the full network (e.g. 21 → 25, 23→27, and 27→32) or certain links for which a directed path from the link’s starting node to its end node exists (e.g. 3→(7)→13, 13 →(17)→21, and 23→(28)→(33)→38).aLinks that contradict the full network by connecting nodes in the opposite direction of information flow in the feed-forward network (e.g. 13 → 3, 21 → 3, and 35 → 3) are implausible, for example. Also, a common trigger can cause co-ordinated firing between nodes, which are con- nected via the triggering node only (e.g. plausible link 6→11 due to common trigger (2): 6←(2)→(5)→11). (For full details on the plausibility concept please see section 6.3.2.)
a Start- and end-nodes underlined, nodes on path in full network italic, hidden nodes in brackets.
grates synaptic inputs over time that are received from its parents (temporal summation). A parameter for synaptic efficiency determines whether 2,3,4, or 5 excitatory inputs are necessary in order to evoke a spike in the receiving neuron. The parameters controlling the spontaneous activity and synaptic ef- ficiency are equal for all neurons. As discussed in detail later (Section 7.3.4), neurons have been simulated without a leakage component, such that synaptic inputs would be integrated over infinite time. However, this does not happen in practice, because spontaneous spiking resets the membrane potential (just as activity evoked by synaptic inputs does). Since each simulated neuron exhibits spontaneous activity, its time-window for summing synaptic inputs is effectively limited.
In accordance with practical situations it is assumed that the simulated system is not completely observable, but that only spike trains from a subset of units (observable nodes) can be collected and used for learning networks (grey shaded nodes in Fig. 7.1a). In order to assess inferred networks, the effects of partial observability are taken into account by applying the plausibility concept (Section 6.3.2). The algorithmic implementation (Algorithm 1) of the plausibility definition 6 was applied to the full network (Fig. 7.1a) with plausible lags (lmin, lmax) = (1,3) in order to determine all plausible links (Fig. 7.1b).
The minimal plausible lag lmin has been chosen according to the smallest lag
observable nodes can show non-spurious correlation andlmaxsuch that a modest
percentage of links will be plausible.1
Two different series were run in which both the parameters of the neural simulation and the score were altered systematically in order to assess their influence on the quality of the recovered networks (Fig. 7.2). Each combination of parameters is simulated 10 times for different data lengths (5,10, and 30 seconds, 1,5,and 10 minutes), whereby the full network, observable nodes, and plausible links are left unaltered. Simulating the full network yields spike train data from which observable channels are then used to determine each node’s best scoring parent configuration with up to 3 parents. (Unless noted otherwise, the parameters of the SSS wered= 3−1and ∆t= 1. The LAT hat been determined from configurations with 3 parents.) Composing parent configuration yields the recovered network, which is analysed with respect to its plausibility: Links of learned networks are classified as either hit or miss (depending on whether they are plausible or not). The recovery rate and precision were determined together with the corresponding P-value in order to account for the fact that the probability of finding plausible links by chance depends on their percentage (Section 6.3.1).
1Forl
min= 1 there exist 12 (≈6.6% out of 182 possible links,lmax= 1), 27 (≈14.8%,
∈!10−1,15−1,25−1,30−1,40−1,50−1" spontaneous activity rate = 10 repetitions = 1 shift constant ∈{5−1,4−1,3−1,2−1,1} decay constant ∈{5,10,30 sec,1,5,10 min} data length
∈{2,3,4,5 excitations per spike}
synaptic efficiency
a b
Figure 7.2: Parameter overview for the assessment framework. Two series a
and b were run in which the marked parameters have been combined in all possible ways. For each of the combinations the neural simulation and the network learning step has been repeated 10 times in order to average out random effects.
Note that parameters of the neural simulation (spontaneous activity level, synaptic efficiency) do not have any physical correspondence in this sparsely connected network; their influence on network inference is thus not shown indi- vidually as their combined effect is fully reflected in the simulated spike train. Instead, the simulation output, the spike train, is characterised by the amount of information it contains about the structure of the network by calculating its
impetus(Section 6.2.2). Determining the impetus for every data set that is used
for network learning and relating it to the corresponding success rates, recovery rate and precision, shows its relationship to the performance of the SSS. The results of this investigation are presented in the next section.