Validation with simulated data

Spike data from the network described in the aboveSection 6.2.2.2 were recorded every millisecond from the simulation model performed in MATLAB 2018b and exported as a CSV(Comma Separated Values) file. The average firing rate was stable over the whole duration of the simulated network as shown in Figure 6.4. Gamma oscillations (30Hz) were present as reported in [192]. Due to the STDP rule for synapse weight updates, synaptic weights tended to converge either to the maximum i.e. 10mV or the minimum i.e. 0mV over a longer period (typically after tens of minutes) of simulation.

The spike times extracted for the last 30 minutes of simulation were used to compute pair wise TE values for all the 100 subsampled neurons. TE values were computed for three different delays - 10, 20 and 30 time bins, which is the parameter d in equation 6.2. Peak values of the TE (TEPk) were considered to infer connectivity. Ito’s method [15] was used to compute the delayed TE values. To investigate the general trend of TEPk values inferred against the ground

Figure 6.4: Firing rate for the first 1 hr of simulation activity. Red lines represent the mean firing rate averaged at 50 ms time bins.

truth synaptic weight, the TEPk values computed were plotted for each pair against the synaptic weights. Figure 6.5 shows the TEPk value and correspond- ing ground truth synaptic weight for each pair of neurons in a linear scale. As evident from Figure 6.5, the TEPk values increase with the synaptic weights in an exponential manner for the subsampled neurons. The log scale shows a linear relationship. Only the synaptic weights between 0mV and 10mV are included in the plot for better visualisation. Most of the connections converge either to the maximum or minimum - including those weights result in large clutter towards both ends.

A higher synaptic weight between two neurons represents a higher probability of effective connectivity which could potentially cause the post synaptic neuron to fire. TEPk values corresponding with synaptic weights from the same pair indicate that the inferred connectivity is actually the connection that is present in the simulation model. However, the plot only shows a general trend and not an empirical metric to measure the accuracy of the method. Receiver Op- erating Characteristic (ROC) analysis was performed to calculate the accuracy performance of the algorithm. Essentially, the problem becomes a classification problem to classify true connections from spurious connections. ROC analysis

Figure 6.5: Relationship between in- ferred connectivity strength with the synaptic weights. X-axis represents the synaptic weights and Y-axis shows the TEPk values.

Figure 6.6: Performance comparison of effective connectivity estimation with the transfer entropy. TPR is plotted against FPR for 10,20 and 30 time bins delay.

provides an objective approach to quantify performance of a binary classifier al- gorithm [194–196].

After computation of TEPk, the task is to distinguish “True Positive(TP)” from “True Negative(TN)” and similarly also to identify “False Positives(FP)” and “False Negatives(FN)” . TP is the case when the algorithm identifies a connectivity between two neurons, which in fact exists in the model as well. TN is the case when the algorithm identifies no connection between two neurons, and in fact there is no connection in the model as well. FP is the case when the algorithm identifies a connection but in reality there is no connection in the model for a particular pair of neurons. Similarly, FN is a scenario when an algorithm identifies no connection between two neurons but there does exist a connection in the model. TP+FN actually reports total number of true connections identified by the algorithm and FP+TN reports the total true unconnected pairs.

ROC space analysis is done by plotting true positive rate(TPR) against false positive rate(FPR) . TPR and FPR are computed as,

T P R = T P

F P R = F P

F P + T N (6.9)

To analyse the effect of the delay parameter,TEPk values for three different delays i.e. 10ms, 20ms and 30ms time bins were computed and corresponding TPR and FPR for each of these delays were plotted. In the model, the axonal delay is in the range from 1 to 20 ms. Figure 6.6 clearly shows a significant improvement in performance from 10 to 20 time bins delay which demonstrates that the delayed TE method is able to capture delayed interactions effectively. With a delay of 10 time bins, the algorithm misses a lot of delayed interactions resulting in poor performance whereas with delay of 20 time bins i.e. 20 ms, most of the delayed interactions are taken into account resulting in a much improved accuracy. A delay of 30 time bins improves the performance slightly compared to the 20 time bins, but not significantly. This is presumably due to the fact that the model has delays in the range of 1-20ms, which is mostly covered by TE delay of 20 ms.

To measure the performance, it is important to measure the TPR against FPR at a very low FPR rate. Accordingly, the TPR was measured at a constant FPR rate of 0.01 which is a very small percentage of FPR so that the performance is not compromised. For delays of 20 and 30 time bins, the TPR at FPR=0.01 was measured to be around 82±2% and 84±2% respectively, showing the effectiveness of the approach.

The delayed TE based method of inferring connectivity is hence accurate in the range of mid 80th _{percentile, even for a biologically realistic model of a} spiking neural network. The experiment on the simulated data thus established the reliability of the method for a realistic network with delays, STDP synapses and complex interactions. However, real experimental neuronal cultures are far more difficult to analyse, with complex interactions but without the ground truth against which the results can be empirically compared.