Evaluating Model Fit on Real-World Networks

Well-fitting network models give insights into understanding the rules gov- erning the emergence and evolution of real-world networks. In order to assess the fit of a network model to a given network, the network should be compared with the networks that can be generated from the model. In particular, given an input networkG, the first step of assessing the model fit is generating several networks from the evaluated network model. Each net- work model is capable of producing a di↵erent range of networks; e.g., less parametric models such as ER are theoretically capable of producing any

observable network overnnodes, while more stringent models that require more parameters can only generate a small range of networks. The range of observable networks also changes depending on the size and density of the generated networks. For this reason, the number of model networks that need to be generated from a model for model-fitting experiments should be chosen to allow observation of a significant range of di↵erent config- urations. On the other hand, generating more model networks increases the required computational time for the model-fitting tests. Generating a minimum of 30 networks per model was previously accepted to be suffi- cient for observing a significant range of networks that can be generated from a model [82, 135, 159]. After generating a sufficient number of model networks, the topologies of the generated networks are compared with the input network, G [157]. As explained in Section 1.4, topological network comparison is a NP-Complete problem, for which there are only approx- imate polynomial-time solutions [35]. Therefore, the comparison between the topologies of the input network and the model generated networks are performed using the heuristic approaches. Any of the global or local net- work properties that are explained in Section 1.4 can be used to perform these comparisons; e.g., degree distribution, clustering coefficient, shortest path length distribution, graph spectra, network motifs, and graphlets.

The most intuitive method for comparing the topologies of the input network and model networks is contrasting their global network proper- ties (e.g., degree distribution, spectrum of shortest path lengths). A visual model-fitting assessment can be obtained by computing the averages and standard deviations of the global network properties for all generated model networks, and plotting them together with the properties of the input net- work. This method was previously applied for evaluating the fit of ERGM models in Statnet package [66]. However, global network properties are not detailed enough to capture the exact topologies of networks. For example, a graph that is composed of 3 disconnected triangles and a 9-node cycle have the same degree distributions while their topologies are completely di↵er- ent. For this reason, testing the model based on global network properties is not a strong model-fit assessment method. Furthermore, the results of these tests do not quantify the level of topological correspondence between two networks.

graph patterns better than the global network properties. Therefore, the comparison of these network properties produce more accurate model iden- tification results. It is hard to interpret the spectral statistics of a net- work, since the spectrum of a graph cannot be translated into everyday language directly. Furthermore, more than one graph may have the same spectral profile, resulting with the failure of spectral methods in network comparison [201]. The information encoded in network motif and graphlet statistics can be translated into everyday language easily, as they represent which subgraph patterns appear in the network and which patterns do not. Among these two network properties, we focus on the graphlet statistics, since the interpretation of the motif-based methods is highly dependent on the chosen random network model to identify the over-represented and under-represented patterns [7].

Przulj et al. use graphlet-based network distance measures (i.e., RGF distance [157] and GDD-Agreement [156]) for identifying the best fitting network model among a number of alternatives. They compute the RGF distances and GDD-Agreements between the input network and the gen- erated model networks, and accept the the model with the minimum av- erage distance to the input network as the best-fitting model. Note that, although this method is suggested and widely-applied using the graphlet- based network distances measures, any other network distance heuristics can be applied in a similar way.

Rito et al. [163] criticizes the methodology of Przulj et al. [156], claiming that the method is good for comparing alternative models with each other but the network model that is at minimum distance to the input network does not necessarily fit the network. In other words, the obtained results are all relative to the compared models; even if none of the models actually fit the data, a well-fitting model is identified with this method. They sug- gest a non-parametric methodology for testing whether a model truly fits a network. This methodology is based on two distributions: (1) distribu- tion of data-vs-model distances: represents the distances between the input network and the model networks, (2) distribution of model-vs-model dis- tances: represents the distances between all model network pairs. If these two distributions intersect, this indicates that the model di↵ers within itself as much as it di↵ers from the input network. Therefore, the intersection between the two distributions is an indicator of model fit. Later on, Hayes

et al. [76] apply the non-parametric method to analyse the topologies of the seven network models that are listed in Section 1.5. They find out that the topology of the model networks are unstable below a certain sizes and edge densities.

The above discussed methods assess the network models for their ability to reproduce the observed structure of an input network. Another problem in network modelling is assessing the trade-o↵ between the complexity of a model (i.e., the number of parameters that are necessary to define the model) and its goodness-of-fit. Network models that are able to reproduce the observed topology of an input network with less number of parameters are desired over more complex models. Given two network modelsM1 and

M2, the trade-o↵between the goodness-of-fit and complexity of the models

can be assessed by two statistical measures that are based on information theory: (1) Akaike Information Criterion (AIC) [1], and (2) Bayesian In- formation Criterion (BIC) [168]. Akaike information criterion is defined as:

AIC = 2k 2 ln(L), (1.28)

wherek is the number of model parameters, andLis the maximized value of the likelihood function for the estimated model. AIC penalizes the high number of parameters while rewarding the goodness-of-fit determined by the maximum likelihood. Therefore, network models that have smaller AIC val- ues are preferred. Bayesian Information Criterion (BIC) is another measure that evaluates the trade-o↵between the model complexity and its goodness- of-fit. BIC penalizes the number of model parameters more strongly than AIC, and it is defined as:

BIC = 2 ln(L) +kln(n), (1.29) whereL is the maximized value of the likelihood function for the estimated model, k is the number of model parameters, and n is the number data points in the observed data. Unlike AIC, BIC depends on the number of data points in the observed data; e.g., number of nodes in the modelled networks. Similar to AIC, models with lower BIC scores are desired. For both models, the likelihood function of the estimated model is defined based on the goodness-of-fit statistics for the networks generated from the models. It should be noted that AIC and BIC scores only quantify the trade-o↵

between the goodness-of-fit and the model complexity; they do not evaluate the fit of a network model. For this reason, these scores should only be used when making a comparison between two well-fitting network models. We use AIC and BIC scores to compare the estimated exponential-family random graph models in Chapter 5.