It is important to mention here the limitations of the current study due to the small dataset for exploring the test-retest reliability statistics. In the era of open science resource where multisite worldwide neuroimaging labs share neuroimaging datasets, it is important to demonstrate novel techniques that improve the reliability of connectomics in common neuroimaging data (Zuo et al., 2014a). A recent Consortium for Reliability and Reproducibility (CoRR) is working to address this gap and establish test-retest reliability as a minimum standard for methods development in functional connectomics (Zuo et al., 2014a,b) and morphological measurements (Maclaren et al., 2014). Reliability is important to build reliable connectomic biomarkers across multi-site (Nielsen et al., 2013 ; Abrakam et al.,2016) and also longitudinal trajectories of structural and functional brain networks across the life-span (Zuo et al., 2017).
Our future goal is to test the proposed methodology in a larger sample for validating the test-retest reliability of our scheme and also on multi-site diffusion-based structural brain networks for building reliable connectomic biomarkers.
4.3 Conclusion
Reliability analysis of both node and network-wise network metrics in IWSBNand its topological filtering version revealed: 1) similar ICC values for all the network metrics on the network level for IWSBNTF compared to the best NWS; 2) higher ICC of network metrics node-
30
wise for both IWSBNand IWSBNTF compared to each NWS with higher values succeedeing based on IWSBNTF;and 3) higher discrimination of each subject compared to the rest of the cohort based on the IWSBNTF derived from each scan compared to IWSBNand the best NWS which was the NSTR. We thus provided a new approach to identifying highly reliable and discriminative network metrics that can be the basis for studies of interindividual differences, longitudinal trajectories and pathological changes in structural brain connectivity.
Acknowledgements
SID, DEJL and DKJ were supported by a MRC grant MR/K004360/1 (Behavioural and Neurophysiological Effects of Schizophrenia Risk Genes: A Multi-locus, Pathway Based Approach).
SID is also supported by a MARIE-CURIE COFUND EU-UK Research Fellowship.
DKJ is supported by a Wellcome Trust New Investigator Award and Wellcome Trust Strategic Award.
We would like to acknowledge the Cardiff University RCUK funding scheme for support with publication costs
Conflict of interest statement
The authors declare that there is no conflict of interest.
Author contributions
Conception of the research: SD; Methods and design: SD; Data analysis: SD, MD, GP, SB; Drafting the manuscript: SD; Critical revision of the manuscript: MD, GP, SB, DL, DJ; Every author read and approved the final version of the manuscript.
31
References
Abraham, A., Milham, M., Di Martino, A., Craddock, R.C., Samaras, D., Thirion, B., Varoquaux, G., 2016. Deriving robust biomarkers from multi-site resting-state data: an Autism-based example. bioRxiv (p. 075853),
Antonakakis M, Dimitriadis SI, Zervakis M, Micheloyannis S, Rezaie R, Babajani-Feremi A, Gouridakis G, Papanicolaou AC. 2016. Altered cross-frequency coupling in resting-state MEG after mild traumatic brain injury. Int J Psychophysiol 102:1–11.
Basser PJ, Mattiello J, LeBihan D (1994) Estimation of the effective self-diffusion tensor from the NMR spin echo. J Magn Reson B 103(3):247–254
Basser, P.J. & Pierpaoli, C. (1996). "Microstructural and physiological features of tissues elucidated by quantitative-diffusion-tensor MRI". Journal of Magnetic Resonance, Series B, 111, 209-219.s
Basser PJ, Pajevic S, Pierpaoli C, Duda J, Aldroubi A (2000) In vivo fiber tractography using DT-MRI data. Magn Reson Med 44(4):625–632
Behrens, T., Woolrich, M., Jenkinson, M., Johansen-Berg, H., Nunes, R., Clare, S., Matthews, P., Brady, J., Smith, S., 2003. Characterization and propagation of uncertainty in diffusion- weighted MR imaging. Magn. Reson. Med. 50, 1077–1088
Bracht, T., Jones, D., Bells, S., Walther, S., Drakesmith, M., &Linden, D.Myelination of the right parahippocampal cingulum is associated with physical activity in young healthy adults.Brain Structure and Function.2016 : DOI: 10.1007/s00429-016-1183-3
Buchanan, C.R.; Pernet, C.R.; Gorgolewski, K.J.; Storkey, A.J.; Bastin, M.E. Test-retest reliability of structural brain networks from diffusion MRI. NeuroImage 2014:86;231–243. Bullmore, E.T., Bassett, D.S., 2011. Brain graphs: graphical models of the human brain
connectome. Annu. Rev. Clin. Psychol. 7, 113–140
Bullmore,E.T., Sporns,O. Complex brain networks: graph theoretical analysis of structural and functional systems.Nature Reviews Neuroscience 10, 186-198
Castellanos FX, Di Martino A, Craddock RC et al (2013) Clinical applications of the functional connectome. Neuroimage 80:527–540
Cheng, H., Wang, Y., Sheng, J., Kronenberger, W.G., Mathews, V.P., Hummer, T.A., Saykin, A.J., 2012. Characteristics and variability of structural networks derived fromdiffusion tensor imaging. NeuroImage 61, 1153–1164.
32
Dell’acqua, F., Scifo, P., Rizzo, G., Catani, M., Simmons, A., Scotti, G. and Fazio, F. (2010) ‘A modified damped Richardson-Lucy algorithm to reduce isotropic background effects in spherical deconvolution.’, NeuroImage, Elsevier Inc., vol. 49, no. 2, pp. 1446–58.
Deoni SCL (2007) High-resolution T1 mapping of the brain at 3T with driven equilibrium single pulse observation of T1 with high-speed incorporation of RF field inhomogeneities (DESPOT1-HIFI). J Magn Reson Imaging 26(4):1106–1111
Deoni SCL, Ward HA, Peters TM, Rutt BK (2004) Rapid T2 estimation with phase-cycled variable nutation steady-state free precession. Magn Reson Med 52(2):435–439
Deoni SC, Rutt BK, Peters TM (2006a) Synthetic T1-weighted brain image generation with incorporated coil inninesity correction using DESPOT1. Magn Reson Imaging 24(9):1241– 1248. doi:10.1016/j.mri.2006.03.015
Deoni SCL, Rutt BK, Peters TM (2006b) Synthetic T1-weighted brain image generation with incorporated coil inninesity correction using DESPOT1. Magn Reson Imaging 24(9):1241– 1248
Deoni SCL, Rutt BK, Jones DK (2008c) Investigating exchange and multicomponent relaxation in fully-balanced steady-state free precession imaging. J Magn Reson Imaging 27(6):1421–1429
Deoni SC, Matthews L, Kolind SH (2013) One component? Two components? Three? The
effect of including a nonexchanging “free” water component in multicomponent driven
equilibrium single pulse observation of T1 and T2. Magn Reson Med 70(1):147–154. de Reus, M.A., van den Heuvel, M.P., 2013. Estimating false positives and negatives in brain
networks. Neuroimage 70, 402–409.
Descoteaux, M., Deriche, R., Knösche, T.R., Anwander, A., 2009. Deterministic and probabilistic tractography based on complex fibre orientation distributions. Med. Imaging IEEE Trans. 28, 269–286.
Dimitriadis SI, Laskaris NA, Tsirka V, Vourkas M, Micheloyannis S, Fotopoulos S (2010).Tracking brain dynamics via time-dependent network analysis. Journal of Neuroscience Methods Volume 193, Issue 1, 30 October 2010, pp. 145-155.
Dimitriadis SI1, Kanatsouli K, Laskaris NA, Tsirka V, Vourkas M, Micheloyannis S. Surface EEG shows that functional segregation via phase coupling contributes to the neural substrate
of mental calculations. Brain Cogn. 2012 Oct;80(1):45-52. doi:
33
Dimitriadis SI, Sun Y,Kwok K,Laskaris NA,Thakor N,Bezerianos A. Cognitive workload assessment based on the tensorial treatment of EEG estimates of cross-frequency phase interactions. Ann Biomed Eng. 2015a Apr;43(4):977-89.
Dimitriadis SI, Laskaris NA, Bitzidou MP, Tarnanas I and Tsolaki MN (2015b) A novel biomarker of amnestic MCI based on dynamic cross-frequency coupling patterns during cognitive brain responses. Front. Neurosci. 9:350. doi: 10.3389/fnins.2015.00350
Dimitriadis SI, Zouridakis G, Rezaie R, Babajani-Feremi A, Papanicolaou AC.Functional connectivity changes detected with magnetoencephalography after mild traumatic brain injury. Neuroimage Clin. 2015c Sep 21;9:519-31. doi: 10.1016/j.nicl.2015.09.011. eCollection 2015.
Dimitriadis SI, Yu Sun, Laskaris NA, Thakor N, Bezerianos A (2016a).Revealing cross- frequency causal interactions during a mental arithmetic task through symbolic transfer entropy: a novel vector-quantization approach. IEEE TNSR 2016 Oct;24(10):1017-1028. Dimitriadis SI, Laskaris NA, Simos PG, Fletcher JM and Papanicolaou AC (2016b) Greater
Repertoire and Temporal Variability of Cross-Frequency Coupling (CFC) Modes in Resting-State Neuromagnetic Recordings among Children with Reading Difficulties. Front. Hum. Neurosci. 10:163. doi: 10.3389/fnhum.2016.00163
Dimitriadis SI, Sun Yu, Thakor N, Bezerianos A (2016d). Causal Interactions between Frontalθ - Parieto-Occipitalα2 predict behavioral performance in a mental multilevel EEG task. Dimitriadis SI, Tarnanas I, Wiederhold M, Wiederhold B, Tsolaki M (2016c).Mnemonic-
Strategy Training in Mild Cognitive Impairment enhances integration of information processing via cross-frequency-coupling. Alzheimer's & Dementia: Translational Research & Clinical Intervention. Volume 2, Issue 4, November 2016, Pages 241–249
Dimitriadis SI,Salis C,Tarnanas I,Linden D.Topological Filtering of Dynamic Functional Brain Networks Unfolds Informative Chronnectomics: A Novel Data-Driven Thresholding Scheme Based on Orthogonal Minimal Spanning Trees (OMSTs). Front. Neuroinform., 26 April 2017a | https://doi.org/10.3389/fninf.2017.00028
Dimitriadis SI, Antonakakis M, Simos P, Fletcher JM, Papanicolaou AC.Data-driven Topological Filtering based on Orthogonal Minimal Spanning Trees: Application to Multi-Group MEG Resting- State Connectivity. Brain Connect. 2017b Sep 11. doi: 10.1089/brain.2017.0512. [Epub ahead of print]
Dosenbach NU, Nardos B, Cohen AL et al (2010) Prediction of individual brain maturity using fMRI. Science 329:1358–1361
34
Drakesmith M, Caeyenberghs K, Dutt A, Lewis G, David AS, Jones DK.Overcoming the effects of false positives and threshold bias in graph theoretical analyses of neuroimaging data. Neuroimage. 2015 Sep;118:313-33.
Drakesmith M, Caeyenberghs K, Dutt A, et al. Schizophrenia‐ like topological changes in the structural connectome of individuals with subclinical psychotic experiences. Human Brain
Mapping. 2015;36(7):2629-2643. doi:10.1002/hbm.22796.
Estrada E. 2011. The structure of complex networks: Theory and Applications. Oxford University Press.
Fouss F, Francoisse K, Yen L, Pirotte A, Saerens M. 2012. Neural Netw 31:53-72.
Gigandet, X., Hagmann, P., Kurant, M., Cammoun, L., Meuli, R., Thiran, J.-P., 2008. Estimating the confidence level of white matter connections obtained with MRI tractography. PLoS One 3, e4006
Girard, G., Whittingstall, K., Deriche, R., Descoteaux, M., 2014. Towards quantitative connectivity analysis: reducing tractography biases. Neuroimage 98, 266–278
Goldberg D, Huxley P (1980) Mental illness in the community: the pathway to psychiatric care. Tavistock Publications, London
Hagmann, P., Cammoun, L., Gigandet, X., Meuli, R., Honey, C.J., Wedeen, V.J., Sporns, O., 2008. Mapping the structural core of human cerebral cortex. PLoS Biol. 6, e159
Klein S, Staring M, Murphy K, Viergever MA, Pluim JP (2010) elastix: a toolbox for inninesity-based medical image registration. IEEE Trans Med Imaging 29(1):196–205. Kruskal JB. 1956. On the shortest spanning subtree of a graph and the traveling salesman
problem. Proc. Am. Math. Soc. 7 (1):48–50.
Jenkinson M, Smith S (2001) A global optimisation method for robust affine registration of brain images. Med Image Anal 5(2):143–156
Jenkinson M, Bannister P, Brady M, Smith S (2002) Improved optimization for the robust and accurate linear registration and motion correction of brain images. Neuroimage 17(2):825– 841
Jeurissen B, Leemans A, Tournier JD, Jones DK, Sijbers J (2013) Investigating the prevalence of complex fiber configurations in white matter tissue with diffusion magnetic resonance imaging. Hum Brain Mapp 34(11):2747–2766.
Jeurissen B, Leemans A, Jones DK, Tournier JD, Sijbers J (2011) Probabilistic fiber tracking using the residual bootstrap with constrained spherical deconvolution. Hum Brain Mapp 32(3):461–479.
35
Jones DK, Horsfield MA, Simmons A (1999) Optimal strategies for measuring diffusion in anisotropic systems by magnetic resonance imaging. Magn Reson Med 42(3):515–525 Leemans A, Jones DK (2009) The B-matrix must be rotated when correcting for subject motion
in DTI data. Magn Reson Med 61(6):1336–1349.
Leemans A, Jeurissen B, Sijbers J, Jones DK (2009) ExploreDTI: a graphical toolbox for processing, analyzing and visualizing diffusion MR data. In: Proceedings of the International Society for Magnetic Resonance in Medicine 17th Annual Meeting; Honolulu, Hawaii, p 3536
MacLaren,J Z. Han, S.B. Vos, N. Fischbein, R. BammerReliability of brain volume measurements: a test–retest dataset Sci. Data, 1 (2014), p. 140037
Metzler-Baddeley C, O’Sullivan MJ, Bells S, Pasternak O, Jones DK (2012) How and how not to correct for CSF-contamination in diffusion MRI. Neuroimage 59(2):1394–1403.
Newman MEJ (2006). "Modularity and community structure in networks". Proc. Natl. Acad. Sci. U.S.A. 103 (23): 8577–8582.
Nielsen JA, Zielinski BA, Fletcher PT, et al. Multisite functional connectivity MRI classification of autism: ABIDE results. Frontiers in Human Neuroscience. 2013;7:599. doi:10.3389/fnhum.2013.00599.
Pasternak O, Sochen N, Gur Y, Intrator N, Assaf Y (2009) Free water elimination and mapping from diffusion MRI. Magn Reson Med 62(3):717–730.
Perry, A., Wen, W., Lord, A., Thalamuthu, A., Roberts, G., Mitchell, P., Sachdev, P., Breakspear, M., 2015. The organisation of the elderly connectome. Neuroimage 114, 414– 426
Prim RC. 1957. Shortest connection networks and some generalizations. Bell Syst. Tech. J. 36: 1389–1401.
Roberts JA, Perry A, Roberts G, Mitchell PB, Breakspear M4.Consisninecy-based thresholding
of the human connectome. Neuroimage. 2016 Sep 22. pii: S1053-8119(16)30531-6. Shoukri, M.M., Asyali, M.H., Donner, A., 2004. Sample size requirements for the design of reliability study: review and new results. Stat. Methods Med. Res. 13, 251–271
Shrout, P.E.; Fleiss, J.L. (1979). Intraclass correlations: Uses in assessing rater reliability. Psych. Bulletin, 86(2), pp. 420-428
Smith, R.E., Tournier, J.D., Calamante, F., Connelly, A., 2012. Anatomically-constrained tractography: improved diffusion MRI streamlines tractography through effective use of anatomical information. Neuroimage 62, 1924–1938
36
Oldfield RC (1971) The assessment and analysis of handedness: the Edinburgh inventory. Neuropsychologia 9(1):97–113
Thomas, C., Frank, Q.Y., Irfanoglu, M.O., Modi, P., Saleem, K.S., Leopold, D.A., Pierpaoli, C., 2014. Anatomical accuracy of brain connections derived from diffusion MRI tractography is inherently limited. Proc. Natl. Acad. Sci. USA 111, 16574–16579
Tzourio-Mazoyer, N., Landeau, B., Papathanassiou, D., Crivello, F., Etard, O., Delcroix, N.,
Mazoyer, B. and Joliot, M. (2002) ‘Automated anatomical labeling of activations in SPM
using a macroscopic anatomical parcellation of the MNI MRI single-subject brain.’, NeuroImage, vol. 15, no. 1, pp. 273–89.
van den Heuvel MP1, Sporns O. Rich-club organization of the human connectome. J Neurosci.
2011 Nov 2;31(44):15775-86. doi: 10.1523/JNEUROSCI.3539-11.2011.
Wu M, Chang LC, Walker L, Lemaitre H, Barnett AS, Marenco S, Pierpaoli C (2008) Comparison of EPI distortion correction methods in diffusion tensor MRI using a novel framework. Med Image Comput Comput Assist Interv 11(Pt 2):321–329
Yeh FC, Badre D, Verstynen T. Connectometry: A statistical approach harnessing the analytical potential of the local connectome. Neuroimage. 2016;125:162–71. pmid:26499808
Yo, T.S., Anwander, A., Descoteaux, M., Fillard, P., Poupon, C., Knösche, T.R., 2009. Quantifying brain connectivity: a comparative tractography study. Med. Image Comput. Comput. Assist. Interv. 12, 886–893.
Zalesky, A., Fornito, A., 2009. A DTI-derived measure of cortico-cortical connectivity. IEEE Trans. Med. Imaging 28, 1023–1036.
Zalesky, A., Fornito, A., Bullmore, E.T., 2010a. Network-based statistic: identifying differences in brain networks. NeuroImage 53, 1197–1207.
Zalesky A1, Fornito A2, Cocchi L3, Gollo LL3, van den Heuvel MP4, Breakspear M5.
Connectome sensitivity or specificity: which is more important? Neuroimage. 2016 Jun 28. pii: S1053-8119(16)30284-1. doi: 10.1016/j.neuroimage.2016.06.035.
Zuo XN, Anderson JS, Bellec P et al (2014a) An open science resource for establishing reliability and reproducibility in functional connectomics. Sci Data 1:140049
Zuo XN, Xing XX (2014b) Test–retest reliabilities of resting-state FMRI measurements in human brain functional connectomics: a systems neuroscience perspective. Neurosci Biobehav Rev 45:100–118
Zuo, X.N., He, Y., Betzel, R.F., Colcombe, S., Sporns, O., Milham, M.P., 2017. Human connectomics across the life span. Trends Cogn. Sci. 21, 32–45