Limitations and Future Works

Despite the technical contributions made by the work presented in this thesis, there are some limitations. Principally is the limitation of evaluation. For each of the framework and two visual extensions, we evaluated using case studies. In doing so, we made a conscientious effort to compare our approaches against quantitative measures. Similarly, by engaging and receiving feedback from fMRI experts, we have increased the strength of our evaluation. However, we were unable to present quantitative results ourselves. Likewise, our evaluation for the proposed Image FCN, while quantitative, was not exhaustive. This limitation on evaluation is largely derived from the purpose of the work in this thesis. Instead of presenting a single technical contribution and evaluating it with, e.g. a user study or clinical trial, our motivation was to define and establish the fMRI-UVA field with a framework and set of core works. These are based on the strong literature analysis in Chapter 2 which is not a common aspect of similar works and presents novel details and findings for fMRI-UVA. Thus our works are developed with a strong set of design guidelines and requirements analysis as a foundation. Therefore our framework is designed to be a base that is improved in future, by us and by encouraging other researchers to present their own solutions or perform robust evaluation.

There are numerous interesting areas for future works related to the visual analytics approaches presented in this thesis. These begin with further investigating the impact and usability of our framework (Chapter 4) with a user study. A full user study will be able to determine which aspects of the framework best minimise the human aspects of uncertainty, while maintaining the benefits in minimising and exposing data uncertainty. Similarly,

150

while we designed the framework for analysis and interpretation of fMRI FCN data, the parameters are not limited to this domain. Other 4D data types, with spatio-temporal dimensions, such as other medical imaging modalities, temporal astronomical observations and weather data could be visualised in the three components of the framework to minimise their own data uncertainties. The usability and impact of our framework can also be assessed on wider data types such as these.

Use of advanced display and human computer interface technologies, such as virtual, mixed and augmented reality (VMAR), are rapidly gaining momentum in clinical and medical research environments. As proposed in our paper [180], we believe that our framework and fMRI research will transition well to use with such technologies. The benefits of VMAR, will also apply to the two targeted extensions to our framework. Therefore, adapting the works presented in this thesis to best suit the technologies will be a focus of future work. Such efforts will need to overcome limitations of VMAR, such as the robustness to large volumes of data, infancy in presenting medical images, increased cognitive load and navigational problems.

In our first targeted extension to the framework (Chapter 5), we made a number of design decisions to minimise the statistical processing of the data as it introduced uncertainty. However, as more is understood about fMRI and the brain, statistical processing will be able to be implemented. Additionally, we can explore alternative presentations of the anatomical data in the transition images. Finally, while the tracks metaphor was presented in the context of fMRI, the technique may be applicable to a number of other areas, including other 4D medical images and complex spatiotemporal data, such as weather pattern data.

151

The threshold visual analytics solution (Chapter 6) is founded upon the k-core decomposition algorithm. However, this algorithm is not designed to consider weighted networks. Thus, in future we can further extend the algorithm for the purpose of fMRI threshold analysis by adapting it to consider edge weights. Moreover there may be improvements that can be made to the interactivity of the visualisation. These will be explored in connection to the solution. As with the other visualisation works presented in this thesis, our threshold viewer may be suited to other data types.

Our proposed coactivation measure (Chapter 7) was shown to improve the creation of FCNs. This method relied on image comparison, using traditional algorithms, such as comparing the texture properties from the grey level co-occurrence matrix or scale invariant feature transform descriptors. With the advent of widespread deep learning for image classification and comparison, we can explore the use of such techniques on our approach. However, to do so, we will either need to determine how to define labelled data sets, which will take research, or explore the range of unsupervised and, possibly pre-trained, methods to see which of them are well suited to the problem. Further, we can explore visualisation of the process, in which users can view and adjust comparison parameters to optimise the coactivation measure.

152

Appendix A

Small World Propensity Thresholding

Figures

153 Subjects 21 to 30

154 Subjects 31 to 40

155 Subjects 41 to 50

156 Subjects 51 to 60

157 Subjects 61 to 70

158 Subjects 71 to 80

159 Subjects 81 to 90

160

161

162

163

164

165

166

References

[1] D. S. Bassett and M. S. Gazzaniga, "Understanding complexity in the human brain," Trends in cognitive sciences, vol. 15, no. 5, pp. 200-209, 2011.

[2] E. B. Falk et al., "What is a representative brain? Neuroscience meets population science," Proceedings of the National Academy of Sciences of the United States of America, vol. 110, no. 44, pp. 17615-22, Oct 29 2013.

[3] S. Simpson, F. Bowman, and P. Laurienti, "Analyzing complex functional brain networks: fusing statistics and network science to understand the brain," Statistics surveys, vol. 7, pp. 1-36, 2013.

[4] S. M. Smith, "Overview of fMRI analysis," British Journal of Radiology, vol. 77 Spec No 2, pp. S167-75, 2004.

[5] M. P. Van Den Heuvel and H. E. H. Pol, "Exploring the brain network: a review on resting-state fMRI functional connectivity," European neuropsychopharmacology, vol. 20, no. 8, pp. 519-534, 2010.

[6] National Institute of Health. (2015). The Human Connectome Project. Available: http://www.humanconnectomeproject.org/

[7] National Institute of Health. (2016). The BRAIN Initiative. Available: http://www.braininitiative.nih.gov/

[8] M. Filippi and Filippi, fMRI techniques and protocols. Springer, 2009.

[9] R. Hindriks et al., "Can sliding-window correlations reveal dynamic functional connectivity in resting-state fmri?," NeuroImage, vol. 127, pp. 242-256, 2016. [10] S. Sarraf and G. Tofighi, "Classification of alzheimer's disease using fmri data and

deep learning convolutional neural networks," arXiv, 2016.

[11] J. Sui and V. D. Calhoun, "Multimodal Fusion of Structural and Functional Brain Imaging Data," fMRI Techniques and Protocols, pp. 853-869, 2016.

[12] D. M. Alexander, C. Trengove, and C. van Leeuwen, "Donders is dead: cortical traveling waves and the limits of mental chronometry in cognitive neuroscience," Cogn Process, Jul 3 2015.

[13] R. Yuste and A. Fairhall, "Temporal dynamics in fMRI resting-state activity," Proceedings of the National Academy of Sciences of the United States of America, vol. 112, no. 17, pp. 5263-5264, 2015.

[14] W. Huf et al., "On the generalizability of resting-state fmri machine learning classifiers," Frontiers in Human Neuroscience, vol. 8, no. 502, 2014.

[15] B. Thirion, G. Varoquaux, E. Dohmatob, and J. B. Poline, "Which fMRI clustering gives good brain parcellations?," Frontiers in Neuroscience, vol. 2, p. 167, 2014. [16] V. D. Calhoun, R. Miller, G. Pearlson, and T. Adalı, "The chronnectome: time-

varying connectivity networks as the next frontier in fMRI data discovery," Neuron, vol. 84, no. 2, pp. 262-274, 2014.

[17] M. H. Lee, C. D. Smyser, and J. S. Shimony, "Resting-state fMRI: a review of methods and clinical applications," American Journal of Neuroradiology, vol. 34, no. 10, pp. 1866-1872, 2013.

[18] D. S. Margulies, J. Böttger, A. Watanabe, and K. J. Gorgolewski, "Visualizing the human connectome.," NeuroImage, vol. 80, pp. 445--61, 2013.

[19] O. Sporns, "Contributions and challenges for network models in cognitive neuroscience," Nature Neuroscience, vol. 17, pp. 652-660, 2014.

167

[20] P. Hagmann et al., "MR connectomics: Principles and challenges," J Neurosci Methods, vol. 194, no. 1, pp. 34-45, Dec 15 2010.

[21] O. Sporns, Networks of the Brain. MIT Press, 2010.

[22] E. Bullmore and O. Sporns, "Complex brain networks: graph theoretical analysis of structural and functional systems," Nature Reviews Neuroscience, vol. 10, no. 3, pp. 186-198, 2009.

[23] A. Eklund, T. Nichols, and H. Knutsson, "Can parametric statistical methods be trusted for fMRI based group studies?," Proceedings of the National Academy of Sciences of the United States of America, vol. 113, no. 28, pp. 7900-7905, 2016. [24] E. S. Finn, D. Scheinost, X. Shen, X. Papademetris, and R. T. Constable,

"Methodological Issues in fMRI Functional Connectivity and Network Analysis," in Brain Mapping: An Encycopedic Reference, vol. 1, A. W. Toga, Ed.: Academic Press, 2015.

[25] M. T. R. Stevens, R. C. Darcy, G. Stroink, D. B. Clarke, and S. D. Beyea, "Thresholds in fmri studies: reliable for single subjects?," Journal of neuroscience methods, vol. 219, no. 2, pp. 312-323, 2013.

[26] J. Carp, "On the plurality of (methodological) worlds: estimating the analytic flexibility of FMRI experiments," Frontiers in Neuroscience, vol. 6, p. 149, 2012. [27] A. Eklund, T. E. Nichols, and H. Knutsson, "Cluster failure: why fMRI inferences

for spatial extent have inflated false-positive rates," Proceedings of the National Academy of Sciences, p. 201602413, 2016.

[28] A. Eklund, M. Andersson, C. Josephson, M. Johannesson, and H. Knutsson, "Does parametric fMRI analysis with SPM yield valid results? An empirical study of 1484 rest datasets," NeuroImage, vol. 61, no. 3, pp. 565-78, Jul 2 2012.

[29] S. P. David et al., "Potential reporting bias in fMRI studies of the brain," PLoS One, vol. 8, no. 7, p. e70104, 2013.

[30] M. D. Lieberman and W. A. Cunningham, "Type I and Type II error concerns in fMRI research: re-balancing the scale," Social cognitive and affective neuroscience, vol. 4, no. 4, pp. 423-428, 2009.

[31] K. Murphy, J. Bodurka, and P. A. Bandettini, "How long to scan? The relationship between fMRI temporal signal to noise ratio and necessary scan duration," NeuroImage, vol. 34, no. 2, pp. 565-574, 2007.

[32] S. Strother, "Evaluating fMRI preprocessing pipelines," EMBC, vol. 25, no. 2, 2006.

[33] C. Stark and L. Squire, "When zero is not zero: The problem of ambiguous baseline conditions in fMRI," Proceedings of the National Academy of Sciences of the United States of America, vol. 98, no. 22, pp. 12760-12766, 2001.

[34] K. J. Gorgolewski et al., "NeuroVault. org: a repository for sharing unthresholded statistical maps, parcellations, and atlases of the human brain," NeuroImage, vol. 124, pp. 1242-1244, 2016.

[35] M. de Ridder, K. Klein, and J. Kim, "TemporalTracks: visual analytics for exploration of 4D fMRI time-series coactivation," in Proceedings of the Computer Graphics International Conference, 2017, p. 13: ACM.

[36] G. Ristovski, T. Preusser, H. K. Hahn, and L. Linsen, "Uncertainty in medical visualization: Towards a taxonomy," Compters and Graphics, vol. 39, pp. 60-73, 2014.

[37] L. Hernandez-Garcia, S. Peltier, and W. Grissom, "Introduction to Functional MRI Hardware," fMRI Techniques and Protocols, pp. 29-67, 2016.

168

[38] S. M. Smith et al., "Functional connectomics from resting-state fMRI," Trends in cognitive sciences, vol. 17, no. 12, pp. 666-682, 2013.

[39] R. Swenson, Review of Clinical and Functional Neuroscience. Dartmouth MEdical School, 2006.

[40] V. S. Fonov, A. C. Evans, R. C. McKinstry, C. R. Almli, and D. L. Collins, "Unbiased nonlinear average age-appropriate brain templates from birth to adulthood," NeuroImage, vol. 47, p. S102, 2009/07/01/ 2009.

[41] M. E. Raichle, A. M. MacLeod, A. Z. Snyder, W. J. Powers, D. A. Gusnard, and G. L. Shulman, "A default mode of brain function," Proceedings of the National Academy of Sciences of the United States of America, vol. 98, no. 2, pp. 676-682, 2001.

[42] D. M. Barch et al., "Function in the human connectome: task-fMRI and individual differences in behavior," Neuroimage, vol. 80, pp. 169-189, 2013.

[43] C. D. Correa, C. Yu-Hsuan, and K.-L. M., "A framework for uncertainty-aware visual analytics.," presented at the Visual Analytics Science and Technology, 2009. [44] D. Sacha, H. Senaratne, B. Kwon, G. Ellis, and K. DA, "The Role of Uncertainty,

Awareness, and Trust in Visual Analytics," IEEE Transactions on Visualization and Computer Graphics, vol. 22, no. 1, pp. 240-249, 2016.

[45] A. Saad, T. Möller, and G. Hamarneh, "ProbExplorer: Uncertainty‐guided Exploration and Editing of Probabilistic Medical Image Segmentation," in Computer Graphics Forum, 2010, vol. 29, no. 3, pp. 1113-1122: Wiley Online Library.

[46] G. P. Bonneau et al., "Overview and State-of-the-Art of Uncertainty Visualization," in Scientific VisualizationLondon: Springer, 2014.

[47] K. Potter et al., "Ensemble-vis: A framework for the statistical visualization of ensemble data," in Data Mining Workshops, 2009. ICDMW'09. IEEE International Conference on, 2009, pp. 233-240: IEEE.

[48] L. Harrison, X. Hu, X. Ying, A. Lu, W. Wang, and X. Wu, "Interactive detection of network anomalies via coordinated multiple views," in Proceedings of the Seventh International Symposium on Visualization for Cyber Security, 2010, pp. 91-101: ACM.

[49] H. Li, C.-W. Fu, Y. Li, and A. Hanson, "Visualizing large-scale uncertainty in astrophysical data," IEEE Transactions on Visualization and Computer Graphics, vol. 13, no. 6, pp. 1640-1647, 2007.

[50] X. Yang, L. Shi, M. Daianu, H. Tong, Q. Liu, and P. Thompson, "Blockwise Human Brain Network Visual Comparison Using NodeTrix Representation," IEEE Transactions on Visualization and Computer Graphics, vol. 23, no. 1, pp. 181-190, 2017.

[51] B. Bach, N. Henry Riche, T. Dwyer, T. Madhyastha, J. Fekete, and T. Grabowski. (2015). Small MultiPiles: Piling Time to Explore Temporal Patterns in Dynamic Networks. Available: http://www.aviz.fr/~bbach/multipiles/

[52] J. Böttger, A. Schäfer, and G. Lohmann, "Three-Dimensional Mean-Shift Edge Bundling for the Visualization of Functional Connectivity in the Brain," IEEE Transactions on Visualization and Computer Graphics, vol. 20, no. 3, pp. 471-480, 2014.

[53] O. Demirci et al., "A review of challenges in the use of fmri for disease classification / characterization and a projection pursuit application from a multi- site fmri schizophrenia study," Brain Imaging and Behavior, vol. 2, no. 3, pp. 207- 226, 2008.

169

[54] M. G. Bright and K. Murphy, "Is fMRI "noise" really noise? Resting state nuisance regressors remove variance with network structure," NeuroImage, vol. 114, pp. 158-69, Jul 1 2015.

[55] S. A. Huettel, A. W. Song, and G. McCarthy, Functional magnetic resonance imaging. Sinauer Associates Sunderland, 2004.

[56] A. C. Silva and H. Merkle, "Hardware considerations for functional magnetic resonance imaging," Concepts in Magnetic Resonance Part A, vol. 16, no. 1, pp. 35-49, 2003.

[57] R. Sladky, K. J. Friston, J. Tröstl, R. Cunnington, E. Moser, and C. Windischberger, "Slice-timing effects and their correction in functional MRI," NeuroImage, vol. 58, no. 2, pp. 588-594, 2011.

[58] S. Clare, R. Bowtell, and P. Morris, "Ghost artefact in fMRI: comparison of techniques for reducing the N/2 ghost," in Proceedings of the ISMRM, on CD-ROM, 1998, p. 2137.

[59] Y. Zhang et al., "Robust brain parcellation using sparse representation on resting- state fMRI," Brain Structure and Function, vol. 220, no. 6, pp. 3565-3579, 2015. [60] M. F. Glasser et al., "A multi-modal parcellation of the human cerebral cortex,"

Nature, vol. 536, no. 7615, pp. 171-178, 2016.

[61] M. Giraldo-Chica and N. D. Woodward, "Review of thalamocortical resting-state fmri studies in schizophrenia," Schizophrenia Research, vol. 6, 2016.

[62] N. D. Woodward, M. S. Karbasforoushan, and S. Heckers, "Thalamocortical dysconnectivity in schizophrenia," American Journal of Psychiatry, vol. 169, no. 10, 2012.

[63] A. Fornito, A. Zalesky, and E. T. Bullmore, "Network scaling effects in graph analytic studies of human resting-state FMRI data," Frontiers in systems neuroscience, vol. 4, p. 22, 2010.

[64] X. Liu and J. H. Duyn, "Time-varying functional network information extracted from brief instances of spontaneous brain activity," Proceedings of the National Academy of Sciences of the United States of America, vol. 110, no. 11, pp. 4392- 4397, 2013.

[65] A. Riaz et al., "FCNet: A Convolutional Neural Network for Calculating Functional Connectivity from functional MRI," in International Workshop on Connectomics in Neuroimaging, 2017, pp. 70-78: Springer.

[66] R. T. Constable et al., "Potential use and challenges of functional connectivity mapping in intractable epilepsy," Frontiers in Neurology, vol. 4, no. 39, 2013. [67] M. Rubinov and O. Sporns, "Complex network measures of brain connectivity: uses

and interpretations.," NeuroImage, vol. 52, no. 3, pp. 1059--69, 2010.

[68] J. Dubois and R. Adolphs, "Building a science of individual differences from fMRI," Trends in cognitive sciences, vol. 20, no. 6, pp. 425-443, 2016.

[69] S. Oviatt, "Human-Centered Design Meets Cognitive Load Theory : Designing Interfaces that Help People Think," pp. 871-880, 2006.

[70] Harvard Laboratory for Computational Neuroimaging. (2017). FreeSurfer. Available: https://surfer.nmr.mgh.harvard.edu/

[71] O. U. FMRIB Analysis Group. (2016). FSL. Available: http://fsl.fmrib.ox.ac.uk/ [72] Wellcome Trust Centre for Neuroimaging. (2014). SPM12. Available:

http://www.fil.ion.ucl.ac.uk/spm/software/spm12/

[73] National Institute of Health. (2016). AFNI. Available: https://afni.nimh.nih.gov/ [74] B. Bach, C. Shi, N. Heulot, T. Madhyastha, T. Grabowski, and P. Dragicevic, "Time

170

Transactions on Visualization and Computer Graphics, vol. 22, no. 1, pp. 559-568, 2016.

[75] R. Inano et al., "Visualization of heterogeneity and regional grading of gliomas by multiple features using magnetic resonance-based clustered images," Scientific Reports, vol. 6, 2016.

[76] A. Irimia, M. C. Chambers, C. M. Torgerson, and J. D. Van Horn, "Circular representation of human cortical networks for subject and population-level connectomic visualization," NeuroImage, vol. 60, no. 2, pp. 1340-51, Apr 2 2012. [77] J. S. Gao, A. G. Huth, M. D. Lescroart, and J. L. Gallant, "Pycortex: an interactive

surface visualizer for fMRI," Frontiers in Neuroinformatics, vol. 9, 2015.

[78] J. Böttger, R. Schurade, E. Jakobsen, A. Schaefer, and D. S. Margulies, "Connexel visualization: a software implementation of glyphs and edge-bundling for dense connectivity data using brainGL," Frontiers in Neuroscience, vol. 8, p. 15, 2014. [79] D. A. Angulo, C. Schneider, J. H. Oliver, N. Charpak, and J. T. Hernandez, "A

multi-facetted visual analytics tool for exploratory analysis of human brain and function datasets," Frontiers in Neuroinformatics, vol. 10, 2016.

[80] M. de Ridder, K. Klein, and J. Kim, "CereVA-Visual Analysis of Functional Brain Connectivity," in IVAPP, 2015, pp. 131-138.

[81] M. Xia, J. Wang, and Y. He, "BrainNet Viewer: a network visualization tool for human brain connectomics," PLoS One, vol. 8, no. 7, p. e68910, 2013.

[82] M. P. Milham, D. Fair, M. Mennes, and S. H. Mostofsky, "The ADHD-200 consortium: a model to advance the translational potential of neuroimaging in clinical neuroscience," Frontiers in systems neuroscience, vol. 6, p. 62, 2012. [83] P. Bellec, C. Chu, F. Chouinard-Decorte, Y. Benhajali, D. S. Margulies, and R. C.

Craddock, "The neuro bureau ADHD-200 preprocessed repository," Neuroimage, vol. 144, pp. 275-286, 2017.

[84] The Mathworks Inc., "MATLAB," ed. Natick, Massachussetts, United States, 2017. [85] J. Shen. (2011). Tools for NIfTI and ANALYZE image. Available:

https://au.mathworks.com/matlabcentral/fileexchange/8797-tools-for-nifti-and- analyze-image

[86] A. Vedaldi and B. Fulkerson. (2008). VLFeat: An Open and Portable Library of Computer Vision Algorithms. Available: http://www.vlfeat.org/

[87] J. Faouzi. (2017). pyts: a Python package for time series transformation and classification. Available: https://github.com/johannfaouzi/pyts

[88] K. Group. (2012, 15/04/2014). WebGL: OpenGL ES 2.0 for the Web. Available: http://www.khronos.org/webgl/

[89] M. Bostock, "D3. js-data-driven documents (2016)," URL: https://d3js. org, 2016. [90] D. Hähn, N. Rannou, B. Ahtam, P. E. Grant, and R. Pienaar, "The X Toolkit:

WebGL for Scientific Visualization," ed, 2014.

[91] Research Imaging Institute University of Texas Health Science Center. (2017). Papaya. Available: http://ric.uthscsa.edu/mango/papaya.html

[92] jQuery Team, "JQuery," ed, 2006.

[93] S. Wang et al., "Abnormal regional homogeneity as a potential imaging biomarker for adolescent-onset schizophrenia: A resting-state fMRI study and support vector machine analysis," Schizophrenia research, vol. 192, pp. 179-184, 2018.

[94] K. Friston, H. R. Brown, J. Siemerkus, and K. E. Stephan, "The dysconnection hypothesis (2016)," Schizophrenia research, vol. 176, no. 2, pp. 83-94, 2016. [95] Y. I. Sheline and M. E. Raichle, "Resting state functional connectivity in preclinical

171

[96] C. Bürger et al., "Differential Abnormal Pattern of Anterior Cingulate Gyrus Activation in Unipolar and Bipolar Depression: an fMRI and Pattern Classification Approach," Neuropsychopharmacology, vol. 42, no. 7, p. 1399, 2017.

[97] B. Rashid et al., "Classification of schizophrenia and bipolar patients using static