Pattern Recognition Approaches in Biomedical and Clinical Magnetic Resonance Spectroscopy ^
3.4 Approaches to MRS data analysis
3.4.9 Genetic and evolutionary computing
Genetic algorithms (GAs) are a class of heuristic search algorithms inspired by the mechanism of evolution and population genetics in biological systems. The data is represented by population of individuals whose parameters are encoded into ’chrom osom es’. The individuals in the population then go through a process of simulated evolution to optimise the encoded parameters by ’crossover’ and ’mutation’ -like manipulations. Holland (1992) and Goldberg (1988) are perhaps the most cited references in the field of GAs; although the idea of using simulated evolution for optimisation goes back to the 1950s (Fraser, 1957; Friedman, 1959; Bremermann, 1962). GAs are an attractive optimisation tool for spectral analysis since they are able to search large parameter space using multiple optimisation criteria (Wienke et al, 1992), are less sensitive to initial conditions and, to some extent, can cope with local minima (Hibbert, 1993). Clouser and Jurs (1995) use GAs to select the best descriptors of a molecular structures to predict the spectra. Feature selection is perhaps one of the natural applications of GAs (Leardi at al, 1992; Kubinyi, 1996), though optimal solutions are not guaranteed. Deweijer at al (1995) developed a curve fitting procedure to resolve the overlapping peaks of IR-spectra from polymer yarns. However, in spite of the above advantages, GAs, by their very nature as a random search technique, are considerably slow and will only converge to suboptimal solutions.
In comparison to the growing interest in GAs in analytical and computational chemistry (Hibbert, 1993; Jouan-Rimbaud at al, 1995; Judson, 1996) and biomolecular NMR (Zimmerman and Montelione, 1995), development of GAs in biomedical MRS has not been as fast. Fisher at al (1995) use GAs to optimise the design parameters of a Z- gradient coil for MRI. Gray at al (1996) used genetic programming to generate different objective functions that utilise combinations of the 2 0 first principal components of "'H spectra of brain tumour extracts. Their results suggests that combinations of glutamine and glutamate or glutamine and alanine are important to differentiate between meningiomas and other brain tumours. Somorjai ef a/(1995a) used genetic programming as part of the consensus classification strategy to generate decision rules and select optimal classifiers, while Nikulin e fa /(1 9 9 5 ) used the algorithms as a pre-processing tool for selecting dominant regions in ^H ax-vlvo spectra of high and low grade astorocytoma. Metzger at al 996) and Patel at al 995) used GA to optimise the frequency and decay parameters of the function governing the shape of the peaks in nonlinear peak fitting for quantification both in the time and frequency domains.
Chapter 3: Pattern recognition in MRS , 62
Summary
In this chapter we have attempted to cover the basic theory and application of the majority of pattern recognition techniques commonly used and reported in the literature of biomedical MRS. We have focused on the importance of data representation and dimensionality reduction to design robust classifiers, discussed the strengths and weaknesses of various classification techniques and suggested new approaches to further experiments. In so doing several points become clear:
• Complex processes, like cellular and molecular biochemistry, are difficult for traditional statistical models to capture (Burke, 1995). As the decision rules for classification or prediction get more complex, the amount of training data required to construct the decision boundary increases. With small sample size and large dimensionality, the decision becomes ill-posed with the risk of overfitting and poor generalisation.
• In the case of data driven models for discrimination or regression, the data solely estimate the parameters of the pattern recognition system and determine its domain of operation. In the case of modelling MRS, where the data are complex and noisy, adaptive techniques that attempt to capture the inherent structure of the data or the underlying distribution are more suited for the analysis.
• Classical analysis of only few major peaks may miss important clinical information. Large and apparently dominant peaks in a spectrum are not necessarily important when it comes to a specific classification task.
• Neural networks are a class of statistical pattern recognition techniques with flexible complexity making them easy to adapt for a variety of MRS applications. The next chapter expands on some properties of feedforward networks that are of special interest to MRS/pattern recognition applications.
References
Abu-Mostafa, Y.S., 1989. The Vapnik-Chervonenkis dimension: information versus complexity in learning. Neural Computation ^, 312-317.
Akay, M., 1995. Wavelets in biomedical engineering. Annals of Biomed. Eng. 23, 531-542.
Ala-Korpela, M., Hiltunen Y. and Bell, J.D. 1995. Quantification of biomedical NMR data using artificial neural-network analysis - lipoprotein lipid profiles from ^H-NMR data of human plasma,
NMR Biomed. 8, 235-244.
Ala-Korpela, M., Hiltunen, Y. and Bell, J.D. 1996. Classification of plasma lipid abnormalities by "'H MRS and self-organising neural networks. Proc. of ISMRM, p.1172.
Anderson, J.A. and Rosenfeld, E. (eds.) 1988. Neurocomputing: foundations of research.
Cambridge, MA: MIT Press.
Andrasi, E., Varga, I., Dozsa, A., Reffy, A. and Nagy, G.J. 1994. Classification of human brain parts using pattern recognition based on inductively coupled plasma atomic emission spectroscopy and instrumental neutron activation analysis. Chemometrics and intelligent Laboratory Systems. 22,
Anthony, M.L. 1993. High resolution NMR spectroscopic and pattern recognition approaches to the biochemical characterisation of experimental nephrotoxicity states. Ph.D. thesis, University of London.
Anthony, M.L., Rose, V.S., Nicholson, J.K. and Linden, J.C. 1995. Classification of toxin-induced changes in NMR spectra of urine using an artificial neural network. J. Pharmaceutical and Biomedical Analysis 13, 205-211.
Anthony, M.L., Sweatman, B.C., Beddell, C.R., Linden, J.C. and Nicholson, J.K. 1994. Pattern recognition classification of the site of nephrotoxicity based on metabolic data derived from proton nuclear magnetic resonance spectra of urine. Molecular Pharmacology ^6, 199-211.
Antz, C., Neidig, K.P. and Kalbitzer, H.R. 1995. A general Bayesian method for an automated signal class recognition in 2D- NMR spectra combined with a multivariate discriminant-analysis J. Biomolecular NMR 5, 287-296.
Aslaksen, E.W., Klauder, J.R. Unitary representations of the offline group. 1968. J. Math. Physics.
9, 206-211.
Aston, M.L. and Wilding, P. 1992. Application of neural networks to the interpretation of laboratory data in cancer-diagnosis. Clinical chemistry 38, 34-38.
Battiti, R. 1992. First- and second-order methods for learning: between steepest descent and Newton method. Neural computation 141-166.
Belue, L.M. and Bauer, K.E. Jr. 1995. Determining input features for multilayer perceptrons.
Neurocomputing 7, 111-121.
Ben-Bassat, M. 1980. On the sensitivity of probability of error rule for feature selection. IEEE Trans. PAMI2, 57-60.
Ben-Bassat, M. 1982. Use of distance measures, information measures and error nounds in feature evaluation. In: Handbook of Statistics(2), Classification, Pattern Recognition and Reduction of Dimensionality {Kr\shna\ah, P.R. and. Kanal, L.N. eds.) Amsterdam: North-Holland.
Bishop, C. M. 1995. Neural networks for pattern recognition. Oxford: Clarendon Press.
Boberg, J. and Salakoski, T. 1993. General formulation and evaluation of agglomerative clustering methods with metric and non-metric distances. Pattern Recognition 26, 1395-1406.
Bos, M. and Vrielink, J.A.M. 1996. The wavelet transform for pre-processing IR spectra in the identification of mono- and di-substituted benzenes. Chemometrics and Intelligent Laboratory Systems 23, 115-122.
Branston, N.M., Maxwell, R.J. and Howells, S.L. 1993. Generalization performance using backpropagation algorithms applied to patterns derived from tumor ^H-NMR spectra. J. Microcomputer Appi 16, 113-123.
Bremermann, H.J. 1962. Optimization through evolution and recombination. In: Self-Organizing Systems (M.C. Yovits etal, eds.) Washington, DC: Spartan Books.
Bridle, J.S. 1990. Probabilistic interpretation of feedforward classification network outputs, with relationships to statistical pattern recognition. In Neurocomputing: algorithms, architectures and applications (F. F. Soulié, and J. Hérault, eds.) New York: Springer-Verlag.
Bruce, A. Donoho, D. and Gao, H.Y. 1996. Wavelet analysis. IEEE Spectrum, 33, 26-35.
Bryan, J.G. 1951. The generalized discriminant function: mathematical foundation and computational issues. Harvard Educational Review 2A, 90-95.
Buckheit, J., Chen, S., Donoho, D., Johnstone, I., Scargle, J. and Yu, T. 1996. WaveLab .701, http://playfair.Stanford.EDU/~wavelab/.
Burke, H.B. 1995. The importance of artificial neural networks in biomedicine. WCNN, Washington DC, p 11-725-730.
Burns, J.A. and Whitesides, G.M. 1993. Feedforward neural networks in chemistry - mathematical systems for classification and pattern-recognition. Chemical Reviews 93, 2583-2601.
Caianiello, E.R. 1961. Outline of a theory of thought-processes and thinking machines. J. Theoretical Biology 1, 204-235.
C hapter 3; Pattern recognition in MRS 64 Calderon, A.P. 1964. Intermediate spaces and interpolation, the complex method. Studia Math. 24, 113-190.
Carrara, E.A., Pagliari, F. and Nicolini, C. 1993. Neural networks for the peak-picking of Nuclear Magnetic Resonance spectra. Neural Networks 6, 1023-32.
Chabrol, B., Salvan, A.M., Confort Gouny, S., Vion Dury, J. and Cozzone, P.J. 1995. Localized proton magnetic resonance spectroscopy of the brain differentiates the inborn metabolic encephalopathies in children. Comptes Rendus de (’Académie des Sciences - Serie III. 318, 985- 992.
Chau, F.T., Shih, T.M., Gao, J.B. and Chan, C.K. 1996. Application of the fast wavelet transform method to compress ultraviolet-visible spectra. Applied Spectroscopy. 50, 339-348.
Cheng, B. and Titterington, D.M. 1994. Neural networks: a review from a statistical perspective.
Statistical Science9, 2-30.
Choakjarernwanit, N. 1992. Feature selection in statistical pattern recognition. Ph.D. Thesis, University of Surrey, UK.
Chui, C. K. 1992. An introduction to wavelets. Boston, MA: Academic Press.
Cios, K.J. and Liu, N.A. 1992. Machine learning-method for generation of a neural network architecture - a continuous ID3 algorithm. IEEE Trans.on Neural Networks 3, 280-291.
Clouser, D.L. and Jurs, P.C. 1995. Simulation of the nuclear-magnetic-resonance spectra of trisaccharides using multiple linear-regression analysis and neural networks. Carbohydrate Research 271, 65-77.
Confort Gouny, S., Vion Dury, J., Nicoli, F., Dano, P., Donnet, A., Grazziani, N., Gastaut, J.L., Grisoli, F. and Cozzone, P.J.A. 1993. Multiparametric data analysis showing the potential of localized proton MR spectroscopy of the brain in the metabolic characterization of neurological diseases. J. the Neurological Sciences, 118, 123-133.
Corne, S.A. 1996. Artificial neural networks for pattern recognition. Concepts in Mag. Res. 8, 303- 324.
Corne, S.A. and Johnson, A.P. 1992. An artificial neural network for classifying cross peaks in two- dimensional spectra. J. of Mag. Res. 100, 256-266.
Corne, S.A., Fisher, J., Johnson, A.P. and Newell, W.P. 1993. Cross-peak classification in 2- dimensional nuclear magnetic resonance spectra using a 2 layer neural network. Analytica Chimica Acta 278, 149-158.
Daubechies, I. 1988. Orthonormal bases of compactly supported wavelets. Comm, in Pure and Applied Math. 41, 909-996.
Daubechies, I. 1990. The wavelet transform: time-frequency localization and signal analysis. IEEE Trans. Info. Theory 5,961-1005.
Derome, A.E. 1987. Modern NMR techniques for chemistry research. Oxford: Pergamon Press Ltd. Devijver, P.A and Kittler, J. 1982. Pattern recognition: a statistical approach. Englewood Cliffs: Prentice-Hall.
Deweije, A.P., Buydens, L., Kateman, G. and Heuvel, H.M. 1995. Spectral curve-fitting of infrared- spectra obtained from semicrystalline polyester yarns. Chemometrics and Intelligent Laboratory Systems28, 149-164.
Dolenko, B. and Somorjai, R.L. 1995. Time well spent: preprocessing of MR spectra for greater classification accuracy. Proc. ISMRM, p.1936.
Duda, R.O. and Hart, P.E. 1973. Pattern classification and scene analysis. NY: Wiley.
Ediund, U. and Grahn, H. 1991. Multivariate data analysis of NMR data. J. of Pharma, and Biomed. Analysis9, 655-658.
El-Deredy, W. 1997. Pattern recognition approaches in biomedical and clinical magnetic resonance spectroscopy: A review. NMR Biomed. 10, 99-124.
Esteban, D. and Galland, 0. 1977. Application of quadrature mirror filters to split-band vioce coding schemes. Proc. IEEE Int. Conf. Acoustics Signal and Speech Processing, Hartford, Connecticut, 191-195.
Everitt, B.S. 1980. Cluster analysis. 2nd edition, London: Heinemann.
FAQ fuzzy logic and fuzzy expert systems. 1996. http://www.cis.ohio-state.edu/hypertext/faq /Usenet/ fuzzy-logic/
Farrant, R.D., Lindon, J.C., Rahr, E. and Sweatman, B.C. 1992. An automatic data reduction and transfer method to aid pattern recognition analysis and classification of NMR spectra. J. of Pharma, and Biomed. Analysis10, 141 -144.
Fisher, B.J., Dillon, N., Carpenter, T.A. and Hall, L.D. 1995. Design by genetic algorithm of a z- gradient set for magnetic-resonance-imaging of the human brain. Measurement, Science & Technology 6, 904-909.
Fisher, R.A. 1936. The use of multiple measurements in taxonomic problems. Annals of Eugenics 7, 179-188.
Florian, C., Preece, N.E., Bhakoo, K.K., Williams, S.R. and Noble, M. 1995. Characteristic metabolic profiles revealed by ^H nuclear magnetic resonance spectroscopy for three types of human brain and nervous system tumours. NMR Biomed. 8, 253-264.
Fluge, 0 ., Gilje, K.S., Sletten, E., Kvalheim, O.M., Skaarland, E., Halvorsen, J.F., Farstad, M.S. and Soreide, O. 1996. Proton nuclear magnetic resonance spectroscopy of serum from patients with colorectal neoplasia. Euro. J. of Surgical Oncology 22, 78-83.
Fossel E.T., Carr J.M. and McDonagh, J. 1986. Detection of malignant tumors. Water-suppressed proton nuclear magnetic resonance spectroscopy of plasma. New England J. of Med. 315, 1369- 1376.
Fraser, A.S. 1957. Simulation of genetic systems by automatic digital computers. Australian J. of Biological Sciences, 10, 484-491.
Friedman, G.J. 1959. Digital simulation of an evolutionary process. General Systems Yearbook, 4:171-184.
Fukunaga, K. 1990. Introduction to statistical pattern recognition. 2nd Edn., Boston: Academic Press.
Gartland, K.P.R., Beddell, C.R., Lindon, J.C. and Nicholson, J.K. 1990. A pattern recognition approach to the comparison of PMR and clinical chemical data for classification of nephrotoxicity. J. of Pharma, and Biomed. Analysis 8, 963-968.
Gartland, K.P.R., Beddell, C.R., Lindon, J.C. and Nicholson, J.K. 1991. Application of pattern recognition methods to the analysis and classification of toxicological data derived from proton nuclear magnetic resonance spectroscopy of urine. Molecular Pharmacology 39, 629-642.
Gezelter, J.D. and Freeman, R. Use of neural networks to design shaped radiofrequency pulses. J. Mag. Res.90, 397-404.
Ghauri, F.Y.K., Nicholson, J.K., Sweatman, B.C., Wood, J., Beddell, C.R., Lindon, J.C. and Cairns, N.J. 1993. NMR-spectroscopy of human postmortem cerebrospinal-fluid: distinction of Alzheimers- disease from control using pattern recognition and statistics. NMR Biomed.6,163-167.
Girosi, F., Jones, M. and Poggio, T. 1995. Regularization theory and neural networks architectures.
Neural Computation 7,219-269.
Goldberg, D.E. 1988. Genetic algorithms in search, optimization, and machine learning, MA: Addison-Wesley.
Gray, H.F., Maxwell, R.J., Martfnez-Pérez, I., Arus, C. and Cerdan, S. 1996. Genetic programming for classification of brain tumours from ^H NMR biopsy spectra. Proc. of ISMRM. p. 411.
Grossberg, S. 1968. Some nonlinear networks capable of learning a spatial pattern of arbitrary complexity. Proc. Natl. Acad. Sci. USA59, 368-372.
C hapter 3: Pattern recognition in MRS 66 Hagberg, G., Burlina, A.P., Mader, I., Roser, W., Radue, E.W. and Seelig, J. 1995. In vivo proton MR spectroscopy of human gliomas: definition of metabolic coordinates for multi-dimensional classification. Mag. Res. Med.34, 242-252.
Hanaoka, H., Yoshioka,Y., lto,l., Niitu, K. and Yasuda, N. 1993. In vitro characterization of lung cancers by the use of 1H nuclear magnetic resonance spectroscopy of tissue extracts and discriminant factor analysis. Mag. Res. Med.29, 436-440.
Hare, B.J. and Prestegard, J.H. 1994. Application of neural networks to automated assignment of NMR-spectra of proteins. J. Biomolecular NMR 4, 35-46.
Hebb, D.O. 1949. The organization of behavior: a neuropsychological theory. New York: Wiley. Hibbert, D.B. 1993. Genetic algorithms in chemistry. Chemometrics and Intelligent Laboratory Systems19, 277-293.
Highleyman, W.H. 1962. Linear decision functions, with applications to pattern recogntion. Proc. IRE. 50, 1501-1514.
Hiltunen, Y., Heiniemi, E. and Ala-Korpela, M. 1995. Lipoprotein lipid quantification by neural- network analysis of '’ H-NMR data from human blood-plasma. J. Mag. Res. Series B,106, 191-194. Holland, J.H. 1992. Adaptation in natural and artificial systems. Cambridge, MA: MIT Press.
Holmes, E., Bonner, F.W., Sweatman, B.C., Lindon, J.C., Beddell, C.R., Rahr, E. and Nicholson J.K. 1992. Nuclear magnetic resonance spectroscopy and pattern recognition analysis of the biochemical processes associated with the progression of and recovery from nephrotoxic lesions in the rat induced by mercury(ll) chloride and 2- bromoethanamine. Molecular Pharmacology. 42, 922- 930.
Holmes, E., Foxall, P.J.D., Nicholson, J.K., Neild, G.H., Brown, S.M., Beddell, C.R., Sweatman, B.C., Rahr, E., Lindon, J.C., Spraul, M. and Neidig, P. 1994. Automatic data reduction and pattern recognition methods for analysis of ^H nuclear magnetic resonance spectra of human urine from normal and pathological states. Analytical Biochemistry.220, 284-296.
Holschneider, M. 1995. Wavelets: an analysis tool.Oxford: Clarendon Press.
Hopfield, J.J. 1982. Neural networks and physical systems with emergent collective computational abilities. Proc. Natl. Acad. Sci. USA79, 2554-2558.
Hornik, K., Stinchcombe, M. and White, H. 1989. Multilayer feedforward networks are universal approximators. Neural Networks2, 359-366.
Hornik, K., Stinchcombe, M. and White, H. 1990. Universal approximation of an unknown mapping and its derivatives using multilayer feedforward networks. Neural Networks. 3, 551-560.
Hotelling, H. 1933. Analysis of a complex of statistical variables into principal components. J. Educational Psychology. 24, 417-441 498-520.
Howells, S.L., Maxwell, R.J., Peet, A.C. and Griffiths, J.R. 1992a. An investigation of tumor ^H nuclear magnetic resonance spectra by the application of chemometric techniques. Mag. Res. Med.
28, 214-236.
Howells, S.L., Maxwell, R.J. and Griffiths, J.R. 1992b. Classification of tumour ^H NMR spectra by pattern recognition. NMR Biomed.5, 59-64.
Howells, S.L., Maxwell, R.J., Howe, F.A., Peet, A.C., Stubbs M., Rodrigues L.M., Robinson, S.P., Baluch, S. and Griffiths, J.R. 1993. Pattern recognition of 3 ip magnetic resonance spectroscopy tumour spectra obtained in vivo. NMR Biomed. 6, 237-241.
Johnson, R.A. and Wichen, D.W. 1992. Applied multivariate statistical analysis. 3rd ed, London: Prentice-Hall International.
Jolliffe, I.T. 1986. Principal component analysis. New York: Springer-Verlag.
Jones M.C., and Sibson, R. 1987. What is a projection pursuit? J. of the Royal Statistical Society series A150, 1-36.
Jouan-Rimbaud, D., Massart, D.L., Leardi, R. and Denoord, O.E. 1995. Genetic algorithms as a tool for wavelength selection in multivariate calibration. Analytical Chemistry S7, 4295-430.
Judson R.S. 1996. Genetic algorithms and their uses in chemistry. Reviews in Computational Chemistry AO, 1-73.
Karhunen, K. 1947. Uber lineare methoden der Wahrscheinlichkeitsrechnung. Annales Academiae Scientiarum Fennicae, Series A1: Mathematica-Physica 37, 3-79 (Trans.: RAND Corp., Santa Monica, GA, Rep. T-131, 1960).
Kari, S., Olsen, N.J. and Park, J.H. 1995. Evaluation of muscle diseases using artificial neural network analysis of 3ip MR spectroscopy data. Mag. Res. Med. 34, 664-672.
Kittler, J. 1975. Mathematical methods of feature selection in pattern recogntion. Int. J. Man- Machine Studies 7, 609-637.
Kohonen, T. 1972. Correlation matrix memories. IEEE Transactions on Computers 0-21 353-359. Kruse, S., Kvalheim, O.M., Gadeholt, G., Halsteinslid, L. and Sletten, E. 1991. Multivariate analysis of proton NMR spectra of serum from rabbits. Monitoring progressive growth of implanted VX-2 carcinoma. Chemometrics and Intelligent Laboratory Systems 11, 191-196.
Kruskal J.B. 1964. Non-metric multidmensional scaling: a numeral method. Psychometrika 29, 115- 129.
Kubinyi, H. 1996. Evolutionary variable selection in regression and PLS analyses. J. Chemometrics
10, 119-133.
Kuesel, A.C., Sutherland, G.R., Halliday, W. and Smith, I.C.P. 1994. MRS of high grade astrocytomas: mobile lipid accumulation in necrotic tissue. NMR Biomed. 7,149-155.
L'awley, D.N. and Maxwell, A.E. 1971. Factor analysis as a statistical method. London: Butterworths. Leardi, R., Boggia, R., and Terrile, M. 1992. Genetic algorithms as a strategy for feature selection.
J. Chemometrics6, 267-281.
LeCun, Y. 1985. Une procedure d’apprendissage pour reseau a seuil assymetrique. Cognitive 85, 599-604.
Lisboa, P.J.G. and Mehridehnavi, A.R. 1996. Sensitivity methods for variable selection using the