Pattern Recognition

From birth, human beings and other animals utilise their inherent object recognition abilities for survival in their environment (Marques de Sa, 2001). The objects of concern may be physical or conceptual. The recognition task may be simple or complex. Whatever the case, our ability to develop object recognition skills is crucial to learning and progressing, and thus surviving.

The skills to perform object recognition are developed from experience (Mar- ques de Sa, 2001). For example, when we rst encounter an object, we observe and identify certain information about that object and store it in our memory. When next we encounter that object, we again identify information about the object and compare that information to our memory of the object.

An example would be recognising another person. When we encounter someone for the rst time, we identify certain characteristics about that person and store them in our memory. When next we encounter that person, we again identify the characteristics and compare them to those in our memory.

Pattern recognition is analogous to object recognition, however it refers more specically to those tasks performed by automated systems to imitate or simulate the human ability for object recognition (Marques de Sa, 2001).

Pattern recognition had its origins in theoretical researchin the area of statistics prior to the 1960's, but with the advent of the computer age has led to the devel- opment of practical automated methods for recognising patterns in many dierent applications areas (Theodoridis and Koutroumbas, 2006). Some of the application areas for this scientic endeavour are machine intelligence or learning, machine vi- sion, character recognition, computer-aided diagnosis, and speech recognition.

In order to observe and subsequently recognise an object, that object needs to be accurately described (Friedman and Kandel, 1999). This description of an object is referred to as a patternthat is, a pattern describes an object.

Pattern recognition is a discipline whose goal is the classication of patterns into dierent categories or classes (Theodoridis and Koutroumbas, 2006). That is, it is concerned with discriminating between dierent populations of patterns (Friedman and Kandel, 1999). In a practical sense, it involves assigning an query input pattern to one of a nite number of M known or exemplar patterns (Lippmann, 1989).

For example, there may be four dierent types of road vehiclescars, buses, trucks, motorcycles. In this case, there are four categories or classes to which a road vehicle can belong. Given an input or query pattern, allocation to a class is often referred to as classication. The decision to allocate a particular pattern to a particular class is based on the features that are used to describe the object (and thus obtain the pattern) and the features that specify or characterise each class (Friedman and Kandel, 1999).

Dierent classes may be described by some common features, though the values of these features should dier enough to cause an input pattern to be allocated membership to one specic class. For example, all human beings have weight and height features and the values for these features could be used to formulate dierent classes; tall and light, tall and heavy, short and light, short and heavy.

Feature selection involves determining the features to use when describing an objectto form a patternand also to use for class dierentiation. More precisely, identifying the features that make patterns distinct, and the measurable attributes that make the distinction apparent for the dening of classes or categories to which a pattern may belong (Theodoridis and Koutroumbas, 2006). The process of ob- taining the actual measurements or values for the selected features is called feature extraction.

A pattern recognition system employs the available informationthat is, ex- tracted featuresto classify patterns or data based either on a priori knowledge or on statistical information extracted from the patterns. The patterns to be classied

are usually groups of measurements or observations, dening points in an appropri- ate multi-dimensional space.

A typical pattern recognition system should consist of (Wang, 2002): • A sensor that gathers the observations to be classied or described.

• A scheme that discerns the salient features, denes the appropriate classes, and describes the pattern.

• A feature extraction mechanism that computes numeric or symbolic informa- tion from the observed selected features.

• A classication or description scheme that performs the actual task of classi- fying or describing observations, based on the extracted features.

However, there are practical concerns, that can cause error in a pattern recogni- tion system, that need to be taken into account when designing and implementing the system (Marques de Sa, 2001):

1. The selected features that describe a pattern should be sucient to truly represent the object being observed.

2. The pattern samples used to dene a class should be truly representative of that class.

3. The classication scheme should be eective in separating the classes.

4. The classication scheme should be able to dierentiate between classes, given that some classes may contain naturally occurring feature overlap.

The next section discusses classication schemes in more detail. 2.4.1.1 Classication Schemes

A pattern recognition system has as its ultimate goal the correct classication of an input pattern (Friedman and Kandel, 1999). Classication occurs after relevant features (that characterise the pattern) have been identied and extracted. Once

the features have been extracted, they are presented to the classier to determine if the input pattern is a member of a given class.

A classier is a `machine' that is designed to allocate a given input pattern to the most appropriate of the available or known classes (Marques de Sa, 2001). To do this, the classier applies its computational algorithm to the given features and indicates the likelihood of class membership. Note that indicates implies that the classier is making a best estimate, and that estimate may or may not be correct.

The classication scheme is usually based on the availability of a set of patterns and known classes that have been described or categorised by human experts (Mar- ques de Sa, 2001). The set of patterns is termed the training set, and the resultant learning strategy is characterised as supervised learning. Supervised learning pro- vides an association between input data and decision making. Learning can also be unsupervised, where the system is not given an a priori labeling of patterns; instead the system itself establishes the classes based on the statistical regularities of the input patterns.

There are various methods for classication, appropriate to the data being clas- sied and the application to which it is being applied. The classication schemes discussed below assume a priori knowledge of classes (Friedman and Kandel, 1999): • Decision Functions. This scheme deals with a known number of training pat- terns that are geometrically separable. Input patterns are classied by decision functions. For example, a simple linear classier could be dened for classes C1 and C2 by the decision function d(x)according to Equation 2.7:

d(x)>0 ⇒ x∈C1

d(x)<0 ⇒ x∈C2

(2.7)

where d(x) = 0 is the `decision boundary'. If decision boundaries can be de-

ned such that they separate numerous classes in real space, then the classes are said to be linearly separable. Non-linear classiers exist that use gener- alised decision functions to distinguish between classes that are not linearly separable.

• Minimum-Distance Classiers. This scheme usilises distance functions (for classication) when patterns in a training set graduate toward a number of dif- ferent clusters15_{. If each class consists of a single cluster, a minimum-distance}

classier can be used to classify an input pattern. If each class consists of multiple clusters, the nearest-neighbour classier can be used. Here, the dis- tances from the input pattern to the patterns in the training set are measured, and the input pattern is classied as being a member of the same class as its nearest neighbour (in the training set).

• Statistical (or decision theoretic) Approach. When the patterns of multiple classes exhibit feature overlap, a statistical approach (for classication) can be adopted. Statistical pattern recognition is based on statistical characterisation of patterns. That is, the patterns that originate from statistical distributions. A statistical classier incorporates the risk or probability of mis-classication. • Fuzzy Classiers. The result of classication may not always be certain. That is, there may be doubt about the result, or the input pattern could be classied as a member of more than one class. Feature selection may dene an ambigu- ous metric. For example, a general class denition for height could be `tall' or `short'. However, the exact measurement, 173 cm, may be considered to belong to both classes or neither. To alleviate such problems, fuzzy classiers assume an input pattern to be a member of every class to varying degrees; the grade (strength or weakness) of membership in each class is expressed as a value in the continuous interval [0,1]. By allocating the appropriate grade

of membership, classication can be determined.

• Syntactic (or structural) Approach. Syntactical pattern recognition is based on the structural inter-relationships between the features of patterns. Examples of research areas where structural components are used for classication are character recognition and ngerprint recognition. It is common to dene a syntax language that describes the structural components of a pattern.

Syntax classiers are then used to process the resultant symbolic strings rep- resenting the pattern.

• Articial Neural Networks. This approach assumes a set of training patterns and their correct classications. The correct classications (for each training pattern) are used as `targets' when training the neural network. The patterns are supplied to the neural network via the input layer, and the targets are used to guide the learning algorithm of the neural network toward correct classication. Articial neural networks are discussed in more detail in the next section 2.4.2.