Matching Algorithm

using the formula:

Curvature(u, σ) = ˙x(u, σ)¨y(u, σ) − ¨x(u, σ) ˙y(u, σ)

( ˙x(u, σ)2_{+ ˙y(u, σ)}2₎3/2 (2.3.3)

Once curvature of the boundary is determined, inflection points (zero-crossings) are determined and noted. As iterations progress and the scaling parameter increases the boundary will approach a fully convex curve, and in the process inflection points will converge as concavities are smoothed out of the boundary. It is these convergence points that are recorded as descriptors for the boundary.

Figure 2.11 shows a CSS representation of the otolith boundary shown in Fig- ure 2.12. The three largest structures on the image denote the three largest cur- vature features of the boundary. For clarity Figure 2.13 shows the boundary part

Figure 2.11: CSS representation image of a herring otolith. Horizontal axis represents point along the boundary, Vertical axis represents an arbitrary smoothing increment. The horizontal line across the image is the point at which the smoothed boundary shown in figure 2.13 was produced.

way through the smoothing process. It can be seen that a shallow but long concav- ity is still present on the boundary (structure three) whilst structure two, another prominent concavity, has already been smoothed to convexity.

Once the smoothing process is complete and the boundary has been reduced to a fully convex curve, we extract from the CSS image the maxima. These are extracted as coordinate pairs for each concavity, where each pair consists of a point/distance along the boundary, and number of smoothing increments to convergence (the point at which the concavity disappears). Point along the boundary is a comparative figure as CSS representations are invariant to rotation, or to start point on boundary. A boundary can be rotated simply by performing a circular shift on all the boundary points/distances simultaneously. Figure 2.14 shows four different rotations of the same otolith boundary. Marked on each boundary is the start point from which coordinates are extracted from the boundary (following a counter-clockwise direction).

Figure 2.12: Initial boundary of a herring otolith. Three major features/structures are marked on the image: structures 1 and 2 are obvious large concavities, however structure 3 is a long concavity that, despite its shallow depth, takes a large number of smoothing increments to remove.

Figure 2.13: The same boundary as in figure 2.12 after several smoothing incre- ments. Produced at the same point as marked on figure 2.11 it shows that struc- ture 3 (as well as structure 1) is still present in the boundary.

Figure 2.14: Four (clockwise) rotations of the same otolith boundary. Left to Right: 0◦ rotation, 90◦, 180◦, 270◦. Under each is depicted the resulting CSS image given the marked boundary start points (stars).

It can be seen that a clockwise rotation of the image results in a (circular) shift of the CSS image to the left. It is also the case (not depicted) that designating a differing boundary start position on the non-rotated otolith boundary results in a similar shift of the CSS image.

2.3.2

Matching Algorithm

With each boundary stored as the locations of its curvature-crossing maxima, images can be compared to one another by finding the Euclidean distance between respective points. This results in a measure of similarity or dissimilarity between the two sets of maxima and can be used to compare one image to a number of model s in a database and find the model with the greatest similarity (or lease dissimilarity) by finding the error between respective points.

Alignment Points

The current literature (including Abbasi et al. [1]) suggests that the CSS images should be aligned by their greatest (filter level) maximum to do this. It is also said that maxima within a certain ratio of said maximum should also be considered as the alignment point. This is to account for possible boundary variation where there may be multiple points of similar significant curvature. Previous works [1, 2, 66] suggest that this ratio be eighty percent of the greatest evolution (of the individual boundary), and any maxima of greater magnitude than this be considered as an alignment point.

Each set of coordinates are shifted so that their greatest maxima are at the ‘start’ of the curve to make the first alignment complete. Further alignments are identified by searching for any of the maxima with a filter level greater than the recommended 80% level. This is done for both image and model, and error must be calculated for each possible alignment between the two sets of flagged maxima. Figure 2.15 shows

two highly simplified sets of maxima, alongside the four alignments that would be processed given an 80% alignment cut-off. From the figure it can be seen that, for this simplified example, there are four possible alignments of maxima given the 80% cut-off.

Figure 2.15: Two sets of maxima from hypothetical boundaries (top-left and bottom- left) alongside four possible alignments of maxima greater than 80% of the maximum.

Matching Order

To calculate the error of an alignment we process each of the maxima in the im- age, from greatest filter level to the smallest. We find the nearest of the maxima in the model to each image point in turn and check whether it is within a pre- defined (Euclidean) distance of our image point. This maximum distance is defined as 20% [66, 73] of the maximum distance possible (remembering that the CSS image itself wraps around). If the distance is within the 20% threshold then it (the Euclidean distance) is added to a cumulative error score for that alignment and the ‘matching’ point is removed from the model so as to not match it to further image points. If it is not within the threshold then the filter level (or evolution number) of the image point is added to the error. Figure 2.16 shows the order in which maxima are matched with this process, in a simplified theoretical image. Point 1 in the image is matched to A from the model, Point 2 is matched to B. Note that no model maxima are within the distance threshold from image point 3 (shown by the large circle) therefore its

magnitude (2) is added as error. Once all points in the image are processed then the filter levels of any remaining model maxima (point C in our example) are added to the error for that alignment.

Figure 2.16: Second alignment of maxima from Figure 2.15 showing the order in which image maxima (circles) are matched to model maxima (stars).

The process must then be repeated for the same alignment but with the roles of image and model reversed (points are matched in order of model point magnitudes). This is required as when matched in order of image magnitude, point matches may occur where the model point is closer to a separate image point. Reversing the image and model roles solves this issue. Figure 2.17 shows the differencing between matching in order of image-maxima magnitude and model-maxima magnitude. It can be seen that when matched in image order image point 1 is matched to model point B, which itself is better matched to image point 2. However when image point 2 is processed point B has already been matched and removed, so 2 is matched to the much further away point A (outside the 20% limit). It can be seen that in this simplified example, the model order match results in a lower error than the image order match.

Figure 2.17: Showing difference in matching between image maxima order and model maxima order.

Mirror Images

To account for reflected boundaries, when calculating alignments we simply flip the CSS image that we are finding matches for. To ‘reflect’ the image we need only amend the maxima array by subtracting the ‘non-reflected’ boundary point value from the total boundary length plus one. The reflection array is then processed in the same way as the non-reflected array. This ensures that were a query image to be in the reverse position, it would still be accurately matched to models it is compared to. Figure 2.18 shows a CSS image being flipped in this manner; marked are two maxima in the image and mirrored image and the coordinate pairs for each are shown.

Once each possible alignment has been processed, the alignment with the least error is selected and the error returned. The alignment itself is unimportant for the process but the error is used as a measure of similarity/dissimilarity for that model from the database. Once the image has been compared to all models in the database and their respective errors logged, we are able to determine which boundaries in the database (or reflection of) are most similar to the search image.

Figure 2.18: Mirroring a CSS image of boundary length BL. Two maxima are marked (stars) and their stored coordinate pairs shown. BL - image boundary length; L1/L2 and E1/E2 - respective point along boundary and evolution number for the marked (unmirrored) maxima.