Basic Template Matching implementation

Template matching techniques are used with Lucas Kanade algorithm to get a set of distorted templates in such a way that the letter contour behind these new images is more similar to the children’s handwriting in the word under consideration. In this phase, template matching is linked to SSD similarity measure, and the allowed the degrees of freedom for the image warping are translation and scaling. More details of this approach are discussed below.

This first phase is considered to be one of the key elements of the method. But moreover, template matching is also used in this work in a later phase to assess the similarity between these new templates and the word image for a finer alignment. In this second phase, it is implemented based on correlation approaches and only translation is allowed.

It can be concluded then that template matching as a whole is a core part of this work. Its implementation and validity have been deeply analysed for the SSD LK variant; but most of the reached conclusions are also applicable to NCC template matching. This section firstly presents the discussion that has led us to define these two different template matching phases and to understand its common implementation issues. And secondly gives a more detailed clarification of the Lucas Kanade method implementation, which has implicated complex analyses and validation phases to reach its final implementation.

a) Basic implementation of the template matching

Multiple template matching techniques are being used nowadays. To better understand the performance of the different approaches we have carried out a comparison between them. With this analysis we also aim to validate the state of the art methods applicability to our work.

In first place, we compare the two main branches of the Template Matching techniques: SSD- base and Correlation-based, concretely under NCC. For this, we design a pair of images to be used as Source image and template, respectively, and solve the registration problem with the two method implementations and translational DOFs.

The complete test can be found in the Appendix A, including the images that we have designed, the implementation details and the results that we have obtained.

After the analysis, we can conclude that the two strategies being compared are able to solve the registration problem: The optimal solution is obtained under SSD and NCC. However, in the SSD approach, the results manifold is remarkably smooth with a clear peak for the optimal solution. On the other hand, the results manifold under the NCC approach is more abrupt; with a

62 Chapter 5: ALGORITHM KEY ELEMENTS

bigger slope between consecutive positions, giving place to a larger region where the results indicate a null matching.

This difference is taken into account for the final choice; but at this point what is interesting is that we can validate the applicability of these techniques in the current work, and more specifically, we can validate our implementation. In next sections, the specific techniques to be implemented in our method are further discussed.

b) General considerations for template matching implementation

After general template matching implementation is validated, it is applied to the particular context of this work: images of handwritten words and letters. The first attempts to apply the previous algorithm with the word images were not successful and the results were not looking reliable at all: They were sometimes very abrupt, provided very inaccurate solutions or did not converge. However the previous simple test had proven that the algorithm was applicable and that its implementation on the program was right.

We have analysed the main points that may be a significant difference on the algorithm characteristics between the two sets of images, (i) simple test images and (ii) handwriting distance maps; and we have found out the most substantial differences are the image dimensions and the greyscale range in the template and image window being compared at every step. This has led us to the decision of deeply analyse multiple options of image normalization and mean- subtraction. The tests are applied to the same pair of source image and template under three template matching approaches: CC, NCC and SSD. The selected pair of images belongs to the reference data of this work, hence handwriting images DM. The complete list of strategies that we have compared and the detailed analysis, with the corresponding results, can be found in Appendix B.

The conclusion of this analysis is, first of all, that it is necessary to be aware of the details in the algorithm implementation because its impact on the solution is significant. And more particularly, that the best strategy depends on the similarity expression used for template matching.

In a more specific level, we conclude that performance on Template Matching improves drastically if the mean-subtraction of the image function is executed locally, only for the IW being compared with the template. This means that this step has to be applied within the iterative part of the algorithm the template and the Image window. Under this strategy, normalization of the output values does not have a significant impact; however, this depends on the specific technique being used.

CC NCC SSD

Section 5.2. Template warping through Lucas Kanade optimization algorithm 63

The conclusion of this particular analysis has been essential for the method development, particularly, for the LK algorithm implementation. The impact of choosing a correct support to estimate the sum during LK optimization is decisive on the global algorithm’s performance. Besides, it has made us gain control on template matching techniques.

The selection of the most appropriate technique to proceed is done according to further analysis to compare the three matching assessment approaches. However, this same analysis also gives us a clue for the assessment method selection, because it proves that the available options respond differently to image variations; being NCC more robust to greyscale changes than CC or SSD.