Results on reference data

The main configuration showed a mean value of 73.67% of success in letters identification among the tests. This means that approximately the 74% of the letters in the reference words were segmented in an accurate way. However, this percentage is linked to only the 17,87% of words completely good-segmented. This means that the algorithm is able to correctly identify the letters in most of the cases in general; but it rarely provides a whole word good segmented. In most of the cases, at least one of the letters in the word is miss-matched. This fact reinforces the suspect that working in a global-word level is advantageous for the objective accomplishment; but further work is required.

On the other hand, the use of the quadratic DM instead of the simple linear one represents the most significant improvement in the method performance for the reference data. The main configuration achieves a18.48% more of success in letter good-matchings with respect to the approach in which the linear DM used.

In second place, the fact of using the graph-formulation and combine the NCC information with the relative position between letters also has proven to have big impact. The effect of using this approach against the simple approach where distance is not taken into account represents an increase of the 14.13% in successful letter’s matching. Actually, without the mixed contribution of costs, the algorithm probability of performing a whole word correct segmentation in letters is below the 5%.

88 Chapter 6: RESULTS VALIDATION

Finally, with the reference words, it hasn’t been found a significant effect of including the Lucas Kanade preliminary step matching; the improvement has been quantified as a 2,16%. This fact is consistent with the context of the work, because in these reference tests, the templates and the words are written by the same subject, in a very regular way given the fact that it is a psychological researcher and not one of the children. In consequence, variability between the letters is very scarce, what makes the Lucas Kanade template warping not to have significant impact; oppositely, templates are barely modified.

This comparison between the 4 different approaches can be better appreciated in the following image.

Figure 59: Boxplot graph of the results’ statistics of word segmentation results under the 4 different strategies

Figure 59 reflects the percentage of letters that have been correctly identified in each word, for various cases. To obtain this information, for each analysed word we count the number of well- segmented letters and express it’s as a percentage of total number of letters in the word; thus we get a success ratio. We plot the statistic results as a boxplot diagram, in which it is observable, not only the mean value of good-matches’ ratio but also the most common interval of percentage achieved. This second fact also gives us an idea of the solution correctness variability.

As it has been pointed out, the success ratio of words segmentation in letters for approaches (A) and (B) is very similar, not being statistically significant. Nevertheless, a substantial decrease in performance is observed in the mean value of success ratio in approaches (C) and (D) with respect to main approach (A).

We have compared the graphical result of the same words between approaches (A) and (C) in more detail to also analyse the effect of the spatial constraints in a qualitative way. Interestingly, we observe that when we include the inter-letters distance cost, the algorithm systematically displaces the vertical lines as we expect, reducing the letter’s overlapping and large link distances. This usually leads to a significant improvement in the result.

Section 6.2. Reference data 89

Figure 60: Segmentation result of the word “bouche” when spatial coherence is ignored (left) and when it is used to constraint the solution (right)

One more fact that can be appreciated in Figure 59 is that when the liner DT is used, not only the mean value of the success ratio decreases, but also its variability vastly increases. This is not desirable for the program because accuracy would be more difficult to predict and improve. Similarly though, it can be noticed that the graph-based formulation of the problem adds variability to the solution effectiveness, what has to be taken into account for new changes that may be added to the method.

Another remarkable observation of the main configuration is that the failures in letter identification are mostly spread across the words. We do not find many errors gathered in a few words, instead we find few errors spread across many words. Actually, with the program configuration (A) we have a 16.71% of words completely good segmented and an 8.34% of words with more than half of the letters being poorly identified, hence most of the words segmentation accuracy percentage belongs to the 50-73%. This clearly demonstrates our previous remark.

Another conclusion of the previous fact is the advantage that an incorrect letter identification in the word does not have a significant impact on the whole word segmentation. If that was the case, we would find that the percentage of good matches per word is inverted; being mostly below the 50%. In contrast, in most of the words we find few segmentation mistakes and it does not lead to a complete miss-matching of the word. However, we observe that actually, an error in one letter’s identification does impact to the identification of its immediately subsequent letter(s), probably due to the spatial constraints. After one or two letters usually the error is counteracted by the graph constraints.

Figure 61: Segmentation of a word where a mistake in one letter’s identification leads to a mistake in the subsequent letter

For example, in Figure 61 we can observe that a mistake in the identification of the first letter leads to a mistake in the identification of the subsequent letter. But in the third letter, the error

90 Chapter 6: RESULTS VALIDATION

has been already counteracted and it does not affect the rest of the words’ segmentation in letters.

6.3 C

’

We have proceeded in a similar way to analyse the results of word segmentation results on children’s handwriting. However, in this case, the quadratic version of DM is always used because it has already been justified. In this new phase, the main objective is to assess performance of the program in its real working context. Consequently, the approaches to compare in this section are:

Approach /

Features Reference (A) Omitted LK (B)

Null Distance Cost (C)

LK Included Omitted Included

Cost NCC + distance NCC + distance NCC Success

ratio 70.76% 43.70% 52.08%

Table 7: Scheme of the 4 algorithm configurations that we compare for the children’s data set

As before, the same words have been loaded to the program under the three different configurations and the segmentation result is obtained. Following the quantification methodology described above, the number of good letter-matches for each word is counted.

6.3.1 Results on children’s handwriting

This main program configuration shows a mean value of the success ratio in letter identification of the 70.76% among the tests. Using this dataset, the variation induced for the different approaches (B) and (C) has also proved to have a relevant effect.

Firstly, the fact of using the graph-based segmentation gives a notable improvement on the results. Concretely, the effect of using this approach (A) against the alternative approach (C) where distance is therefore not taken into account represents an increase of the 18.68% in the success ratio. This statement is very relevant because it justifies the whole work included in this method. The construction of a mixed graph with template matching results and 2D spatial information is the core of this work; with this idea, we introduce a new methodology that is not being used in the state of the art. So, this is the main concept that has to be validated, being a new practise for handwriting character recognition. Hence, this improvement, in fact, is the proof that the proposed methodology is a valid tool on words’ segmentation and that the designed strategy for handwriting digitalization is applicable. However, these results are still below the desired threshold of achieved performance. For a better performance of the algorithm, there are still some modifications that might be necessary to add to this work. This is discussed in next chapter.

In second place, an impact is also observable with respect to the inclusion of the Lucas Kanade template warping phase. This improvement has been quantified as a 27.06% more in the success ratio in segmentation. This result is very different from the impact that this same variation has shown for the reference handwriting.