Base classifier optimization

In section 4.1.4 on page 40, we mentioned that the normalized scatter plot (figure 4.4) shows a lot of outliers in the genuine scores. These outliers score low on both the LDA and LBP axis, indicating that the cause may not lie in the classifier algorithms but in the preprocessing step. To investigate this we examine the automatic detector and registrator modules.

The goal of the preprocessing step is to align all faces and correct the intensity of the lighting. The alignment is based on the position of the eyes, which are automatically detected using the Viola-Jones Haar cascade algorithm. If this not done accurately, the result of both the LDA en LBP classifiers is negatively influenced. As the Viola-Jones algorithm is known to have a good detection rate but a low accuracy, this might be a cause for the found outliers.

To test this hypothesis, we set up a manually registered query set and an automatic

registered set that originate from the same underlying input images. Both are tested against a manually aligned target set. Using this data, we perform an LDA-only verification experiment and to minimize the influence of the LDA training for this experiment, we use

the FRGCtraining set as the validation set. The results are shown in figure 4.6.

Figure 4.6: ROC curve of the manual and automatic registration experiments. It can be seen that the manual aligned images are recognized much better.

It can be seen from the figure that the automatic registrator has a rather large influence on the performance. A significant performance gain is feasible for the recognition system by improving the registration module. Therefore, we recommend that, in a future research, special attention is paid to the improvement of the automatic registration process.

Conclusion

In this report, we have proposed a framework for the recognition of faces. The design is well documented in chapter 2 and appendix C. Also, the framework is tested to meet up with its specifications in chapter 3.

The resulting framework can be the basis for future studies in the field of face recognition, as it standardizes the way in which recognizer modules are implemented. We have shown how future researchers can use the framework to simplify their work and, more importantly, how their implemented modules can be stored and documented in a way that subsequent studies will benefit from their work.

We have further shown how the framework can be used to set up a complex, fusion, recognizer. From chapter 4 we have learned that, by using fusion, multiple base classifiers can be combined to form a better one, albeit that future research should probably focus more on creating better base modules than fancier fusion algorithms.

Although the currently implemented face recognizer modules do not meet up with the state-of-the-art in face recognition, the framework provides all the means to gradually improve this as each module can be separately developed and tested. We expect that, in a few years, newly created modules will be the basis for a contemporary competitor in the field of face recognition. To help achieve this goal we provide some recommendations for future work on the following page.

5.1

Recommendations for future work

These recommendations are sorted from most significant to least significant.

Improve the registration algorithm As we have seen in section 4.3, the automatic registrator module does perform very well. As the registration can be considered as the foundation for a face recognizer, we strongly suggest that this module be among the first to be improved.

As a workaround, a manually labelled and registered images can be a good solution. There- fore, we advise to keep a read-only copy of such a set next to the original FRGC database.

Implement XML parser for FRGC data The framework uses the FRGC as its

main validation database. The FRGC originally provided the experiments setups in XML- formatted signature sets, but we have used derived TXT-formatted lists instead. To be able to really use the FRGC database, an XML input parser is necessary that can handle the signature sets directly.

Expand ImageReader to accept multi image input sets The FRGC provides ex- periments that consider multi-image input sets. The ImageReader class should thus be redefined to handle this aspect.

Implement score matrix data interpretation The current ScoreMatrix class only stores the outcomes of each recognition experiment. Data interpretation is then done

outside the framework using, for instance, Matlab or the roc-tool. It would be more

efficient if the interpretation of the data could be done directly from the framework, so that (among others) equal error rates, verification rates, rank-n scores and ROC-curves can be calculated for any experiment without further hassle.

Expand ScoreMatrix to hold unique (per-image) identifiers Currently, ScoreMa- trix does not record any image identifiers which prevents that a distinction can be made between the different images of the same person during data interpretation. While in all- vs-all experiments this distinction can be deducted from the score’s position in the matrix (on-diagonal or off-diagonal), this is not true for experiments in general. Therefore, it would be wise to keep track of the image numbers as well.

Implement database indexing function At the moment, the Database class can only

be used to match a given query to all targets in the database. For verification purposes

we would like to be able to retrieve only a single target record. Therefore, an indexing or ‘search by target ID’ function needs to be implemented.

File and folder structure

The legacy of this project is added to this report as a disk. To help future researchers find their way around this disk, we will provide an overview of its contents here.

In the root, there are two files: UTSurface.sln and UTSurface.ncb. These are required

by Microsoft Visual C Studio to set up a solution (=project).

Debug

After compilation of the source code, this folder will contain the compiled program.

Resources folder

This folder contains everything that either the framework or any of the modules need to work properly. The subfolders have names that correspond to the class name of the modules that uses them, as well as a three-letter prefix for the function of that class. Currently, the following resources are present:

• db Database files with target feature vectors of existing modules.

• sigsetsSignature Files for the FRGC experiments 1 and 4 in TXT-format.

• det.violajones The Haar cascade wavelets for the ViolaJones detector.

• reg.haarcascade The Haar cascade wavelets for the HaarCascadeand ImprovedEyeFinder registrators.

• ill.maskMask files for the Mask applier.

• fex.lda Training files for the LDA feature extractor and LdaLikelihood comparator.

Sfireader folder

This folder contains a tool to show SFI-formatted images. As the framework uses this as its standard output for images, and there is no tool available for reading such files from

the internet, we have developed it ourselves. Please read the readme.txt file for more

information.

Docs folder

This folder contains the full documentation of the project in both HTML and LaTeX

forms. Open the HTML variant from the /html/index.html, or the LaTeX variant using

/latex/refman.pdf.

UTSurface folder

This folder contains all source files for the program. Inside you will find the following files:

• UTSurface.cpp The main file of the program, where the program is started.

• stdafx.cpp /.h The precompiled header files for MS Visual Studio.

• ... All other files are less relevant, but are required by MS Visual Studio.

UTSurface/debug folder

Contains all debug files that are created during compilation, except for the final executable, which is found in the Debug folder in the root directory.

UTSurface/docsrc folder

This folder contains the source files for the generation of the Doxygen documentation, including the Doxyfile configuration file for this.

UTSurface/modules folder

All the source files for modules that are designed for the framework are located in this folder. All the modules are declared in the *.h files of the appropriate stage in this folder. The *.cpp files are organized in subfolders.

UTSurface/supFunctions folder

Main file for large scale tests

This program compares a list of queries to an existing database of targets (not per se all-vs-all). It assumes that the query input images have been registered beforehand. /* * @ f i l e U T S u r f a c e . cpp

* @ b r i e f M a i n . T h i s is w h e r e the e n t r y p o i n t for the c o n s o l e * a p p l i c a t i o n : m a i n () is d e f i n e d . */ // I n c l u d e the p r e c o m p i l e d h e a d e r f i l e to s p e e d up c o m p i l e t i m e s # i n c l u d e " s t d a f x . h " // T h i s h e a d e r can be i n c l u d e d to t r a c e m e m o r y l e a k s # i n c l u d e " c :/ dev / vld / i n c l u d e / vld . h "

// T h e s e i n c l u d e the h e a d e r for the m o d u l e s # i n c l u d e " M o d u l e s / u t b p r _ m o d u l e . h " # i n c l u d e " M o d u l e s / u t b p r _ d e t e c t o r . h " # i n c l u d e " M o d u l e s / u t b p r _ r e g i s t r a t o r . h " # i n c l u d e " M o d u l e s / u t b p r _ i l l u m i n a t o r . h " # i n c l u d e " M o d u l e s / u t b p r _ f e a t E x t r a c t o r . h " # i n c l u d e " M o d u l e s / u t b p r _ c o m p a r a t o r . h "

// T h e s e are the h e a d e r s for the s u p p o r t i n g f u n c t i o n s # i n c l u d e " S u p F u n c t i o n s / u t b p r _ d a t a b a s e . h "

# i n c l u d e " S u p F u n c t i o n s / u t b p r _ i m f e e d . h "

/* * @ b r i e f The m a i n e n t r y p o i n t for the c o n s o l e a p p l i c a t i o n . * T h i s is w h e r e it all b e g i n s .

*/

int m a i n (int argc , c h a r* a r g v []) {

std:: c o u t < < " \ n W e l c o m e to U T S u r F a c e .\ n ";

// S t a r t by e n c a p s u l a t i n g e v e r y t h i n g in a try - c a t c h b l o c k

try {

// D e f i n e the p a t h s t h a t are n e c e s s a r y for the m o d u l e s c h a r* s o u r c e B a s e P a t h = " C :/ dev / i n p u t F o l d e r ";

c h a r* o u t p u t B a s e P a t h = " C :/ dev / o u t p u t F o l d e r ";

c h a r* q u e r y L i s t P a t h = " C :/ dev / r e s o u r c e s / s i g s e t s / f r g c 1 _ q u e r y . txt ";

c h a r* l d a T r a i n F i l e = " C :/ dev / r e s o u r c e s / fex . lda / l d a _ 9 0 _ 1 2 8 ";

c h a r* l o g F i l e P a t h = " C :/ dev / o u t p u t F o l d e r / log . log ";

c h a r* o u t p u t P a t h = " C :/ dev / o u t p u t F o l d e r / l d a _ e x p ";

c h a r* d b F i l e P a t h = " C :/ dev / r e s o u r c e s / db / f r g c _ e x p 1 . db ";

// Set the v e r b o s i t y l e v e l and c r e a t e an l o g F i l e p o i n t e r int v e r b o s e = 1;

F I L E* l o g F i l e = u t b p r :: F i l e I O :: o p e n O u t p u t F i l e ( l o g F i l e P a t h ,1) ;

// C o n s t r u c t the m o d u l e s of the f a c e r e c o g n i z e r

u t b p r :: i l l u m i n a t o r :: H i s t E q ill ( verbose , l o g F i l e ) ;

u t b p r :: f e a t E x t r a c t o r :: LDA fex ( verbose , logFile , l d a T r a i n F i l e ) ;

u t b p r :: c o m p a r a t o r :: L d a L i k e l i h o o d com ( verbose , logFile , l d a T r a i n F i l e ) ;

// C r e a t e the s c o r e m a t r i x and d a t a b a s e c o n t a i n e r s u t b p r :: S c o r e M a t r i x scm ( o u t p u t P a t h , true , f a l s e ) ; u t b p r :: D a t a b a s e d b o n c e ( d b F i l e P a t h ,’ r ’) ; u t b p r :: D a t a b a s e R A M db ; // C r e a t e a l i s t of s u b j e c t IDs ( b a s e d on the f i l e n a m e ) std::vector<std::string> i m a g e L s t ; i m a g e L s t = u t b p r :: F i l e I O :: r e a d L i s t F r o m F i l e ( t a r g e t L i s t P a t h ) ; std::vector<std::string> i d L i s t ; u n s i g n e d int sz = i m a g e L s t . s i z e () ; i d L i s t . r e s i z e ( sz ) ; p r i n t f (" \ n S i z e : % i ", sz ) ; for(u n s i g n e d int i =0; i < sz ; i ++) i d L i s t [ i ] = i m a g e L s t [ i ]. s u b s t r ( i m a g e L s t [ i ]. f i n d _ l a s t _ o f (" / ") +1 ,5) ;

// L o a d the f i l e n a m e l i s t and s u b j e c t IDs in the I m a g e R e a d e r

u t b p r :: I m a g e R e a d e r imr ( i m a g e L i s t , i d L i s t ) ; imr . s e t B a s e P a t h ( s o u r c e B a s e P a t h ) ; imr . s e t L o g F i l e ( l o g F i l e ) ; // C r e a t e m e m o r y p l a c e h o l d e r s u n s i g n e d int s u b j e c t I d ; std::vector<cv::Mat> f e a t V e c t o r s ;

// L o a d all r e c o r d s f r o m the d i s k d a t a b a s e to the RAM d a t a b a s e

p r i n t f (" \ n R e a d d a t a b a s e to RAM ") ;

b o o l n e o f = d b o n c e . g e t N e x t F e a t u r e (& s u b j e c t I d ,& f e a t V e c t o r s ) ;

// W h i l e not end of d a t a b a s e file , do : w h i l e( n e o f ) { db . s t o r e F e a t u r e ( s u b j e c t I d , f e a t V e c t o r s ) ; n e o f = d b o n c e . g e t N e x t F e a t u r e (& s u b j e c t I d ,& f e a t V e c t o r s ) ; } p r i n t f (" \ n # D a t a b a s e R e c o r d s :% i ", db . s i z e () ) ;