Biometric Performance Variables and System Errors

In an empirical study where classication is being assessed (in the case of biometrics, verication accuracy), there are four possible outcomes1 _{(Bradley, 1997; Fawcett,}

2006; Flach, 2004):

1. True positive. The sample of a valid user is correctly accepted as belonging to that user.

2. False positive. The sample of a non-valid user, is incorrectly accepted as belonging to a valid user. This is referred to as a Type I error.

3. True negative. The sample of a non-valid user, is correctly rejected as not belonging to a valid user.

4. False negative. The sample of a valid user, is incorrectly rejected as not be- longing to that user. This is referred to as a Type II error.

In experiments related to authentication, it is the error in classication that is of interest. Therefore, classication outcomes 2 and 4 above (i.e. Type I and II errors) are measured by the following rates:

• The rate at which non-valid users are falsely accepted as valid users. That is, false positives or Type I errors.

• The rate at which valid users are falsely rejected. That is, false negatives or Type II errors.

The two performance variables used to express these rates are generally termed the False Acceptance Rate (FAR) and the False Rejection Rate (FRR) respectively2_.

Thus, the FAR is expressed as the ratio of samples from non-valid users that are falsely accepted, and the FRR is expressed as the ratio of samples from valid users that are falsely rejected.

1_{These classication outcomes are discussed in greater detail in Chapter 6 section 6.2.1.} 2_{In some literature, these are termed the False Match Rate (FMR) and the False Non-Match}

In a typical authentication system, the primary concern is to minimise access to the system by non-valid users. The degree of minimisation will be dependent on the nature of the information being protected. For example, the military may have top secret information to be protected, and would insist on eliminating any unauthorised access. However, in achieving zero tolerance access, valid users may be inconvenienced by being falsely rejected numerous times.

In the case of a top secret system, this may be considered an acceptable trade-o. In contrast, a home computer used by family members would not necessarily require the same level of restriction, and numerous false rejections may be considered an unreasonable inconvenience.

In an experiment, the ultimate goal when evaluating verication accuracy is to achieve a FAR of 0%. Such a rate means that no non-valid user has been accepted as a valid user, and indicates that the experiment performed to the highest expec- tations. The goal for the FRR is to achieve a rate that is appropriately low without negatively impacting on the FAR. Previous research has shown that a FRR of 0% is dicult to attain without having some detrimental eect on the FAR (Maltoni et al., 2003; Qi and Wang, 2005).

As an example, a FAR of 0.4% and a FRR of 5% indicates that four in one thou- sand non-valid users could expect to be successful in gaining unauthorised access, while a valid user could expect to be rejected once in twenty attempts. Reducing the FRR to 1% would mean a valid user could expect to be rejected only once in one hundred attempts. However, this may, for instance, increase the FAR to about 2.5%, which means that twenty ve in one thoudand non-valid users could expect to be successful in gaining unauthorised access. This would not generally be considered a wise trade-o.

In practice, no biometric authentication system can be expected to absolutely verify the identity of an individual (Matyas Jr and Riha, 2000). For example, a password system involves the comparison of the hashes of two passwords (one being the query sample and one being the registered template in a database). If there is an exact correspondence, verication is conrmed.

However, a biometric system can only indicate the likelihood or probability that two samples are from the same person. This is because biometric characteristics are determined by sensors, and there are various factors associated with human interaction with sensors that aect the accuracy of sensor readings. This means that two biometrics samples from the same person are most unlikely to be absolutely identical.

Some of the error rates that reect or impact on this uncertainty are (Maltoni et al., 2003; Nandakumar, 2008):

• Failure To Capture Rate (FTCR): the percentage of times that a biometric capture device fails to automatically capture the intended biometric trait. This usually occurs as a result of poor quality or malfunctioning sensing devices. • Failure To Enrol Rate (FTER): the percentage of times that users of the

biometric authentication system are unable to enrol in the system. This may occur as a result of quality control checks on the enrolment procedure, a poor quality or malfunctioning sensor, inappropriate interaction between the user and the sensor, or other environmental factors (such as ambient conditions, background noise, etc.).

• Equal-Error Rate (EER): denotes the (classication) error rateat a given threshold twhere the FAR and the FRR are equivalent. Though this may seem to be an ideal trade-o point (that is, an appropriate point of equal accuracy between the performance metrics), for authentication purposes this is seldom the case. Most often the threshold requires adjustment to provide a more stringent control over the FAR.

The next section provides an overview of the dierent biometric characteristics that are available for use in a biometric authentication system.