Methodology

A methodology was developed so that important aspects of the framework could be assessed and evaluated in practice. Apart from monitoring the operation by collecting information from the system that would help the evaluation such as number of samples and authentication requests, a questionnaire was also available to assess the user experience and acquire any comments from the users. The questionnaire is available in Appendix C. The user trial was split to two phases:

Enrolment Phase: The participants used the prototype to provide face, voice and keystroke biometric samples that would be subsequently used to create their biometric profiles. They were also asked to define two cognitive questions that could be used for secret knowledge questions in case the biometric authentication failed and they reached the last intrusive request as described in the previous section. A simple to use and intuitive interface was used to capture the samples. 8 samples for face and 9 for voice were captured. For the face samples they were asked to capture 8 samples with slight variations on their position when facing the camera. For voice they were called to repeat the name of 3 applications that they were going to call subsequently as part of the scenario under the assumption that the system was using a voice recognition application. Also 15 keystroke samples were captured for each cognitive response they gave, in order to create the keystroke profiles for the two-factor authentication. The enrolment process took no more than 15 minutes per person and at the end the participants were asked to complete the first questionnaire.

113

Usability Phase: Each participant was asked to follow a series of steps that would force an interaction with the device while the authentication prototype was running on the background. This would enable for biometric samples to be captured transparently as well as force access to services set to be of high security in order to test the operation of the alert level algorithm and the authentication mechanism in general. The length of this phase varies as each user had a different interaction with the device and therefore took differing times to complete each task. The average time of this phase was 45 minutes. After completion of the scenario, the user was asked to fill in a questionnaire assessing their experience and the system. After that the users were asked to play the role of an impostor on the same device using the profile of another person and by using the same steps see how quickly the system will recognise that they were not the legitimate users. Again the users were asked to fill a questionnaire and assess their views again based on the new experience of the system, as now they had a further perspective from the security side rather than the usability of the system when they were acting as the legitimate users.

The user trial involved 27 participants, with all of them having at least a basic knowledge of using a computer. In order to ensure that the users would have something to do during the ‘usability’ phase of the trial and to ensure that contexts would occur in which different aspects of the prototype could be utilised, each user was asked to work through a given set of tasks (see Appendix C). The rationale for each stage of user activity is shown in Table 4-3.

114

Activity Rationale Search for contact details of the other

participant.

Involves the use of a local application with mildly sensitive data.

Establish an IM session and exchange initial greetings.

Involves keyboard and/or voice interaction.

Each user opens a web browser and searches for hotels in Las Vegas. Each user should find 3 options.

Involves the prolonged use of an unsecured web browsing session.

Users discuss the options they discovered via IM and agree a choice.

Returns the user to IM and provides a basis for a reasonably involved discussion.

Users visit a secure ‘travel agent’ site, and provide the name of the agreed hotel plus other booking details.

Involves the use of a secure browsing session (thus demanding stronger authentication assurance from NICA) and gives the users a basis for entering keyboard information.

Each user opens a local ‘expenses’ file and record the estimated costs of the trip.

Involves the use of a sensitive local file and requires keyboard entry.

Each user creates a Word document that presents a biography statement, and types a standard disclaimer that permits the conference to post the details online.

This ensures a prolonged period of typing activity in a less bursty context that IM.

The aim of getting the users to type a biography is that it allows them to type free text on a topic that they should be able to say something about. The aim of getting them to type a disclaimer statement is that it will represent known text in which we can ensure representation of profiled keywords.

Email the document to the other user as an attachment for checking.

Involves the use of a further application context (i.e. email).

Each user checks and edits the other’s document as appropriate and sends it back.

Continues the use of Word and email, and thereby prolongs the overall session to give Face and Keystroke metrics more opportunity for testing.

Table 4-3: User trial activity and rationale

115

At the outset of each trial session the participants were briefed about the purpose of the experiment and what each phase would involve. Each user used the same device in both phases to mitigate any effect on the biometric samples from the device hardware.