Methodology

2.4.1 Participants

The participants consisted of all the test candidates who took the UK theory-driving test during the month of February 2012. The decision was taken to only analyse data from test candidates who were taking the HP test for the first time in order to avoid any affects multiple attempts may have had on test performance. After the data was sorted and the required information was extracted the total number of participants was 26,452. A total of 32 test candidates did not provide a gender. A total of x passed the HP test, while x failed to pass the test. A more detailed breakdown of the participant information can be found in Table 2.1.

Table 2.1. Details of candidates included in the HP test analysis. video clips contain 1 score-able hazard and one of the clips contains 2 score-able hazards. A maximum of 5 points is awarded for each score-able hazard. Points awarded are determined on the basis of scoring windows, which define points in

time in which the hazard becomes visible and when responding to the hazard would be too late to avoid having a collision. A button press in the first fifth of this time window provided the candidate with a score of 5, while scores in later windows receive lower scores, down to 1 for the last window. Any button responses made outside the scoring window receive no points. The HP test sets are assigned to test candidates at random. There are 20 HP test sets and 150 different HP videos in total (including 4 videos that contain 2 soring hazards), with some HP videos shared between test sets. The HP test data contained:

• Registration number (to identify the test candidate)

• Gender of test candidate

• Age of test candidate

• Test attempts

• The test set the candidate was assigned to

• The individual hazard videos the test candidates saw

• Scores for each hazard video

• Total score for the HP test

• What grade the test candidate received for the HP test

2.4.3 Design

The main analysis of the data involved an ALSCAL multi-dimensional scaling on the scores test candidates received for the HP video clips they saw.

ALSCAL is a multidimensional scaling algorithm developed by Young, Takane and Lewyckyj (1978). ALSCAL is a metric and nonmetric multidimensional scaling method that allows for a number of individual difference options.

ALSCAL uses the alternating least squares approach (Takane, Young and Lewyckyj, 1977). ALSCAL unfolds the data in order to construct a Euclidian space, which has points for each object. Rows and columns of the data are represented as points within that Euclidean space.

ALSCAL analyses the similarity of values within a proximity matrix in order to create a scaling solution that represents the similarity/dissimilarity of points within Euclidian space. If data points (in this case scores for particular HP test videos) are similar, then they are placed close together, while if the data points

are dissimilar, the points are placed far apart. The ALSCAL goes through an iterative process of unfolding the data to produce the scaling solution. Each step involves making slight adjustments to the scaling solution by moving the data points in order to improve the reliability of the solution. The iterations are stopped when any further iteration results in an improvement in stress (i.e. the reliability of the scaling solution) of less than .001. The end result of the multidimensional scaling is that the data are placed into a plot in order to provide a visual representation of the scaling solution (i.e. how similar/dissimilar the data points are in the dimensions of Euclidian space). The analysis also outputs goodness-of-fit values (the proportion of variance accounted for by the multidimensional scaling) and stress values for the scaling solution.

The data were analysed in 2-dimensions of Euclidian space. This value was chosen, as this is the maximum number of dimensions possible to produce reliable results with the amount of data from each HP test set (i.e. the number of data points in Euclidean space). Separate scaling solutions were created for candidates who passed the HP test and those who failed the test.

2.4.4 Procedure

The first step taken was to place the data into an MS Access database as this enabled easy extraction of specific parts of the data set for analysis. From this database, 20 sets of data were extracted, one for each of the HP test sets. These contained the scores that each test candidate received for the 14 hazard videos that make up each test set. From these test sets, data was extracted for test candidates who were attempting the HP for the first time. This was done in order to avoid any practise effects that may have existed in candidates who had attempted the HP test multiple times. This allowed for a clean and direct comparison between test candidates without any confounding effects of multiple testing.

The resulting data sets were then analysed in SPSS using an ALSCAL multidimensional scaling in order to establish proximity values between the HP video clips contained within each of the 20 HP test sets. Separate proximity matrices were calculated for those who had passed the HP test and those that had failed the HP test. In total, 40 proximity matrices were created, 2 for each of the 20 HP test sets (one for the participants who had failed the HP test and another for

those that had passed the HP test). After inspecting these 40 proximity matrices, it became apparent that including the video hazard clips that contained double hazards resulted in the scaling solutions being distorted by the presence of the double hazards (as the dimensions of the scaling solutions were highly skewed by the double hazards). As a result the multidimensional scaling was repeated, this time with the double hazard clips removed, leaving 13 scores for each test candidate in the analyses.