TAD & TADAM

The final example of a task loading approach to workload assessment discussed here is perhaps the most complex, although it arguably has the greatest potential benefit for workload assessment in a maritime context. The approach is known as the target audience descriptor (TAD) model and a TAD assessment is achieved through application of the TAD Application Methodology (TADAM). TAD and TADAM were developed to assist in workload evaluations of air traffic control scenarios in the mid 1990s and were used particularly to explore the implications of automation on the ATCO role. TAD had three main purposes:

• To help identify possible areas of shortfall in the ability of ATCOs to address demand specific task requirements (especially those involving performance of safety critical tasks)

• To evaluate changes to the ATCO role implied by proposed deployment of new technology

• To help specify skill bases and profiles useful for selection and recruitment of ATCO trainees

The basic premise of the method is that prediction of human performance needs to consider the demands placed on the operator in the light of the abilities (or resources) that he or she possesses to meet those demands. Essentially, human error is viewed as arising from a demand/ resource mismatch. From this perspective, if a method is developed for measuring the

cognitive demands arising from a task, and the abilities of people to meet these demands, it would be possible to identify possible mismatches for existing or redesigned tasks. This, in essence, is what the TAD attempts to do. Any mismatches identified can then be overcome using the two classical human factors approaches. The ‘Fitting the person to the job’ approach would either select or train the operator to eliminate the specific skill deficiency. The ‘Fitting the job to the person’ strategy would involve a redesign of the task to eliminate the demands for which the required abilities were inadequate. In either case, a method for explicitly identifying the nature of the demand/resource mismatch would be essential.

Implementation of a TAD analysis via the TAD Application Method (TADAM) involves three stages of work.

TAD Construction

In the first stage, a Target Audience Description (TAD) is prepared. The TAD is a profile of measures that are used to describe populations of stakeholders responsible for the completion of a particular job or work role. The ATCO- TAD, for example, is a description of a given population of Air Traffic Controllers (ATCOs). Any TAD is made up of two different types of measures.

The first set of measures is used to denote cognitive abilities. That is, the range of cognitive functions which all people possess but in different proportions (etc., memory, convergent/divergent thought, perceptual speed/accuracy). Cognitive abilities are deployed in response to prevailing task demands and will influence how well the individual is able to respond to those demands. Cognitive abilities are generally defined and informed by cognitive theory, empirical study within the target population and error data. A second set of descriptor measures are used to characterise an individual’s personality: Measurable demographic factors that describe an individual’s personality style which are expected to influence their willingness and/or ability to stay on task (e.g., Decisiveness, tenacity, stress tolerance, etc). Again, identification and selection of appropriate personality attributes are informed by cognitive theory, assessment of the target population and error data.

Estimate of task demands (task demand profiling)

In the second stage, a task analysis is performed to assess the cognitive demands that are implicit within a task or proposed task. The results of the task analysis are captured in a Task Demand Profile (TDP) in which the task is described in terms of a set of task demand descriptors (TDDs) - A description of the cognitive primitives (e.g., detects, assess, diagnose) used to describe cognitive tasks in work environments. A partial list of TDDs is shown in Figure 28 for reference purposes.

Task Demand Descriptors

Definition

Detect Become aware of initial stages of problem Ascertain Observe or collect data from instruments

based upon a specific need (e.g. confirmation of an initial diagnosis)

Recognise Identify system state directly from a pattern of indicators

Interpret Evaluate the implications of an event that has occurred

Assess Evaluate and select alternative goals

Formulate Plan path by which goal will be successfully achieved

Decide Choose or formulate a procedure to achieve required objective

Act Execute the chosen procedure following its selection

Initiate/Comply Select appropriate procedure

Check Perform visual examination to determine status

Monitor Status Continually visually or aurally evaluate status in order to detect deviance from the expected state

Tracking event Continually track a sequence of events in order to detect deviance from an expected plan

Diagnose Identify a pattern of indicators and associate them with an underlying cause or causes

Communicate Send information to another person or obtain or acquire an incoming message Liaise Communicate with more than one person or

agency

Negotiate Co-ordinate and agree activities with more than person or agency

Figure 28 Sample list of Task Demand Descriptors (TDDs)

Development of a Task Demand Profile involves moving from the description of the task elements (captured in task analysis) to an alternative description based on the Task Demand Descriptors defined in Figure 28. The reason for this process is to enable this controlled set of terms (a lexicon) to express the mental and physical demands that are common to all tasks. The terms in the lexicon set out in Figure 28 can be used in a number of ways for this purpose. For example, if the terms are linked together by means of a flow chart, they can be used to express the way in which Task Demand Descriptors (TDDs) such as detection, communication, action and checking occur as a task progresses. Typically, such a process will be both iterative and interactive, with certain demands such as action checking and event tracking occurring repeatedly. Such a representation of a task structure is useful for identifying the nature of the demands on the operator and where these may give rise to

errors. The number of times that the various task demand descriptors occur within the task provides a simple measure of the overall level of demands. A more sophisticated estimate of task demands are obtained via the creation of the task demand profile – as shown in Figure 29.

The abbreviations in the table are as follows: * RR : Resource Requirements

F : Frequency of Occurrence of a TDD in a task

N : Sum of Frequencies of occurrence of each TDD in column 1 F/N : Relative Frequency of each TDD in Task (Calculated as F/N for each TDD Frequency)

The frequencies and relative frequencies of the TDDs present in a sample task are shown in the second and third columns of Figure 29 (see page 88) and can be interpreted as follows. The most extensive demands arise from the TDDs ‘Act’ and ‘Communicate’, each of which account for 20% of the total task demand (see column 3). Diagnose and event tracking each account for 1% of the demands and the rest are distributed across the TDDs as shown in column 3 of the table. This process provides a comprehensive profile of the demands arising from the tasks. However, this does not yet describe workload levels, are evaluated by examining the interactions from the deployment of resources to meet these task demands.

Cognitive Skills and Abilities Resource requirements Task Demand Descriptors (TDD) Fre q . of TDDs in Standard C Task D emand Profil e

(F/N for each TDDs) Memory Attention Verbal Skills Spatial Visualisation Perce

p tual S p eed a n d

Accuracy Motor Skills Decision Making Convergent Thinking

Average Resource Requirements per TDD Detect 7 0.03 0.40 0.75 0.15 0.55 0.70 0.10 0.45 0.45 0.444 Ascertain 12 0.05 0.40 0.80 0.40 0.50 0.60 0.15 0.40 0.50 0.469 Recognise 10 0.04 0.70 0.60 0.20 0.70 0.50 0.20 0.30 0.30 0.438 Interpret 16 0.07 0.70 0.70 0.30 0.60 0.40 0.20 0.60 0.60 0.512 Assess 11 0.05 0.55 0.60 0.20 0.20 0.40 0.20 0.80 0.70 0.456 Formulate 10 0.04 0.50 0.70 0.20 0.40 0.25 0.15 0.75 0.65 0.450 Decide 16 0.07 0.40 0.60 0.35 0.60 0.45 0.20 0.90 0.85 0.769 Act 46 0.20 0.60 0.40 0.20 0.35 0.45 0.90 0.30 0.30 0.438 Initiate/Comply 11 0.05 0.75 0.55 0.30 0.30 0.60 0.60 0.75 0.60 0.556 Check 13 0.06 0.55 0.70 0.15 0.45 0.70 0.30 0.35 0.20 0.425 Stat. Monitoring 11 0.05 0.60 0.75 0.40 0.60 0.60 0.25 0.55 0.45 0.525 Event Tracking 3 0.01 0.70 0.80 0.40 0.55 0.54 0.30 0.60 0.55 0.555 Diagnose 2 0.01 0.60 0.70 0.35 0.60 0.44 0.15 0.90 0.75 0.561 Communicate 45 0.20 0.40 0.70 0.70 0.20 0.35 0.20 0.30 0.30 0.394 Liaise 9 0.04 0.40 0.60 0.75 0.30 0.30 0.15 0.40 0.40 0.413 Negotiate 7 0.03 0.40 0.60 0.80 0.25 0.30 0.10 0.70 0.60 0.469 Abilities Requirements Profile (Average RR across all TDDs) 0.54 0.64 0.62 0.41 0.47 0.37 0.49 0.46

Figure 29 Demand-Resource Matrix of Cognitive Skills and Abilities for an example task showing

Resource Requirements weighted by frequency of demands Task Demand

Descriptors (TDD)

Memory Attention Verbal

Skills Spatial Visualisation Perceptual Speed and Accuracy

Motor

Skills Decision Making Convergent Thinking Overall Demand/ Resource Profile (DRP) Detect 0.01 0.02 0.00 0.02 0.02 0.00 0.01 0.01 0.09 Ascertain 0.02 0.04 0.02 0.03 0.03 0.01 0.02 0.03 0.20 Recognise 0.03 0.03 0.01 0.03 0.02 0.01 0.01 0.01 0.15 Interpret 0.05 0.05 0.02 0.04 0.03 0.01 0.04 0.04 0.28 Assess 0.03 0.03 0.01 0.01 0.02 0.01 0.04 0.03 0.18 Formulate 0.02 0.03 0.01 0.02 0.01 0.01 0.03 0.03 0.16 Decide 0.03 0.04 0.25 0.04 0.03 0.01 0.06 0.06 0.52 Act 0.12 0.08 0.04 0.07 0.09 0.19 0.06 0.06 0.71 Initiate/Comply 0.04 0.03 0.01 0.01 0.03 0.03 0.04 0.03 0.22 Check 0.03 0.04 0.01 0.03 0.04 0.02 0.02 0.01 0.20 Stat. Monitoring 0.03 0.04 0.02 0.03 0.03 0.01 0.03 0.02 0.21 Event Tracking 0.01 0.01 0.01 0.01 0.01 0.00 0.01 0.01 0.70 Diagnose 0.01 0.01 0.00 0.01 0.00 0.00 0.01 0.01 0.50 Communication 0.08 0.14 0.14 0.04 0.07 0.04 0.06 0.06 0.63 Liaise 0.02 0.02 0.03 0.01 0.01 0.01 0.02 0.02 0.14 Negotiate 0.01 0.02 0.03 0.01 0.01 0.00 0.02 0.02 0.12 Average DRP for whole task (CMWL measure) 0.31

Workload assessment

Workload assessment involves the creation of two further profiles - the Abilities Requirements Profile (ARP) and the Demand/Resource Profile (DRP).

The first stage in the process is to evaluate the proportion of the abilities in the TAD likely to be required to meet the demands represented by each of the Task Demand Descriptors in Figure 28. This uses a structured expert judgement process in which a small team of psychologists and human factors experts independently rate the proportion of each TDD that would be required to perform the task effectively on each occasion that the TDDs are invoked. These ratings are set out in Figure 29. Thus, if the TDD ‘Detection ‘ was required, each time a radar display had to be scanned, and this demand occurred seven times during a watch keeping task, these demands are rated as involving 40% of the available memory resource, 75% of the available attention resource and so on, as shown in the first row of Figure 29. It is assumed for the purpose of this example that the proportion of the resources required remains the same during each demand, but this assumption could me modified if a more detailed analysis were required. Figure 29 contains a subset of the TDDs shown in Figure 28, as it is assumed that only these are relevant to the example task assessment provided.

If the Resource Requirements (RR) in Figure 29 are averaged vertically for each TDD, the resulting scores (the bottom row) represent the proportions of each ability that are required for each of the TDDs that are required in the task. These averages are called the Abilities Requirements Profile (ARP) for the Task (or Task Set).

If the Resource Requirements in Figure 29 are averaged horizontally, i.e. within TDDs (See the last column of Figure 29), this provides a measure of the average resource requirements to satisfy each of the TDD demands. However, in order to develop a CMWL index from these interactions between demands and resources, this profile of resource requirements needs to take into account the frequency of occurrence of the TDDs within the Task or task set being evaluated. Each of the Resource Requirement entries in Figure 29 (columns 4 - 11) is multiplied by the Task Demand Profile (Column 3), which is the relative frequency that each TDD demand is invoked. The results of this process are the entries in columns 2-9 of Figure 30. The entries in the rows for each TDD in Figure 30 are summed to give the Demand/Resource Profile (DRP) in the last column.

The Demand Resource Profile (DRP) is an attractive approach to CMWL measurement as it directly measures the interaction between demands and resources, which is central to the concept of CMWL. This column specifies the separate contribution of each of the task demands (TDDs) to the workload. Thus the amount of workload generated by the ‘Act’ component of the task demands (0.71) is considerably higher than the ‘Recognise’ component.. Thus, this analysis would suggest that reducing the action

demands in the task would have a greater impact in reducing workload than reducing the recognition demands. Alternatively, interventions to reduce workload by improvements in resources, e.g. by means of training, would best be directed at improving the motor skills component of the ‘Act’ demand, since this is the greatest contributor (0.19) to the greatest demand (0.71) in the Demand Resource Profile in the last column of Figure 30. A simple way to create a CMWL index for the task considered in this example is to average the values in the DRP column, to give a value of 0.31. If the TAD approach to workload evaluation were used in this way, it would be very important to apply the process at an appropriate level of aggregation for the task set. For example, if a very large task set were assessed in this way, the overall level of loading, as measured by the index, might appear to fall within an acceptable range. However, this might conceal the presence of certain tasks that that exceed the acceptable workload, and which therefore have a higher probability of failure. Thus, the workload analysis needs to be preceded by a comprehensive task analysis stage.