This section addresses the question of what generality means, specifically with respect to the derived working definition of a model. The aim will be to arrive at a conceptualization which, with the additional constraints in the following section, can be used to specify a (heuristic) method to optimize the development of a model with the desired properties.
i^napter z
6.1. What does ‘generality’ mean?
Generality means different things to different people, and, as will shortly become clear, different things to the same person.
The derived definition of a model presented earlier relied in part on Long’s (1987a) characterisation of models, expressed in its short form below. Marked on this expression are the three points where it is thought the term generality could apply:
M -> R(E) for purpose P and utiliser U
T T T
This section will address each of the above in turn.
6.2. Generality of Scenario (E) represented by the model.
Long’s (1987a) characterisation of a model as a representation of an entity assumes what is here called a scenario, i.e. it is a representation of the interaction between an object and its scenario (situation), specifically in this case, a person doing a job. Generality across people is assumed by much of psychology - i.e. it is assumed that there is a degree of consistency across people, although the experimental practice of looking at behaviour across a group of subjects is a concession to supporting this assumption. There are branches of psychology concerned with the opposite, ie individual differences, but these will not be reviewed here.
The alternative is generality of scenario - and it is proposed that it is this which can be optimised, even if not guaranteed. This is possibly one of the more important
categories of generalisation; after all, we are always trying to extrapolate our knowledge to new instances, thus Broadbent (1980) comments that the point of a model is to avoid the need for experimentation. (He is arguably assuming that generality of scenario is implicit within models, which, as will become clear, is not disputed here. The aim here will be to establish some basis for this such that it is not surprising when an extrapolation fails [or will it be more surprising?]).
6.3. Generality of Purpose of the model.
A previous section has described the practices of science and engineering and thus the purpose of models within each. It was noted that there need not be a clash in
scientific purpose and engineering purpose for a model (although there could be). However, it is suggested that a given representation may not necessarily support more than a restricted set of purposes within a given paradigm. This may be called the representational problem, or ‘tools for the job’. To take an analogy used by Marr (1982), there are many systems for representing numbers - Roman, Arabic and so
^.napter z
on. The point is that these representations are not equal with respect to the purposes they support. The Roman system of numbers does not support long division very well at all. The Romans never really developed much in the way of mathematics, however, they were good civil engineers, so it could be hypothesised that their number system did have some purpose. This problem of generality of purpose applies to scientific models as well as applied models. The level of description of a model can have important consequences for its purpose. It may well be possible (one day) to describe the functioning of a human being at the level of the interaction of molecules, but this representation would not realistically support discussion of, for example, perception of language. Possibly it could be done, but it would require a disproportionate amount of effort
To summarize, it is suggested that the right tool for the job will be more efficient than a single general all-purpose tool.
6.4. Generality of Utiliser of the model.
Different users of models could be expected to require models to be expressed in different forms, even for the same purpose. Given a model of behaviour, for example, some users, with previous experience of Cognitive Science, would be able to use a model in a form which relied on such experience by utilising known
concepts. Other users, without experience, would need the model to be expressed more completely. One could construct a model that was intended to be used by any utiliser, but as in the case of generality of purpose, it is a case of being efficient. To create a model usable by all, one would have to choose the lowest common
denominator and concede nothing for the benefit of experts. This would be both excessively demanding on the constructor of the model, and it is felt, constrain some users unnecessarily.
6.5. The type of generality to be addressed.
Whilst it is possible to provide generality in all three ways, this PhD will only attempt to address Generality of Scenario. This is because a) there is a mechanism which allows, it to a degree (the following heuristic), and b) it is implicit in the subject that is to be modelled. The following sections of this chapter attempt to provide a mechanism for this. It has been stated above that a model’s scope and purpose should be explicit. To this list it is necessary to add the intended user. For the present model, this will be a behavioural scientist.
Unless otherwise stated, all following references to a ‘general model’ can be taken to refer to generality of instance.
v^napicr z.
6.6. Conceptual Variables and Generalization.
This section is intended to show that generality of scenario is already implicit in the way we talk about entities. The task of this chapter then becomes to make this explicit as well as to optimise it in some way.
Chapanis (1988) discussing generalization, makes the interesting and important point that all things bearing the same label are not necessarily equivalent. He takes the example of the study of fatigue - not all studies address the same phenomenon simply because their titles contain the word. The problem lies in that fatigue is what is known as a Conceptual Variable. Conceptual Variables cannot be studied directly - they have to be operationalized and operationalizations of the same Conceptual Variable may differ. It is this differing operationalization which he argues acts against generalization. The problem is easily illustrated by the timeworn scientists’ qualification: “ah, well, it depends what you mean by ...”
Behaviour expressed as a conceptual variable is common to many different instances, so it is tautological to talk of a general model of a conceptual variable. To take an example, Multitasking is a conceptual variable, but one couldn’t develop a ‘General Model of Multitasking’ (where general refers to behavioural instance). Rather it would automatically be a general model of multitasking. Here lies the point that possibly Chapanis could have pushed further - he was right that because of the necessary operationalization, generalization does not follow to all instances, but it can be taken a step further to say that by constraining the operationalization (i.e. stating what one means by ...), then generalization across instances is possible (to a degree). There can be no predictable, guaranteed, relation between the characteristics of one study and those of future studies (or applications) to which one might want to generalize. Whatever is done can thus only be an optimization, a heuristic, based on some criteria (such that its failure will be at least understandable). The heuristic method, presented below, for developing a model which is general across instances will be based on the idea of constrained, explicit, operationalization.
6.7. A basis for generalisation
The basis for generalisation of instance will be Constrained Operationalization, in other words, saying what is meant by a conceptual variable and only expecting generalisation if the definition fits.
A scenario will be equated with a job, and expressed as a set of attributes. A
conceptual variable is thought of as relating a scenario to a particular set of attributes. Thus the same job would have many attributes, but different subsets of these would relate to different conceptual variables. As an example, consider the job of a sonar
operator on a submarine. Some of the attributes o f such a job would be time o f day, target frequency, equipment characteristics (e.g. screen brightness) and level o f general illumination. If one were interested in the the conceptual variable vigilance, the attributes of interest would probably be time of day and target frequency. If, however, one were instead interested in visual reaction times, the one would be more concerned with the equipment characteristics and prevailing levels of illumination. Note that these attributes are not the same as those of Long discussed above which refer to the model/person and the input together, rather than just the input alone. The reason for this is that the present concern is with behaviour, which is exhibited when a person does something, and so it is only reasonable to expect that behaviour will vary (in some way) with the properties (i.e. attributes) of what people are given to do.
Any one scenario will contain a subset of attributes relevant to a given conceptual variable - so vigilance in terms of submarine sonar operations may not be precisely the same as vigilance in the case of a nightwatchman when expressed in terms of job attributes. The assumption is that a number of such attributes will nevertheless be common to both. Generalising a model across scenarios thus requires that the new scenario be expressible in terms of attributes already known. Obviously then, the larger the set of Job attributes addressed by the model the better - this will be the basis of the heuristic method for developing a model with this type of generality. In short, such generality could be summarised as generality by commonality.
6.8. Generality breaking down
Generality could be said to have broken down if the model no longer produces a behavioural output which agrees with the behaviour of the person (see the original ‘Black Box’ diagram of Fig 2.2). This could occur for two reasons:
a) the input being outside its limits (this ought not to be surprising)
b) the model itself being wrong, but the input supposedly within its limits. This might be the case if the model is internally incorrect - analogies break down sooner or later. The job specification is in terms of attributes, but the model is a system which has to represent their interactions. It is entirely possible that this particular
constellation of attributes (and thus set of interactions) has not been met before, and so might not be represented correctly in the model. This is fine from the point o f view of the model as a scientific device since it is an accepted part of the development process, but in the context of application, there is a need to optimise a model’s useful life against this.
Minimising both of the above factors requires a development program which draws on as many different, partially overlapping instances as possible, plus a requirement
L^napter z
that a model come with an explicit definition of its conceptual variable which it purports to accept as its input, in terms of attributes.
Note that the precise definition of a conceptual variable can be expected to develop alongside the model which refers to it.
6.9. A model for generalizing models.
S u b sets of Attributes corresponding to different Scenarios (i.e. behavioural instances)
Universe of all possible Job 'Attributes'. ^
Subset of Attributes
addressed by Model M N.B. The circular nature of the sets should not be taken to imply anything (at all).
Fig. 2.3. Schematic representation of the relationship of a model to different scenarios.
Figure 2.3 is intended to represent the scheme for generalisation of models based on the commonality of Job Attributes as discussed. This diagram will be returned to later, when the approach to developing such a model is presented (Section 8).