Problems with the measurement of skill
Chapter 4 The data source
4.2. The skill-related variables
The HILDA survey, as should be clear from section 4.1, was not primarily intended as a data source for the analysis of skill or other qualitative job characteristics. In the overall instrument design, skill represents only a very subsidiary element of the “labour market dynamics” topic. The small group of relevant indicators appears in the SCQ and has been largely overlooked by researchers so far. (Exceptions are covered in Section 5.3.)
These questions are asked only of respondents employed at the time of survey, and refer to the respondent’s current main job. The core set, asked over all six waves, consists of six variables which break into two logical subsets, one referring to the skill demands of the job, the other to the degree of control or discretion which the respondent exercises over how s/he goes about the work. Each variable is listed below under three elements. The first element is an intuitively meaningful variable name that has been coined for the purposes of this thesis. The second is the variable name in the HILDA dataset, with the underscore standing for a letter identifying the item to a particular wave (a for Wave 1, b for Wave 2, etc). The third is the corresponding question in the self-completion questionnaire
COMPLEX (_jomcd) - My job is complex and difficult
NUSKILLS (_jomns) - My job often requires me to learn new skills
USESKILL (_jomus) - I use many of my skills and abilities in my current job OWNTASK (_jomfd) - I have a lot of freedom to decide how I do my job HAVESAY (_jomls) - I have a lot of say about what happens in my job
WORKFLOW (_jomfw) - I have a lot of freedom to decide when I do my work These questions form part of a sequence of twelve in which respondents are asked to rate their agreement with statements about aspects of their main job on a 7-point response scale. The other items in the sequence, which precede the skill-related set, relate to the
stressfulness of the job, whether it pays fairly, the security of the respondent’s employment, the security of the job itself and the likelihood that the employing business will still be trading in five years’ time. In each case the scale is presented on the questionnaire form as a set of tick boxes, with verbal anchors only at the extreme points (“strongly disagree”, “strongly agree”). This format is common in several sections of the SCQ, and respondents would have answered five similarly structured sequences using unanchored scales (some with a different number of response options) before reaching this point in the questionnaire. From Wave 5 onwards, a supplementary set of nine variables was added to this sequence:
• I have a lot of choice in deciding what to do at work
• My working times can be flexible
• I can decide when to take a break
• My job provides me with a variety of interesting things to do
• My job requires me to take initiative
• I have to work fast in my job
• I have to work very intensely in my job
• I don’t have enough time to do everything in my job.
These additional data items have both broadened the scope of the data available for analysis and helped to clarify the meanings placed by respondents on the individual core variables, while also making it possible to construct more reliable and sensitive composite scales for the different dimensions of both skill and job stress. Their contribution will be outlined below, after the core variables have been discussed in more detail.
The six core variables correspond closely to two out of the three generic dimensions of skill identified in Chapter 3. The first subset can be regarded as more or less direct and
complementary indicators of the extent to which a job requires skill in a generic sense, i.e. independently of the level or field of competence. COMPLEX, at least on first sight, comes closest to capturing the substantive complexity criterion, while NUSKILLS and USESKILL cover different aspects of the skill-intensity criterion. OWNTASK and HAVESAY respectively capture the individual and collective aspects of task discretion, while WORKFLOW corresponds to a qualitatively different aspect of the same dimension relating to the timing of tasks and the sequence in which they are done.
This close correspondence opens up the potential for the individual variables to be combined into scales which could be used to measure the key dimensions of skill in the model developed in Chapter 3. This process, and the tests applied to validate the resulting scales, are described in detail in Chapter 6. The preliminary assessments set out
immediately below refer only to the face and construct validity of each indicator.
4.2.1. COMPLEX
COMPLEX stands out as the most problematic and ambiguous of the six. To the extent that it does capture the substantive complexity dimension, it suffers more than the others from the core problem with self-report that was identified in Chapter 3, namely that it effectively assesses the match between the respondent's skills and the demands of the job. Thus there is a potential ambiguity over whether the item score reflects the nature of the job or the adequacy of the respondent’s skill base. This is acceptable so long as COMPLEX is treated as an element of skill-intensity, since the latter construct as defined in Chapter 3 is specifically concerned with the issue of match. However, it detracts from the face validity of this variable as a measure of substantive complexity, seen as an objective characteristic of jobs. A second ambiguity goes to the more fundamental issue of whether increasing complexity in the colloquial sense can always be treated as an indicator of higher skill. In foreseeable circumstances a rising score on this variable over time could result from work intensification rather than a growth in task complexity as
understood by Spenner or an increased coordination requirement in the sense used by Nelson and Winter. In other words, a job could become more “complex” simply because more, perhaps unrelated tasks now need to be done in the same time, even though each of those tasks is individually straightforward. This would be the case,
been downsized and the surviving employees are required to fill in for lower-skilled support staff who no longer exist. In such cases, COMPLEX might more logically be associated with job stress than with any absolute or relative measure of the skill required.
The third potential ambiguity stems from the wording of the question. Read literally, the question asks about two things, complexity and difficulty. This could be seen as an advantage if one accepts the argument in 3.4.1 that difficulty is one aspect of the “level” of tasks in Spenner’s original definition of substantive complexity, since combining the two terms in the same question should help the respondent to identify the kind of complexity that is intended. But difficulty could just as legitimately be seen as the key indicator of the “stretch” dimension of skill- intensity, in which case the question can be seen as straddling the two constructs. If this is not clear to the analyst, there can be little certainty about how the average respondent will interpret it.
4.2.2. NUSKILLS
NUSKILLS refers directly to the learning element of skill-intensity. Specifically, it describes the extent to which the job itself requires the employee to keep learning. It is indirectly informative about the extent to which the workplace provides opportunities or support for learning, but only insofar as a respondent would
logically be unable to give a positive score if it were altogether impossible to do the learning required. Taken at its face value, it says nothing about whether the worker is encouraged or facilitated to engage in learning beyond the immediate
requirements of the job. However, a positive trend can be taken as fairly unambiguous evidence of growth or change in the skill requirement of a job1. The implications of a negative or declining score on this variable are less straightforward. It could mean that the firm in which the job is located is not innovating, and consequently that product lines and production processes have remained stable over several years, though those processes might still require considerable skill. Conversely, it could mean that after decades of learning, the respondent has at last fully mastered a highly skilled occupation where the needs and techniques change relatively little over three or four years, e.g. some kinds of surgery. Indeed, if one accepts Attewell’s proposition that “a virtuoso recognises fewer exceptions than a novice”, it is reasonable to expect that someone who has achieved virtuosity in her job will do less conscious learning than a normally
competent practitioner would over the same time. The important thing to remember is that in such cases, perceived stability in the skill requirement could well be reflected in a declining score for this variable over the time a worker remains in the same job. In this respect NUSKILLS can be expected to behave differently from the other indicators in the set, and this needs to be taken into account when
interpreting any trends that emerge. Of course, the intuitively obvious conclusion – that the job is getting less skilful over time – may just as well be true in other cases, but it would require different evidence to determine whether that is happening.
1
One obvious qualification is that individual scores will be more reliable when averaged out over several years, as the initial period of transition into a new job is bound to involve some amount of learning for the recruit even if the actual skill profile of the job is static.
4.2.3. USESKILL
This variable captures the third element of skill-intensity, the utilisation of available skills. Like the others, it is strictly an indicator of skill match, specifically whether the respondent considers herself to be overskilled or appropriately skilled for her present job. It is thus more likely to capture either underutilisation of skills or substantive mismatch (i.e. the worker has good skills, but the wrong ones for the job) than skill gaps or deficits.
The one obvious problem with this variable lies in its wording: “I use many of my skills and abilities.” Just how many is “many”? Since respondents are given no guidance on how to make this assessment, it is unrealistic to expect much consistency in the response. Some may quite reasonably interpret it as meaning “more than normal”, in which case they will need to guess what is a “normal” amount to use; in other words, their response may be determined by guesses as to how others have answered the same question, rather than just by the respondent’s assessment of her own skills and her own job.
Another caution that needs to be voiced about this variable is that over the first six waves it has been consistently scored much higher, and with less wave-on-wave variation in the mean, than the other two core skill variables. One might reasonably take this at its face value, except that comparisons with UK evidence (Mavromaras et al 2007b, discussed in detail in section 5.3) show a prevalence of reported
overskilling in that country so much higher than the level revealed by HILDA that it cannot credibly be explained by known differences between the two labour markets. USESKILL also shows the weakest inter-item correlation with the other two core skill indicators and loads in a counter-intuitive way in the factor analysis discussed in Chapter 6. These are possibly indications that the item is insufficiently
“difficult” in the terminology of Item Response Theory, i.e. that the level of actual positive perception required for the average respondent to record a high score is too low for the item to discriminate effectively between respondents.
The discussion of these first three variables reinforces the warning that has already been given several times about the distinction between skill-intensity and skilfulness. What is being measured here is not depth or quality of skill, but rather skill-related aspects of the job-worker match. These indicators tell nothing about the actual amount of skill required in each job, either in an absolute sense or relative to other jobs. “Learning new skills” should logically demand on average a great deal more application, prior knowledge and underlying aptitude for a surgeon than it does for a low-level clerk. A worker with low overall skills is more likely to have to use most of them in order to repay the cost of employing her than one with a diversity of highly advanced skills (who could still be employed profitably on a wage far lower than the full potential value added by his skills). These problems
emphasise that without adequate indicators of substantive complexity, the other two dimensions can provide at best an impressionistic picture of the actual skill content of a given job at any point in time.
The next three indicators correspond in general intent, but not precisely, to a set of five used in the UK Skills Surveys. Whether fortuitously or by design, the three variables provide complementary perspectives on the autonomy-control dimension. As was argued in Chapter 3, this is an advantage given that jobs in which some aspects of this dimension
are highly evident may be less strong on other aspects because of the nature of the work itself.
4.2.4. OWNTASK
This variable refers specifically to individual worker autonomy, the least
definitionally problematic aspect of task discretion. This is the aspect which has been most prominent in the labour process debate, being the one that comes closest to Braverman’s concept of mastery. It was argued in 3.4.2 above that individual task autonomy is often associated with a high-skilled job, since it implies that there are different ways of going about a given set of tasks, the choice will make a significant difference to the efficiency or effectiveness with which the job is performed, and the most appropriate choice is dependent on the individual
circumstances in which the operation is performed, so that it cannot be codified into a protocol. If one accepts the argument put forward there that the choice of means is a matter of coordination, individual task autonomy arguably comes closest of any among the six core indicators to capturing at least a part of the substantive
complexity dimension.
However, individual autonomy is also a form of work organisation, and only some kinds of job lend themselves to this form. Where the coordination involved is coordination among persons – that is, in any environment where work takes place in a closely coupled team, or where there is tight interdependence between supplier and customer or members of a supply chain – it may be simply inappropriate for individuals to act wholly on their own initiative and judgement. This interpersonal coordination may be a major element of the skill required by the job, and jobs characterised by highly interdependent working can be more skilful than ones that require little cooperation. Despite this, when an individual’s job is redesigned to fit into a more cooperative work arrangement, the worker may see the loss of
autonomy as deskilling. This is especially so if the job in its previous form was a specialist or supervisory one.
4.2.5. HAVESAY
This variable provides some compensation for the limited applicability of
OWNTASK. Even if a worker cannot decide independently for himself how the work is done, he can still have a degree of control over its organisation through collective decision or input to job design. Indeed, the need to negotiate mutually optimal arrangements may increase the skill demands of the job over a comparable one where individuals are free to find the way that best suits themselves. However, HAVESAY incorporates its own kind of ambiguity because it can accommodate a large range of possibilities including individual autonomy, devolved
decisionmaking and inclusive hierarchical decisionmaking. A job can be quite rigidly prescribed and controlled in the exercise, but involve workers regularly in the process of job definition and review on which the controls are based. This participatory-bureaucratic style of work organisation contrasts in practice and ethos with one where individuals work with high levels of autonomy but in circumscribed roles which are defined without any input from them. Both differ just as much from the situation where a work team has latitude to structure its own tasks but little say on how those tasks are defined. The three forms of work organisation require different skills, and perhaps different levels of skill. Yet all three could attract an equally high score on this variable.
4.2.6. WORKFLOW
Control over workflow, as already argued in Chapter 3, is a central aspect of substantive complexity in the kind of job where a large number of tasks with conflicting or uncertain priority have to be handled in tight timeframes. However, the name assigned to this variable is not wholly accurate. As the only variable in the core set that refers to the time control dimension, it also needs to accommodate responses that refer to things other than actual control over the sequencing of tasks; indeed, this is not the most literal reading of the question as asked. Some
respondents could interpret it as referring either to flexibility in their overall hours of work, or to flexibility in the way they schedule their working time over a standard working day or week. Until supplementary variables were introduced in Wave 5 to expand the coverage of time control, it was difficult to interpret the response or read any skilling implications into it.
The time control dimension in general suffers from the same limitation as the personal autonomy dimension, namely that it is not an option for many kinds of otherwise skilled work. A style of work organisation based on intensive, highly collegial teamwork could well attract a low score on this question. So would the type of work which is highly demand- or event-driven, e.g. in a hospital emergency ward, even though many of the highest-skilled jobs fall into that category.