The development of a test of occupational functional reading ability is predicated upon the assumption that, in some way, the measurements obtained from its use are positively related to adequate performance of job-related reading tasks. If no such relationship exists, there is little point in developing such a test. Further, because of the temporal difference between testing and job performance, that relationship, if existing, may be considered one of prediction: the use of test scores to predict the level of performance. One expects a high functional reading test score to be associated with, and predict, high performance measures on job-related reading tasks.
The picture is not so simple, however. School-leavers, like everyone else in the labour market, enter different sorts of jobs which have different entrance requirements and different tasks
involved. Many such jobs have requirements based on academic stand ards or judgemental assessments of general ability, and these are largely determined by the content of the job. Thus, insurance
companies tend to employ school leavers of proven academic achievement (usually only those with four or more O-level passes), on the basis that the work demands abilities associated with that level of achiev ement. Similarly, van boys are recruited with little consideration of general ability as the job is undemanding. Non-academic criteria, such as reliability, perseverence, appearance and manners take on
much greater weight. It follows, therefore, that if different sorts of school-leaver are taken on into jobs with different requirements, it is not valid to compare, say, a highly intelligent child doing a difficult job with a moderately able child doing an easy job, or with a less able child doing a moderately difficult job. One must compare like with like; to do so suggests that the job should be held constant (e.g. all apprentices) and then any investigation of a predictive relationship should proceed using the test scores of school-leavers and measures of performance on that job. Similarly, this should be undertaken for each job-type.
This brings us to a consideration of the measures of performance on job-related reading tasks. Ideally, one requires a technique for observing and assessing the performance on such tasks of a number of school-leavers, for whom functional reading test scores are also
available. This suggests that either another test be used or some set of objective criteria be developed and applied over the first six weeks of the school-leaver’s employment. The former represents a cir cular problem, as one would need to validate such a test by the same procedures. The latter implies that every working moment of a number of school-leavers would be under observation, or that some objective timed observation schedule be used. There are many objections to this, however, not least of which is the large amount of time needed for one person to undertake the observation versus the limit of six weeks in which to do so. The major objection is the disruption such an invest igation would mean. In that initial period, everyone involved in the induction, training and supervision of a school-leaver, including the leaver himself, are heavily committed and very unlikely to welcome or tolerate the prescence of an observer. (A similar proposal was di s cussed with one company, to provide a detailed analysis of the reading tasks of engineering apprentices as part of the early fieldwork. A courteous refusal was received, on the grounds given above. This was from a firm otherwise prepared to give every possible assistance; less co-operative firms could certainly not be expected to be any more helpful, and probably less courteous!) The same objection
applies to the possibility of training supervisors to undertake the observations themselves.
As it is necessary to have some assessment of performance, other methods must be considered. The most convenient method would seem to be a rating scale of some simplicity, requiring little time to complete by a supervisor, but phrased to elicit as much useful inform
upon today it should be only because of inability to find or devise a better criterion and after systematic steps have been taken to make them as reliable as possible" (p. 38). It would be unfair to suggest that the use of rating scales indicated in this study for reasons of necessity rather than choice. Although other means would be preferable, the judgemental assessment of performance by supervisors is the very assessment that a firm is most likely to make itself, and the one
which is most familiar and best undertaken by the supervisors. Ghiselli reports that "Experience indicates that criteria of occupational success generally have a reasonably substantial degree of reliability, although the reliability is by no means perfect" (1966, p. 24). He mentions that test-retest correlations of ratings tend to lie in the range 0.7 to 0.8. Supervisors* ratings have been widely used and it seems at least just ifiable to continue that use here. In a related area, selection testing for engineering apprentices, Frisby, Vincent and Lancashire (1959) used supervisors* ratings, although they, too, were dissatisfied that no other measure of on-the-job performance was available.
Sticht et al (1972) used a Supervisor's Questionnaire, consisting of five or six point scales on 14 attributes such as conduct, job performance, co-operativeness etc.; plus twelve items such as "Does he
need more supervision on the job than most?". They report, however,
negligable correlations of reading performance with these scales, for the four categories of occupation with which they were dealing. Their scales, however, were aimed at assessing job proficiency, rather than specifically tasks involving reading, and it is a reasonable assumption to expect that ratings of this latter will be related to functional reading test scores. In jobs with high reading requirements in the initial period of employment, one would expect testees with high scores to do well in the job as a whole and, conversely, for jobs with low requirements, there to be no relationship between test score and an assessment of the whole job. Information to substantiate this view would be useful to compare with Sticht’s findings, and it would seem worthwhile to ask for a rating not only on performance in tasks involving
reading but also on tasks not involving reading.
One expects the prediction of performance in job-related reading tasks from functional reading test scores to be linear, within job types. If the test items have been correctly developed to reflect, insofar as possible, typical job-related reading tasks, there should be a directly proportional relationship between the two. Perhaps this view is too
simple, however. The use of Supervisors* Ratings allows one to suggest
that the supervisors may themselves introduce some intervening or modify ing variable into the relationship. For instance, a 8halo* effect may occur, where the perception of overall adequate job performance may lead the supervisor to neglect specific performance deficits in reading. The reverse effect could also occur. Time on the job may also be a factor. Although ideally, rating should be at the same time (the end of the initial six weeks of employment) for each testee, it would be beyond the control of the researcher to ensure this condition was met. Age of the testee may affect the relationship, in that older testees may be perceived as more mature and/or given higher-level tasks to undertake. Situational factors in the job market may introduce biases in the number and quality of entrants into various jobs and industries.
All in all, then, whilst a linear prediction seems to have face validity, at least in the first instance, it will be necessary to consider the possibility of increasing the quality of prediction via a multi-factor approach. Of course, such an approach has technical and operational problems of its own, not least of which include the questions of additivity and linearity of the factors, and the inter pretation of test scores.
This presupposes the actual measurement of the relationship between performance and test scores, and the statistical problems involved. The recurrent problems of restricted variability and skew of scores again need consideration. Linear regression of one variable on another is given by