GENERAL-PURPOSE AND SPECIFIC-PURPOSE LANGUAGE TESTING

6.8.1 Having established the importance of a code of ethics and best practice for testing and having laid the groundwork for identifying or developing appropriate language tests for pilots and controllers, testing administrators (providers or developers) will next need to consider content. The provisions of the ICAO language proficiency requirements that address test content are as follows:

• Annex 10, Volume II, Chapter 5, 5.1.1.1 refers to the need for both phraseologies and plain language;

• Annex 1, Appendix, holistic descriptors 2 b), 2 c), and 2 d) refer in turn to “work-related topics”, “work-related contexts”, and “routine work situation”; and

• Annex 1, Attachment, under Vocabulary and Comprehension, refers to “work-related topics”.

6.8.2 The objective of the ICAO language proficiency requirements is to ensure that flight crew and air traffic controllers have sufficient language proficiency in whatever language they use for radiotelephony communications to manage all of the potential communicative needs related to pilot and controller communications, ranging from routine phraseologies, to routine communications not encompassed by phraseologies, to non-routine situations (aircraft lost or low on fuel), to outright emergencies. It is known that

Chapter 6. Aviation Language Testing 6-11 pilots and controllers can memorize phraseologies, and it is known that phraseologies are often not sufficient, and so it is clear that a language test for aviation communications should not be limited to testing knowledge of phraseologies.

6.8.3 The purpose of direct or semi-direct language testing is to elicit a speech sample for assessment in order to make predictions or generalizations about future language use. It follows, then, that a language test for the aviation industry should replicate as far as possible the work-related communicative requirements. In addition, experience shows that professional pilots and air traffic controllers want to be, and even insist on being, tested on the language they need on the job.

6.8.4 General-purpose English testing attempts to evaluate language proficiency regardless of the place or manner in which it was acquired. It is person-oriented and topic-independent. Specific-purpose English language testing seeks to evaluate a test-taker’s ability to perform tasks in ways that closely approximate job-related tasks. It is mission-oriented and may be more limited in scope. Proponents of general-purpose English testing hold that language ability is not significantly constrained by context, and that inferences about future language ability in one context can be made from language performance in another context (a conversation or interview, for example).

6.8.5 While we tend to talk in terms of “specific-purpose” testing versus “general-purpose” testing, it is more accurate to posit all tests on a specificity continuum. Challenges to the notion of general-purpose language testing have been made on the grounds that all language testing is conducted in one context or another and that there is little value to the notion of general competence because, it is argued, language competence ever only exists in relation to specific contexts; i.e. all language use occurs in a context and the context impacts test performance. Rather than a strict division between two types of tests, a continuum of specificity for language tests is posited, with all language tests falling somewhere on the continuum from very general to very specific. A specific-purpose test has been defined as one in which test content and methods are derived from an analysis of a specific-purposed target language use situation.

6.8.6 In the unique context of pilot and controller communication, it is evident that both types of proficiency tests — general purpose and aviation-specific — may have a role, with, however, two cautionary notes.

a) Aeronautical radiotelephony communications are based in large part on standardized phraseologies. However, in addition to unexpected emergency or urgency situations, there are many non-routine but not necessarily unusual circumstances that are not covered by phraseologies. An appropriate speaking/listening test designed to assess compliance with the ICAO language proficiency requirements will not be limited to testing knowledge of standardized phraseologies but will assess a broader range of communicative abilities. b) A speaking or listening test developed for another context may not be appropriate to the

context of aviation communications.

6.8.7 The content of a proficiency speaking test, or the degree of aviation specificity, will be determined in part by at what point in the training or career the test is given. Naturally, if the language proficiency of candidates is assessed before they have received aviation training, then a specific-purpose test requiring knowledge of aeronautical radiotelephony communications is not appropriate. There is a need for a number of different tests; for example, a language test appropriate for ab initio pilots or controllers will be different from that required by professional pilots or controllers A useful model may include a test comprised of a mix of both aviation-specific content alongside less aviation-specific content. Such a test would ensure that the training content remains strongly guided by performance objectives. It also gives face validity to the testing process in that it involves ATC professionals as testers alongside the language specialists. The less specific content (or more general content) gives access to a wider sample of student

speech/interaction and allows training to focus on a wider variety of language contexts so as to maintain interest and motivation. In addition, if we assume that all language use occurs in a context and the context impacts test performance, then the context is important. Why should pilots or controllers be tested, say, on their ability to describe a favourite movie, or to order pizza, when what they need to be able to do is to safely negotiate all communicative language functions related to flying and controlling aeroplanes.