Criteria of the quality of interpreting

The essential problem of quality can be summarised by Shlesinger et al.’s (1997: 123) two questions: ‘Quality according to what criteria? Quality for whom?’. Several studies with varied methodological lines such as surveys, experiments, corpus-based observations and case studies (cf. Pöchhacker 2001; 2002) have identified a number of criteria which can be divided into two main groups: professional standards and user criteria21 or what is generally referred to as professional norms (Chesterman 2000: 67) and ‘expectancy norms’ (ibid: 64) which in turn reflect a deeper argument over who is qualified to assess interpreting quality.

2.2.2.4.1.1 Professional criteria

Bühler (1986) has conducted an empirical study into the criteria of interpreting quality by

21 _{See Schjoldager (1996) and Riccardi (2002) for proposals of criteria for the assessment of students’}

administering a questionnaire containing 16 criteria22 to 47 AIIC members (six of whom are members of AIIC’s Committee on Admissions and Language Classification23) to give their ratings for these criteria when evaluating a candidate for AIIC membership. She assumes that her study provides a system of criteria that reflects ‘the requirements of the user as well as fellow interpreter in a […] well-balanced mixture’ (1986: 233). She has found out that

practically all linguistic criteria, namely fluency of delivery (3), logical cohesion of utterance (4),

sense consistency with the original message (5), completeness of interpretation (6), correct grammatical usage (7), and use of correct terminology (8) were rated high by informants, and that among the extra-linguistic criteria reliability (14), followed by thorough preparation of conference documents (10) and ability to work in a team (15), were the leaders. The criteria of native accent (1),

pleasant voice (2), use of appropriate style (9), endurance (11), poise (12), and pleasant appearance

(13) were considered desirable in most cases, but not essential (1986: 233, emphasis in original). Sense consistency with the original message has been given the highest rating of importance (Bühler’s 1986: 231) by 96% of the respondents.

Déjean Le Féal (1990: 155) argues for what she calls commonly shared standards of what to be considered professional interpreting for quality evaluation of interpreter output:

What our listeners receive through their earphones should produce the same effect on them as the original speech on the speaker’s audience. It should have the same cognitive content and be presented with equal clarity and precision in the same type of language. Its language and oratory quality should be at least on the same level as that of the original speech, if not better, given that we are professional communicators, while many speakers are not, and sometimes even have to express themselves in languages other than their own.

What is also noticeable in the two studies is the fact that while professional criteria of quality are being discussed, user expectations are given prominence.

2.2.2.4.1.2 User criteria

Many studies have followed to test Bühler’s standards of quality from a user-oriented perspective. Inspired by Bühler’s study, Kurz has conducted a series of empirical studies (cf. e.g. 1989a; 1993; 2001) to compare Bühler’s findings from a user-oriented perspective

22 _{(1) native accent, (2) pleasant voice, (3) fluency of delivery, (4) logical cohesion of utterance, (5) sense}

consistency with the original message, (6) completeness of interpretation, (7) correct grammatical usage, (8) use of correct terminology, (9) use of appropriate style, (10) thorough preparation of conference documents, (11) endurance, (12) poise, (13) pleasant appearance, (14) reliability, (15) ability to work in a team and (16) positive feedback from delegates (Bühler 1986: 234).

by administering a number of questionnaires containing the first eight criteria in Bühler’s survey to three different user groups24 to give their ratings of the relevant importance of these criteria, setting from the assumption that different user groups might give different ratings of importance to different criteria (2001: 398).

Yet, there have been high correlations not only in the expectations among different user groups, but also between user expectations and AIIC criteria in Bühler’s study in the ratings of importance of such important criteria for communication as sense consistency with the original message, use of correct terminology and logical cohesion of utterance, in this order (Kurz 1993: 317; 2001: 398). This has led Kurz (1993: 319) to argue that the results confirm the validity of the theory of sense (cf. Seleskovitch and Lederer 1995; Chapter 4).

AIIC interpreters and different user groups, however, have differed in their ratings of other criteria such as native accent, pleasant voice, correct usage of grammar, fluency of delivery

and completeness of interpretation (Kurz 1993: 317; 2001: 398).

Kurz has concluded that AIIC interpreters in Bühler’s study generally impose higher demands on interpreting quality than users and that the findings validate theories which view translation and interpreting as an intercultural communication process and stress the importance of situationality and communicative context, which in turn shows that TL receivers must be seen as an important element in the interpreting process (1993: 323).

A number of other researchers have conducted similar user-oriented surveys including Vuorikoski (1993; 1995; 1998), Kopczyński (1994), Mack and Cattaruzza (1995), Moser (1995), etc. Generally, these surveys emphasise the importance of users’ opinions regarding quality, which is commonsense. Seleskovitch (1986b: 236) has anticipated this a long time ago, arguing that ‘interpretation should always be judged from the perspective of the listener’ because the ‘chain of communication does not end in the booth’.

However, each survey has addressed a given audience, in a given situation, in a given

24 _{47 delegates of an international medical conference in Klagenfurt, Austria in 1988 (cf. Kurz 1989a: 144),}

29 delegates of an international conference on quality control in September 1989, and 48 participants in a Council of Europe meeting on equivalences in Europe in 1989 (cf. Kurz 1993: 316).

language combination and focused on different variables, which highlights, in Vuorikoski’s (1998: 189) words, the considerable heterogeneity of audience’s opinions even within the same audience, though almost all surveys provide a consensus on the importance of faithfulness to the sense of the original as the most important criterion of quality. Thus, some (Shlesinger et al. 1997: 128f) have criticised user surveys for their unreliability resulting from the lack of comparability of findings, inability to make up for the basic lack of clear definitions of quality components (Mack and Cattaruzza 1995: 47), and for being ‘more liable’ to subjectivity and ‘ideological distortion than research based on a rigorously descriptive approach’ Garzone (2002: 118).

Thus, there have been calls for, first, specific definitions of criteria (cf. Vuorikoski 1998: 193); second, broadening research into quality to include information, communication and pragmatic issues to define the communicative situation as a whole including the special demands of the situation, speakers, organisers, audience and interpreters, modes or conferences, etc. (cf. Kahane 2000); and third, coordination of the way surveys are being carried out, and a multidisciplinary approach to the evaluation of results to obtain reliable information on users’ perceptions and expectations (cf. Mack and Cattaruzza 1995: 47).

2.2.2.4.1.3 Who can evaluate the quality of conference interpreting?

The dichotomy between criteria defined by professional interpreters and standards defined by users of the service reflects an even deeper chasm in the question of who should assess interpreting quality: is it the user or professional? If it is the former, are users capable of providing quality assessment? What makes it more complicated is that the user is not only the listener, but could also be the speaker (Kopczyński 1994: 190) or client who might not be a speaker or listener (Gile 1995d: 36).

Shlesinger et al. (1997: 127) argue that listeners cannot be judges of interpreting quality for two reasons. First, they lack the ability to know the SL, which is one of the fundamental means for assessing fidelity/quality. Kurz (2001: 403), however, argues that this does not necessarily apply to all audiences since some audiences, such as Finnish delegates (Vuorikoski 1993: 324), know the SL but use the interpreting service only for support. The present researcher’s experience shows that some use it even for prestige. However, Seleskovitch (1978a: 121) argues that experience has shown that multilingual delegates

unfamiliar with interpretation problems may ‘grossly misjudge the quality of the interpretation’ by looking at individual words rather than sense. Moreover, some studies have empirically shown that users cannot be judges of the fidelity of interpreting even though they might happen to know the SL because subjects have failed to detect a considerable number of errors (Collados Aís 1998: 336; Gile 1995c: 160f).

Second, as shown above, different user groups or listeners have different priorities, tastes, attitudes, etc.; therefore, satisfying listeners might not necessarily result in high-quality performance as Cartellieri (1983: 213) rightly argues in his call for finding reliable parameters for assessing quality, in particular, the quantity of some features that might develop into quality criteria such as keeping a certain rate of speed whatever that of the speaker is might result in a loss of some original material (quantity) to serve the smoothness of presentation (quality) (see also Pöchhacker 1994).

If users cannot be reliable judges of interpreting quality, then can interpreters themselves judge the quality of their output? Déjean Le Féal (1990:154) and Gile (1995d: 37) dismiss the interpreter’s on-line self-evaluation of his/her output as unreliable because the complex nature of the interpreting process and efforts involved prevent the interpreter from giving much needed attention to monitor the output. Déjean Le Féal’s (1990: 156) idea of self- assessment of output by tape-recording is also dismissed as subjective. Viaggio’s (unpublished script) idea of judging quality by a professional interpreter who does not know or is not given access to the SL cannot be accepted either for the purposes of fidelity or sense consistency with the original.

Seleskovitch (1986b: 236) recognises the importance of user satisfaction, but she argues that ‘competent’ conference interpreters or chief interpreters should assess an interpreter’s performance as long as they are ‘able and objective’.

It might be argued that while user satisfaction should be taken on board in any project of quality assessment, interpreting quality should always be assessed by independent

conference interpreters or teachers of conference interpreting, because they are experts in the field. In all areas of research or practice, quality assessment is usually assigned to experts in a given field, which is commonsense. There is no reason why conference

interpreting quality should be singled out and assigned to other people than experts. Since on-line assessment is very difficult, if not impossible, by audience or interpreters alike, Déjean Le Féal’s (1990: 156) idea of self-evaluation by tape-recording can be made less subjective by extending the assessment of the recording to other more experienced professionals or chief interpreters as Seleskovitch (1986b: 236) suggests.

As for the question of what criteria (professional or user criteria?) should be assessed, some argue that these should be norms that concern all players in any conference (interpreters, users, listeners, speakers, clients, conference organisers, etc.) (cf. Gile 1999; Garzone 2002) including factors, prerequisites, conditions and requirements that are not only related to the interpreter’s output, but also to all stages of the process, before, during and after the interpreting process (cf. Kalina 2002; 2005) though, as Kalina admits, research is needed to determine which ones can be measured and how (see also Pöchhacker 1994).