4.1 Performance assessment
8 Analysis of self-assessments in relation to five CEFR scales
8.2 Sufficient CEFR levels for lecturing engineering subjects
8.2.4 Non-native features are acceptable
On accuracy, one factor that may have influenced lecturers’ views is an article distributed in the mentoring program, which discussed speaker credibility. The article presents errors from a deficit view, maintaining that systematic errors (especially from the mother tongue) threaten NNS credibility and thus should be eliminated from their language24.
From the grammatical-accuracy scale, two levels were suggested as sufficient for lecturing technical subjects: B2 (upper) and B1 (upper). When asked about their perceptions of the overall scale, lecturers commented on the descriptors toward the top of the scale reflecting native-like proficiency. This impression appears to be related to the absence of the phrase ‘mother tongue influence’ in the B2 – C2 descriptors. Moreover, the account in 8.16 implies that mother tongue influence extends beyond the A1-B1 levels. (8.16) <I> and can you pinpoint something specific here that helped you determine your
choice </I>
<E> well the description of c1 and b2 upper is a bit similar, but i think there are there are this this er slips or <READING> minor flaws in the structure </READING> i would i mean i was thinking between or deciding between c1 and b2 upper and i chose b2 upper because it feels that or i feel that the slips and minor flaws are there and they are rare but not very rare </E> <I> all right </I>
<E>and they are not difficult to spot i think difficult to spot may be pending that
the c level is more erm i feel that they are more for native native speakers maybe </E>
<I> and what made you makes you think that </I>
<E>@@well@@er well obviously they are, maybe they are not from the concept point of view but the <READING> consistent grammatical control of
complex language </READING> is not easy to acquire as for a foreign language i think so that’s@@that’s just my analytical point of view@@
</E>
<I> all right erm was there anything else that in the scale that er made you think that this upper level could be targeting a near native speaker</I>
<E> this b1 this mother tongue influence </E> <I> yeah</I>
24
I did not know about this article (i.e. Lavelle 2008) until it was pointed out in one of the interviews. Since it was distributed to all mentees, I assume they have all read it.
<E> well that of course i’m aware that i have mother tongue influence but i so i
didn’t consider that strong enough to be@@to put me to b1@@ </E>
(8.17) <I> all right, ok, and now the title of this table grammatical accuracy erm what does this word accuracy mean to you </I>
<D> grammatical accuracy, it means you er you don’t make mistakes it’s a very discrete thing you either make a mistake or you don’t in this case for grammar </D>
<I> ok erm so that’s very clear erm what level on this scale do you think would be sufficient for lecturing in english to non-native speakers for example here in finland </I>
<D> in finland i think that b1 is actually the one in that case </D> <I> and why would you choose that level </I>
<D> erm because a2<CEFR LEVEL> isn’t good enough because then the errors
you make are going to distract the listener </D>
<I> all right </I>
<D> with upper you i mean it’s clear that you’re not a native speaker but it
doesn’t harm it doesn’t get in the way </D>
The two accounts, 8.16-.17, demonstrate the difference between a deficit view and a neutral view to grammatical accuracy. In 8.16, the account illustrates how deficit definitions can cause confusion for assessing the SLU of English: E recognizes that his grammar has mother tongue influence but considers his level higher than B1, thus chooses descriptor B2 (upper). In 8.17, the account does not seem to consider whether mother tongue influence extends beyond level B1, but does consider this feature to be acceptable for lectures and thus chooses this level. This difference raises the question of whether it is necessary to describe grammatical mistakes with deficit terms in the lower half of the scale, marking features from the mother tongue as undesirable. A critical discussion of this prescriptive aspect of the CEFR grammatical accuracy scale is presented in 5.3.1.1.
Another inconsistency that concerns the harmonizing of the scale is a shift in focus. According to the next account, the descriptors in the lower part of the scale focus on communication and those in the upper part on accuracy.
(8.18) <I> (…) how is grammatical accuracy being judged here in the scale </I>
<C> ok, yeah . (xx) how well it's understood, anyway on these levels <SIGHS>
on these levels <INDICATING B1 AND BELOW> i think it's more about the understandability but here on the higher levels more towards accuracy
</C>
Such a shift in the internal consistency in a scale can cause ambiguity in professional assessment, where communication is primary for accomplishing work-related tasks and accuracy secondary. An inconsistent scale can thus lead to an unreliable or invalid assessment, especially for SLU where the focus is on efficient communication not
accuracy. Likewise, the NS-referencing is also problematic as it is irrelevant to professional communication in an international context (8.19).
(8.19) <I> ok, all right and er how about when you read through these would these grammatical descriptions would they be something that you could apply to your work situation </I>
<C> <SIGHS> ok i can evaluate myself based on these definitely this is in a way
clear but this this is not necessarily exactly er relevant from my my work perspective </C>
<I> ok you said this is not relevant particularly </I>
<C> not not exactly b-because there is only one native speaker in my class
anyways so it's er it's not so relevant what is the difference between my myself and native speakers they <THE STUDENTS> are not native
anyways ok some of them may may be able to assess that aspect but (xx)
what's the difference between native spoken english and er but in that situation when i'm giving the lectures that's not really the main point </C>
On neutrality, the accounts in 8.16-.19 suggest that there is no reason to bias scales with NS norms. In other words, it is not the aim of professionals communicating in ELF to achieve native-like proficiency or to erase mother tongue influence from their spoken grammar (regardless of their level). These accounts provide further support to the irrelevance of native-likeness to the evaluation of professional communication.
On this point, one account differed, where the speaker explicitly refers to the NS- biased article mentioned in the beginning of this section. The example in 8.20 first identifies level B1 as adequate for lecturing, a descriptor that mentions noticeable mother tongue influence. However, the account then suggests this level to be insufficient for speaker credibility – according to the NS-biased article.
(8.20) <I> and the last one grammatical accuracy what level do you think would be sufficient <FOR LECTURING ENGINEERING SUBJECTS> </I>
<A> grammatical yeah, i would say that even b1 would be enough here but there is one aspect here the credibility of of the teacher and if you want to have a
good credibility you probably need to have at least b2 or c1 so that's i think
it comes from that article <GOES TO GET IT; SEE EXPLANATION IN BEGINNING OF THIS SECTION> so the more systematic mistakes you
make in the classroom when you are teaching er the less credibility you get from the students so so in some cases er the gram the grammar i er find that the grammar is very important in language i think it's the basis of language the grammar if you if you don't handle the grammar well then you run into more problems actually </A>
In this example, speaker credibility is viewed as a desirable trait for lecturing. It also appeals to the NS-biased article, and this speaker accepts that view. The account stresses the importance of good grammar and eliminating systematic mistakes as defining traits for
professional credibility. In contrast to this view, the following account contains no mention of the Lavelle article and links credibility to non-linguistic skills.
(8.21) <I> ok erm how about credibility are you ever concerned about that when lecturing </I>
<C> @@of course so i think it's important to have deep knowledge (xx) i should be able to answer questions but some sometimes these asian students are difficult to understand then then it's difficult </C>
From the accounts, the picture that emerges is that grammatical descriptors for assessing professional communication need to be consistent in their focus. The accounts also imply that some NNS features are acceptable, even at high levels of proficiency. Moreover, they indicate speaker credibility to be a desirable trait, and suggest the ability to demonstrate in-depth knowledge as important as well as having good grammar.