7.2 RIF: the Response Interpretation Framework
7.2.2 Applying the RIF to four FL learning tasks
7.2.2.3 Task type III
The task that exemplifies Task type III corresponds to Activities 1 and 2 of the Final Task in Education and Training in the English version of the ALLES materials. The task’s title is Registering for a course. We first present the TAF analysis:
Description The learner has to write an email to the human Resources department of the company for which she works and register for a course, taking into account a piece of information given in the input. Focus Meaning.
Outcome An email requiring to be registered in a course.
Processes Writing a formal email; registering one for a course; being able to argue for the suitability and appropriateness of the course to one’s own interests and restrictions; understanding messages from one’s boss; understanding course descriptions; understanding calendar pages. Input The task includes three pieces of input data: a calendar page with
one’s fictive month schedule; a recorded message from the department manager; and a description of the available courses. The instructions include also information on what has to be included in the email.
Response type Extended production response: an email. Teaching goal Structured communication.
Assessment Summative and formative.
According to the TAF, this is a writing task focusing on meaning that has both a communicative goal and a communicative outcome. It is again an individual role play activity in which the learner is put in a setting where s/he has to integrate information provided by third parties (manager’s voice message and course information) and her/his own (personal calendar) to be able to require for the registration per email to an in-company department. All of these processes are compatible with a professional setting.
As for the response, it would be qualified as an extended production response, an email, whose thematic content is considerably restricted by the input data and the task instructions (see the RIF analysis below). Since it is a final task in ALLES, it requires both summative and formative assessment. Compared to the two previous task types this task type has a longer response than Task type I, and a more complex response in terms of processes than both Task types I and II.
7.2.2.3.1 Characterisation of the response in pedagogical and linguistic terms
Table 7.10 shows the application of the RIF to characterise the expected response for Activities 1 and 2 in the Final Task of Education and Training in terms of instructions and input. The prompt requires the learner to send an email to the human resources department of her company. According to this, s/he has to express her interest for a course, explain that this course suits the advice given by her manager, as well as her monthly schedule. As seen in the image included in the Input data, to complete it the learner is provided with a calendar page, a recorded voice mail from her boss with information relevant for the decision, and a list of courses offered by the department (the two latter are reflected in the image in two icons in the lower right corner of the image: its actual contents can be found in Annex C).
As for the instructions, they require the learner to make sure that certain pieces of information are provided, most of which are included in or inferable from the information in the input data. Instructions also remind learners to include certain text elements in the email, which are mainly oriented to guide learners in providing the requirements of the text genre.
Prompt Now, you are an employee of the marketing department at Inteltrans. You just got an email from the Human Resources Department. In this email several courses for training and education of employees are listed. First, you need to check your calendar to see if you have some time free to take some courses. Then, you have to read the email from Human Resources and check for the schedule of the courses. Finally, you need to check your voice mail to listen to an important message from your manager who will give you recommendations for your training and the advancement of your career at Inteltrans. Once you have decided what courses to take, proceed to write the email to register for the courses.
Instructions Now you are ready to reply to that email from Human Resources. Don’t forget to specify the course or courses you are taking, the reason and whether you have checked with your manager this training. Below you can find a short list of items you need to address in the email:
• Address recipient
• Introduce yourself and specify your department • State courses you are planning to take
• State whether it’s OK for your schedule, from whom you got authorisation, and why you are taking this training
• Specify other course(s) you would like to take in the future
• Your signature
Table 7.10: Application of the RIF to Activities 1 and 2 in the Final Task in Educa- tion and Training in ALLES (continues).
Input data
From: To: Subject:
Table 7.10: Application of the RIF to Activities 1 and 2 in the Final Task in Educa- tion and Training in ALLES.
As for the relationship between input and response, the activity has a notably broad scope both in the input and in the response. They are both complex in language terms and lengthy. The learner is required to process a considerable amount of information (an audio message, a course list and a calendar page) to extract from them the relevant information. The response to be produced is an email with a considerable amount of information.
As for the directness of the relationship, this is again relatively close in terms of topical knowledge but rather open in terms of the expected language knowledge. The lexical contents to describe the relations between entities that are expected in the response are not guided by the instructions, maybe a bit by the input data, although the instructions include specific information on the email structure.
Table 7.11 includes a detailed analysis in terms of the thematic and the linguistic content of the response to this activity. The first block in the table presents the expected entities and relations:
• Your name and the name of the department in which you work • The course(s) in which you are interested
• The authorisation from your manager for registering
• The profit that you expect to gain from attending the course • Other courses you would be interested in in the future
Thematic content of the expected response Entities – I, as a sender of the request: my name
– Related to the department: Marketing Department, David Altman
– The names of the courses: Business Communication and E-commerce and E-business.
– Related to time: schedule, days of the week, day times, availability
– Related to manager approval: manager, approval, per- mission, ...
– Related to why the course is relevant for the applicant: improve, improvement, better position, future position, future projects, ...
Relations – State the applicant’s affiliation to the Marketing De- partment
– State the courses that suit your needs and availability – Argue decision in connection with future plans
– State agreement with or approval by your supervisor – Express interest in other courses in the future
Linguistic content of the expected response
Functional – Asking others to perform an action – register one for a course
– Expressing an interest or an intention – Arguing for decisions
– Understanding pieces of advice – Understanding course descriptions – Understanding calendar pages
Syntactic – Present tense to express current states of affairs – Future tenses (be + gerund, going to + infinitive, will + infinitive, ...) to express future plans
– Causative sentences to express one’s own or third-party interests in making decisions
– All the syntactic phenomena related with the lexical choices included in the response
Table 7.11: Thematic and linguistic content according to the RIF for Activities 1 and 2 in the Final Task in Education and Training (continues).
Lexical Work (in), Marketing Department, name, register, sign up, attend, course, match, affect, schedule, free, time ex- pressions, authorisation, manager, intend, apply, posi- tion, career, future, useful, project, take a course, etc. Pragmatics Provide information in a coherent and cohesive manner
– Use adequate discourse markers and pronouns to glider the text
– Email structure: greeting, body, complimentary close, signature
Graphology – Observe grammar and spelling required in private pro- fessional contexts
Table 7.11: Thematic and linguistic content according to the RIF for Activities 1 and 2 in the Final Task in Education and Training.
The information to be provided is mostly in the input data: that you work in the Marketing Department, that your manager’s name is David Altman, and that he would be happy that you take the course on Business Communication or the one on E-commerce and E-business, and so on.
The second block in Table 7.11 describes the linguistic contents of the response. The task aims at eliciting from the learner linguistic expressions to express a will, to introduce herself/himself and her/his department’s name, to argue on the basis of third-party advice’s, to express interests, and to justify decisions.
In terms of syntax, learners are expected to use the present tenses to describe states of affairs; to use future tenses to express future intentions and expectations; to use subordination or coordination to express causes; and to use appropriately the functional words – prepositions, conjunctions, etc. – as required by the linguistic items relevant for the response.
As for lexical content, there are several expressions within the semantic field of training courses, career plans, company structure, etc. that learners could use to produce a response. Some of these expressions are found in the input data, but most of them are practised through the unit. In any case, they can hardly be taken as is from the different places where they appear and learners need to resort to their own creative processes to produce the correct linguistic structures.
As for the pragmatic contents, learners are required to provide typical informa- tion in emails, some of which is already provided, and ensure that it contains (i) a greeting, (ii) the body of the message containing the expected information, (iii) a complimentary close, and (iv) a signature.
7.2.2.3.2 Assessment
We first present the criteria for correctness, which determine the feedback corre- sponding to formative assessment, and in the following subsection we present the summative assessment strategy proposed.
A correct response to this task should include:
• An email with addressee, subject – provided as input data – and text • The text email should contain:
– Greeting to address colleagues in the human resources dept. – Body, with the required information – see above and Table 7.11 – Complimentary close
– Signature
• In terms of language knowledge the response has to include:
– The appropriate expressions to introduce oneself, describe intentions, re- quire actions, report opinions, etc.
– Use the appropriate syntactic structures in accordance with the lexical choices
– Structure the text in a coherent and cohesive manner
With all this information a RIF-based sample response can be generated. We present it and comment on it in Section 7.2.2.3.3, after we present the summative assessment criteria for this task.
Summative assessment criteria The criteria for summative assessment are rooted in the pedagogical goals of the unit. The purpose of this assessment in ALLES is that of a low-stakes assessment: to grade and to evaluate learner’s progress. To do so, in collaboration with SLA experts in ALLES, we defined indicators based on textual cues that can be used for the assessment of the responses. These indicators correlate with the four aspects we mentioned in Section 6.2.6.2: communicative contents, lexical contents, sentence structure and accuracy, and overall text layout.
These indicators as well as how they combine and link to specific grades and feedback messages are shown in Tables 7.12, 7.13, 7.14, and 7.15. Since these are four very large and wide tables they are presented in landscape orientation after com- menting on them, which we do in the following paragraphs. The implementation of the feedback generation strategy of these assessment criteria is described in Chapter 8.
First of all, the tables present two to three columns containing indicators for the linguistic items to be identified for each of the four aspects. In the following two columns, a grade ranging from 4 to 0 is linked to each possible combination of values for each indicator. The different combinations of the values of the indicators are correlated with different grades and different “canned” feedback messages. The feedback that the learner gets results from adding up all the messages obtained for each of the dimensions.
As for the communicative contents, Table 7.12 shows two indicators that take into account the presence of the expected thematic content (TC), as well as the
expected linguistic content (LC) at the level of text genre. Indicators are based on the identification of pieces of information (language chunks) that correlate with the expected elements of the response, linguistic or thematic. In this task, elements corresponding to thematic contents are expressions relating to introducing yourself, saying what department you work for, and so on. As for elements corresponding to linguistic content, these are the greeting, the complimentary close, etc.
Using this table, when a response obtains the best possible value for both indica- tors gets the message “Very good. You use the expected functions adequately” (first row in Table 7.12). In contrast, if a response obtains the best value with respect to the thematic content but the worst one with respect to the linguistic content, then gets a message as “Careful: the exercise has enough contents, but it is not polite” (fourth row).
As for lexical contents, Table 7.13 shows two indicators related to the use of specific vocabulary (SV) and word fluency (the number of words, NW). Assuming a set of reference values for these two indicators, a learner response can be analysed in a way that two values for each of the indicators are associated with the response. The response values are compared to the reference values and the percentage of overlapping between them is reflected in columns SV and NW and linked to specific grades and feedback messages. Again, good indicator values in both SV and NW yield higher grades and more positive messages, but good value indicators in one of the two combined with low values in the other yield less positive messages.
As for sentence structure and accuracy, Table 7.14 shows the three indicators used to assess it: the number of sentences (NS), the number of discourse markers (NDM), and the number of grammar and language usage errors (NGE). This table has absolute values that have been defined by ALLES content designers according to their definition of the task and their experience as FLTL teachers.
The feedback that can be generated thanks to these three indicators are based on formal aspects of written communication. Three sample of messages are: (i)“Great. Your text is correct and adequate. There are no mistakes.”, (ii) “Careful, the text is adequate but there are too many errors.”, and (iii) “Careful. Your text is adequate but you are not using any connecting words.”
As for overall text layout, Table 7.15 shows the indicators used to assess it: the number of paragraphs (NP) and the number of spelling errors (NSE) in the whole response. As it happens in Table 7.14, Table 7.15 presents absolute figures that have been defined by ALLES contents designers on the basis of their experience. Value combinations yield different sorts of feedback messages.
Communicative contents
TC LC Grade Message
6 4 4 Very good. You use the expected functions adequately.
6 3 3 Very good. You use almost all of the expected functions adequately.
6 2 3 Good. Although you adequately use the expected functions, review the courtesy. 6 1 1 Careful: the exercise has enough contents, but it is not polite.
5 4 3 Very good. You use almost all of the expected functions adequately. 5 3 ≤ x ≤ 2 2 Good. Although you use almost all of the functions, review the courtesy. 5 1 1 Careful: there is some information missing in the exercise.
4 4 2 Careful: there is some information missing in the exercise. 4 x ≤ 3 1 Are you sure you have understood the purpose of this exercise? 3 x ≤ 4 0 Are you sure you have understood the purpose of this exercise?
Table 7.12: Indicators for the assessment of communicative contents: thematic content (TC) and linguistic content (LC) at the level of text genre.
Lexical contents
SV NW G Message
80% ≤ 100% 90% ≤ 100% 4 Excellent. Your text reads well and is precise. You are using the (...) 80% ≤ 100% 50% ≤ 89% 3 Good. Your text is pertinent but you should be more fluent.
80% ≤ 100% 0% ≤ 49% 2 Careful: You are using adequate vocabulary but the text does not read well. 50% ≤ 79% 90% ≤ 100% 3 Excellent. Your text reads well, but you should use specific vocabulary. 50% ≤ 79% 50% ≤ 89% 2 Try to be more fluent and use specific vocabulary.
50% ≤ 79% 0% ≤ 49% 1 Careful: your text does not read well and you should use more (...) 30% ≤ 49% 90% ≤ 100% 2 Good. Your text reads well, but you should use specific vocabulary. 30% ≤ 49% 50% ≤ 89% 1 Careful; Try to be more fluent. Check the vocabulary you are using. 30% ≤ 49% 0% ≤ 49% 0 Careful. Try to write a text that reads well. Check the vocabulary.
0% ≤ 29% 90% ≤ 100% 1 Good. Your text reads well, but you should use specific vocabulary. 0% ≤ 29% 50% ≤ 89% 1 Careful; Try to be more fluent. Check the vocabulary you are using! 0% ≤ 29% 0% ≤ 49% 0 Careful. Your vocabulary is inappropriate and the text does not read well.
Table 7.13: Indicators for the assessment of lexical contents: use of specific vocabulary (SV) and the number of words (NW) as a fluency measure.
Sentence structure and accuracy
NS NDM NGE G Message
10 ≤ x ≤ 9 10 ≤ x ≤ 9 0 ≤ x ≤ 1 4 Great. Your text is correct and adequate. There are no mistakes. 8 10 ≤ x ≤ 9 0 ≤ x ≤ 1 4 Great. Your text is adequate, but there are some minor errors. 7 ≤ x ≤ 6 10 ≤ x ≤ 9 0 ≤ x ≤ 1 3 Good. But some information is missing.
5 ≤ x ≤ 0 10 ≤ x ≤ 9 0 ≤ x ≤ 1 2 Careful. Your text is too short or has too long sentences, though it is adequate.
10 ≤ x ≤ 9 10 ≤ x ≤ 9 2 3 Great. Your text is adequate, but there are some minor errors. 8 10 ≤ x ≤ 9 2 3 Good. Your text is adequate, but there are some minor errors. 7 ≤ x ≤ 6 10 ≤ x ≤ 9 2 2 Good. But some information is missing, and there are some minor
errors.
5 ≤ x ≤ 0 10 ≤ x ≤ 9 2 2 Careful. Your text is too short or has too long sentences, and it has some grammatical mistakes.
(...)
10 ≤ x ≤ 9 10 ≤ x ≤ 9 x ≥ 5 3 Careful, the text is adequate but there are too many errors. 8 10 ≤ x ≤ 9 x ≥ 5 2 Careful, the text is adequate but there are too many errors.
7 ≤ x ≤ 6 10 ≤ x ≤ 9 x ≥ 5 2 Careful, some information is missing, and there are too many errors. 5 ≤ x ≤ 0 10 ≤ x ≤ 9 x ≥ 5 1 Careful, the text is not adequate and there are too many errors. 10 ≤ x ≤ 9 8 ≤ x ≤ 6 0 ≤ x ≤ 1 3 Good. Your text is adequate, but there are some minor errors.
8 8 ≤ x ≤ 6 0 ≤ x ≤ 1 3 Good. But some information is missing. Check it out.
7 ≤ x ≤ 6 8 ≤ x ≤ 6 0 ≤ x ≤ 1 2 Careful; some information is missing and there are some minor er- rors.
5 ≤ x ≤ 0 8 ≤ x ≤ 6 0 ≤ x ≤ 1 1 Careful, some information is missing and the text is not adequate. (...)
Table 7.14: Indicators for the assessment of sentence structure and accuracy: number of sentences (NS), number of discourse markers (NDM), and number of grammar and usage errors (NGE) in the response (continues).
8 9 ≤ x ≤ 10 x ≥ 5 2 Careful: there are global and local syntactic problems in your text. Check it out, please.
8 6 ≤ x ≤ 8 x ≥ 5 1 Careful: there are global and local syntactic problems in your text. Check it out, please.
8 6 ≤ x ≤ 5 x ≥ 5 1 Careful: you are not using any connecting words. There are too many errors as well.
8 ≤ x ≤ 10 x ≤ 5 0 ≤ x ≤ 2 1 Careful. Your text is adequate but you are not using any connecting words.
6 ≤ x ≤ 7 ∀ x x ≥ 5 0 Careful: some information is missing. There are too many mistakes as well.
5 ≤ x ≤ 0 6 ≤ x ≤ 10 x ≥ 3 1 Careful. Some information is missing and the text has some gram- matical mistakes.
0 ≤ x ≤ 7 x ≤ 5 x ≥ 3 1 Careful. Some information is missing and you are not using any connecting words.
(...)
Table 7.14: Indicators for the assessment of sentence structure and accuracy: number of sentences (NS), number of discourse markers (NDM), and number of grammar and usage errors (NGE) in the response.
Overall text layout
NP2 NSE G Message
9 0 ≤ x ≤ 1 4 Excellent. Your text has an adequate structure and no spelling mistakes.