Testing Procedures

Testing procedures were done in three- steps:

4.9.4.1 Pre-test

1. In the first week of the field study and after reading the consent forms that showed students what the researchers was doing and her purpose of doing this research; students from both classes (experimental group and control group) did the pre-test. They did the writing test during the normal classroom hours within a specific period of time. Chairs were arranged in a different way from how it is normal in their lecture; the arrangement was in such a way that enough space had to be created between them. They were asked not to use dictionaries for any reason and their phones had to be switched off in order for them not to use their electronic dictionaries. They were also notified that this task was not to be assessed as part of their final result in the course. The students were asked to write an argumentative essay about “watching TV’’ where they were to give two different opinions on this matter (see Appendix).Students were told to feel relaxed and write about the topic. They were also told to support their argument with proper examples from their experiences, reasons, and evidence if they had any, in order to produce an informative piece of writing. In general the purpose of the test was to measure core writing skills that are required in an effective piece of writing which are: the main idea, supporting details, organization and coherence as well as grammar and spelling conventions.

2. The test took up between 30-45 minutes and it was monitored carefully by both the researcher and their teacher.

3. Once they finished their writing, students wrote their names on the paper and handed in their papers to the researcher in order to be evaluated.

4.9.4.2 The treatment stage (intervention)

The treatment/intervention stage was started a week after the pre-test. The students in the experimental group were exposed to teacher-student face-to-face where verbal comments were given. The comments were on the students’ writing performance and included identification of errors, and also showed areas for improvement in terms of the relevance, coherence and organization of the content as well as language form and mechanics of the writing..

Students received feedback on each particular piece of work in the week following its submission. The treatment group received group feedback in normal class time. The teacher indicated the major problems that most students faced and asked them questions that helped them to re-evaluate their own work and to offer their own suggestions in order to revise productively, with students learning from each other’s mistakes. The in-class time spent on group conferences was between 25 and 40 minutes on each occasion. In addition, one-to-one FACE-TO-FACE FEEDBACK sessions took place outside class time. The number of these during varied based on the individual student’s need (e.g., 4 times for the strongest students to 12 times for the weakest during the term). To these figures for teacher-time must be added the time spent reading the student scripts (about 3 minutes per script)

Participants in the control group were provided with written feedback. The nature of written of feedback was various, sometimes it was correction of the error and sometimes it was indications of the type of error. There were typically about 10 interventions on each script. The time spent by the teacher on this activity was approximately 5-6 minutes per script. The students in both groups had the same teaching materials and were taught on the same day by the same teacher. Both groups practiced writing for the same amount of time and had to submit their revised paper after performing the corrections based on the feedback they got.

Students who were not part of the face-to-face are control group who got written feedback only, they amend their task based on the comments they got from their teachers which were regarding content, organization, and language use of their writing. They also did other activities regarding writing. In addition, sometimes they worked in a group and learned from one another by working together on some tasks.

Furthermore, as the purpose of feedback is to keep students revising their work, grades were not assigned in their first drafts.

 Principles and theory

 students learn how to write through face-to-face feedback  research results

 Goals

 To develop students’ writing in academic context.  To increase students’ motivation and participation.

 To enjoy writing and makes it less frustration

 To increase general knowledge about the writing structure, organization and unity.

 To encourage students participate and ask questions either in one to one face- to-face or as a group in the class.

 Procedure

 Students learn how to write through face-to-face feedback and practice writing. The face-to-face sessions were used regularly and with different styles sometimes individually and sometimes in a group based on the students’ need and the time. Each conference lasted for about 10 minutes or more. The teacher gave oral comments on the required assignments in the term. The feedback included different aspects such as relevance, coherence, organization of the content and the mechanics of students’ writing. The students were involved in the face-to-face and they are required to do different tasks.

 Requirements

 Students had to write different assignments during the term.

 Students had to participate either one on one with their writing teacher; or as a group on face-to-face feedback where the teachers highlighted the major errors that were made by the students in their writing and they identified their own errors and discussed how to improve their writing. Successful conferences were identified through literature as the ones where students participated actively in their writing evaluation as Walker & Elias (1987) indicate.

 Students were motivated to produce a better quality of their writing rather than correct their drafts.

4.10.4.3. Post Test

1- The design of post-test was similar to the pre - test design. Identical tests were used for both the control group and the treatment group; identical techniques and data collections were used at the end of the term together with the post face-to-face interview and that was on the 17th _{of January 2013.}

The entire control group attended the post- test. However, there were four students from the treatment group that were absent for the post test.

2. The post-test consisted of the same task as in the pre-test, and in addition other writing tasks which were used for the final exam of the term were used. (see Appendix 7). This latter test was known as the delayed post- test. The reason for including both elements in the post test was to evaluate students

what they should change and improve, while the other tasks was to gain more insight into students’ more general abilities in writing. It was important to bear in mind the timing of the post-tests because timing has a critical impact on the results obtained. As Newton (1999) highlights, there is relation between the result of post testing and how soon the test is administered. He states “ideally if the test is administered immediately after the learning session additional follow up tests at later dates should be used to provide some evidence of application and impact of what has been learned’’. In general, both pre and post-test are used to estimate the change in the student’s abilities that can be attributed to the intervention of using face-to-face feedback in the student’s learning.

4.11 Rating Scale

According to Srivastava et al (2013) assessment of students’ academic achievement is one of the fundamental and crucial steps in any educational project; the assessment tool is used to provide information about attainment of specific learning objectives (p.47)

Evaluation is one of the elements in this study used to reveal the students’ proficiency in their writing as well as to measure their improvement. Both groups-experimental and control- were required to make writing corrections based on either the FACE-TO-FACE FEEDBACK or written feedback. Although the purpose of this study was not concerned with the final result of students’ writing, the multiple trait scoring used for evaluation was used as it is one of the holistic methods for writing assessment (Hamp-Lyons 1991). Hamp-Lyons indicates that “Multiple trait scoring implies scoring any single essay on more than one facet or trait exhibited by the text’’ (1991, p. 247).

4.11.1. Rating

Those who did the rating of the work were both the researcher and the teachers of writing who taught the groups under the study. Both teachers have their PhD from UK with many years of experience in teaching English in Libyan universities. They marked pre- and post-tests sample essays using both holistic and analytical scoring criteria “Research has shown that reliable and valid information gained from both analytic and holistic scoring instruments can tell teachers much about their students’ proficiency levels’’ (Bacha, 2001, p.371). Holistic scoring assesses students’ papers as a whole and balances strengths and weaknesses on certain criteria to assign the score for the effectiveness of a piece of writing, whereas analytic scoring provides students with a rating score for each criterion and analytic scoring feedback can be given. The assessments were made using certain marking criteria, once all criteria were taken into consideration an overall mark was given for each paper. The scale rating for the test was from 0 to 10 (total of 10 scores). As writing is usually scored ‘subjectively’ and not ‘objectively’ as either right or wrong (such as in math subject where 1+1=2); there were some procedures that were used to reduce the subjectivity as much as possible. The procedures were as follows:

The researcher marked the essays first but did not put the mark on the paper, she wrote the mark beside the name of each student in the students’ list. Then the essays were handed over to their teacher to mark them again following the same procedure that the researcher did in that; the marks were written in the student’s list. The reason for evaluating the essays with such method was to avoid any bias in judgment. Then, the researcher and the teacher compared the marks which were mostly the same the mark. Generally, the average of the two raters’ scores was considered as the final grade. It was just in very rare cases that there were score discrepancies, the marks of both markers were summed up and the average (mean) was considered.

However, for the delayed post-test the marking was done by their teacher only and it was blind marking (blind marking is usually used in Libyan exams system starting from primary school to university level at the end of the term or school year). Blind marking is anonymous marking where the students’ names are not shown on the exam paper. Students ID and desk number are usually used to identify the students later. This type of marking policy is used in order to ensure equality of marking process and it is usually applied at the end of term or year exam. This kind of marking confirms that it had been done objectively and the result is considered impartial.

On the whole, the evaluation was based on the general rules that are needed in writing which are content and language accuracy. Thus, in this study, the essay were evaluated and scored out of ten on how well the essay showed the following:

A: Content , organization, style and format; and the evaluation is about:

1- Clear and logical organization as well as coherence in presentation and a well- constructed paragraph and the writing as a whole.

2- Specific supporting examples and details where the ideas in the piece of writing must be relevant to the topic.

3- Is the format of writing appropriate to the question?

B: Language accuracy which includes mechanics and the evaluation is about: 4- Proper grammar, punctuation and spelling

5- Proper use of a variety of vocabulary and the choice of the words and level of formality.

Generally, the evaluation measured: firstly, the students’ ability to produce coherent texts within common university expectations and requirements where the piece to writing should flow coherently and logically; starting from an introduction that attracts a reader to a well- demonstrated conclusion. Secondly, it measured the students’ ability to develop an argument and organize supporting details where the paragraphs flow coherently and examples are provided to support the ideas presented. In addition, especially in regard to the post test which was done after applying face-to-face feedback, the improvement in the evaluation measured the students’ ability to revise and improve on their writing skill and show how their writing is clear and easy to read and follow.

In general to evaluate students’ writing, it is important to look at the following features of Academic writing style:

1. Precise

The information in the text is expressed exactly and does not use expressions that could be misunderstood by the readers.

2. Concise

Concise – expresses complete ideas using as few words as possible 3. Objective/ impersonal

Objective/ impersonal – emphasises the information rather than the writer

4. Responsible – uses sources without plagiarising (if there is need for sources, although in this study the writing was essays)

5. Conventional

Conventional – formal, impersonal and follows the expectations of the academic community it is written for, e.g. uses formulae, cites sources accurately

6. Accessible

Accessible – clearly structured and planned 7. Reader friendly

Reader-friendly – ideas are expressed explicitly using logical links to help the reader

Appendix 8 shows the criteria that were used as a guide for teachers to evaluate students’ writing since there are no certain criteria that are used in Libyan higher education.