Improvement from pre to post-test based on topic

When the results of each question were compared, it could be seen that Class A students scored higher than Class B students in most of the questions in all topics (Figure 6). Only in one question did Class B students score higher than Class A students in the post-test. This overall finding for each question aligns with the improvements by each student outlined in Table 13. As can be seen in Figure 6, in 4 out of 6 of the post-test questions, Class A students did better than Class B students. In the remaining 2 questions, in only one did Class B students score higher than Class A students, and in the other (circle best sentence completion), students from both classes had almost equal results. In order to explain what might have contributed to this, in this section, each question and the respective scores are analysed based on the teachers’ transcriptions.

Figure 6: Distribution of scores in both classes in pre- and post-tests by question

Clauses

For more than seven weeks, students were taught about clauses in class. The types of clauses taught ranged from noun and adverbial clauses to dependent clauses. As this was the largest topic covered, two of the six question sets were based on it.

By looking at the scores, it can be seen that students in Class A did better than students in Class B in both of these questions. Although the pre-test the scores of the two classes were similar, in the post-test, all Class A students scored at least 1 point higher than the Class B students.

To find out how students from both classes were exposed to this topic, an excerpt of the classroom transcripts related to clauses is presented below. In Class A, the teacher introduced “noun clauses” as the topic for the day, followed by asking students to read out the definition. The teacher then explained in detail about noun clauses. During this explanation, the teacher linked it to their previous discussion about dependent and independent clauses.

The Class B transcripts demonstrate a similar teaching method. The teacher discussed dependent clauses, told them how to identify them, and told them how to use one. The difference, as can be seen, is that the teacher of Class A only used English to explain the concept, while the teacher of Class B used both English and Arabic.

Class A

1 T: Let’s have a look at noun clauses today. It’s a dependent clause, lets look at the definition here.

2 T: Who would like to read the definition for me please?

3 S: A noun clause is a dependent clause that functions as a noun in a sentence.

4 T: It is used as a subject, an object of a verb, an object of a preposition or complement.

5 T: Never separate a noun clause from the main clause with comments or other punctuation marks since noun clauses are part of main clauses.

6 T: So you do have a main clause and part of that main clause is your dependent clause and it can function as a subject, as an object and verb, object and preposition in the complement.

7 T: Noun clauses come at the beginning of a sentence or after certain nouns, adjectives and verbs such as if you see any of these nouns, adjectives or verbs usually do have a noun clause after them or these are the verbs that usually start with them.

Class B

1 T: A subordinating conjunction starts as a dependent clause, ṭayb. [Ok.]

2 T: fyn nḥuṭuh? [Where should we put it?] Beginning ….end …doesn't matter.

3 T: ālmuhim yṣyr lahā ma΄nā., [what’s important, it must have a meaning.]

4 T: āysh ālfikrah áily áiḥnā nbghāhā ? [What is the idea that we want?]

5 T: We want to know how do you act. You know how to change things.

6 T: fy kul ālāḥwḥl ānty rāḥ āitḥuṭy qā΄idātayn. [In all circumstances, you will apply only two rules.] āláūlā hy taḥwyl jumlah aw suáāl

āilā "noun clause" …. [The first one is to change a sentence or a

question into a "noun clause"]

7 T: āysh ḥatsaūy.. b΄dyn trūḥy ΄lyhā kulhā ātḥuṭylhā "dependent clause" [after that you go through all sentences again and you add to them a "dependent clause"]

8 T: biḥayth tṣyr "independent", [Then it will become "independent"] 9 T: ṭab΄an. wā āy jumlah āinty ḥatkhailyhā ḥatkūn "independent" wā

ātḥuṭylhā "dependent part" wā ātḥuṭylhā. [Of course. And any

sentence you want to change is "independent" and you add a "dependent part" and change it.]

By looking at the test questions, it can be seen that they were indeed discussed in the class. For example, one of the questions was, “She is not sure whether she fed the cat or not”, where students were asked to circle whether this was an adverbial clause, noun clause, or an adjectival clause. As can be seen from the class transcript above, noun clauses were discussed in both classes, but Class B students often failed to identify the noun clause. Class A students, however, almost always correctly identified it. This finding suggests that there is a likely correlation between students’ performance and the teacher explaining grammar topics only in English or explaining them in Arabic and English. Whether or not this can be seen in other topics is examined next.

Direct and indirect statements

Under the topic of direct and indirect statements, the students were asked to “Choose the correct indirect statements that report the following sentences.” For example, one of the questions was, “He replied, I am on my way to work”. The options were the following:

a. He said that he was on my way to work. b. He said that he was on his way to work.

Overall, the results suggest a similar pattern to that which was discussed earlier. On average, Class A students scored 1 mark, or 25% higher than Class B students in the post-test in this topic as well. In comparison, in the pre-test, Class A and B scores were almost equal. Although Class A students scored higher than Class B students in the pre-test, on average the scores were only 0.07, or 1.75%, higher.

Looking at the transcription, it can be seen that discussion about direct and indirect statements in Class A used English, and the discussion in Class B was in both Arabic and English. Even though the teachers in both classes used “reported speech” to discuss indirect statements throughout the class, Class A students were able to answer questions about indirect and direct statements in the post-test more accurately than Class B students. However, a more interesting observation from the class transcription was that, even though Class B students were introduced to indirect speech in a more thorough manner than Class A students, they scored lower than Class A students. For instance, in the example below, it can be seen that the Class B teacher not only introduced how they can change a direct statement to an indirect statement, she even conducted multiple practice exercises to make the students familiar with the concept. The Class A teacher, on the other hand, explained only the bigger picture and provided just one example.

Based on the extract below, it could be expected that Class B students would score higher than Class A students because they were exposed to the concept and were given a more detailed explanation. However, post-test scores show that, even though students from Class B had more explanation, they did not necessarily learn the concept well.

Class A

1 T: Now the idea of the present perfect just emphasises that the action had been completed at a certain time, that the action is complete or it finished especially with the past perfect.

2 T: When you have two actions happening both of them in the past, the first action that was completed first you'll have it in the past perfect but here with the reported speech you're going to have to do this. 3 T: “I have watched TV,” he said that he had watched TV. “I went

Class B

1 T: liānuh mmkn āfham "stop it" ΄alā āinhā "strong command” [it is possible that I understand "stop it" as "a strong command”] ānā āithā

ḥaūaltahā "reported speech"; ḥaqūl [so when I want to change it into

"reported speech", I will say, she said that you must stop it].

2 T: tamām, khalṣnā [Ok, we’ve finished.] lisā bāqynā waqt, mnkamil "practice" wa "quiz" kamān [we still have time, so we will do both the "practice" and "the quiz."].

3 T: Change the commands to reported speech “be at your desk by nine”

4 T: Practice number three, how would you report, “be at your desk?” Rana? S: I must be at my desk by nine.

5 T: I must be. I should be hiya t’tamid ‘ala [it depends on] iḥsasik [what you feel].

6 T: mumtāz ḥlū āttaghyyr [Excellent! The change is good.]

Sentence type

There was one topic where Class B students scored higher than Class A students in the post-test. In the pre-test, both groups had the same average - on average, both classes scored 2.07 out of 4, but, in the post-test, Class B students scored much higher than Class A students. On average, their scores were 14.25% higher than Class A students.

The question was about identifying sentence type. Students were asked to specify the type of sentence: whether it was simple, complex, compound or complex- compound. Contrary to the overall improvement in scores for other questions, students from Class B scored higher than Class A students on average in the post-test for this topic.

Scores for the post-test were not the only anomaly in this situation. From the overview of class transcripts, it can be seen that, for this class, this was the only topic where teaching and learning in Class B was conducted in English at least 50% of the time. This can be seen from the following example. The only part where the teacher used Arabic was to congratulate or encourage a student. The rest of the explanation was conducted in English. Class A, on the other hand, was conducted completely in English.

Class A

1 T: So simple, compound, complex and compound complex. We talked about the simple sentence, what kind of clause is it?

2 S: Independent.

3 T: It’s one independent clause.

4 T: How can you determine if it’s a clause or not? What do you have to the clause?

5 S: Subject and verb.

6 T: How would you know it’s an independent clause? 7 S: The meaning is complete.

8 T: So it can stand alone, it doesn’t start with a subordinator.

9 T: We have already completed the first part. And now how I can change it into a "complex sentence".

10 T: Imagine that the sentence I will bring in the exam is a question and I ask you to “change it into a noun clause then to add an independent clause to the complex sentence.” ok.

11 T: What does "complex" mean? It means "dependent" plus "independent" and this is what you will say in the exam.

Class B

1) T: “She wanted to know when I will start my new job” Is this a dependent or independent part?

2) S: Independent.

3) T: Independent clause. How would I change it to the complex sentences?

4) S: Add dependent.

5) T: yā sālam ‘liki [well done].

This raises the question: if the Class B teacher had conducted the class in English instead of switching to Arabic so much, would the students have been able to score higher? This is because in the two previous topics discussed, although Class B students were given more detailed explanation of the topic in class, their performance was worse than Class A students. However, when the teacher only used English to explain concepts, students’ scores improved when everything else, including teaching style, remained the same.

Conjunctions

In the topic about conjunctions, again Class A students scored higher than Class B students in the post-test. The transcripts of teachers’ talk show that grammatical concepts were introduced using Arabic or a mixture of Arabic and English in Class B. Even though it was discussed in detail, Class B students scored fewer marks than Class A students.

Class A

1 T: Now for adverbial conjunction, you can join two main clauses. In this situation, the conjunctive adverb behaves like a coordinating conjunction, connecting two complete ideas. Notice,

however, that you need a semicolon, not a comma, to connect the two clauses.

Class B

1 T: ya’ni whether it is a coordinator "conjunctive clause" āw "adverbial conjunction" kuluha ṣahyhah. [I mean whether it is “conjunctive clause" and "adverbial conjunction" both of them are correct.]

2 T: inqūl [We say] conjunctive adverb can appear ……., tamām,

áidhan [Ok, so we have] ndanā wāḥid yarbuṭ wiḥdatyn mutasawiatyn

[We have one that connects two equal units] "grammatically".

3 T: āily hū [which is] "coordinating conjunction" wāḥid yyjy fy

bidāyat āl [one comes at the beginning of the] "independent clause".

4 T: wa wāḥid yyjy fy bidāyat aw muntaṣaf aw nihāyat āl [and one comes, at the middle, or at the end of the] "subordinating clause" aw "strong clause" wa áismuh [and its name is] "adverbial conjunction"

aw "conjunctive adverb".

5 T: fa áinty āysh ḥatsaūy [So you what will you do] when you have

such sentences? …..

6 T: ḥaṭāl΄y wa tshūfy hū marbūṭ bi āysh wa hū jāy fy bidāyat , fy nihāyat, fy muntaṣaf āysh bḍḍabṭ, ṭayyib. [You will look and see with

what it is connected and whether it comes at the beginning, at the middle, or at the end of what exactly, ok.]

As was seen in previous sections, the Class B teacher often explained topics in more detail compared to the Class A teacher. She not only introduced the concept but also explained it by providing various examples that the students could use. She suggested how students could extend their knowledge to other scenarios by asking them to look for patterns in the sentences. When the Class A teacher discussed prepositions, however, she explained them only in brief.

Even though the Class B teacher conducted a thorough introduction and explanation of grammatical concepts, Class B students were unable to answer the questions correctly. Since everything else was the same for both classes, this could suggest that explaining grammatical concepts using Arabic may not have been as effective as doing so in English.

Sentence completion

Out of the six topics covered in the tests, sentence completion was the only one that was not covered in the syllabus, but this topic was asked to gauge students’ existing grammatical knowledge in the pre-test and to follow up in the post-test. In addition, four of the questions asked were considered easy to answer by students in

the Grammar 3 class, as they were already familiar with some English grammar from earlier classes. Based on the results, it can be seen that there was a minimal difference between the two classes in the results of their pre-tests and post-tests. As this was not part of the syllabus, there was no transcription for this topic.