Summary of the findings

This chapter summarises the findings from the two strands of the study to answer the research questions posed at the start of this thesis.

12.1 Is there evidence that instruction in CT can help develop CT skills of ELL (those whose first language is not English) in higher education?

The findings from the review suggest indicative evidence that teaching CT to ELL in higher education can help develop their CT skills. However, because of weak research design and methodological flaws in all the studies, the evidence is not conclusive. But the small-scale RCT suggests that explicit instruction in CT can improve the CT skills of ELL in higher education.

12.2 What is/are the most promising approach(es) to teaching CT skills to ELL in higher education?

The review synthesised evidence from 36 studies that evaluated different approaches to teaching CT skills in higher education language classrooms. Of all the strategies examined, explicit instruction in CT skills appears to be the most promising. It has been evaluated by a bigger number of studies than other approaches (like debate, assessment techniques, journal writing, etc.) and all the higher quality studies reported positive results. There is no evidence as yet that other approaches such as debates, use of literary and narrative texts, brainstorming and scaffolding are beneficial even though all the studies claimed positive effects. This is largely because few studies were found that evaluated these approaches with ELL in higher education settings. So it is not possible to confirm if positive results could be replicated. Also all these studies were very weak in design, therefore no conclusive statements can be made either way. But this does not mean that these approaches do not work in practice. Debates and similar approaches, for example, have been found to be effective in promoting CT among school children (e.g. Gorard, Siddiqui & See, 2017). It just means that there is not enough evidence to support the claim of positive effect.

12.3 Is it feasible to teach CT skills in the Lebanese context?

and religious interference in the education curriculum, it may not be feasible to introduce CT in the classroom.

The results of the RCT suggest that contrary to common perceptions there is evidence that CT can be taught in the Lebanese context. The intervention was generally well received by teachers and students. There were no objections or resistance to the intervention. Overall staff and students responded well to the teaching module and the teaching resources. Feedback was generally positive. Students reported that they have benefited from the instruction and that it had helped them to assess credibility of sources and information. They found these skills useful and relevant not only in their studies, but also in real life application. All in all, the participants welcomed such instruction and were appreciative. This suggests that there is a role for CT instruction in higher education in Lebanon.

This study also demonstrates that CT can be easily introduced into the regular curriculum and no elaborate formal training is needed.

12.4 Does instruction in CT improve the CT skills of ELL students in a higher education institution in Lebanon?

Both the systematic review and the trial indicate that instruction in CT has the potential to develop ELL students’ CT. A subsidiary question is whether the teaching of CT can be effectively and easily embedded within the regular curriculum in higher education in Lebanon. And if so, does it help to develop the CT skills of students. For this reason, the trial examined the effectiveness of the general and explicit approach to teaching CT.

The results of the trial suggest that instruction in CT could raise the levels of CT skills of undergraduate students even if they had no previous exposure to CT instruction in school. Despite criticisms of the general approach by many theorists (McPeck, 1984; Bailin et al., 1999; Moore, 2014), the findings of this trial provide evidence that CT instruction can be easily integrated into existing curriculum and the general approach could work well in a language classroom. But there is no reason why it cannot be used in history, geography or science classes. A little bit of creativity and innovation may be needed on the part of the teachers.

The trial also shows that effects could be realised in just one term. But feedback from teachers questioned the sustainability of the effects, and the transference of skills to other areas of students’ life. There is also the question of whether teachers will continue using the teaching materials or develop their own teaching materials after the trial.

12.5 Does the impact of CT instruction differ for different groups of students? The trial also tries to find out if the instructional approach used is equally effective for all groups of students. The findings showed that girls, foreign students, students who do not work part-time, students coming from Life Science background in school, and students majoring in Engineering at university made the biggest progress between pre- and post-tests.

However, the findings should be read with caution as the subgroup contained only a small number of students. Small samples are particularly vulnerable to volatility of small changes.

Performance on each subskill was also examined. The analyses show that students performed particularly well on questions about ‘credibility’, whereas they performed less well compared to control students on test of Assumptions. However, the very small number of questions per subskill and the fact that many of the skills overlap, make it difficult to accurately assess this.

12.6 What are barriers/challenges to the successful implementation of CT strategies in the classroom?

The systematic review and the trial revealed a number of potential challenges and barriers to successful implementation of CT strategies. These can be summarised as follows:

Short exposure to CT instruction

The short duration of the intervention is cited in a number of studies reviewed as a barrier to successful implementation. The experimental teachers that were interviewed in the trial also raised this same issue. As CT comprises a set of complex skills, which are often not familiar to EFL/ESL learners, constant reinforcement and application of those skills is needed. Therefore, we suggest that evaluations of CT skills approaches

Student attitude

The process evaluation in the trial revealed that some students were resistant to the idea of learning CT as they did not perceive it as relevant to their course. This potentially could be a barrier to successful implementation of the intervention.

Students’ background knowledge

Feedback from teachers suggests that teaching CT can be challenging because of students’ lack of general knowledge, their low level of English proficiency and their dislike for reading. A lot of prior work may be needed to prepare students to receive CT instruction.

Appropriate and interesting teaching materials

Students and teachers fed back about the need to present the teaching materials in a wider range of media to suit students’ learning styles. Suggestions included video clips, audio recordings and visual images. Some of the examples used in the trial were also found to be above the level of the students. These could affect students’ interest and their responsiveness to the intervention.

12.7 Factors that facilitate the teaching of critical thinking

The findings of this study highlight some factors that may be necessary for effective delivery of CT instruction in a second language classroom.

Training and support of teachers

Training of teachers is essential for effective delivery of CT instruction. Teachers do not necessarily have the CT skills themselves. Asking teachers to teach CT without training may create resentment and resistance. In this study, the researcher modelled the teaching strategies and provided regular support by making herself available for consultations. This has helped allay any apprehension that teachers may have.

Availability of teaching resources

The provision of ready-made teaching materials and the structured module used in this study also facilitated the delivery of the lessons. For effective delivery of the intervention, such teaching resources are very useful. It saves teachers time in preparing for the lesson and to source for relevant and appropriate materials.

Longer duration of instruction

For effective implementation it was suggested that CT instruction should be taught over more than a term to allow for the concepts to be reinforced and time to practise them.

Integration into the regular curriculum

Another factor that could facilitate implementation is the integration of CT within the regular curriculum so that teachers and students see its relevance within the course. Teachers are also more likely to be receptive to the idea if it is seen as part of the curriculum and not something additional that they have to do.

Teachers’ attitude

The attitude of the teachers towards CT is another factor that needs to be considered. If the teachers do not believe in the usefulness of CT or the need to teach CT, it can jeopardise the successful implementation of the intervention.

12.8 Implications for future research

The trial in this study was a small-scale feasibility study. Moving from this small study a larger and well-funded study could be commissioned to evaluate the generic explicit approach to CT more widely to include a range of higher education institutions and faculties. This study could be extended to other Arab countries to test if it also works in other similar jurisdictions. The small sample size in this current trial did not allow for subgroup analysis. Perhaps future larger-scale studies could evaluate the impact of the approach on different subgroups.

Because of the short duration of the study it was not possible to test the long-term retention of CT skills and the transfer of skills to other areas. A longer trial could be conducted to determine the long-term effects on CT instruction, and whether the skills learnt could be transferred to other contexts or subjects.

12.9 Recommendations for policy and practice

This study indicates that it is feasible to teach CT in Lebanon and that instruction in CT even to students who have had little prior exposure to such concepts can help develop their critical ability. Just one term of instruction can bring about some changes. Therefore, if the government is serious about fostering CT among its citizens, conscious

can be easily and cheaply integrated into the regular curriculum. There is no need for new textbooks or expensive teaching resources. The course resources are relatively low cost (cost of photocopying) and can be easily produced. Everyday resources like newspaper articles and internet materials can be easily adapted to teach CT. The course materials may be updated periodically with more current materials.

The findings in this thesis also have implications for initial teacher training and teachers’ professional training. Preservice teacher training on the integration of CT in the curriculum and continual professional development workshops may be considered as part of the reform to prepare teachers for the new curriculum.

For the reform in Lebanon to achieve its objectives, education policy therefore should be free from the dominance of political parties and religious sects, which are currently key hindrances to progress in education. CT should not be left to an independent initiative advanced only by some teachers. It should be a national initiative mandated by the government. Different CT programmes could be piloted and evaluated as part of the reform. The assessment system also needs to be revamped to test higher order thinking skills rather than simple recall of information. Only if and when such changes take place, no real reform can happen.

In the long-term education reform in Lebanon should also consider improving the quality of research and research skills of its researchers. Training researchers and teachers to judge the quality of research and to read research reports critically is useful. Such skills can be cascaded to training of students in schools and colleges. Similar initiatives like the Q-step in the UK that offers a range of training modules, programmes of study, seminars, studentships, and conferences could be introduced.

12.10 Recommendations for improving research in this field

The systematic review revealed that current empirical research relating to CT in the English language classroom in higher education is rather immature and particularly weak. Similar weaknesses were also highlighted in other systematic reviews (Ten Dam & Volman, 2004; Torgerson at al., 2006; Behar-Horenstein & Niu, 2011).

intervention-related measures. Where standardised test instruments were used they tended to be modified by the researchers, for example, translating the instrument to the language of the students rather than the language of instruction or selecting certain items.

A common problem encountered in this review is lack of adequate reporting in most studies making it difficult for the reader to judge the credibility and trustworthiness of the evidence. For example, not enough detail was provided as to how groups were identified, if and how they were randomised, what the intervention consisted of, how fidelity to treatment was ensured, or how diffusion of treatment was countered. Almost all did not report attrition.

Given the large number of small-scale studies in the review, often carried out by researchers themselves involving their own students, what is now needed for clearer evidence is well-designed, large-scale, independently evaluated randomised controlled trials using standardised tests of CT in the language of instruction.

Therefore, to improve future research in this field the following recommendations are suggested:

• Well-designed, large-scale evaluations, should be conducted ideally by independent evaluators.

• Studies of less evaluated approaches should be replicated.

• Assessments should be conducted by independent assessors who are blind to treatment allocation.

• The licensed version of the test instrument in the language of instruction should be used to avoid the problem of language transference. This also minimises the possibility of researchers teaching to the test if an adapted or modified version is used.

• Where approaches involve the use of unconventional strategies such as computer software or video recording (as in the concept map approach), an alternative innovative treatment could be used to ensure that any impact is not due to the novelty effect.

1994; Trafimow & Rice, 2009; Colquoun, 2014, 2016; Perezgonzalez, 2015; Gorard, 2016). The irony is that teachers/researchers of CT themselves fall for the common fallacies of significance tests. Instead, calculation of effect size is recommended. Data analyses should include basic information like the mean pre- test scores and the mean post-test scores of the two groups being compared as well as the standard deviation.

• Research reports should include any missing data, missing values and attrition. Missing specific variables or missing data in a dataset, as well as attrition, should be handled with caution as they could affect the validity and generalizability of findings. Gorard (2013b) warns that missing data cannot be considered random as participants who drop out from a study or whose data are missing could be different from those who comply. Therefore, attrition or missing data should be clearly reported.

• Where there is missing data, attrition or non-compliance, both intention-to-treat and compliance average causal effect analysis are recommended.

• Process evaluations should form part of the evaluation especially in complex interventions so that if the programme works we can identify the mechanism that brings about change, or factors that are necessary for successful implementation. And if the programme fails, process evaluation is useful in identifying those factors that may have hindered effective implementation.

• Clear, complete and transparent reporting is necessary if research in this field is to advance.

References

Abouchedid, K., R. Nasser, & J. Van Blommestein. (2002). The limitations of inter- group learning in confessional school systems: The case of Lebanon. Arab

Studies Quarterly, 24(4), 61–82.

Abrami, P. C., Bernard, R. M., Borokhovski, E., Wade, A., Surkes, M. A., Tamim, R., & Zhang, D. (2008). Instructional interventions affecting critical thinking skills and dispositions: A stage 1 meta-analysis. Review of Educational Research,

78(4), 1102-1134.

Abrami, P. C., Bernard, R. M., Borokhovski, E., Waddington, D. I., Wade, C. A., & Persson, T. (2015). Strategies for teaching students to think critically a meta- analysis. Review of Educational Research, 85(2), 275–314. DOI: 10.3102/0034654314551063

Akar, B., Shuayb, M., & Hamadeh, N. (2016). Towards a disciplinary approach to History Education: The experience of the Lebanese Association for History (LAH). The Peace Building in Lebanon.

Anaya, G. (1999). College impact on student learning: Comparing the use of self- reported gains, standardised and college grades. Research in Higher Education,

40(5), 499-526. https://doi.org/10.1023/A:1018744326915.

Andrews, R. (2007). Argumentation, critical thinking and the postgraduate dissertation.

Educational Review, 59(1), 1-18. DOI: 10.1080/00131910600796777

Andrews, R. (2009). Argumentation in Higher Education: Improving Practice Through

Theory and Research. New York, NY: Routledge. ISBN 978-0415995016

Andrews, R. (2015). Critical thinking and/or argumentation in higher education. In M. Davies & R. Barnett (Eds.), The Palgrave handbook of critical thinking in

higher education (pp. 49-62). New York, NY: Palgrave MacMillan

Andrews, R., Bilbro, R, Mitchell, S., Peake, K., Prior, P., Robinson, A., See, B.H. and Torgerson, C. (2006). Argumentative skills in first year undergraduates: A pilot study. York: The Higher Education Academy.

Arab Human Development Report. (2016). Youth and the prospects for human

development in a changing reality. Retrieved from http://www.arab-

hdr.org/Reports/2016/ 2016.aspx

Armitage, P. (1979). The analysis of data from clinical trials. The Statistician, 28, 171- 183.

Association of American Colleges and Universities. (2015). Falling Short? College

Learning and Career Success. Washington, D.C. Retrieved from http://www.aacu.org/sites/default/files/files/LEAP/2015employerstudentsurvey. pdf

Atkinson, D. (1997). A critical approach to critical thinking in TESOL. TESOL

Quarterly, 31(1), 71–94. DOI: 10.2307/3587975

Bailin, S., Case, R., Coombs, J. R., & Daniels, L. B. (1999). Common misconceptions of critical thinking. Journal of Curriculum Studies, 31(3), 269-283.

Bailin, S., & Siegel, H. (2003). Critical thinking. In Blake, N., Smeyers, P., Smith, R. & Standish, P. (eds.). The Blackwell guide to the philosophy of education. Oxford, UK:

Blackwell, (pp. 181–193).

Bashshur, M. (2005). History teaching and history textbooks in Lebanon. Paper presented at the workshop Learning about the Other and Teaching for Tolerance in Muslim Majority Societies. 10-12 November Istanbul, Turkey.

Baytiyeh, H. (2017). Has the educational system in Lebanon contributed to the growing sectarian divisions?. Education and Urban Society, 49(5), 546-559.

Behar-Horenstein, L. S., & Niu, L. (2011). Teaching critical thinking skills in higher education: A review of the literature. Journal of College Teaching & Learning

(TLC), 8(2), 25-42.

Benesch, S. (1999). Thinking critically, thinking dialogically. TESOL Quarterly, 33(3), 573-580. Retrieved from JSTOR Database.

Berrill, D. (2000). In Mitchell, S. and Andrews, R. (ed.) Learning to argue in

highereducation. Hull: Heineman/Boynton-Cook.

Blackmore, J. (2001). Universities in crisis? Knowledge economies, emancipatory pedagogies, and the critical intellectual. Educational Theory, 51(3), 353-370. Boujaude, S. (2003). Scientific inquiry in the Lebanese science curriculum. In the

seminar onInquiry in Science Education: International Perspectives (sponsored by ICASE and the NARST International Committee). Presented at the annual conference of the National Association for Research in Science teaching, St. Louis, MI, March 25 – 28, 2001.

BouJaoude, S. (2004). Scientific inquiry in the Lebanese curriculum. In E. Krugly- Smolska and P. C. Taylor (Eds.). Inquiry in Science Education: International