Chapter 6 – Summary and Recommendations
6.6 Concluding Remarks
This study set out to gain a better understanding of students’ conception of the use of a student response system to provide formative feedback a large university class. The approach taken was to deliver the feedback using a proven theoretical framework, and then to gather the experiences of the students in order to understand their perception of the experience.
The results indicate that feedback is important to both teachers and students. Feedback creates connections by helping students understand more about learning through assisting them to focus on their learning goals. It helps students to know where they are and where they are going, and prompts them to think about how to close the gap.
University of Cape Town
The key to delivering effective feedback is to ensure that the pedagogy is emphasised above the technology, and not the other way around. It is important to first understand the needs of the teachers and learners, and to then determine what the technology can do to support the learning environment.
After all, “technology works best when it has to meet a challenge, rather than being merely a solution looking for a problem” (Laurillard, 2008, p. 14).
University of Cape Town
References
Akerlind, G. S. (2005). Variation and commonality in phenomenographic research methods. Higher Education Research and Development, 24(4), 321–334.
doi:10.1080/07294360500284672
Allin, L., & Fishwick, L. (2009). Engaging sport students in assessment and formative feedback. Hospitality, Leisure, Sport and Tourism Network, June, 1–29.
Ashworth, P., & Lucas, U. (1998). What is the “world” of phenomenography? Scandinavian Journal of Educational Research, 42(4), 415–431. doi:10.1080/0031383980420407 Banks, D. A., & Werner, M. (2008). Audience response systems in higher education:
Applications and cases. Educase Quarterly, 2, 83–84.
Beatty, I. D. (2004). Transforming student learning with classroom communication systems. EDUCAUSE Research Bulletin, 2004(3), 1–13.
Beatty, I. D. (2007). What is TEFA? (pp. 1–2). Massachusetts.
Beatty, I. D., & Gerace, W. J. (2009). Technology-enhanced formative assessment: a research- based pedagogy for teaching science with classroom response technology. Journal of Science Education and Technology, 18(2), 146–162. doi:10.1007/s10956-008-9140-4 Beatty, I. D., Gerace, W. J., Leonard, W. J., & Dufresne, R. J. (2006). Designing Effective Questions for Classroom Response System Teaching. American Association of Physics Teachers, 74(1), 31–39. doi:10.1119/1.2121753
Berge, Z., & Collins, M. (2005). Providing effective feedback online. In ICDE International conference (pp. 1–10). New Delhi.
Beutel, D. A. (2010). The nature of pedagogic teacher-student interactions: A phenomenographic study. Australian Educational Researcher, 37(2), 77–91.
Black, P., & Wiliam, D. (1998). Inside the Black Box: Raising Standards Through Classroom Assessment. Assessment (pp. 1–22). London: GL Assessment.
Boud, D., & Molloy, E. (2012). Rethinking models of feedback for learning: the challenge of design. Assessment & Evaluation in Higher Education, 1–15.
doi:10.1080/02602938.2012.691462
Bruff, D. (2009). Multiple-choice questions you wouldn’t put on a test: Promoting deep learning using clickers. The Professional & Organizational Development Network in Higher Education, 21(3).
Caldwell, J. E. (2007). Clickers in the large classroom: Current research and best-practice tips. CBE - Life Sciences Education, 6(1), 9–20.
University of Cape Town
Carnaghan, C., & Webb, A. (2007). Investigating the effects of group response systems on student satisfaction, learning, and engagement in accounting education. Issues in Accounting Education, 22(3), 391–409. doi:10.2308/iace.2007.22.3.391
Chickering, A. W., & Ehrmann, S. C. (1996). Implementing the Seven Principles: Technology as a Lever. AAHE Bulletin, October, 3–6.
Cohen, L., Manion, L., & Morrison, K. (2007). Research Methods in Education (6th ed., pp. 1– 633). Oxon: Routledge.
Collins, L. J. (2007). Livening up the classroom: Using audience response systems to promote active learning. Medical Reference Services Quarterly, 26(1), 81–89.
doi:10.1300/J115v26n01
Covey, S. R. (2004). The 8th Habit (pp. 1–409). London: Simon & Schuster.
Crouch, C. H., & Mazur, E. (2001). Peer instruction: Ten years of experience and results. Am. J. Phys., 69(9), 970–977. doi:10.1119/1.1374249
Cutts, Q. I., Carbone, A., & Van Haaster, K. (2004). Using an electronic voting system to promote active reflection on coursework feedback. In International Conference on Computers in Education (pp. 1–7).
Cutts, Q. I., & Kennedy, G. E. (2005). Connecting learning environments using electronic voting systems. In Australasian Computing Education Conference (Vol. 42, pp. 1–6). Newcastle, Australia.
Deal, A. (2007). Classroom response systems. Teaching With Technology, 1–14.
Draper, S. W., & Brown, M. I. (2004). Increasing interactivity in lectures using an electronic voting system. Journal of Computer Assisted Learning, 20, 81–94.
Duncan, D. (2004). Clickers in the Classroom: How to Enhance Science Teaching Using Classroom Response Systems (pp. 1–72). San Francisco: Benjamin Cummings.
Duncan, D. (2006). Clickers: A new teaching aid with exceptional promise. The Astronomy Education Review, 5(1), 1–19.
Fies, C., & Marshall, J. (2006). Classroom response systems: A review of the literature. Journal of Science Education and Technology, 15(1), 101–109. doi:10.1007/sl0956-006- 0360-l
Freeman, M., & Blayney, P. (2004). Promoting interactive in-class learning environments: A comparison of an electronic response system with a traditional alternative. In 11th Australasian Teaching Economics Conference (pp. 23–34).
Gebru, M. T., Phelps, A. J., & Wulfsberg, G. (2012). Effect of clickers versus online homework on students’ long-term retention of general chemistry course material. Chemistry Education Research and Practice, 13, 325–329. doi:10.1039/c2rp20033c Hancock, T. M. (2010). Use of audience response systems for summative assessment in large
University of Cape Town
Harlen, W. (2003). Enhancing Inquiry Through Formative Assessment. (J. Brand & R. Brown, Eds.) (pp. 13–44). Exploratorium.
Harris, L. R. (2008). A phenomenographic investigation of teacher conceptions of student engagement in learning. The Australian Educational Researcher, 35(1), 57–79. doi:10.1007/BF03216875
Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81–112. doi:10.3102/003465430298487
Heinrich, E., Milne, J., & Moore, M. (2009). An investigation into e-tool use for formative assignment assessment - status and recommendations. Educational Technology & Society, 12(4), 176–192.
Hoekstra, A. R. (2008). Vibrant student voices: exploring the effects of the use of clickers in large college courses. Learning, Media and Technology, 33(4), 329–341.
doi:10.1080/17439880802497081
Hoekstra, A. R. (2009). A Socio-Cultural Analysis of The Use of Clickers in Higher Education (pp. 1–24).
Hu, J., Bertol, P., Hamilton, M., White, G., & Duff, A. (2006). Wireless interactive teaching by using keypad-based ARS. In D A Banks (Ed.), Audience Response Systems in Higher Education (pp. 209–221). Hershey, PA: Information Science Publishing.
Immerwahr, J. (2009). Engaging the “thumb generation” with clickers. Teaching Philosophy, 32(3), 233–245.
Johnson, T., & Meckelborg, A. (2009). Student response systems: A cure for lecturalgia? Revista De Informática Aplicada, V(01), 51–60.
Judson, E., & Sawada, D. (2002). Learning from past and present: Electronic response systems in college lecture halls. Journal of Computers in Mathematics and Science Teaching, 21(2), 167–181.
Kay, R. H., & LeSage, A. (2009a). A strategic assessment of audience response systems used in higher education. Australasian Journal of Educational Technology, 25(2), 235–249. Kay, R. H., & LeSage, A. (2009b). Examining the benefits and challenges of using audience
response systems: A review of the literature. Computers & Education, 53(3), 819–827. doi:10.1016/j.compedu.2009.05.001
Kift, S. M., & Moody, K. E. (2009). Harnessing assessment and feedback in the first year to support learning success, engagement and retention. In ATN Assessment Conference 2009 Proceedings (pp. 1–11). Melbourne, Australia.
King, S. O., & Robinson, C. L. (2009a). “Pretty lights” and maths! Increasing student engagement and enhancing learning through the use of electronic voting systems. Computers & Education, 53(1), 189–199. doi:10.1016/j.compedu.2009.01.012 King, S. O., & Robinson, C. L. (2009b). Staff perspectives on the use of technology for
University of Cape Town
King, S. O., & Robinson, C. L. (2009c). Formative teaching: A conversational framework for evaluating the impact of response technology on student experience, engagement and achievement. In ASEE/IEEE Frontiers in Education Conference (pp. 1–6). San Antonio, TX.
Landis, C. R., Ellis, A. B., Lisensky, G. C., Lorenz, J. K., & Meeker, K. (2001). Chemistry ConcepTests: A Pathway to Interactive Classrooms. San Francisco: Prentice-Hall. Lantz, M. E. (2010). The use of “clickers” in the classroom: teaching innovation or merely an
amusing novelty? Computers in Human Behavior, 26(4), 556–561. doi:10.1016/j.chb.2010.02.014
Lasry, N. (2008). Clickers or flashcards: Is there really a difference? The Physics Teacher, 46(April), 242–244. doi:10.1119/1.2895678
Laurillard, D. (1998). How can interactive multimedia enhance learning? In IV Congresso RIBIE (pp. 1–13). Brasilia.
Laurillard, D. (1999). A conversational framework for individual learning applied to the “learning organisation” and the “learning society.” Systems Research and Behavioral Science, 16(2), 113–122.
Laurillard, D. (2002). Rethinking University Teaching (pp. 1–268). London: RoutledgeFalmer. Laurillard, D. (2008). The teacher as action researcher: Using technology to capture pedagogic
form. Studies in Higher Education, 33(2), 139–154. doi:10.1080/03075070801915908 Laurillard, D. (2009). The pedagogical challenges to collaborative technologies. International
Journal of Computer-Supported Collaborative Learning, 4(1), 5–20. doi:10.1007/s11412- 008-9056-2
Laxman, K. (2011). A study on the adoption of clickers in higher education. Australasian Journal of Educational Technology, 27(8), 1291–1303.
Lilley, M., & Barker, T. (2007). Students’ perceived usefulness of formative feedback for a computer-adaptive test. The Electronic Journal of e-Learning, 5(1), 31–38.
MacArthur, J. R., & Jones, L. L. (2008). A review of literature reports of clickers applicable to college chemistry classrooms. Chemistry Education Research and Practice, 9(3), 187– 195.
Maietta, R. C. (2009). The life of an ATLAS.ti quotation (pp. 1–6). Bohemia, NY.
Marton, F. (1981). Phenomenography - Describing Conceptions of the World Around Us. Instructional Science, 10, 177–200.
Marton, F. (1986). Phenomenography - a research approach to investigating different understandings of reality. Journal of Thought, 28–49.
Marton, F. (1988). Phenomenography: Exploring different conceptions of reality. In D. Fetterman (Ed.), Qualitative Approaches to Evaluation in Education: The Silent
University of Cape Town
Marton, F., & Booth, S. (1996). The learners’ experience of learning. In D. R. Olson & N. Torrance (Eds.), The Handbook of Education and Human Development (pp. 534–564). Oxford: Blackwell Publishers.
Marton, F., & Pang, M. F. (1999). Two faces of variation. In 8th European Conference for Learning and Instruction. Goteborg, Sweden.
Mayer, R. E., Stull, A., Deleeuw, K., & Almeroth, K. (2009). Clickers in college classrooms: Fostering learning with questioning methods in large lecture classes. Contemporary Educational Psychology, 34(1), 51–57.
Mazur, E. (1997). Peer Instruction: A User’s Manual. Upper Saddle River: Prentice Hall. Mazur, E., & Watkins, J. (2007). Just-in-time teaching and peer instruction. In Just-In-Time
Teaching: Across the Disciplines, Across the Academy (pp. 39–62).
Menon, A. S., Moffett, S., Enriquez, M., Martinez, M., & Dev, P. (2004). Audience response made easy: using personal digital assistants as a classroom polling tool. Journal of the American Medical Informatics Association, 11(3), 217–220.
Merriam, S. B. (1998). Qualitative research and case study applications in education. San Francisco, CA: Jossey-Bass Publishers.
Mollborn, S., & Hoekstra, A. R. (2010). “A meeting of minds”: Using clickers for critical thinking and discussion in large sociology classes. Teaching Sociology, 38(1), 18–27. Moreau, N. A. (2009). Do clickers open minds? Use of a questioning strategy in developmental
mathematics. Capella University.
Mula, J. M., & Kavanagh, M. (2009). Click go the students, click-click-click: The efficacy of a student response system for engaging students to improve feedback and performance. e- Journal of Business Education & Scholarship of Teaching, 3(1), 1–17.
Nicolle, P. S., & Lou, Y. (2008). Technology adoption into teaching and learning by
mainstream university faculty: A mixed methodology study revealing the “how, when, why, and why not.” Journal of Educational Computing Research, 39(3), 235–265. Pargas, R. P. (2005). Using MessageGrid to promote student collaboration. In Cognition and
Exploratory Learning in the Digital Age (pp. 1–9).
Patton, M. (1990). Designing Qualitative Studies. In Qualitative Evaluation and Research Methods (pp. 169–186). Beverly Hills, CA: Sage.
Penuel, W. R., Yarnall, M. K., & Roschelle, J. (2004). Meeting teachers in the middle:
Designing handheld computer-supported activities to improve student questioning. In 6th International Conference on Learning Sciences (pp. 404–411).
Poirier, C. R., & Feldman, R. S. (2007). Promoting active learning using individual response technology in large psychology classes. Teaching of Psychology, 34, 194–196.
University of Cape Town
Sandberg, J. (1997). Are phenomenographic results reliable? Higher Education Research & Development, 16(2), 203–212. doi:10.1080/0729436970160207
Sanseverino, M. (2010). Pedagogy That clicks: “Clickers” in the CSC classroom. In WCCCE’10 (pp. 1–5). Kelowna, Canada.
Scott, G. (2005). Using CEQuery to identify what retains students and promotes engagement in productive learning in Australian higher education (pp. 1–116).
Shapiro, J. A. (1997). Electronic student response found feasible in large science lecture hall. Journal of College Science Teaching, 26(6), 408–412.
Shute, V. J. (2008). Focus on formative feedback. Review of Educational Research, 78, 153– 189. doi:10.3102/0034654307313795
Sjöström, B., & Dahlgren, L. O. (2002). Applying phenomenography in nursing research. Journal of Advanced Nursing, 40(3), 339–345.
Søndergaard, H., & Thomas, D. (2004). Effective feedback to small and large classes. In 34th ASEE/IEEE Frontiers in Education Conference (pp. 9–14).
Taras, M. (2005). Assessment - summative and formative - some theoretical reflections. British Journal of Educational Studies, 53(4), 466–478. doi:10.1111/j.1467-8527.2005.00307.x Taylor, J. A. (2008). Assessment in first year university: A model to manage transition.
Journal of University Teaching & Learning Practice, 5(1), 19–53.
Trees, A. R., & Jackson, M. H. (2007). The learning environment in clicker classrooms: Student processes of learning and involvement in large university-level courses using student response systems. Learning, Media and Technology, 32(1), 21–40.
Yorke, M. (2003). Formative assessment in higher education: Moves towards theory and the enhancement of pedagogic practice. Higher Education, 45(4), 477–501.
Zimmerman, B. J. (2000). Attaining self-regulation: a social cognitive perspective. In M. Boekaerts & P. R. Pintrich (Eds.), Handbook of Self-Regulation (pp. 13–39). San Diego, CA: Academic Press.
University of Cape Town
Appendices
Appendix A – Clicker Lesson Plan Template
Activity Description Time Allocation
Create Set up the computer and hand out the clickers 15 minutes
Open Present an introduction to the lesson 5 minutes
Explore Lecture and 1st clicker session 15 minutes
Explore Lecture and 2nd clicker session 15 minutes
Close Present a brief summary of the lesson 5 minutes
Collect Collect the clickers as the students leave class 5 minutes Connect Connect with students who want individual attention 5 minutes
End Save the session and shut down the computer 5 minutes
Time Activity Description
10:50 Create Set up the computer and start the TurningPoint software 10:55 Create Hand out the clickers to students as they arrive
11:00 Create Students answer a check-in question as latecomers arrive 11:05 Open Present an introduction to the lesson
11:10 Explore Brief lecture – Topic 1 11:15 Explore Brief lecture – Topic 1 11:20 Explore Clicker questions – Topic 1 11:25 Explore Brief lecture – Topic 2 11:30 Explore Brief lecture – Topic 2 11:35 Explore Clicker questions – Topic 2
11:40 Close Present a brief summary of the lesson
11:45 Collect Collect the clickers as the students leave class 11:50 Connect Connect with students who want individual attention 11:55 End Save the TurningPoint session and shut down the computer
University of Cape Town
Appendix B – Interview Request Letter
To the students of INF1002H
As partial fulfilment towards a Masters degree (ICT in Education) I am researching the use of clickers in large classes and am particularly interested in understanding more about how students experience the use of clickers in the classroom.
The title of my research is: Using a student response system to provide formative feedback in
large classes: A phenomenographic study at the University of Cape Town.
In order to achieve my objective, I am wanting to interview students who have used clickers in lectures and who are willing to talk to me about their experience. Was it useful to you? Did you enjoy using them? Did it help you in your studies? Anything that will help me understand your experience. Each interview will be recorded, and the transcript will become part of the research presented to the University. All names of individuals will be anonymised using pseudonyms in the transcribed documents. The interview will take no longer than 30 minutes and can be done individually or with one or two other students if you prefer.
Your participation in this study will remain confidential, and all information gathered about any individual will be treated as strictly confidential. This study is conducted for academic purposes only. If the research findings prove to be useful to the broader community, the results will possibly be presented in a journal or at a conference.
Participation is voluntary, and you may withdraw from the process at any time.
The interviews will be conducted from Friday 3rd June to Friday 10th June 2011 at UCT.
If you are willing to be interviewed, then please let me know by email or by writing your name
on the sheet provided. Please could you also indicate a date and time that suits you. Thank you for your time and cooperation, your participation is greatly appreciated.
Ian Barbour (Researcher)
Mobile: 082 824-2172
Email: [email protected]
Room 3.03.2 Leslie Commerce Building University of Cape Town
Phone: 021 650-4387
Associate Professor Cheryl Hodgkinson-Williams (Supervisor)
Email: [email protected]
Phone: 021 650-5030
Convenor & Lecturer: INF1002H
Ian Barbour
Room 3.03.2
Leslie Commerce Building
Upper Campus, University of Cape Town Tel: +27 (0) 21 650-4387
E-mail: [email protected] Internet: www.uct.ac.za
University of Cape Town
Appendix C – Research Consent Form
University of Cape Town Faculty of Humanities Research Consent Form
Title of research project: Using a student response system to provide formative feedback in large classes: A phenomenographic study at the University of Cape Town.
Name of principal researcher: Ian Barbour
Department address: School of Education, University of Cape Town Private Bag X3, Rondebosch 7701 Republic of South Africa
Telephone: 021 650-2769
Email: [email protected]
Name of participant:
Nature of the research: A phenomenographic qualitative study, using semi-structured interviews to gather the data. Participant’s involvement ~
What’s involved: A 30 minute interview to talk about your experience of using clickers in the INF1002H lectures.
Risks: None
Benefits: None
Costs: None
Payment: None
I agree to participate in this research project.
I have read this consent form and the information it contains and had the opportunity to ask questions about them.
I agree to my responses being used for education and research, on condition that my privacy is respected, subject to the following:
o I understand that my personal details will be used in aggregate form only, so that I will not be personally identifiable.
I understand that I am under no obligation to take part in this project.
I understand I have the right to withdraw from this project at any stage.
Signature of participant: Name of participant: Signature of principal researcher:
Name of principal researcher: Ian Barbour Date:
University of Cape Town
Appendix D – Interview Protocol and Contextual Question
Interview introduction
I am doing a study on clickers as part of my Master’s degree, and I am talking to students that have used them in the classroom. I do not have a set of pre-determined questions to ask you, as the approach I'm taking is having a conversation to talk about your experience of clickers in the classroom. And when I say ‘clickers’, I do not mean only the little hand-held device – I am interested in the whole experience of you being shown a question, using the clicker to having to answer it, and then seeing the results etc. Does that make sense to you?
Some formalities – I'm recording this conversation, are you okay with that?
Afterwards I will get someone to type up what we discuss, and the transcript will then become part of the research. Your participation in this study is confidential, and any names including your own will be replaced with pseudonyms. And at any time you are free to withdraw from the process if you want to.
Are you happy with that? Any concerns?
Okay, let’s start.
Contextual question
When the clickers were first handed out in class, what did you think?
Types of probing questions to ask:
How did you feel? (or What did you like about that?)