• No results found

Lecture Capturing to Transform Student Learning Opportunities in Large Classes

N/A
N/A
Protected

Academic year: 2021

Share "Lecture Capturing to Transform Student Learning Opportunities in Large Classes"

Copied!
13
0
0

Loading.... (view fulltext now)

Full text

(1)

Lecture Capturing to Transform Student Learning Opportunities in Large Classes

Stephanie B. Corliss, Ph.D.

James Henson, Ph.D.

Joel Heikes, Ph.D.

Michael Heidenreich

(2)

Executive Summary

Title: Lecture Capturing to Transform Student Learning Opportunities in Large Classes Institution: The University of Texas at Austin

Abstract: This proposal is designed to examine the impact of traditional uses of lecture capturing in courses of varying sizes and disciplines on students’ learning and course and instructor satisfaction, and to explore innovative instructional uses of lecture capturing to re-design traditional classroom activities in large enrollment lecture courses. Through the proposed research, we strive to identify types of courses that could benefit most from traditional uses of lecture capturing and the student uses of the captures that lead to positive learning outcomes. We also hope to engage faculty in non-traditional uses of lecture capturing to provide time for interactive student-centered learning opportunities in the classroom, with the expectation of improving students’ learning. Results of this research have implications for instructional technology administrators, designers, and users in higher education. To investigate the first research question, additional data analyses will be conducted on existing data from instructors and students who participated in a fall 2009 pilot evaluation of classroom capture using Echo360 installations. If instructors taught the same course the previous year with little to no changes in content covered, book and instructional materials used, and classroom structure, course instructor survey data and common exam item and/or rubric-based assignment scores will be compared across semesters. This will allow for a comparison of student learning and satisfaction with the course and instructor in the same course, one with lecture captures and one without. To investigate the second research question, three to five instructors from different disciplines who teach large enrollment classes (100+ students) will be chosen to redesign an existing “lecture” of their course. Instructors will work with an instructional designer to pre-record an existing lecture to be delivered to students as outside class work and a student-centered application activity/activities to be delivered during the classroom session time when the original lecture would have been delivered. Various methods will be used to gather data about the effectiveness of the activity on students’ learning, satisfaction, the instructors’ practices, and the classroom environment.

(3)

Overview

Description

In the fall of 2009, Liberal Arts Technology Services (LAITS) at the University of Texas at Austin installed the Echo360 lecture capturing system in five campus classrooms. LAITS partnered with the Division of Instructional Assessment and Innovation (DIIA) to evaluate the impact of the Echo360 system on instructors and students. Sixteen instructors teaching a class in an equipped classroom participated in the pilot evaluation. The evaluation assessed the functionality and usability of the system for instructors and students, user satisfaction, instructor expectations of lecture capturing and perceptions of the effects on teaching and learning.

Results revealed that overall student and instructor satisfaction of the Echo360 lecture capturing system was very high. Instructors found the tool easy to use, and few technical problems were reported by either instructors or students. Using the system required minimal time of the instructors and their concerns about attendance, security, and technical problems lessened by the end of the semester. Both instructors and students felt the lecture captures were helpful for student learning. Students used the recordings for note-taking, reviewing, and studying. Although many students also used the recordings to get information they missed when absent from class, lecture captures did not appear to be used as a reason to skip class. Overall, the Echo360 system integrated well with current UT systems and was viewed favorably by faculty and student users (Corliss, Heikes & Heidenreich, 2010).

We want to expand on the results of the pilot evaluation to determine how the use of traditional lecture capturing impacts students’ learning and their overall satisfaction with courses and instructors that offer lecture captures. On self-report measures, students are largely positive about participating in classes with lecture capture options, whether in the form of “rich media” or simply an audio-only podcast, citing the convenience, flexibility, and positive impact on learning as the main reasons to have recorded lectures (Nagel, 2008; Fernandez, Simo, & Sallan, 2009, Copley, 2007.) Students report using lecture captures in a variety of ways: as study and review aids prior to exams (Zupancic & Horz, 2002); to gain a better understanding of confusing lecture material after class (Soong, Chan, Cheers, & Hu, 2006); and as an opportunity to access information when absent from classes (Veeramani & Bradley, 2008). Results of our pilot evaluation align with these findings (Corliss, Heikes & Heidenreich, 2010).

Although student perceptions of lecture capturing are highly favorable, studies that have examined student learning outcomes related to traditional uses of lecture capturing have had mixed results. Brotherton and Abowd (2004) found no statistically significant differences in mid-term and final exams between classes with webcasts available to students and classes without access to webcasts. Harley and colleagues (2003) found that the relationship between the self-reported webcast usage and course grade was not statistically significant, and students who reported using lecture webcasts as a replacement for the in-person lecture had lower scores in the course overall. The proposed study will allow us to determine the effects that traditional uses of lecture capturing have on UT students’ learning and their perceptions of the instructional experience in courses of various sizes and disciplines.

(4)

In addition to its traditional uses, lecture capturing also has the potential to transform the ways instructors utilize in-class time. If captured lectures are viewed prior to class, in-class time can be spent on student-centered activities that allow students to practice the skills and concepts covered in the lectures. In a four-year longitudinal study on DVD-based lectures for self-directed learning, Ambikairajah , Epps, Sheng, & Celler (2008) found that lecture captures increased active learning for engineering students, especially those within large classrooms. They discovered that delivering lectures solely through the use of pre-recorded lecture material allowed in-class time to be spent on focused, critical discussions. Day and Foley (2006) conducted a quasi-experimental study with students studying HCI (human-computer interaction). They found that the final grades of students who used pre-recorded lecture captures were significantly better than those in the traditional lecture classes; additionally, the lecture capture students reported positive attitudes about the use of lecture-capture technology, and about the course in general.

The purpose of this proposal is to expand on the findings from the pilot evaluation to: 1) examine the impact of traditional uses of lecture capturing in courses of varying sizes and disciplines on students’ learning and course and instructor satisfaction; and 2) to explore innovative instructional uses of lecture capturing to re-design traditional classroom activities in large enrollment lecture courses. Through the proposed research, we strive to identify types of courses that could benefit most from traditional uses of lecture capturing and the student uses of the captures that lead to positive learning outcomes. We also hope to engage faculty in non-traditional uses of lecture capturing to provide time for interactive student-centered learning opportunities in the classroom, with the expectation of improving students’ learning. Results of this research have implications for instructional technology administrators, designers, and users in higher education.

Objectives

The objectives for this project are to answer the following research questions:

1. What are the effects of traditional uses of lecture capturing on students’ learning and their satisfaction with the course and the instructor?

2. Can lecture capturing be used innovatively and effectively in large enrollment lecture courses to transform classroom instructional activities and students’ success rates?

Methods

Research Question 1:

To investigate the first research question, additional data analyses will be conducted on existing data from instructors and students who participated in the fall 2009 pilot evaluation. If instructors taught the same course the previous year with little to no changes in content covered, book and instructional materials used, and classroom structure, course instructor survey data and common exam item and/or rubric-based assignment scores will be compared across semesters. This will allow for a comparison of student learning and satisfaction with the course and instructor in the same course, one with lecture captures and one without.

(5)

Research Question 2:

To investigate the second research question, three to five instructors from different disciplines who teach large enrollment classes (100+ students) will be chosen to redesign an existing “lecture” of their course. Instructors will work with an instructional designer to pre-record an existing lecture to be delivered to students as outside class work and a student-centered application activity/activities to be delivered during the classroom session time when the original lecture would have been delivered. Various methods will be used to gather data about the effectiveness of the activity on students’ learning, satisfaction, the

instructors’ practices, and the classroom environment.

Table 1 summarizes the multiple methods that will be used to answer the two research questions. A description of each component follows the table.

(6)

Table 1: Research Questions and Data Collection Methods

Research Questions 1 and 2

Course Instructor Surveys (CIS): CISs contain questions related to overall satisfaction of a course

and the instructor, and direct questions related to instruction, workload, and classroom assessment. The surveys are administered at the end of each semester. CIS scores will be compared across semesters.

Student Course Add/Drop Rates: Student drop rates will be compared across semesters.

Exam Item/Assignment Scores: Common exam items and/or rubric-based assignments will be

identified and scores will be compared across semesters.

Research Questions Course Instructor Survey Student Course Add/Drop Rates Exam Item/ Rubric-based Assignment Scores Instructor Pre-Intervention

Interview Observation Classroom

Student Post-Intervention Survey Instructor Post-Intervention Survey Instructor Post-Intervention Consult What are the

effects of traditional uses of lecture capturing on students’ learning and their satisfaction with the course and the instructor? X X X Can lecture capturing be used innovatively and effectively in large enrollment lecture courses to transform classroom instructional activities and students’ success rates? X X X X X X X X

(7)

Research Question 2

Intervention: The intervention will consist of redesigning one traditional classroom lecture session. Instructors will capture the traditional lecture normally give during class time, using either the Echo360 classroom capture system or the personal lecture capture tool. Students will be assigned to view the lecture capture outside of class and to come to the next classroom session prepared to practice applying the skills and concepts from the lecture during classroom activities. Individual faculty members will work collaboratively with instructional designers from the research team to design the student-centered classroom activities. Faculty members will be provided with a small stipend ($400-600 each, depending on final number of participants) to participate.

Instructor Pre-intervention Interviews: Faculty participants will be interviewed about their

expectations and goals for using lecture capturing in their course, as well as their anticipated impact on teaching and students’ learning. Interviews will take place at the beginning of the fall 2010 semester.

Classroom Observations: A checklist/rating form evaluating perceived student engagement,

teacher-centered approach, and student-teacher-centered approach will be used during classroom observations. Researchers will observe 1 or 2 classroom sessions before the intervention to gather baseline scores. Researchers will observe again during the intervention classroom session, where the student-centered activities replace the traditional lecture format. Observation data will be compared across classroom sessions.

Student Post-intervention Survey: Students will complete a survey to document their perceptions of

the lecture capture assignment and the classroom intervention activities. Students will be asked to comment on how lecture capturing affected their learning, the instructors’ teaching, and the classroom environment. The survey will be administered at the end of the intervention classroom session.

Instructor Post-intervention Survey: Faculty participants will complete an online survey to

document their perceptions about creating the lecture capture, designing the student-centered classroom activities, and the effects both had on teaching, the classroom environment, and students’ learning. The survey will be administered the week following the intervention classroom session.

Instructor Post-intervention Consult: Faculty participants will be interviewed to gather more

in-depth information about their experiences using lecture capturing. Student learning data and CIS data will be reviewed prior to the interview, and instructors will comment on their perceptions on how lecture capturing impacted their teaching, the classroom environment, and students’ learning. Initial expectations and goals will be reviewed to determine if expectations were met. Plans for future uses of lecture capturing will be discussed. Interviews will take place in January 2011.

Dissemination

Results of this research will be disseminated to both the University of Texas at Austin community and the national learning technology community in the following ways:

(8)

The University of Texas community:

• The lecture capturing system will be demonstrated, best practices discussed, and the results of this

research presented to faculty at the Instructional Technology Showcase hosted by DIIA in the spring of 2011 and the Faculty Instructional Technology Academy hosted by LAITS during the summer of 2011.

• Results of the research, along with student and faculty comments and/or case studies will be

presented on the LAITS website.

• The final project report will be downloadable from the DIIA website.

The national learning technology community:

• Researchers will submit a proposal to share findings at that the EDUCAUSE Annual Conference

to take place October 12-21, 2011. EDUCAUSE is a nonprofit association whose mission is to advance higher education by promoting the intelligent use of information technology.

• Researchers will share findings in 1 to 2 articles submitted for publication to national higher

education and/or instructional technology journals in the summer of 2011.

References

Ambikairajah, E., Epps, J., Sheng, M., & Celler, B. (2008). A New Mode of Lecturing for Self- Directed Learning—Virtual Classroom on a DVD. AIP Conference Proceedings, 1007(1), 152-161.

Brotherton, J. A., & Abowd, G. D. (2004). Lessons learned from eClass: Assessing automated capture and access in the classroom. Transactions on Computer-Human Interaction 11(2), 121-155.

Copley, J. (2007). Audio and video podcasts of lectures for campus-based students: Production and evaluation of student use. Innovations in Education and Teaching International, 44(4), 387-399.

Corliss, S. B., Heikes, J. & Heidenreich (2010). The University of Texas at AustinEcho360 Pilot Fall 2009 Results: Evaluation Report.

Day, J., & Foley, J. (2006, April). Evaluating Web Lectures: A Case Study from HCI. Paper

presented at the Conference on Human Factors in Computing Systems, Montreal, Canada. Fernandez, V., Simo, P., & Sallan, J. M. (2009). Podcasting: A new technological tool to facilitate good

practice in higher education. Computers & Education, 53, 385- 392.

Harley, D., Henke, J., Lawrence, S., McMartin, F., Maher, M., Gawlik, M., et al., (2003). Costs,

culture, and complexity: An analysis of technology enhancements in a large lecture course at UC Berkeley" Retrieved June 2, 2010, from University of California, Berkeley, Center for Studies in Higher Education Web site: http://cshe.berkeley.edu/publications/publications.php?id=36. Nagel, D. (2008, September). Lecture capture: No longer optional? Campus Technology. Retrieved on

(9)

Veeramani, R., & Bradley, S. (2008). U-W Madison online learning study: Insights regarding

undergraduate preference for lecture capture. Retrieved from http://www.uwebi.org/news/uw-online-learning.pdf

Zupancic, B., & Horz, H. (2002, June). Lecture Recording and Its Use in a Traditional University Course. Paper presented at the Annual Joint Conference Integrating Technology into Computer Science Education, Aarhus, Denmark.

(10)

Appendix A: Project Team

STEPHANIE CORLISS is the Instructional Technology Evaluation Specialist with the Division of Instructional Innovation and Assessment (DIIA). She consults with faculty, staff, and administration in the areas of instructional technology evaluation, study design, sampling, data collection, and statistical analyses. Dr. Corliss has been researching the use of instructional technology to enhance the teaching and learning experience for over nine years. She previously worked at the University of California, Berkeley, where she led professional development, research, and assessment efforts with teachers integrating technology-enhanced instruction into their curriculum. Additionally, Dr. Corliss has taught educational psychology and instructional technology courses at the undergraduate and graduate level. She has a Ph.D. (2005) in educational psychology focused on learning, cognition, and instruction, a master’s degree in educational psychology focused on program evaluation, and a bachelor’s degree (1998) in psychology from The University of Texas at Austin.

MICHAEL HEIDENREICH is the lead audio engineer for the College of Liberal Arts at the University of Texas at Austin. He holds a B.S. degree in Radio-Television-Film from the University of Texas. He has been working as an audio engineer for 15 years. He specializes in the delivery of new and remastered archival audio via online means. He has worked on numerous online projects including a reading of the original script of EL Cantar de Mio Cio (http://www.laits.utexas.edu/cid/), The Onda Latina Collection

(http://www.laits.utexas.edu/onda_latina) and Liberal Arts ITS Audio Catalog

(http://www.laits.utexas.edu/itsaud/), and he has provided archival services to the Dolph Briscoe Center

for American History, the Harry Ransom Center and the Blanton Museum of Art. He also digitized and remasterd hundreds of field recordings for Dr. Megan Biesele’s National Science Foundation grant Documenting the Khomani and Ju/hoan Languages. He severs as the College of Liberal Arts

Instructional Technology Services project manager for the pilot program using Echo360 lecture capture appliances.

JOEL HEIKES is the DIIA Instructional Program Evaluation Coordinator specializing in program evaluation, instructional technology evaluation, evaluation design, grant evaluation, and qualitative methods and analysis. He regularly consults with faculty and staff in the areas of evaluation planning, instructional assessment, survey construction, focus group facilitation, and qualitative data analysis. Dr. Heikes has ten years of experience conducting large-scale program evaluations and was previously an Assistant Professor of Sociology with eleven years of university-level teaching experience. He holds Ph.D. (1993) and master’s (1989) degrees in sociology from The University of Texas at Austin and a bachelor’s degree (1986) in business administration from Drake University.

(11)

JAMES HENSON is the Associate Director of the College of Liberal Arts Instructional Technology Services at The University of Texas at Austin, and produces instructional media projects for the largest college at the largest public university in the United States. He also directs the Texas Politics Project (http://texaspolitics.laits.utexas.edu) and teaches in the Department of Government. The Texas Politics Project produces an open access multimedia education resource that combines an online textbook with original high quality video and multimedia content, and produces original public opinion research and public events focused on politics and policy in Texas. Henson helped design public interest multimedia for the Benton Foundation in Washington D.C. in the late 1990’s, and his academic writing about American and international politics has appeared in The Harvard International Journal of Press/Politics, Theory & Event, and elsewhere. In 2008, with Daron Shaw and colleagues in the Government

Department, he collaborated in establishing the only statewide publicly available public opinion poll for Texas, which is now co-sponsored by The Texas Tribune, where Henson is also now a regular contributor. He received his Ph.D. from the Department of Government at The University of Texas at Austin.

(12)

Appendix B Project Plan

Date

Task

Begin End

8/30/2010 12/17/2010 Research Question 1: Data Collection & Initial Analysis 8/30/2010 9/30/2010 Identify courses for analysis

9/30/2010 10/15/2010 Compile existing CIS data 9/30/2010 11/17/2010 Gather new student learning data 10/18/2010 11/17/2010 Compile existing student learning data 11/22/2010 12/6/2010 Data analysis of compiled data 12/6/2010 12/17/2010 Gather new CIS data

Research Question 2: Materials Development & Data Collection 8/30/2010 9/3/2010 Conduct faculty interviews

8/30/2010 9/30/2010 Develop lecture captures and accompanying in-class activities with faculty partners

8/30/2010 9/30/2010 Identify sources of student learning data with faculty partners 9/30/2010 10/29/2010 Develop post-use survey

9/30/2010 11/17/2010 Conduct classroom observations 9/30/2010 11/17/2010 Collect student learning data 11/17/2010 11/24/2010 Administer post-use survey

11/17/2010 11/24/2010 Administer instructor post-use survey 12/6/2010 12/17/2010 Gather CIS data

Data Analysis

1/3/2011 1/14/2011 Conduct faculty debrief interviews

1/3/2011 1/21/2011 Initial data analysis (one faculty member for research question 2)

1/3/2011 1/21/2011 Draft proposal for 2011 Educause Annual Conference (based on compiled data for research question 1 and case study for research question 2)

1/3/2011 1/21/2011 Interview transcription 1/21/2011 2/25/2011 All CSI data analysis

2/14/2011 2/28/2011 Draft mid-project progress report

2/28/2011 2/28/2011 Deliver mid-project progress report to Echo360

2/28/2011 4/1/2011 All student learning data analysis 4/4/2011 4/15/2011 All classroom observation data analysis 4/18/2011 4/29/2011 All faculty interview data analysis 5/2/2011 5/6/2011 Student post-use survey data analysis 5/9/2011 5/20/2011 Draw conclusions

5/23/2011 6/03/2011 Develop recommendations Report Results

6/6/2011 6/24/2011 Share findings with UT Austin faculty at Faculty Instructional Technology Academy 6/6/2011 9/2/2011 Draft final report

6/6/2011 9/2/2011 Draft article for publication 9/02/2011 9/02/2011 Project completion date

10/3/2011 10/3/2011 Deliver final project report to Echo360

12/31/2011 12/31/2011 Paper and/or article submission, and presentation at Echo360 Community

(13)

Appendix C

Project Budget & Justification

Items

Cost

Graduate Research Assistant -- wages & fringe

2760

Faculty stipends

3200

Travel for research presentation

1450

Indirect costs (35%)

2594

TOTAL

$10,004

Budget Justification: Echo360 Research Grant Graduate Research Assistant.

A GRA will be hired to assist with the monitoring and data collection in the participating courses. This will require approximately ten hours per week for one semester. Fringe benefits are calculated at 35% of salary.

Faculty stipends

Stipends of $400 each with be offered to eight (8) faculty members teaching in classrooms equipped with the lecture capture equipment and software, the impact of which is being tested in this pilot program. Six faculty members, listed below, have expressed interest in participating. Other faculty members using classrooms with the equipment installed will be recruited from a pool of approximately 18 possible candidates. The total for this budget line is $3200.

Travel

The 2 Primary Investigators will travel to the Educause Annual Conference in Philadelphia, PA, October 12-15, to report on findings from the classroom capture pilot program. The budget allocates $725 per person, with plans to request additional funds from the PI’s respective units.

Indirect costs

Figure

Table 1: Research Questions and Data Collection Methods

References

Related documents

(This is based on the East Midlands, which is used in the minimum income standard to represent a low-cost region outside London.) Figure 3.2 shows that the cost of adding a bedroom

Legacy Application Modernization Legacy Application Modernization Planning Project Selection Solidify Funding SUITE/SEM Process Update Portfolio, Document Benefits

on a WINDOWS 95 system 24 on a WINDOWS 98 system 24 on a WINDOWS NT 4.0 system 28 cleaning pick roller 38 print head 35 printer 39 clearing error messages 40 paper jams 33.

Assimilation is a part of the process by which the individual cognitively adapts to and organizes the environment’’ (p. We can infer that assimilation has a paramount

With the development of point-to-multipoint LSPs and related multiplay technologies, Juniper Networks has taken a leading role in providing the tools to support the transport of

Microsoft Power Point was selected as the platform for the tool, being available to most industry. By working in Power Point the use of internal and external hyperlinks is possible,

Heterologous spheroid culture in which tumor cells are combined with one or more stromal cell types has provided unique insights into stromal and tumor cell re- sponses to hypoxia

Certain material contracts to which the company is a party must be filed with the securities commissions, including contracts with management (other than employment