As of September 2009, the New York University

(1)

Comparative Use of Podcasts vs. Lecture Transcripts as Learning Aids for Dental Students

Kenneth L. Allen, D.D.S., M.B.A.; Ralph V. Katz, D.M.D., M.P.H., Ph.D.

Abstract: The purpose of this project was to describe dental students’ use of lecture podcasts versus written lecture transcripts as learning aids under three different circumstances: studying for an exam, reviewing an attended lecture, and reviewing a missed lecture. Additional analyses were performed to see whether demographic differences (e.g., age, gender, language skills, and computer skills) or grade differences were associated with preferences for using podcast versus written lecture transcripts of class notes. Fifty- one percent (n=171) of the second-year dental students at the New York University College of Dentistry voluntarily participated in this survey. The major findings were that 1) a high percentage of students (70–92 percent) used one or both aids in all three utilization circumstances with a consistent preference for podcast use, especially when reviewing a missed lecture; 2) course grades were not associated with the preferred use of either lecture aid; and 3) over half the students listened to the podcasts at speeds that were one and one-half or two times faster than normal speech, especially younger students. Further studies are warranted to delve into the current student generation’s preferred learning styles and the resultant learning outcomes associated with those preferences.

Dr. Allen is Clinical Associate Professor, Department of Cariology and Comprehensive Care, New York University College of Dentistry; and Dr. Katz is Professor and Chair, Department of Epidemiology and Health Promotion, New York University College of Dentistry. Direct correspondence and requests for reprints to Dr. Kenneth L. Allen, Department of Cariology and Comprehen- sive Care, College of Dentistry, New York University, 345 East 24^th Street, VA 16N, New York, NY 10010; 212-998-9426 phone;

212-995-4889 fax; kenneth.allen@nyu.edu.

Keywords: podcasts, dental education, learning, reading comprehension, listening comprehension Submitted for publication 8/20/10; accepted 12/6/10

A

s of September 2009, the New York Uni- versity (NYU) College of Dentistry made podcasts of all lectures available to students enrolled in the predoctoral Doctor of Dental Surgery (D.D.S.) program. This move was intended to replace the well-established practice of having a student-run written transcript service provide class notes for the entire class. The student-run transcript service provided a verbatim record of the spoken lecture. When the faculty introduced this new service of podcasting all lectures in the D.D.S. curriculum, its goals were to ensure that the lectures taped had good audio and video quality, were readily available to students via the intranet, and provided an appropriate copyright notice to listeners.^1,2

NYU College of Dentistry is not the first school at a major university to podcast. Duke University, in 2004, gave its 1,600 College of Arts and Science bac- calaureate freshmen free iPods for academic work.

This giveaway continues at Duke, but is now called a Digital Initiative with the devices given only to students enrolled in designated courses. A number of U.S. dental schools have adopted podcasting, with some using it as a learning aid. A University of Michi- gan student summarized the thoughts of the so-called

“me generation” when he said, “It’s not a better way of learning or a worse way. It’s just a new way. Our way, it’s about maximizing our time. But, frankly, there have been a couple of lectures, early morning ones, where I just didn’t get up.”³ Students have been sharing notes, taping lectures, and reviewing posted slides since these technologies became available. The podcast is the next iteration as it combines the audio and visual presentation in an easy-to-retrieve format.

The NYU College of Dentistry podcast lectures are available within hours of the lecture via iTunes U on a website available only to students enrolled in the course. Students can download any lecture from multiple computer stations located throughout the dental school building, after which they can listen and view the lectures on their computer, iPod, or any MP3 player.⁴ For the 2008–09 academic year, second-year predoctoral students who had become accustomed to having a transcript service for class notes in their first year decided to continue the transcript service in two selected courses. This transcript service was student-funded and conducted, as it had been in past years, with student workers doing the transcribing.

As these two courses had both class note transcripts and podcasts available, they offered the opportunity

(2)

for a natural experiment: a comparison of student use and preferences regarding these alternative methods for creating supplemental materials for lectures. The two second-year (D2) courses studied were Diagnosis and Treatment of Oral Diseases (DOD) and Complex Restorations I (CRI). DOD is a lecture-only course, while CRI is a lecture and preclinical laboratory course, with the preclinical laboratory component scheduled for six hours per week.

As reported by McCann et al., “Many decisions are being made in dental education about how to use technology for learning, yet there is little in the literature about the preferences of dental and dental hygiene students.”⁵ The purposes of this study, then, were to determine student preferences and outcomes and understand 1) whether students prefer podcasts to transcripts of class notes under three different study utilization circumstances: studying for an exam, reviewing an attended lecture, and reviewing a missed lecture; 2) whether there are any demographic differences (e.g., age, gender, language skills, and computer skills) associated with a preference for podcasts over transcripts of class notes; and 3) whether there are grade differences associated with the preferential use of podcasts versus transcripts of class notes.

Methods and Materials

All 347 D2 students received an e-mail with a hyperlink to take an Institutional Review Board- approved online survey using SurveyMonkey that allowed their responses to be anonymous.⁶ As an incentive to participate, the researchers offered a pair of tickets to a New York Yankees home game to be given to one randomly selected student who completed the survey. The e-mail outlined the design of the study and informed students as to what they would be asked to do.

The survey included ten questions within four areas: personal demographic information, self- reported computer literacy evaluation, self-reported course grade, and comparison of podcasts versus transcripts use with regard to frequency and practices.

This fourth section asked participants to respond to the posed set of comparative questions within three utilization-circumstances: when studying for an exam, when reviewing a missed lecture, and when reviewing an attended lecture.

Because the NYU College of Dentistry has an internationally diverse student body (approximately one-third of the class consists of advanced placement,

AP, students with dental degrees from other coun- tries), language skills became a variable of interest.

All AP students must score a minimum of 234 on the Test of English as a Foreign Language (TOEFL), but we used an additional self-reported spoken English measurement instrument for this study. Specifically, language skills were evaluated by asking two questions: is English your first learned language, and is English your primary language now used at home?

Students who answered yes to both questions were considered to have strong language skills. Students who answered no to both questions were considered to have weak language skills. All others were considered to have moderate language skills.

Categorical data were summarized as rates.

Tests of a difference in preference for podcast versus transcripts as a function of these categorical groupings, as well estimates of the association between categorical groupings, were achieved through use of cross-tabulations, phi coefficients, and chi-squared tests, using SPSS v 17. A type 1 error rate of less than 5 percent was used as the criterion for statistical significance.

Results

The response rate was 51 percent (n=171 respondents). The respondent survey sample was 49 percent male, with an average age of twenty-seven for both male and female respondents. This sample mir- rors the gender and age distribution for that second- year class at NYU College of Dentistry. The average age of those self-reporting excellent computer skills was twenty-six years versus an average age of thirty years for those self-reporting poor computer skills.

The relationship between computer skills and age was not statistically significant (p=.208). Self-reported computer skills also varied by gender as 67.5 percent of males self-reported excellent or very good computer skills compared to 56.3 percent of females reporting this level of skill; this variation by gender was not statistically significant (p=.387).

When comparing self-reported computer skills by self-reported English language skills—that is, English as the first learned language (yes/no) and English as the primary language spoken at home (yes/no)—there were no statistically significant differences observed (p=.326). For those who responded yes to English as their first learned language, 63.3 percent reported excellent or very good computer skills compared to those who responded no to English

(3)

as their first learned language, of which 61.9 percent reported excellent or very good computer skills. For those who responded yes to English as their primary language spoken at home, 70.6 percent reported excellent or very good computer skills compared to those who responded no to English as their primary language spoken at home, of which 62.5 percent reported excellent or very good computer skills.

Student use of podcasts and written transcripts was uniformly high, ranging from 70.1 percent to 91.8 percent across all three utilization circumstances (see Figure 1). Analyses revealed that students preferred the podcasts to the transcripts, regardless of whether they were studying for an exam, reviewing

an attended lecture, or reviewing a missed lecture.

This preference for podcasts was consistent across the two courses surveyed, DOD and CRI. The observed preference for podcasts across both courses was weakest for studying for an exam, slightly stronger when they were reviewing an attended lecture, and strongest when they were reviewing a missed lecture, although these observed differences were not statistically significant.

Podcast versus transcript use did not differ across the three self-reported course grade levels (bottom fourth of class, middle half of class, and top fourth of class) within either the DOD or CRI course (p=.746 and p=.294, respectively; see Figure 2).

Figure 1. Podcast and transcript usage under three different usage circumstances

Figure 2. Self-reported grades by podcast and/or transcript use when studying for an exam

(4)

While the sample size limited the power to conduct meaningful multivariate analyses, inspection failed to suggest any change in the relationship between self- reported grades and preferred use of podcast versus transcript as a function of demographic variables, speed of listening, or English language level.

Podcast listening speed (whether students listened to the podcast in real time, 1.5 times real-time speed, or twice real-time speed) was analyzed as a function of three variables: English language skills, age, and computer skills. Figure 3 shows the result- ing statistically nonsignificant trend, suggesting that stronger English language skills were associated with a greater tendency to listen to the podcasts at faster speeds (i.e., 1.5 times or twice real-time speed).

Figure 4 shows that while 41.2 percent of the oldest students (aged twenty-nine and older) listened to the podcasts at an accelerated speed, about 60 percent of the younger students (those aged ≤24 years) listened at accelerated speeds (usually 1.5 times real speed), but this difference was not statistically significant (p=.283). A final analysis revealed that the majority of students with excellent or very good computer skills (66.6 percent and 55.1 percent, respectively) reported listening at accelerated speeds, while a minority of those with good or poor computer skills listened at accelerated speeds (45.5 percent and 20.0 percent, respectively). Again while this trend was interesting, it was not statistically significant (p=.321).

Figure 3. Podcast listening speed as affected by language strength

Figure 4. Podcast listening speed as affected by age

(5)

Discussion

Technology affords students multiple methods to review and learn material. For this study, we focused on determining whether a distinct student preference exists for reviewing a previously deliv- ered lecture by playing back a podcast over reading a printed transcript.

The method by which students receive information will always be subject to change as technology advances and cultures develop. Before there was the web-based BlackBoard server, there was chalk on a chalkboard. The method of learning must suit the type and time of the student body. Baby boomers now getting close to retirement have been replaced with the “Generation Me” student.⁷

Technology is constantly advancing. The movement from what is now called snail mail to fax transmission was once considered a giant leap. Today, with e-mail, smart phones, and over 140,000 iPhone applications, Generation Me has been programmed for speedy delivery of whatever they want. Accord- ingly, our study found that many podcast users listened to lectures at accelerated speeds: one and a half to two times faster than real time. After the study was completed, we discussed the results with individual students who had expressed an interest in the project.

This led to informal meetings with groups of four to eight students about speed listening in which we discovered that a learning curve exists when listening at enhanced speeds and that the lecturer’s accent and native language play a role in determining the speed at which students listen. This speed-listening phenomenon warrants further investigation; this would include studying whether comprehension is affected by listening speed.

By the time dental students arrive for their first year of predoctoral study, they have proven that they are high achievers. This highly focused group engages in intense internal competition for grades as grades and class rank have a serious impact on coveted postdoctoral specialty programs. In Figure 2, we correlated podcast and transcript usage with self-reported grades. Across all performers, podcast and transcript usage was relatively static. This is consistent with the highly competitive nature of our population, regardless of class standing.

A variable we did not take into account was that other learning materials were available to students for one of the two courses identified in the survey.

In Complex Restorations I, which has a six-hour

per week laboratory component that incorporates a mastery of manual skills as well as didactic material, students also had access to online videos, laboratory guides, and handouts. We could have taken these additional materials into account and surveyed students as to their preferences by adding questions to the survey.

This study focused on only two second-year courses at an institution with large lecture-type classes. A smaller learning environment, in which class size was less than fifty, may have produced different results. In a smaller class, students may feel compelled to attend lectures since faculty members, who may also supervise clinical treatment, are more aware of who is present. Due to the nature of student behavior, however, we feel that the preference for podcasts would be similar in all educational years, D1 through D4, and among all institutions with a large class size.

One question raised in this study was how students listened to podcasts. The only groups that primarily listened at real-time (versus accelerated) speeds were those with self-reported computer skills that were good or poor and those over twenty-nine years of age. This is understandable, as these two groups might well be less familiar and/or comfortable with either accessing podcasts or using the devices for listening. The desire to listen at an increased speed was identified in a focus group of University of Michi- gan dental students reported in 2006. As a report of that group stated, “Students reported using the audio book feature of the iPod to speed up or slow down a lecture. Students specifically pointed to this single feature that made the iPod more useful than other audio players.”⁸ The lack of statistical significance in comparing language skills and speed of listening in our sample is likely due to the sample size.

A current textbook on e-learning⁹ discusses student performance and how podcast usage affects study practices and academic performance, including a pilot study of thirteen undergraduate students using a scale of high, good, or average to identify academic performance. That study found that high-performing students used the podcast less often than those clas- sified as good performers. Another small-scale study (N=28) discussed in the textbook examined the use of an e-lecture, similar to our podcast, both with and without a written transcript. That study found that having a written transcript had no effect on student learning as measured by performance on a ten- question test. In our research, student performance, as measured by self-reported grades, was not associated with preference for podcast or transcript.

(6)

Accordingly, a potential limitation of this study was the sample size. The response rate of 51 percent on our closed panel of 347 eligible students resulted in a final survey sample of 171. However, as our study findings revealed a few trends “of interest” that were not found to be statistically significant, one limitation of the study could be that it was underpowered due to the limited (low-cost) incentive offered to potential participants. We conducted another educational research project (sponsored by a commercial company) on a previous albeit similar class at the NYU College of Dentistry and achieved a 70 percent participation rate when each research subject was offered a significant cash incentive (~$500).¹⁰

It is interesting to note that, even with the preference for podcasts exhibited by the class that took part in this study, the class of D2 students in the subsequent academic year (2009–10) agreed on their own to fund a transcript service. Thus, despite the reported preference for podcasts, the students appear unwilling to abandon access to written transcripts.

There are differing theories on the comparative use and utility of podcasts versus written transcripts at this educational level. As educational institutions invest considerable resources to make podcasts of ev- ery lecture available, there is clear value in obtaining utilization, practices, and satisfaction data directly from students. Doing so will allow both the faculty and administrators to understand the how, why, and for what of this new educational technology. Most research concludes with the suggestion of future research to determine the real impact of this added student resource.^11-13

There is considerable interest and research on the Generation Me students, the group now making up the bulk of those in college and graduate school.

Twenge points out that “Generation Me students have high IQs, but little desire to read long texts.”⁷ Instruction to this group may need to be more interac- tive, including videos and podcasts. The potential for immediate delivery and wide availability (listening on the subway, for instance) appear to give these delivery methods wide appeal, but a study to define actual student preferences and usage is important.

Testing reading versus listening comprehension is one line of investigation worth pursuing. We found no relationship between grades and the choice of learning materials (podcast or transcript). However, as podcasts become more familiar to students in their undergraduate curricula, further investigation into their influence on grades in postgraduate education is essential.

Conclusion

The major findings of this study were that 1) a high percentage of students (70 to 92 percent) used one or both aids in all three utilization circumstances with a consistent preference for podcast use, especially when reviewing a missed lecture; 2) when studying under any one of the three utilization circumstances (studying for exams, reviewing an attended lecture, or reviewing a missed lecture), no statistically significant differences in podcast versus transcript use were detected; 3) course grades were not associated with preferences for either lecture aid; and 4) over half the students listened to the podcasts at rates of one and one-half or two times normal speed, especially younger students. Further studies are warranted to delve into the current student generation’s preferred learning styles and the resultant learning outcomes associated with those preferences. There appears to be a trend for using podcasts more than transcripts, but this trend did not vary in our study based on age, self-reported computer literacy, or English language skills.

REFERENCES

1. Billings-Gagliardi S, Mazor KM. Student decisions about lecture attendance: do electronic course materials matter?

Acad Med 2007;82(10 Suppl):S73–6.

2. Podcasts and classes. Chronicle of Higher Education, June 6, 2008:A31.

3. Michigan dental students able to view lectures on iPods.

Diverse Issues Higher Educ 2006;22(24):37.

4. Young JR. Lectures are recorded, so why go to class?

Chronicle of Higher Education, May 16, 2008:A1, A12.

5. McCann AL, Schneiderman ED, Hinton RJ. E-teaching and learning preferences of dental and dental hygiene students. J Dent Educ 2010;74(1):65–78.

6. Boehle S. Survey says! Training (Minneapolis, MN) 2008;

45(5):72.

7. Twenge JM. Generational changes and their impact in the classroom: teaching generation me. Med Educ 2009;43(5):

398–405.

8. Brittain S, Glowacki P, Van Ittersum J, Johnson L. Forma- tive evaluation strategies helped identify a solution to a learning dilemma. Educause Q 2006;29(3):24–31.

9. Torres PL, ed. Handbook of research on e-learning methodologies for language acquisition. Hershey, PA:

Informational Science Reference, 2009.

10. Allen KL, Norman RG, Katz RV. The effect of chewing gum on learning as measured by test performance. Nutr Bull 2008;33(2):102–7.

11. Harris H. Educational usages of podcasting. Br J Educ Technol 2008;39(3):548–51.

12. Lonn S, Teasley SD. Podcasting in higher education: what are the implications for teaching and learning? Internet Higher Educ 2009;12(2):88–92.

13. Jham BC, Duraes GV, Strassler HE, Sensi LG. Joining the podcast revolution. J Dent Educ 2008;72(3):278–81.