Improving Learning and Reducing Costs

(1)

B

Y

C

A R O L

A . T

W I G G

Improving Learning

and Reducing Costs

Improving Learning

and Reducing Costs

Lessons Learned from Round I of the

Pew Grant Program in Course Redesign

(2)

Improving Learning and Reducing Costs: Lessons Learned from Round I of the Pew Grant Program in Course Redesign by Carol A. Twigg

110 8th Street, Troy, NY 12180 518-276-6519 (voice) 518-695-5633 (fax) http://www.center.rpi.edu

(3)

The Center for Academic Transforma-tion at Rensselaer Polytechnic Institute is conducting a Program in Course Redesign with support from the Pew Charitable Trusts. The purpose of this institutional grant program is to encour-age colleges and universities to redesign their instructional approaches using technology to achieve cost savings as well as quality enhancements. Redesign projects focus on large-enrollment, introductory courses, which have the potential of impacting significant stu-dent numbers and generating substan-tial cost savings. The Center has awarded $6 million in grants to thirty projects in three rounds of ten projects each. The first round of redesign projects began in July 1999 and concluded in July 2001. (Detailed descriptions of the ten redesigns and the outcomes each achieved can be found at

www.center.rpi.edu/PewGrant/rd1award .html.) The ten institutions and the courses they redesigned are:

•

Indiana University-Purdue University at Indianapolis (IUPUI): Sociology

•

Penn State University: Elementary Statistics

•

Rio Salado College: Mathematics

•

University at Buffalo (UB): Computer Literacy

•

University of Central Florida (UCF): American Government

•

University of Colorado-Boulder (UC): Astronomy

•

University of Illinois at Urbana-Champaign (UIUC): Statistics

•

University of Southern Maine (USM): Psychology

•

University of Wisconsin-Madison (UW): Chemistry

•

Virginia Tech: Linear Algebra Round I was constructed as a pilot for the overall program. Rather than open-ing the program to a national competi-tion, the Center staff selected representa-tives from twenty institutions that exhib-ited a high degree of readiness to develop proposals. Ten of the twenty institutions

were selected to receive grants, and they began their redesign projects in fall 1999. Our intention was to teach the principles of redesign that the Center espouses to those institutions that we considered most “ready” and to work closely with them as they developed proposals. We believed that it was necessary to estab-lish exemplars for future redesign efforts since higher education institutions have had little experience with redesign strategies that both improve quality and reduce cost.

What follows is an analysis of the results of the Round I projects, with a focus on the most important quality improve-ment and cost reduction techniques used in the redesigns, the implementa-tion issues they encountered, and the projected sustainability of the course redesigns. The Center will produce a similar analysis for Rounds II and III when they have been completed.

Quality Improvement Strategies and Successes

Five of the ten projects reported improved learning outcomes. Four reported no significant difference, and one was inconclusive. Among the find-ings were the following:

•

At IUPUI, students in redesigned sections had significantly higher (.10 level) grades.

•

Redesign students at Penn State out-performed traditional students at a statistically significant level on a content-knowledge test: 60 percent correct in the traditional format, and 68 percent in the redesigned classes. Students in the redesigned classes also demonstrated a greater under-standing of a number of critical statistical concepts.

•

At UB, the redesign resulted in an increase in the percentage of students earning a grade of A- or higher, moving from 37 percent to 56 percent. The mean grade earned in the course increased by one-third of a letter grade, from a C+ to a B-.

•

At UCF, students in the traditional format posted a 1.6-point

improve-Improving Learning and Reducing Costs: Lessons Learned from Round I

We believed that it was

necessary to establish

exemplars for future

re-design efforts since higher

education institutions have

had little experience with

redesign strategies that

both improve quality and

reduce cost.

(4)

I M P R O V I N G L E A R N I N G A N D R E D U C I N G C O S T S

ment on a content examination, whereas at 2.9, the mean change for students in the redesigned course was almost double that amount.

•

At USM, the redesign resulted in significant improvements in overall understanding of course content as measured by pre- and post-course assessment of important concepts. Six of the ten projects measured changes in drop-failure-withdrawal (DFW) rates; five showed improvement. Among the findings were the following:

•

At Penn State, the DFW rate decreased from a rate of 12 percent in the traditional course to 9.8 percent in the redesigned course.

•

IUPUI reduced the DFW rate from 38.9 percent to 24.8 percent.

•

Rio Salado increased completion rates from 59 percent to 64.8 percent.

•

At USM, a smaller percentage of students received failing grades, moving from 28 percent in tradi-tional sections to 19 percent in the redesigned course.

•

At Virginia Tech, the percentage of students completing the course and achieving grades of D- or better improved from an average of 80.5 percent to an average of 87.25 percent.

All ten projects made significant shifts in the teaching-learning enterprise, making it more active and learner-centered. The primary goal was to move students from a passive, note-taking role to an active, learning orientation. As one math professor put it, “Students learn math by doing math, not by listening to someone talk about doing math.” Lectures were replaced with a variety of learning resources, all of which involved

more active forms of student learning or more individualized assistance. When the structure of the course moves from an entirely lecture-based to a student-engagement approach, learning was less dependent on the conveying of words by instructors and more on reading, explor-ing, and problem solving by students. The following is a list of the most effec-tive quality improvement techniques used by the Round I projects.

•

Continuous Assessment and Feed-back.Six of the projects incorporated automated (computer-based) assess-ment and feedback into their re-designs. Automating assessment and feedback enabled both repetition (student practice) and frequent

feed-back, pedagogical techniques that have repeatedly been documented to facilitate learning. Students were regularly tested on assigned readings and homework using short quizzes that probed their preparedness and conceptual understanding. These low-stakes quizzes motivated stu-dents to keep on top of the course material, structured their studying and encouraged them to spend more time on task. Online quizzing encouraged a “do it till you get it

right” approach: students were allowed to take quizzes until they mastered the material.

Quizzes also provided powerful for-mative feedback to both students and faculty members. Faculty could detect those areas where students were not grasping concepts, thereby enabling corrective actions to be taken in a timely manner. Students received diagnostic feedback that pointed out why an incorrect response was inappropriate and directed them to material that need-ed review. Since students were required to complete quizzes before class, they were better prepared for higher-level activities in class. Consequently, the role of the instruc-tor shifted from one of introducing basic material to one of reviewing and expanding what students had already mastered.

•

Increased Interaction among Students.Seven of the projects took advantage of the Internet’s ability to provide useful and convenient opportunities to increase discussion among students. Students in large lecture classes tend to be passive recipients of information, and student-to-student interaction is often inhibited by class size. In smaller discussion forums, students can participate actively. UCF and IUPUI created small online discus-sion groups in which students could easily contact one another. Students were able to benefit from participat-ing in the informal learnparticipat-ing commu-nities that were created. Software allowed instructors to monitor the frequency and the quality of stu-dents’ contributions to discussions more easily and carefully than in a crowded classroom.

The primary goal was to

move students from a

passive, note-taking role to

an active, learning

orienta-tion. As one math

profes-sor put it, “Students learn

math by doing math, not

by listening to someone

talk about doing math.”

(5)

L E S S O N S L E A R N E D F R O M R O U N D I

/ Five of the projects replaced lecture

time with individual and small-group activities that took place in computer labs, staffed by faculty, graduate teaching assistants (GTAs) and/or peer tutors. In several instances, increased lab hours enabled the students to have more one-on-one assistance. Students welcomed the reduction in lectures and the opportunity to work in groups to apply what they had learned from the resource materials. Students learned from each other and increased their skills in working collaboratively on projects. In addi-tion, peer pressure within groups was a powerful incentive for students to keep up with their work.

•

Online Tutorials.UW and Virginia Tech were the most sophisticated users of online tutorials. Building on substantial experience in using and developing interactive materials, UW has developed thirty-seven Web-based instructional modules in chemistry as of July 2001. Each tutorial module leads a student through a topic in six to ten interac-tive pages. When the student has completed the tutorial, a debriefing section presents a series of questions that test whether the student has mastered the content of that module. UW also incorporated interactive chemistry materials created by Stanley Smith at UIUC. Students found these online tutorials to be very helpful; they particularly liked the ability to link directly from a problem they had difficulty with to a tutorial that helped them learn the concepts needed to solve the problem. Many reported that they found the online material much more accessible than the textbook.

Virginia Tech also used a variety of Web-based course delivery tech-niques such as tutorials, streaming video lectures and lecture notes as the main tools for presenting course materials. Consisting of exercises with solutions that were explained in built-in video clips, tutorials could be used at home or at a campus lab. Tutorials have taken over the main instructional role: 84 percent of the students reported, “The computer presentations explain the concepts well.” Students at UB also found the self-paced tutorials provided by the textbook publisher to be effective and easy to use, and they reported that the materials enhanced their learning.

•

Undergraduate Learning Assistants (ULAs).UC and UB employed ULAs in lieu of GTAs. Both universities found that ULAs turned out to be better at assisting their peers than GTAs because of their understanding of the course content, their superior communication skills and their awareness of the common miscon-ceptions about computers held by the students. At UC, the instructor met weekly with the ULAs and dis-cussed in detail what was working and where the students were having difficulty. The feedback from these

weekly meetings gave the instructor a much better sense of the class as a whole and of the individual stu-dents in it than would be possible with a class of 200 students. Those knowledgeable about the impact of pedagogy on improved student learn-ing will find nothlearn-ing surprislearn-ing in this list. Among the well-accepted “Seven Principles for Good Practice in Under-graduate Education” developed by Arthur W. Chickering and Zelda F. Gamson in 1987 are “encourage active learning,” “give prompt feedback,” “encourage cooperation among students,” and “emphasize time on task.” Good pedagogy itself has nothing to do with technology. What is signifi-cant about these redesigns, however, is that they were able to incorporate good pedagogical practice in courses with very large numbers of students, which would have been impossible without using technology.

Cost Reduction Strategies and Successes

There are a variety of ways to reduce costs and, consequently, a variety of instructional models that can be devel-oped depending upon institutional circumstances. The approach most favored by the Round I projects was to keep student enrollments the same while reducing the instructional resources devoted to the course. Seven of the ten projects employed this approach, which makes sense when student demand for the particular course is relatively stable. The other three projects—IUPUI, USM and Rio—increased student enrollments with little or no change in course expen-ditures. In the first two instances, section size was increased; in Rio’s case, one faculty member, who had tradition-ally taught one section, handled four

➤

These redesigns were able

to incorporate good

peda-gogical practice in courses

with very large numbers

of students, which would

have been impossible

without using technology.

(6)

sections simultaneously with the help of a course assistant. This technique is especially appealing to institutions that face greater student demand than can be met using conventional methods. A third way to decrease costs is by reducing the number of repetitions required to pass the course. In many community colleges, for example, it takes an average of 2.5 enrollments to pass introductory mathematics courses. This means that the institution and the student must spend 2.5 times what it would cost to pass the course on the first try. Five of the ten projects showed a decrease in drop-failure-withdrawal (DFW) rates, ranging from 10 to 20 percent. Of those five, only UCF calculat-ed the cost savings resulting from higher retention rates. Clearly, the other four could calculate those savings, which, in turn, would produce a higher cost-per-student savings than we report. What were the most effective cost savings techniques employed by the Round I projects? Since the major cost item in instruction is personnel, reduc-ing the time faculty and other instruc-tional personnel spend and transferring some tasks to technology-assisted activi-ties is key. By reducing the number of hours spent by faculty and others while keeping credit hours constant with no diminution of learning results, all ten projects were able to reduce costs while maintaining quality. The following is a list of the predominant techniques used by the projects.

• Online Course Management Systems. Course management systems played a central role in eight of the ten redesigns. Some projects used com-mercial products like WebCT and Blackboard; others used homegrown systems created for campuswide use or specifically for the redesigned

course; and others used instructional software that included an integrated management system. Using a course management system radically reduced (or eliminated) the amount of time faculty spent on nonacadem-ic tasks such as recording, calculat-ing and storcalculat-ing grades; photocopycalculat-ing course materials; posting changes in schedules and course syllabi; send-ing out special announcements; and transporting syllabi, assignments, and examinations from one semester to the next.

•

Online Automated Assessment of Exercises, Quizzes, and Tests. Five of the ten projects used automated grading of exercises, quizzes or tests. Some used the quizzing features of commercial products like WebCT; others used homegrown systems created for campuswide use like UIUC’s Mallard; and others used specific quizzing software like TEST-PILOT. The amount of time faculty and/or GTAs spent on the time-consuming process of preparing quizzes as well as on grading, record-ing and postrecord-ing results was sharply reduced. Automated testing systems, comprised of large databases of questions, enabled individualized tests to be easily generated.

•

Online Tutorials.Online tutorials at UW helped structure discussion sections by having students come to class prepared to ask questions. This meant less preparation time for GTAs. Virginia Tech’s use of similar online course delivery techniques enabled a radical reduction in teaching staff. Individual faculty members were no longer required to present the same content in duplicative efforts, nor were they required to replicate exer-cises and quizzes for each section.

•

Shared Resources.When the whole course (or more than one section) is redesigned, substantial amounts of time that faculty spend developing and revising course materials and preparing for classes can be consid-erably reduced by eliminating duplication of effort. Penn State constructed an easily navigated Web site that contained not only the management aspects of the course but also a large number of student aids and resources (solutions to problems, study guides, supplemen-tal reading materials for topics not otherwise treated in the text, self-assessment activities, etc.). Having assignments, quizzes, exams and other course materials on a commu-nity Web site saved a considerable amount of instructional time.

•

Staffing Substitutions.UC and UB found that using ULAs in lieu of GTAs was a key cost-saving device. By replacing expensive labor (faculty and graduate students) with rela-tively inexpensive labor, the teams increased the person-hours devoted to the course while cutting costs. Rio Salado employed a course assis-tant to address nonmath-related questions (which characterized 90 percent of all interactions with students!) and to monitor student progress. This freed the instructor to handle more students and to con-centrate on academic rather than logistical interactions with students. Penn State also used ULAs to grade homework assignments, relieving GTAs of this chore, as well as to as-sist in labs, thus reducing the num-ber of GTAs required for the course.

•

Reduction of Space Requirements. UCF wanted to utilize classroom space more efficiently and thus

(7)

/ reduce the amount of rented space

needed by the university. Delivering portions of a course via the Web as a substitute for face-to-face class-room instruction saved precious classroom space. Two or three sections/courses could be scheduled in the same classroom where only one could be scheduled before. UCF was the only project that detailed the cost savings resulting from reduced space costs, but any of the projects that reduced contact hours could calculate those space savings as well. In regard to cost savings, the redesign methodology was an unqualified success. All ten of the Round I projects reduced their costs. Some saved more than they had planned; others saved less. The Round I projects planned to reduce costs by about 37 percent on average, with a range of 20 to 71 percent. They actually reduced costs by 33 percent on average, with a range of 16 to 77 percent. Final results from Round I show a collective savings of $1,006,506 for ten courses, compared with the original projection of $1,160,706. (For a detailed comparison of planned versus actual savings, please see www.center.rpi.edu/ PewGrant/Rd1saving.html.)

Why is there such a large range of sav-ings among the projects? The differences are directly attributable to the different design decisions made by the teams, especially regarding what to do with the faculty time that was saved. Those with a lower percentage of savings tended to redirect rather than reallocate saved fac-ulty time; in other words, they kept the total amount of faculty time devoted to the course constant but changed the nature of how the faculty spent their time (e.g., lecturing vs. interacting with students.). Others radically reduced the amount of time non-faculty personnel

like GTAs spent but kept the amount of faculty time constant. Those decisions cut down on the total savings. In con-trast, by reallocating faculty time to other courses and activities, Virginia Tech showed the most substantial cost savings. Other projects could have saved more with no diminution in quality had they made different design decisions. Higher education has traditionally assumed that high quality means low student-faculty ratios and that large lecture-presentation techniques are the only low-cost alternatives. By using tech-nology-based approaches and learner-centered principles in redesigning their courses, these ten institutions have demonstrated a way out of higher

educa-tion’s historical trade-off between cost and quality. Some of the projects relied on asynchronous, self-paced learning modes; others used a traditional, synchronous classroom setting but with reduced student/faculty contact hours. Both approaches considered how best to use all available resources—including faculty time and technology—to achieve the desired learning objectives. Moving away from the current credit-for-contact mode of instruction and focusing on how to produce more effective and efficient learning by students were fundamental to success.

Implementation Issues

As part of the grant application process, the Center required institutions to assess and demonstrate their readiness to engage in large-scale redesign by responding to a set of institutional-readiness criteria and to a set of course-readiness criteria, both developed by Center staff. (For a full description of the program’s readiness criteria, please see www.center.rpi.edu/PewGrRdi.html.) Our experience in the program has taught us that some institutions, because of their prior investments and experi-ences, better understand what is required to create these new learning environments and are more ready to engage in redesign efforts. In addition, just as some institutions are more ready than others to engage in large-scale redesign, some faculty members and some courses are more ready than others to be the focus of that redesign effort. Prior experiences with technology-mediated teaching and learning and numerous attitudinal factors give them a head start on the process.

The experiences of the Round I projects corroborated the importance of readi-ness in completing a successful redesign project. The ten institutions involved in Round I exhibited a high degree of readiness, and all successfully completed their redesigns. When project teams encountered implementation problems, however, in almost every instance the problem was directly related to a lack of readiness. The description of implemen-tation problems that follows is organized in relation to the program’s readiness criteria; the italicized portions are taken from commentary about each criterion included in the grant program guide-lines.

•

Course Readiness Criteria #3. Decisions about curriculum in the

➤

When project teams

encountered

implementa-tion problems, in almost

every instance the problem

was directly related to a

lack of readiness.

(8)

department, program, or school must be made collectively. Decisions to engage in large-scale course redesign cannot be left to an individual faculty member. An insti-tution’s best chance of long-term success involves not a single individ-ual but rather a group of people who, working together, are committed to the objectives of the project. Indica-tors that the faculty in a particular unit are ready to collaborate include the following: they may have talked among themselves about the need for change; they may have decided to establish common learning objectives and processes for the course in ques-tion; and they may have instituted pieces of a common approach, such as a shared final examination. The biggest implementation issue for several of the projects was achieving consensus among all faculty teaching the course about a variety of issues. Course development is usually done by a single faculty member working on a single course, and the redesign of a single course by multiple faculty presented several challenges. These challenges included gaining agree-ment on core course outcomes and instructional formats, reaching consensus on textbook selection and topic sequences, and setting up a common Web site. Since instructors were not used to talking about such issues, they needed time to work through them. As one team com-mented, however, this was a “good” problem to have in that it led to exciting discussions and efforts to design a course freed of past conven-tional wisdom.

Individual faculty readiness, not only departmental or program readiness, needs to be an integral part of the

course redesign process. Identifying the “right” faculty members to teach the redesigned courses—that is, those with attitudes open to change and to collaboration—is important. Two of the projects encountered difficulties when they tried to move beyond the initial course designers to enlist other faculty in teaching the redesigned course.

•

Course Readiness Criteria #4.The faculty must be able and willing to incorporate existing curricular materials in order to focus work on redesign issues rather than materials creation.

Faculty who are willing to use an appropriate blend of homegrown (created by local faculty) and pur-chased learning materials in a non-dogmatic fashion will have a head start. Faculty who are susceptible to the “not-invented-here syndrome”— that is, who believe that they must create everything themselves from scratch—will be consumed with materials development and will add large amounts of time to the redesign process. Courses taught by faculty who are willing to partner with other content providers, whether commer-cial software producers or other colleges or universities that have

developed technology-based materials, make better candidates for a large-scale redesign project. In several instances, revising previ-ously developed materials, adapting existing materials and developing new materials turned out to be more work than originally anticipated. As one team put it, these activities were both time- and thought-consuming. One team that decided to develop customized course management software, which was a larger task than they anticipated, now believes that it was a mistake not to adopt a standard course management soft-ware package.

•

Institutional Readiness Criteria #3. The institution’s goal must be to integrate computing throughout the campus culture.

Unlike institutions that have estab-lished “initiatives” without specific milestones, computing-intensive campuses know the numbers. They know the level of availability of net-work access and the level of personal computer ownership (or availability) for students and faculty on their cam-puses because their goal is saturation, and the numbers tell them how close they are to achieving that goal. Ubiquitous networked computing is a prerequisite to achieving a return on institutional investment. Until all members of the campus community have full access to IT resources, it is difficult to implement significant redesign projects.

As they ramped up, two of the projects encountered problems in providing adequate laboratory classroom space and equipment to offer the course in the redesigned format. In both cases, computer lab space on campus was scarce. The institutions involved

Several projects

experienced delays due

to factors beyond their

control having to do with

the current, relatively

immature state of the

commercial software

marketplace.

(9)

/ view these problems as temporary

and see three solutions on the hori-zon: 1) constructing more smart classrooms, 2) adopting wireless solutions in which students bring laptops to traditional classrooms, and 3) using lab facilities in campus housing sites that have experienced a decline in demand as more students bring their own PCs to campus.

•

Institutional Readiness Criteria #7. The institution must have estab-lished ways to assess and provide for learner readiness to engage in IT-based courses.

Learner readiness involves more than access to computers and to the net-work. It also involves access to techni-cal support for using navigation tools and course-management systems. Students also need to be aware of what is required to be successful in technology-intensive courses. Making the change from face-to-face instruc-tion to online learning involves far more than learning to use a computer. Many students are set in their ways after a lifetime (albeit brief) of passive instruction. They need preparation in making the transition to more active learning environments.

Preparing students (and their par-ents) for changes in the way a course is offered turned out to be an impor-tant ingredient for several projects. Students and their home depart-ments were uneasy at best about the new approaches. Inevitable develop-mental problems like system crashes and data-handling errors became visible targets of dissatisfaction. Issues of perception were addressed successfully through active commu-nication with departments and patient replies to student e-mail messages. Software and hardware

problems were resolved through improvements in equipment, programs and system backups, and eventually these problems decreased to near zero. Finally, both novelty and anxiety wore off as succeeding classes of students moved through and the redesigned approaches blended into the teaching and learning scene.

•

Additional Implementation Problems. Several projects experienced problems and delays due to factors beyond their control having to do with the current, relatively immature state of the commercial software marketplace. Course management software, for example, is being continuously changed and updated. Upgrades can bring problems, especially in situations where the software is being stretched, such as occurred in these projects. Upgrades in software required rechecking on-line homework and quiz questions and revising online tutorials. Conse-quently much time was spent redo-ing course materials that had been developed and tested earlier. Similarly, those project teams committed to using instructional software products developed by textbook publishers encountered some problems in adapting the software to meet both faculty and student needs. If a team decided to change texts, for example, they found themselves having to spend considerable time changing linked feedback for students. Some com-mercial materials that were originally planned to be included in the redesigns were rejected because of perceived low quality or because they were unable to accommodate large numbers of students. (Many

soft-ware products assume a small class size in contrast to the projects class sizes of more than 1,000.) One team commented that because the off-the-shelf software they used was not as mature as anticipated, their redesign might have been slightly ahead of its time. They believe, however, that the software will mature sufficiently in the not-too-distant future. Several of the projects experienced sig-nificant backsliding from their original project goals in regard to cost reduction, bringing to mind the importance of Institutional Readiness Criteria #1: The institution must want to reduce costs and increase academic productivity.In one case, the projected cost reduction was to be achieved by increasing section size in order to free faculty to offer additional courses. Despite the lead faculty member’s confidence that this could be done and increased quality could be preserved, the administration failed to follow through on their responsibility to reduce the number of sections. In another case, a department reneged on its commitment to reduce seat time to the projected percentage, again despite the lead faculty member’s confidence that this could be done. Both of these instances suggest a lack of institutional commitment to increasing academic productivity.

Because we did not require the institu-tions to redesign the entire course, as we did in subsequent rounds, several of the projects redesigned a single section as a proof of concept. In one instance, the project proved in one section that students were able to learn effectively in the absence of lectures by using Web-based tutorials. The project leader’s departmental colleagues, however, were reluctant to reduce the number of lectures in the remaining sections.

(10)

Again, this suggests a lack of depart-mental and institutional commitment. Faculty members on their own have shown spectacular success in creating highly effective new learning environ-ments, but in order for these successes to have a real impact on the institution as a whole, administrative leadership needs to play an active and continuing role.

Sustainability

One way to judge the success of a grant-funded project is to assess its potential to be sustained once the grant funding runs out. All ten of the Round I projects are firmly committed to sustaining their redesigns. Comments include “the team is convinced that the redesign is sustainable,” “our efforts are clearly sustainable,” “the department is totally committed to its continuation,” “there is no desire within the department to return to the traditional design,” and “there are overwhelming reasons for continuing with the redesigned course.” Several project leaders have said that the redesigned format is now embedded within the department’s culture or that a new culture for teaching the introduc-tory course has been created, and “from this there is no going back.” As one team put it, “The success of the restructured course—as reflected by cost savings, improved student performance, and instructor satisfaction—ensures that [the redesign] is the preferred mode of instructional delivery.”

A second way to evaluate the success of a grant-funded project is to consider its impact on other courses within the department and within the institution. Again, all ten projects report a signifi-cant impact on other courses. Penn State is redesigning two additional introduc-tory courses and a 400-level statistics

course. In addition, their redesigned statistics course will be distributed for use at Penn State’s twenty-two Com-monwealth Campuses. UB’s redesign methodology is being applied to other courses within the department. Based on what they learned in their initial redesign, Virginia Tech has created a new tutorial system to be used in the Math Emporium in additional courses. UW’s team has also implemented a coursewide redesign in General and Analytical Chemistry based on the same principles employed in General Chem-istry. UCF’s course redesign model is being adopted widely throughout the institution. Three other large enrollment general education courses, English

Composition I and II and College Alge-bra, are in various stages of planning and implementation of the reduced seat-time instructional model.

To what do we attribute the high level of success achieved by the Round I projects? The innovation, dedication and hard work shown by each of the ten project teams was an essential ingredi-ent. In addition, the Center for Academic Transformation provided leadership in choosing the right participants, teaching them the planning methodology, actively supporting them as they developed their design plans, closely monitoring the implementation process, and insisting

on ongoing and final progress reports that include measurable outcomes. The Center created a unique three-stage proposal process that required appli-cants to assess their readiness to partici-pate in the program, develop a plan for improved learning outcomes, and analyze the cost of traditional methods of instruction as well as new methods of instruction utilizing technology. (See www.center.rpi.edu/PewGrant/Tool.html for a description of the Center’s Course Planning Tool, which facilitates this analysis.)

Perhaps the most significant aspect of this process has been the need for the Center to teach the redesign methodol-ogy, especially in regard to cost savings, since neither faculty nor administrators traditionally employ this approach to restructuring courses using IT. Prospec-tive grant recipients were supported throughout by a series of invitational workshops that taught these assessment and planning methodologies and by individual consultations with Center staff. Both faculty and administrators have repeatedly indicated that learning the methodology is key to the effective-ness of the process. Once learned, how-ever, the methodology is easily transfer-able to other courses and disciplines. The pioneering institutions from Round I have established replicable models for those institutions that want to use tech-nology to improve student learning while reducing instructional costs. Building upon that valuable experience, the Round II and Round III projects have made improvements to these initial efforts and appear to be achieving even stronger results. We look forward to producing an analysis similar to this one for Rounds II and III when their projects are complete.