Alternative and Performance-based Assessment in Online Learning: Analysis, Framework, and Strategies to Systematically Assess Student Learning

AECT 2004 Conference Oct. 10–22, Chicago, IL

Alternative and Performance-based

Assessment in Online Learning:

Mahnaz Moallem, Ph.D.

The University of North Carolina at

Wilmington

Purposes of Presentation

{ Analyze Student-centered, Project-based,

Performance-based Assessment system (SPPA) to identify its implementation problems in face-to-face courses

{ Identify potential of online learning

environment for implementation of SPPA system

{ Propose a design framework for SPPA

system in online courses

{ Demonstrate the application of the model

using an example online course

A Conventional Teacher-Centered &

Outcome-based Assessment Model

Implement Instruction Assess Learning Outcomes & Effectiveness of Instruction Identify & Develop Assessment Establish Learning Outcomes

Re-mediate and/or Revise Instruction

S

tudent-centered,

P

roject-based,

P

erformance-based

A

ssessment system

tude nts to disp_lay al th inkin g skil_{ls an} d grea ter d epth of k_now ledg_e

And most importantly………

Provides a diversity of learning opportunities Deve lops a deepe r dialo_g over the co urse m ateria_ls Foste r both indiv idual a_nd group orien ted le_arnin g activit_ies Supports m measu r ore au thent ing learnin g ically Enab_{les s} critic

S

tudent-centered,

P

roject-based,

P

erformance-based

A

ssessment system

{ Integrating

assessment & instruction

{ Progress in learning

& in becoming more sophisticated

learners

{ Pedagogical changes

Student-centered, Project-based, Performance-based

Assessment system

Self-Assessment/

Feedback Expert-Assessment/_Feedback

Peer-Assessment/ Feedback Performance-based, Process-based, Progress-based Assessment Continuous Assessment Continuous Assessm ent Continuous Assessm ent

SPPA System in Face-to-Face

Classrooms: Implementation Problems

of dynamic & continuous assessment is appealing it is difficult to manage timely and formative feedback

SPPA System in Face-to-Face

Classrooms: Implementation Problems

tools & resources for gathering &

documenting day-to-day assessment data constrains the effectiveness of

SPPA System in Face-to-Face

Classrooms: Implementation Problems

It is difficult to: { Be an effective facilitator { Ask appropriate guiding questions { Document student progress when solving problems/projects

SPPA System in Face-to-Face

Classrooms: Implementation Problems

It is difficult to:

• Promote individual

students’ growth without providing

individual assessment

• Keep the focus of

individual

assignments on self-assessment

SPPA System in Face-to-Face

Classrooms: Implementation Problems

do not seem to work effectively without the instructor’s

feedback & guidance.

• It is difficult to

document the effects of self-assessment tools on individual growth & team

SPPA System in Face-to-Face

Classrooms: Implementation Problems

time and tools, using multiple sources of

assessment is

SPPA System in Online learning

Environment

Online learning environment:

{

Has the highest potential for real

world, authentic performance

assessment

{

Offers new tools and resources that

make implementation of SPPA a

reality

{

Provides a much better environment

for practicing duties of a mentor,

coach and/or facilitator

Online Learning Environment: New

Tools & Resources

• Database and

interactive system to track, monitor & document students’ activities

automatically

• Easy access & easy

development process for conventional

Online Learning Environment: New

Tools & Resources

{ Opportunity to self-assess own knowledge through automatic and instant feedback { Multiple communication tools to facilitate as well as to document dialogue

Online Learning Environment: New

Tools & Resources

Content & communication tools to:

{ Develop projects/tasks

that could incorporate multiple resources & perspectives

{ Complete project/tasks

at various intervals

{ Provide continuous

formal & informal feedback

Online Learning Environment: New

Tools & Resources

{ Unlimited & self-paced

access to course materials & communication tools

{ Interactive & asynchronous

communication tools to promote thoughtful & reflective commentaries

{ Learner’s & instructor’s

access to the record of responses, answers,

feedback, etc. to analyze and interpret one’s own performance & to reflect

A Model for Assessment of Online

Learning

Expert-Assessment Assessment

Self- Peer-Assessment

Initial Assessment

Expert-Assessment Assessment

Self- Peer-Assessment

Progress Assessment

Expert-Assessment Assessment

Self- Peer-Assessment

Product Assessment

Designing Online SPPA System:

An Example

Product Assessment: Designing Real

World Problem/Project

The first step was to develop a

complex and multifaceted real world

problem or project that required

students to

{

Demonstrate achievement of all

course goals

{

Engage in challenging real world

activities

{

Work in teams

{

Work with real clients and/or the

Product Assessment: Designing Real

World Problem/Project

Made sure that the real world

problem/project was complex enough to:

{ Break it into several subtasks or mini

projects to be completed at various intervals

{ Monitor its subtasks

{ Structure the accomplishment of the

major task

{ Develop a timeline for student work

{ Incorporate process-based, continuous

Product Assessment: Designing Self

Assessment Tools

{ A series of self-assessment rubrics was

developed (self assessment)

{ One or two completed projects were linked

to the course project to help set the standards for best performance (self-assessment)

{ The instructor reviewed teams’ progress

and product and provided written feedback

(expert assessment)

{ The instructor scheduled a chat session

with each team and reviewed the written comments at least once for each part of the project (expert assessment)

Product Assessment: Designing Self

Assessment Tools

{

Provided an online space for students

to work on their course project

continuously, and in collaboration with

their team members throughout the

semester

(peer assessment)

{

Used “Student Presentations” area

within WebCT course management

Assessing Progress

The next step in designing SPPA system was to develop related problems or cases for each unit of instruction to:

{ Provide learning experiences for the

completion of the course project

{ Address domain specific knowledge and

skills to complete the course project

{ Identify & further developed resources

and supporting instructional materials

{ Develop and form unit content and

Assessing Progress: Expert, Peer &

Self- Assessment Tools

{ Self-assessment quizzes to assess

background knowledge with automatic

expert feedback (self & expert assessment)

{ Individual assignments and forum

discussion to assess understanding and synthesis of readings and domain specific knowledge with immediate expert and peer feedback (Peer & Expert assessment)

{ Conversation space for teams working on

unit problems (peer & expert assessment)

{ Team evaluation instruments to assess

team’s performance (peer assessment)

{ Opportunity to review and compare other

teams’ products and feedback (peer assessment)

Initial Assessment

The third step was to develop opportunities for assessing students’ initial

understanding of the materials in order to assist:

{ Students in making their initial thoughts

explicit to themselves

{ Students compare their initial

understanding with their later

conceptualization of the materials & identify progress

{ The instructor to identify coaching and

Initial Assessment: Self, Peer & Expert

Assessment Tools

{

Individual assignments that consisted

of a list of thinking questions

{

Non-graded, self-assessment quizzes

with immediate feedback using WebCT

Quiz Maker

{

Team Forum discussion

{

Learning, thinking & multiple

intelligences inventories to identify

Reflections: Enablers

{ Use of SPPA Systems made assessment a

natural extension of instruction

{ Non-graded and reusable instructional

activities gave both students and the instructor useful feedback on teaching-learning process

{ Constant feedback enabled students to be

strategic in their own learning process and enabled the instructor to adapt the

instructional process to meet the needs of students

Reflections: Enablers

{ Assessment of actual performance enabled

students to translate what they had

learned in different units of instruction to a real-life application

{ Within SPPA system it was possible to

provide a wide range of assessment tools and techniques

{ SPPA offered more opportunities and ways

for providing feedback

{ SPPA made the management, collection

and transfer of assessment information easier

Reflections: Barriers

{

Accessibility and responsiveness of

the instructor

{

Instructor’s philosophy of learning

and assessment and his/her

expectations of students

{

Fostering a participatory online