Quality Assessment and Quality
Assurance
in
Online
Learning
Michael A. Mariasingam
Quality Learning Global
USA
28th Annual Conference on Distance Teaching &
Learning, Madison
August 8-10, 2012
Session 1 Topics
Current status of quality assessment and
assurance in online learning
New comprehensive approach to quality
assurance
Benchmarks and measurements for quality
assessment and assurance
Current Status of Quality
Assessment and Assurance in
Online Learning
Focus on Limited Stakeholders
Quality guidelines are based on a quality
concept that has focus on
limited
stakeholders
Stakeholders
normally included
:
Learners, faculty, and institution
Stakeholders
generally not included
:
Employers, professional associations,
society, government, and others
Limited Perspectives
Guidelines have only the functional
perspective – learning effectiveness, student
support, technology support etc
Example:
Sloan-C
Five pillars of quality
- - learning
effectiveness, cost effectiveness, access,
Structure of Quality Frameworks
Frameworks are two-level frameworks.
First level
:
Functional categories, like learning
effectiveness, as quality assessment categories.
Example:
Quality assessment category: “Course
Development Benchmarks”
Structure of Quality Frameworks
Second level
: Guidelines as criteria for
assessment under the quality assessment
category.
Example:
Guideline: “Courses are designed to require
students to engage themselves in analysis,
synthesis, and evaluation as part of their
course and program requirements”.
Major Characteristic
Feature
Quality frameworks, with few exceptions
[Mariasingam (2006)]
, are
guidelines, not
benchmarks or standards
although they are
often called benchmarks.
Example:
“The course or program provides for
appropriate interaction between faculty and
students and among students.“
(IHEP, 2000)
This is just a guideline,
not
a benchmark or
standard
.
Levels of Assessment
Guidelines, with a few exceptions, are
defined to assess quality at program level
Quality must be measured at multiple
levels – institutional level, program
level, and course level.
New Comprehensive Approach
to Quality
Need for an Alternative
Accreditation Model
“
Quality assurance in distance learning should be
sensitive to both learning principles common to all forms
of higher education, and aspects of learning that are
distance specific
”
(Marginson 2002).
Hence, a need for an alternative accreditation
model that has a comprehensive quality
Alternative Customer Focused
Quality Concept
Education now a global economic commodity
To be sold, must meet the requirements
of
all
stakeholders
Must have a quality assurance approach that
has a business orientation
Must have a customer focused industry
oriented approach.
Multiple Dimensions of Quality
(cf. Ehlers 2002a, 2002b, 2003a)
Q
Different meanings
of quality
Different levels of
quality
Different
perspectives
Multiple Perspectives of Quality
Quality, like beauty, is in the eye of the beholder (or the
stakeholder)
“
The achievement of quality can be managed and quality itself be
credibly measured only when it is clear for what purpose the
product or service is to be used and whose quality interests are to
be served
”
(Schweiger, 1996).
There are multiple stakeholders in education; to be of
quality,
education should serve the interests of all
stakeholders
A new approach that meets the requirements from the
perspectives of all stakeholders is needed.
Multiple Levels of Quality
To be precise and meaningful quality must be
measured at multiple levels –
institutional
level, program level, and course level
Quality measures defined for these three levels
will be interrelated, will have some overlap but
the breadth and depth of these performance
measures will vary significantly with levels
Quality frameworks for the three different levels
will be different
Criterion Based Negotiated
Approach
Criterion referenced concept
“Quality is measured by indicators of the performance
of individual institutions or of the worth of the products
on their own rather than in comparison with others”
(Baumgart and Kaluge, 1987)
Negotiated approach
Quality assessment process includes both
quantifiable measures and qualitative
intrinsic
values
Outcomes-based Approach
In
outcomes-based approach
learning
outcomes that are measured should include
learning outcomes like learning skills at
Kirkpatrick’s
“behavior”
level
[Kirkpatrick’s four-level model: ‘reaction‘, ‘learning‘, ‘behavior‘,
and ‘business results’]
Education must provide learners the ability
and skills necessary to transfer knowledge
gained to real life situations.
Benchmarks and Measurements
for Quality Assessment and
Benchmarks
“A
standard
against which something can be
measured”
Specify criteria
that define the element(s) of
quality to be assessed at multiple levels (e.g):
• Threshold
• Modal
Selection of Standard
The standard selected could be:
Absolute standard
based on theory
Examples
:
Quality Assurance Agency [UK] subject benchmarks
http://www.qaa.ac.uk/
Mariasingam, M. A. (2006)
Quality criteria and
benchmarks for online degree programs
(Paperback).
Ann Arbor, MI: ProQuest / UMI
Selection of Standard
Best practice
Example:
Master of Engineering in Professional Practice [MEPP]
program (UW-Madison)
Internal
To suit the institution’s, program’s goals, contexts of
learners, stakeholders. Given the constraints of the
institution, program, the standard selected need not be
either the ideal possible or the best practice, it would be
internal – institution’s or program’s specific benchmark.
Benchmarking process
Eight step process:
• Decide what to benchmark
• Decide the level at which to define the benchmark
• Select the standard – absolute, best practice, or internal
• Identify the current level of quality
• Identify the gap in quality
• Identify and take the necessary steps to improve quality
• Monitor the implementation process and ensure quality
• Adopt the culture of CQI
Quality Assurance
1: Define Quality Criteria and Benchmarks:
They should:
address the requirements of all stakeholders
address the different perspectives of each
stakeholder
be measurable
include metrics to determine the extent to
2: Organize Specific Program Processes
So that the program:
Meets the defined quality criteria and
benchmarks at all levels
Addresses key components of program
planning, program implementation, and
program quality assessment and assurance
Includes multiple perspectives when
3: Establish:
A monitoring system to ensure that the steps
and processes specified in Step 2 are
implemented
A system for periodic review to ensure that
4: Maintain, Monitor, and Improve:
A self-assessment process to determine that
all quality assurance steps taken in [1] to [3]
are yielding intended program outcomes
Processes for correcting any deficiencies that
Quality Frameworks
Tools for Quality Assessment and
Assurance at:
Program Level
Institutional Level
Quality Framework at Program
Level
Quality Framework
Mariasingam, M. A. (2006)
Quality criteria
and benchmarks for online degree
programs
(Paperback). Ann Arbor, MI:
Framework Features
Alternative quality concept
Wider role for education
Multiple dimensions of quality
Multiple perspectives of quality
Multiple levels of quality
Criterion-based approach to quality assessment
Negotiated approach to quality assessment
Quality Framework at Institution
Level
McKinnon, K R., Walker, S H., & Davis, D.
(2000).
Benchmarking: A manual for
Australian Universities
.
Canberra, ACT:
Higher Education Division, Department of
Education, Training and Youth Affairs.
February 2000.
http://
www.dest.gov.au/archive/highered/otherp
ub/bench.pdf
Quality Framework at Institution
Level - Another Framework
Quality Improvement Framework
Inglis, A., Ling, P., and Joosten, V. (2002).
Delivering Digitally: managing the
transition to the knowledge media
(2nd
Choosing Between the Frameworks
The two frameworks have slightly different purposes:
The
Benchmarking Framework
was intended to enable universities
to make comparisons across the full range of institutional functions.
The
Quality Improvement Framework
has a focus on the aspects of
an institution’s functions that are concerned with teaching, learning,
and student support.
For more information see:
Inglis, A.
(
2005).
Quality Improvement, Quality Assurance, and
Benchmarking: Comparing Two Frameworks for Managing
Quality Processes in Open and Distance Learning
.
International
Review of Research in Open and Distance Learning, Vol 6, No1 March
2005.
Quality Framework at Course
Level
Michigan Virtual University (2002).
Standards for quality online courses
.
More Information on Quality
Michael A. Mariasingam
michaelmariasingam@yahoo.com
Consultant
Quality Learning Global
USA
