Elements of program evaluation for administrators

(1)

Program

Evaluation:

The

Administrator’s

Role

Casey Self Arizona State University

February, 2013 Special Thanks to Maura Reynolds and Sallie Paschal for their

significant contributions to this presentation.

NACADA Executive Office Kansas State University 2323 Anderson Ave, Suite 225

The contents of all material in this presentation are copyrighted by the National Academic Advising Association, unless otherwise indicated. Copyright is not claimed as to any part of an original work prepared by a U.S. or state government officer or employee as part of that person's official duties. All rights are reserved by NACADA, and content may not be reproduced, downloaded, disseminated, published, or transferred in any form or by any means, except with the prior written permission of NACADA, or as indicated below. Members of NACADA may download pages or other content for their own use, consistent with the mission and purpose of NACADA. However, no part of such content may be otherwise or subsequently be reproduced, downloaded, disseminated, published, or transferred, in any form or by any means, except with the prior written permission of, and with express attribution to NACADA. Copyright infringement is a violation of federal law and is subject to criminal and civil penalties. NACADA and National Academic Advising Association are service marks of the National Academic Advising Association.

The Global Community for Academic Advising

The purpose for the

Academic Advising Program must be:

•Established

•Communicated

•Validated

as an essential function for student

success.

The Global Community for Academic Advising

Elements

of

program

evaluation

for

administrators

The Global Community for Academic Advising

The Global Community for Academic Advising

•

Vision

•

Mission

•

Goals

•

Objectives

•

Outcomes

Without

these,

there

is

no

baseline

for

assessment

or

evaluation.

The Global Community for Academic Advising

Developing a Vision

Statement

Should reflect the aspirations of

what academic advising can be on

a campus (or unit). It represents a

desired or ideal future.

Campbell, 2008, p. 235.

(2)

Developing a Mission Statement

• It should line up with the Mission Statement of the

Institution.

• What is the purpose of academic advising? • What does this college/unit value about academic

advising?

• It is the critical driving force to the overall assessment plan.

• It defines academic advising on a specific campus or unit.

• It should be displayed throughout the office.

Developing Goals and

Objectives

•Must support the Mission Statement

•Will guide the Learning Outcomes

The Global Community for Academic Advising

Learning Outcomes

Learning outcomes are what we

expect students to know, do,

value, and appreciate as a result

of their academic advising

experience.

The Global Community for Academic Advising

What should students LEARN

through advising?

9How and when should they learn these things?

9Why are these learning goals important to students and to your institution?

9How can advisors and administrators (& students themselves) evaluate whether students have learned these things?

9How can advisors and administrators create the best context for these learning goals?

The Global Community for Academic Advising After the mission statement is created and after learning objectives are set, what’s next?

¾Are these learning goals reasonable? •Limited in number?

•Achievable given advising loads, time, other responsibilities?

¾How will we communicate goals to students & advisors?

¾Do we need to examine the delivery of academic advising?

The Global Community for Academic Advising

Needs Assessment

Tools for constructive and positive

change that is rational, logical and

functional transformation – not

change driven by controversy,

quick fixes and situational crises

(3)

Needs Assessments can be

Proactive or Reactive

•A reactive needs assessment may be

conducted to determine why retention rates

are poor

•A proactive needs assessment can be

conducted before any negative situations or

problems develop to identify future

programming concentrations

After needs are determined,

data must be gathered

•Existing Data approach

•Student-Attitude Survey Approach •Key Informant Approach

•Student Forum •Focus Group Interviews

•No instrument or tool is perfect, so plan for multiple measures.

•Plan for how you will respond to the date before you get it.

The Global Community for Academic Advising

Questions about advising on NSSE and CCSE surveys _{The Global Community for Academic Advising}University of Minnesota, Morris

How

often

have

you

met

with

your

adviser?

(

Choose

one

)

__During orientation only

__During orientation and before registration for spring

__Additional times __Never

3.

My

adviser

is

on

time

for

appointments.

4.

My

adviser

keeps

regular

office

hours.

5.

My

adviser

takes

time

to

become

acquainted

with

me.

The Global Community for Academic Advising 6. My adviser discusses my academic goals and ways to meet them.

7. My adviser provides me with information about academic requirements and choices.

8. My adviser seems interested in helping me succeed academically.

9. My adviser refers me to other staff or offices when necessary.

10. My adviser explains what is expected of me as an advisee.

11. My adviser explains what I can expect from my adviser.

The Global Community for Academic Advising

University Advising Center, University of Kansas

•The front desk staff effectively assisted me and was informed

about services and policies.

•I was prepared for my advising session.

•I understand that advising is a shared responsibility between

me and my advisor.

•The time I spent with my advisor was sufficient to discuss the

questions I had.

•My advisor was able to answer my questions or referred me to

someone who could.

•My advisor communicated effectively and understood my

needs.

•At the end of my appointment, I understood what I needed to

(4)

After needs have been

assessed, an advising

program has been developed

and implemented, the

program must be evaluated.

Program evaluation should

measure the effect of specific

changes on an academic

advising operation.

The Global Community for Academic Advising

Administrators should make it

(Program Evaluation) an

established and recurrent

process such that long-term

changes to academic advising

are identified and measured.

The Global Community for Academic Advising

The purpose of Program

Evaluation is to ensure the

continuous improvement of the

unit as it relates to the

institution’s goals, mission and

strategic indicators.

The Global Community for Academic Advising

The following questions should

be addressed

:

•How efficient is the unit?

•Are students receiving the assistance they need? •How high is their satisfaction?

•How is effective practice measured? •Is the advisor/advisee ratio appropriate? •Are advisors trained at the level of experience needed?

•What plan is in place for continuous professional development?

The Global Community for Academic Advising

In summary, through Program Evaluation,

Academic Advising Administrators can

demonstrate to the institution the ways

that particular advising interventions have

positively affected:

Student GPAs

Length of time to graduation

Persistence of students

Learning that has occurred

(5)

NACADA Resources

• NACADA Concept of Academic Advising

• Council for Advancement of Standards (CAS)

• NACADA Clearinghouse

• NACADA Commissions

• NACADA Consultants and Speaker Service

Concept of Academic Advising

The Global Community for Academic Advising

CAS Standards

The Global Community for Academic Advising

NACADA Clearinghouse

The Global Community for Academic Advising

NACADA Commissions

The Global Community for Academic Advising

NACADA Consultants and

Speaker Service

• Individual or Team of consultants

• Visit campus & provide report on their assessment of your campus/unit advising program.

• Speakers and Workshops

• Address specific topics or campus issues. • Professional development for campus

advisors.

(6)