DEFINITIONS

Program Review

Regular review and evaluation of academic programs to evaluate the quality of the curriculum and learning experience against established MTCU program and quality assurance standards, and make recommendations for future program modifications. 2. Scope

Subject to availability of resources, each program will normally be reviewed fully and formally every 7 to 10 years as outlined below.

The comprehensiveness of program review for certificate, diploma and degree programs may vary to be consistent with relevant standards (such as, the Post Secondary

Education Quality Assessment Board (PEQAB) requirements for applied degree programs and diploma program standards).

3. PROCESS

The program review process will include ongoing formative assessment and intervention as appropriate and as described in the 3 Tier process below (see Chart 1, on page 8 of Appendix 11.1).

Tier 1 – Early Identification System – this includes:

i) ongoing and at least annually, reports on performance indicators (primarily quantitative data) provided by the staff of Institutional Research to the program chair

ii) if the performance indicators remain above or at acceptable levels, no further action is taken until the next Tier 1 review

iii) any performance indicators that drop below acceptable ranges will be flagged by the chair and the review process will be moved to Tier 2 or Tier 3 review as appropriate

iv) the chair will report a summary of the findings of the Tier 1 review to the dean and faculty (each Fall)

Tier 2 – Probing of indicators identified at Tier 1 - this includes:

i) exploration of factors influencing the drop in performance indicators ii) examination of both quantitative and qualitative data as appropriate iii) appropriate and timely response to address the issue(s) identified

iv) if the performance indicators in subsequent Tier 1 reviews remain at or above acceptable levels, no further action is required

v) the chair will report a summary of the findings of the Tier 2 review to the dean and faculty (each Fall)

vi) if the program appears sustainable and it is deemed appropriate by the chair and dean in collaboration with faculty, the review process may be moved immediately to a Tier 3 full program review (i.e., not wait until the normal 7 year review) Tier 3 - Full Program Review:

i) Program Review Teams

a) Every academic program will establish a review team(s) responsible for overseeing the program review and writing the final report of their findings

and recommendations.

b) The membership and committee size will vary according to program nature, needs, locations and delivery modes.

c) The review team will ensure that the review achieves the purpose and intent of the College’s Quality Assurance policies.

d) Internal Review Team for Program Self-Assessment The internal review team will typically include:

! the School's chair or designate

! one or two professors from the program under review

! one professor from the School of English and Liberal Studies

! at least one member external to the program or the college as appropriate • (e.g., a professor or chair from another school or a relevant expert

from outside the college who has no conflict of interest) ! one graduated student ,if feasible

! one representative of Professional Development (CPD), Institutional Research (IR), and the Office of Research & Innovation (ORI) if feasible • one member of the team will be chosen to be the team leader

In addition, a Learning Resources Subcommittee will review and assess the quality of program specific learning resources available through the Library and Learning Commons and provide feedback to the Internal Review Committee: Membership consists of:

• One faculty librarian from the Library and Learning Commons • One professor who is a member of the internal review team e) All members of the review team will receive training in the Quality Assurance

process to be provided collaboratively with the Centre for Professional

Development (CPD), Institutional Research (IR), and the Office of Research & Innovation (ORI).

f) Throughout the review process, centralized support, guidance collection tools will be provided to the review team through CPD, IR and ORI.

g) The program review team will report regularly on their progress to their faculty and staff, to the appropriate chair(s), and to the program advisory

committee.

h) The program review team will establish a communication plan and ensure that all stakeholders, including the program advisory committee are informed on an ongoing basis

i) External Review Team

In addition and as appropriate (e.g., for degree programs) the program review process will include an external review panel consisting of at least

• 3 external academic peers with relevant expertise and free of any conflict of interest. Academic peers may be peers from similar programs outside the College or representatives from relevant accrediting associations and related industries.

The external review will include a site visit.

j) Where an appropriate external accreditation or program recognition process is available, this may be used in lieu of the College's program review process. ii) Data Collection

The Data Collection Template/Worksheet (Appendix A) lists the Elements and Key

Questions to be addressed by the data collected. The program review will include, but will not be limited to the following sources of data collected by the review team:

a) comments and recommendations from faculty teaching in the program, from relevant staff, and from advisory committee members

b) annual performance indicators

c) focus group sessions, interviews or other means as identified by the Note:

Appendix A is available upon request.

review team to ensure effective input from students and graduates. d) consultations with library services staff and others as appropriate

e) as appropriate:

• comments solicited from employers in addition to input from the program advisory committee

• professional organizations

• partners such as universities or colleges participating in collaborative programs with Seneca

• and/or other groups or individuals able to provide perspective and valued advice on program reputation and direction

f) data collected using tools to be provided by Institutional Research and the Office of Research & Innovation

g) with the written approval of the students involved, blinded samples of student work that are representative of the various levels of quality of student work. For summative courses, these samples are particularly

important.

iii) Program Review Report

The Project Team is responsible for authoring the final public report. The report will include

• Title Page – including:

o the name of the program and school o the names of review team

o identification of external review panel member(s) if appropriate o the date of the report

• An Executive Summary - one to two pages, providing an overview of the program's key strengths and areas for improvement

• Responses to Key Questions - this is the main body of the report and includes: o the main body of the report should be a summary of the findings related to

the Key Questions

o Appendix A is a template is to be completed and attached to the report o perspectives of employers, partners, graduates, students, faculty and staff • The Program Profile - each element identified in Appendix A rated according to

strengths and areas for improvement as follows: A - significant strength

B - strength

C - needs some improvement D - needs significant improvement N/A – not applicable

• Recommended Action Plan – summarizes:

! 5-10 specific recommended actions ordered by priority ! supporting narrative (if required)

! proposed timelines for implementation of the recommendations ! budget implications

• Appendices: - available upon request

! Appendix A – Completed Data Collection Template/Worksheet ! Appendix B – Samples of additional survey tools used

! Appendix C – Supporting data

i) The Program Review team shall present to the chair/dean and VP Academic (within a reasonable time as negotiated with the program chair), the final written Program Review Report (described above). This may include a verbal presentation.

ii) The VP Academic will present the review team’s public report to the Academic Planning and Student Affairs sub-committee of the Board of Governors (BOG). iv) Follow-up to the review and the report

i) The Vice President Academic (in collaboration with the faculty and administrative and support staff of the program assessed), will make the decisions and resource allocations (along with implementation timelines) in developing local

operational/action plans to respond to the recommendations made by the review team.

ii) The program chair is responsible for providing to the dean and the VP Academic, regular reports on the progress of the implementation of the approved local operational/action plan(s)

iii) Suggested Timeframe – (may be adapted as appropriate to the specific program) a) September to May – focus is building a portfolio of relevant data using data and tools provided and collected centrally through

Institutional Research

b) January to May – Review Team is formed and trained

c) May-June (8 weeks) – Review Team reviews the cumulative data, completes Appendix A (web based so that the team can work on this

collaboratively), assesses the findings and make recommendations. Some reassignment of May-June/July-Aug vacations may need to be negotiated to free faculty for participation in the review team

Chart # 1

Has it been 7 years since the last formal

review (Tier 3)? Tier 1 Review (Annual) End Obtain Performance Indicators Report from IR Have performance indicators passed acceptable range ? Tier 2 Review Evaluate issues Can issues be Immediately addressed ? Tier 3 Review (Every 7 years or more frequently as required) Assemble program review team

Receive training and support from CPD , IR

and ORI

Collect data - refer to Data Collection/ Template Worksheet Implement Recommendation Plan Submit Program Review Report to Chair/Dean & VPA

Follow up on Recommendation Plan Address issues

Submit Summary Report to Dean &

Faculty

Has program been terminated ? No Yes Yes No No No Yes Yes