Programme Review

DBS manages its own process of programme review under agreed Terms of Reference with QQI. The process is managed in accordance with ‘Provider Monitoring Policy and Procedures - HETAC October 2010’.

The objective of a Programmatic Review is to review the development of programmes over the previous five years, with particular emphasis on the achievement and improvement of educational quality. The focus is principally on the evaluation of quality and the flexibility of the programmes’ responses to changing needs in light of the validation criteria (Section 3 of QQI’s Core Validation Policy and Criteria 2010) and relevant awards standards. Programme Review is a peer review process.

The Objectives of Programme Review are to:

● Analyse the effectiveness and efficiency of each validated programme, including detail of learner numbers, retention rates and success rates

● Review the development of the programmes in the context of the requirements of employers, industry, professional bodies, the Irish economy and international developments ● Evaluate the response of the provider/school/department to market requirements and

educational developments

● Evaluate the feedback mechanisms for learners and the processes for acting on this feedback ● Evaluate the physical facilities and resources provided for the provision of the programme(s) ● Evaluate the formal links which have been established with industry, business and the wider

community in order to maintain the relevance of its programmes

● Evaluate feedback from employers of the programmes’ graduates and from those graduates ● Review any research activities in the field of learning under review and their impact on

teaching and learning

● Evaluate projections for the following five years in the programme(s)/field of learning under review

● Make proposals in relation to updating programmes and modules; proposals in relation to the discontinuation of programmes and the development of new programmes.

51

DBS has the opportunity to have other initiatives taken into consideration at review: ● The introduction of a new programme structures

● The revision of credit weightings ● Approval for a new minor award

● Approval of new centres for delivery (Locations, resources, proposed intakes, etc.) ● The addition of new delivery modes

Programmatic review is ultimately a structured self-evaluation undertaken by the School, whereby the school conducts a critical evaluation of its own activities at programme level and produces a self- evaluation report. Programmatic review normally takes place every five years, but may be undertaken more frequently if circumstances require.

4.4.1 Procedures for Programme Review

Programmes are monitored continuously but the process of Programme Review starts with the agreement of Terms of Reference with QQI. This includes agreement of Peer Review Panel membership, a timeline for the review process and the date of the panel visit.

4.4.1.1 Self-Evaluation Process

The DBS approach to Programmatic Review includes a wide-ranging consultative process, reflection and a series of approval procedures involving all stakeholders. The self-evaluation process is designed to:

● Provide opportunities for reflection on the operation of programme boards and meetings ● Provide opportunities for consultation with learners, employers, staff and external

stakeholders

● Contribute to the strategic plan of the provider and strategic management

● Provide information on strengths and weaknesses, in respect of all aspects of programme provision

● Identify the future direction for the programmes/fields of learning and/or the profession under review

● Identify future challenges

● Identify and eliminate inefficiencies and overlaps between programmes ● Include a review of external examiner reports and actions taken on same ● Provide for an analysis of recruitment statistics, attrition rates, pass rates etc.

52

● Include the review of all programmes and benchmarking these against best practice ● Identify and address resource issues, both physical and human

● Incorporate a review of the operation and effectiveness of current quality assurance procedures.

To ensure that:

● programmes remain relevant to learner needs, including academic and labour market needs ● quality improvements are made to programmes in a timely and effective manner

The self-evaluation process is the responsibility of the relevant Head of School and appropriate PL. The PL is responsible for progression of the review process and for bringing detailed review proposals forward for approval by the relevant school. The PL chairs meetings of the Programme Team, i.e. all of the lecturers/examiners involved in delivery/assessment of modules under the currently approved structure, to consider the outcomes of the consultative processes described below and over a series of meetings, develop draft proposals for consideration by the various parties involved in the approval process.

The self-evaluation report should include the following: ● A statement of its strategic objectives

● A review and critical analysis of the quality systems and processes which are in place to enable the achievement of its objectives

● The views of teaching staff, past and current learners, administrative staff servicing the programme, views of other staff that have any association with the programme or those involved with/on the programme

● An analysis of its strengths and weaknesses

● The identification of potential opportunities and threats, together with the possible actions to be taken

● An analysis of the success of the programme to date, including access statistics, performance of learners at each stage (including grade profiles and trends), completion rates by stage, graduate performance, etc.

● The identification of resources required for the delivery of its programmes ● A review of reports from programme boards and student feedback forms ● A review of employment/advancement opportunities for learners

53

● A review of the teaching, assessment and learning strategy of the provider in the relevant field

● A review of the assessment strategies for each programme

● An analysis of all research activity within the unit and future plans in that regard

● A review of its links with employers, industry, professions, the business and wider community

● Detail of programme changes proposed and the rationale for same ● A review of all modules included in the programmes under review ● Proposed Programme Schedules, incorporating the proposed changes.

The resulting Self-Evaluation Report (SER) is reviewed and approved by: ● The School Executive Board

● The Academic Board

Following completion of the internal review and approval process, the responsibility for the co- ordination of the engagement with the external validating body becomes the responsibility of the Registrar’s Office. The activities include:

● ensuring appropriate programme review documentation has been prepared and gone through the appropriate approval mechanisms

● organising an internal evaluation by a panel of assessors ● liaising with the panel members to arrange the Panel visit ● liaising with validating body

4.4.2 External Programme Review Evaluation for QQI Accredited Programmes

The Registrar’s office co-ordinate the formal evaluation process which is conducted by an independent Peer Review Group (PRG) and is held in DBS. Membership of the PRG should comprise experts from relevant fields of learning, who are capable of making national and international comparisons with regard to the specific suite of programmes.

At a minimum it should comprise the following:

● Chairperson – experienced in higher education and training; preferably with knowledge of Programmatic Review

● Secretary

● Academics (minimum 2), experts in relevant field of learning ● Representatives from industry/relevant profession

54 ● Learner representative

● QQI representative if appropriate

● A member of a relevant professional body if appropriate and by agreement with QQI

Other factors to be considered include, gender balance and international input, in particular the possibility of an academic from another jurisdiction.

PLs provide nominees for the peer review panel to the Registrar. These are agreed with the QQI in the Terms of Reference. Each panel member returns a conflict of interest form to the Registrar to ensure their independence of the evaluation.

The PRG are provided with the following documentation prior to the visit: ● Agreed Terms of Reference

● Programme Self Evaluation Document

● Provider Monitoring Policy and Procedures - HETAC 2010 ● Covering letter- including local arrangements maps etc.

● Participating in an Evaluation Panel as an Expert Assessor: Guidelines - HETAC 2009 ● Conflict of interest form to be signed and posted

● Expenses Claim Form

The role of the Peer Review Group is to evaluate the extent to which the programme review objectives have been fulfilled in the internal process. The specific responsibilities of the PRG are to:

● Study the self-evaluation report

● Visit the provider and meet with teaching staff, learners (past and present if possible), administrative staff, employers and any other category of internal and external stakeholders ● Clarify and verify details in the SER

● Consider how well the identified aims and objectives of the provider are being met

● Consider programme changes proposed in the context of all other information provided and recommend acceptance or otherwise of the proposals

● Consider the quality assurance arrangements which affect the programmes under review ● Present its findings at the end of the visit

● Prepare a report on the findings of the PRG, to include recommendations for the provider in respect of the suite of programmes under review

55

On completion of the external evaluation, the panel submits a draft Programmatic Review report to the Registrar for comment. The report should address the quality of the provision and make recommendations for improvement, and/or change, based on a combination of the SER and findings during the site visit and meeting with relevant stakeholders. The report should also make recommendations in respect of the continuing validation of the programme(s) which are the subject of the review:

● Recommendation for approval without conditions ● Recommendation for approval with conditions ● Referral for resubmission

The maximum period of validation and re-approval in line with QQI regulations is usually five years. The Programme Team is given an opportunity to review and comment on the Programmatic Review report, to amend inaccuracies, and to identify and take action where necessary. All internal feedback and intended actions are formally documented by the Programme Team, and communicated to the PRG.

The chair of the PRG then produces a final Programmatic Review report and documents the consensus of the PRG. This report is forwarded to QQI, together with the Programme Team response to the report, the Proposed Programme Schedule (T4.1) and the Programme Document.

QQI may make further recommendations in light of the documentation received from DBS. The Registrar’s Office within DBS is responsible for ensuring all required actions are implemented in a timeframe agreed with QQI.

Once approved by the QQI Programme and Awards Executive Committee (PAEC), QQI will send a letter confirming re-validation. This is followed by a Certificate of Programme Accreditation and an Approved Programme Schedule. These documents are checked by the Registrar’s Office and the accuracy confirmed with QQI.

The following activities are carried out to facilitate effective implementation of the revised programme:

● The Directory of Programme Validation is updated.

56

● The Certificate of Programme Accreditation is held by the Registrar and an electronic version made available on the S:Drive.