Drafting Program Educational Objectives for Undergraduate Engineering Degree Programs

(1)

Drafting Program Educational Objectives for

Undergraduate Engineering Degree Programs

Ramakrishnan Sundaram Gannon University, [email protected]

Abstract - This paper outlines the process to draft Program Educational Objectives (PEOs) appropriate for undergraduate engineering degree programs at ABET accredited institutions of higher education. In the ECE department at our University, the existing PEOs were deemed to have language that was very similar to that used in the ABET student outcomes. Therefore, it was imperative to distinguish the PEOs from the ABET student outcomes since the PEOs must quantify the expected attainments of graduates a few years after graduation. First, PEOs must reflect the Mission Statement of the institution and serve as a yardstick of student achievement three to five years following graduation. The objectives represent the expectations of the department from its graduates. Active participation by the faculty in defining the PEOs yield clear and concise objectives and promotes ownership of the goals of the Department and ABET process. However, not all faculty members are necessarily familiar with the assessment language and the process to evaluate the PEOs. In order to ensure a meaningful contribution from all faculty members involved in defining the PEOs, this paper presents a framework to define the PEOs that (1) adhere to the Mission of the University (2) achieve consistent and measurable expectations.

Index Terms Program Educational Objectives, Accreditation, Assessment, Bloom taxonomy

INTRODUCTION

Program Educational Objectives (PEOs) for undergraduate engineering degree programs must clearly defme the vision of the program for its graduates three to five years after graduation. These objectives represent the expectations of the department from its graduates and must be crafted such that measurable objective evidence can be obtained through alumni surveys. In addition, there must be in place internal and external review procedures to periodically review and revise the language, assessment, and evaluation of the PE�s. The focus of this paper is on defming the process to draft the PE�s. The framework that we propose is first to provide an assessment language that is similar to the cognitive learning pattern of Bloom's Taxonomy for Knowledge, Comprehension, Application, Analysis, Evaluation and Synthesis during the different stages completed by our graduates in their career path. Faculty members should recognize that the students in the

undergraduate degree program are viewed as learners who must acquire the intended skills over the duration of the baccalaureate program. These students are then expected to demonstrate the application of these skills three years following graduation. They are expected to asswne more responsibilities in management or to lead in their profession and/or society six years after graduation. Ten or more years after graduation, these students should become contributors with significant influence on policy and decision making in their chosen profession and/or society. The assessment language at these different stages of our graduates' career path is different. Hence, it is important to use the proper set of assessment language intended for the PEOs that focuses only on the immediate or first three years to measure the direct impact of our program on young careers, not the six year or the ten-year longer term objectives which would measure learning experiences and contributions beyond the direct impact of our program. Second, the framework requires faculty to extract key words from the Mission Statement of the institution which emphasize the broad and specific intent of undergraduate education along with a set of key words that the faculty see as attributes to gauge graduates in becoming the individuals the PEOs intended. Third, capture the distinction between educational objectives and student outcomes, and formulate each objective recognizing the need for measurability.

The faculty of the Electrical and Computer Engineering (ECE) department at our institution met on different occasions for discussion and a final work session for almost six hours to complete the steps of the process in order to revise the existing set of PE�s. The PEOs of our fully accredited undergraduate ECE program (accreditation period from 2005 to 2011) were deemed to have language that was very similar to that used in the ABET student outcomes. Therefore, it was imperative to distinguish the PEOs from the ABET student outcomes since the PEOs must quantify the expected attainments of graduates a few years after graduation. For instance, the ECE program at one pre-eminent institution [2] develops the PEOs according to three expected outcomes from graduates five years after graduation - professional expertise, innovativeness, and leadership. At another institution [3], continuous education program assessment is addressed by bringing together a working team of departments and education specialists using a Web-based assessment process to gather and evaluate the data. There are also institutions that stipulate

(2)

the PEOs of the program without empowering the faculty to develop, own, and adopt a consistent set of measurable PEOs.

Section 2 provides a broad overview of the PEO assessment cycle at our institution with internal and external review processes. Section 3 describes the framework to formulate the PEOs. Section 4 discusses the PEO assessment and evaluation process, summarizes the mapping of these PEOs to the ABET student outcomes, and the preparation of the alumni survey. The conclusions and future issues appear in Section 5.

SECTION 2: PEO ASSESSMENT CYCLE The following requirements are stipulated by ABET in its Self-Study Questionnaire [1] as part of Criterion 2. Program Educational Objectives.

A. Mission Statement of the Institution B. Program Educational Objectives (PEOs)

C. Consistency of the PEOs with the Mission Statement D. Program constituencies

E. Process for revision of the PEOs

In order to fulfill the requirement of consistency between the PEOs and the Mission Statement of the institution, key words and phrases are identified in the Mission statement that would directly relate to the language used to formulate the PEOs. For example, the following Mission Statement of our institution has words and phrases underlined to indicate their significance in the drafting of PEOs.

This Institution is a Catholic, Diocesan, student-centered University which provides for the holistic development of undergraduate and graduate students in the Judeo Christian tradition. As such, it offers each student outstanding teaching and a value-centered education in both liberal arts and professional specializations in order to prepare students for leadership roles in their careers, society, and church. The University faculty and staff are committed to excellence and continuous improvement in teaching, learning, scholarship, research, and service. The University's environment is to be one of inclusiveness and cultural diversity.

Figure 1 displays the overall Quality Assessment cycle with Level 1 representing the PEO Assessment highlighted. as the focus of this paper. The assessment cycle for Level 1 is once every three to five years to gauge the relevancy of the PEOs to the requirements of the program constituents because the PEOs are measured three to five years after graduation. Student outcomes assessment, identified as Level 2, is in a different cycle but provides the major input to address the attainment of the PEOs. The Level 1 PEO Assessment cycle is shown in Figure 2.

Levell: PEO Assessment

Level 2: SO Assessment Level 3: CO Assessment FIGURE I

QUALITY ASSESSMENT CYCLE

Constituents (EAC) Faculty

defines/refines _r- plan/refines PEO PEO & performance criteria

1

Faculty _Committee

make decisions for

I- analyzes evidence

changes or

implementation and generate action items

Assessment cycle: 3-5 years FIGURE 2

�

LEVEL I PEO ASSESSMENT CYCLE

Collectevidence

forPEO

The program constituents for the PEO assessment process comprise the faculty, staff, students, and the local industry represented by its members on the Engineering Advisory Council (EAC). The toolsets used in the PEO assessment process are as follows:

• Alumni survey (external) • EAC review (external)

• Executive summary report for student outcomes (internal)

SECTION 3: FRAMEWORK TO FORMULATE PEOs The first step was that faculty members as individuals had independent PEO ideas, leveraging our multi-cultural and multi-discipline resources. Then, the faculty members, objectively as a team, had long workout sessions to revise and commit to these new PEOs, thus promoting PEO ownership as a Department. At the beginning of the workout session, the faculty identified and agreed to the following keywords of the Mission Statement of our institution

(3)

-professional development, value-centered, leadership roles, cultural diversity, and lifelong learning. In addition, the faculty identified the categories in which objective evidence shall be obtained to measure each key word keeping in mind the proper set of assessment language. Faculty members then formed break-out groups of two to draft the PEO suitable for each key word. Finally, the faculty reconvened as one group to review and revise each objective as well as to arrange the objectives in the order of chronological impact and accomplishment before reaching agreement for adoption.

The framework for the review and revision of the PEOs at the departmental level involving all the faculty members comprises the following broad stages.

1. Provide the assessment language

2. Use the key words and phrases extracted from the Mission Statement of the institution to identify attributes to gauge graduates

3. Capture the distinction between the educational objective and the student outcomes

4. Formulate each objective to be measurable

The assessment language is patterned along the lines of the Cognitive domain of Bloom's taxonomy [4]. The Cognitive domain (or What should I know?) corresponds to the recall or recognition of specific facts, procedural patterns, and concepts that serve in the development of intellectual abilities and skills.

There are six major categories from simple to complex -Knowledge, Comprehension, Application, Analysis, Synthesis, and Evaluation - to be mastered. Although these categories are covered within every course at all levels (first year to senior year), the focus of the pattern of learning at each level is as follows:

TABLE I FOCUS OF LEARNING

Level of Course Focus of learning

First year Knowled�e

Second year Comprehension

Third year Application & Analysis Fourth year Evaluation & Synthesis

SECTION 4: PEO ASSESSMENT & EV ALVA TION Figure 3 illustrates the components and the flow of the process of PEO assessment and evaluation. The two primary approaches to PEO assessment are (1) internal review, and (2) external review.

The internal review consists of assessing the attainment of student outcomes, through senior exit surveys and

embedded course assessment processes, to determine if the skill sets necessary for attainment of PEOs are being provided within each course and across the curriculum. The external review comprises the assessment of data gathered through the alumni survey to determine how well our graduates are performing in the workforce three to five years following their graduation. This measures the level of attainment of each PE�.

The PEO evaluation process determines the relevancy of each PEO to address and successfully meet the needs of the program constituents. The program constituents are faculty, students, alumni, and industry. The evaluation of the PEOs by both the EAC and the faculty members helps address the needs of program constituents. Action items generated at faculty and EAC review meetings are expected to lead to changes in the PEOs and have significant impact on the shaping of our educational practices.

1---

---Attainment

ChangesloPEOs

Changes to Educational Practices Pteparednessof Students'skillsets FIGURE 3

PEO ASSESSMENT AND EVALUATION PROCESS The mapping of the student outcomes to the PEOs is intended to help us determine if our educational practices within each course and across the curriculum prepare our graduates to meet the intended PEOs. Table II illustrates the mapping.

The PEOs used to complete this mapping are as follows: PEO 1. Sound preparation for adaptation in the exciting,

rapidly-changing areas of technology and a passion for lifelong learning

PEO 2. Ability to understand how engineering solutions affect our society and respond to ethical and public issues, including safety, social, and environmental concerns.

(4)

PEO 3. Ability to apply personal values to daily and professional life, develop the skills necessary for exercising infonned literary and aesthetic judgments, and appreciate diverse cultures and societies

PEO 4. Ability to communicate effectively in both oral and written forms and work efficiently with multi disciplinary/multi-cultural teams which foster leadership qualities

PEO 5. Foundation in problem formulation and problem solving skills to include the following:

• _{Sound preparation in general science and} applied mathematics

• Strong preparation in electrical engineering and applications

• _{Strong preparation in computer and software} systems development

• Effective use of computer-aided design and analysis tools

• Quality experience in engineering design TABLE II

RELATIONSHIP OF STUDENT OUTCOMES TO PEOs

Student Outcomes - N rr, ..

0 0 0 0

'" '" '" '"

� .� � �

ABEICrit"ia:

SO 1: Ability to apply knowledge of mathematics, science, and _X

engineering

S02: Ability to design and condoct experiment, as well as to analyze and inteljlret data

S03: Ability to design a system, component, or process to meet _X desired needs

S04: Ability to function on multi-disciplinary teams X X S05: Ability to identify, formulate, and solve engineering problems X

S06: Understandin,. of professional and ethical responsibility X X S07: Ability to commWlicate effectire1y X S08: Board education necessary to tulderstand the impact of _X _X _X _X

engineering solutions in a global and societal context

S09: Recognition of the need for, and an ability to engage in life-long _X _X learning

SOlO: knowledge of contemporary issuts X X X

SO 11: Ability to use the techniques, skills, and mndero engineering _X

tools necessary for engineering practice ECE Proer.m-Sp,cific Comp,t,nci",

S012: Knowledge and ability to apply mathematics including _X probability, statistics, and discrete mathematics

SOl3: Develop systems containing hardware and software _X

components

SO 14: Analyze and Design of complex electrical and electronic _X devices ,n 0 _'" � X X X X X X X X The language of the PEOs listed above does not make the clear distinction between PEOs and ABET outcomes. The implementation of the process described in this paper resulted in the following set of PEOs.

PEO 1. Demonstrate professional ethics and personal values in daily and professional life that exercise infonned literary and aesthetic judgments by leveraging diverse cultures and societies

PEO 2. Demonstrate teamwork and leadership qualities and/or the attainment of leadership roles in a global work environment

PEO 3. Demonstrate technical competency in applying comprehensive engineering knowledge throughout their chosen profession

PEO 4. Demonstrate passion for life-long learning through engaging in the rapidly changing and emerging areas of technology, and/or continued professional development

This set of PEOs is also mapped to the ABET outcomes as in Table II. Table III summarizes this mapping.

TABLE III

SUMMARY OF REVISED MAPPING

PEO# Student Outcomes

1 S06, S08, SO 1 0 2 S04, S07

3 SOI-3, S05, S08, SOII-14 4 S01-3, S05, S07, S08-14

The alumni survey provides the data for assessment of the PEOs as part of the external review. The survey addresses the following question:

How well are our graduates really doing in the workforce? The survey is administered through secure access on the Web [5] and has quantitative and qualitative sections in the following categories.

• General information • Employment History

• _{Preparedness of Gannon Education} • Soft Skills and Broad Education • Liberal Studies

• Intern Experience and Professional Societies • Rate education based on the PEGs

Table IV lists the questions asked in the quantitative section of the alumni survey for the category titled Preparedness of Gannon Education.

(5)

TABLE IV QUANTITATIVE SECTION - I

Preparedness of Gannon Education

Questions

How well do you feel your undergraduate training helped you to develop

the ability to apply technical knowledge in the solution of specific work