Assurance of Learning for the Fisher College of Business at The Ohio State University Master s of Business Administration

(1)

Assurance of Learning for the

Fisher College of Business at The Ohio State University

Master’s of Business Administration

Draft version 10.1- November 7, 2008

Karen Wruck, Associate Dean, MBA Programs Leslie Fine, Academic Director, Executive MBA

(2)

TABLE OF CONTENTS

I. MISSION STATEMENT ... 1

II. LEARNING GOALS ... 1

A. DEVELOPMENT OF LEARNING GOALS... 2

1. Learning Goal 1: Graduates are able to apply their knowledge & skills to solve business problems ... 3

2. Learning Goal 2: Graduates have a global perspective and an awareness of how cultural differences impact business... 3

3. Learning Goal 3: Graduates are able to work and lead effectively in a team-based environment... 4

4. Learning Goal 4: Graduates demonstrate professional deportment, self-awareness, polish and effective communication skills ... 5

B. LOCATION OF GOALS WITHIN CURRICULUM... 6

C. RELATIONSHIP OF LEARNING GOALS TO MBA PROGRAM MISSION... 7

III. ASSESSMENT PLAN ... 8

A. THE OHIO STATE UNIVERSITY- FISHER COLLEGE OF BUSINESS FULL TIME MBA ASSESSMENT PLAN. 8 B. THE OHIO STATE UNIVERSITY- FISHER COLLEGE OF BUSINESS WORKING PROFESSIONAL MBA ASSESSMENT PLAN... 9

C. THE OHIO STATE UNIVERSITY- FISHER COLLEGE OF BUSINESS EMBA ASSESSMENT PLAN... 10

IV. LEARNING GOAL 1: GRADUATES ARE ABLE TO APPLY THEIR KNOWLEDGE & SKILLS TO LIVE BUSINESS PROBLEMS... 11

A. FULL-TIME TRACK... 11

1. Background ... 11

2. Assessment ... 11

3. Standard for demonstrating achievement ... 11

4. Results... 11

5. Analysis... 13

6. Closing the loop ... 13

B. WORKING PROFESSIONALS TRACK... 15

1. Learning Objective A: The Fisher Case Competition ... 15

2. Learning Objective B: Prior work experience ... 16

C. EXECUTIVE MBA TRACK... 18

1. Introduction ... 18

2. Learning Objective A: Students have at least seven years of significant managerial experience... 18

3. Learning Objective B: Students develop a business plan... 19

V. LEARNING GOAL 2: GRADUATES HAVE A GLOBAL PERSPECTIVE & AWARENESS OF HOW CULTURAL DIFFERENCES IMPACT BUSINESS ... 21

A. FULL-TIME AND WORKING PROFESSIONALS... 21

1. Background ... 21

2. Assessment ... 21

3. Standard of achievement... 21

3. Results... 22

5. Closing the loop ... 22

B. EMBA ... 23

1. Learning Objective A – Students will have international experiences as part of their program .... 23

2. Learning Objective B- Students integrate a diverse body of knowledge to address the issues faced by the multinational firm... 23

VI. LEARNING GOAL 3: GRADUATES ARE ABLE TO WORK AND LEAD EFFECTIVELY IN A TEAM-BASED ENVIRONMENT... 25

A. BACKGROUND... 25

(3)

C. SURVEY ADMINISTRATION... 26

D. DEVELOPMENT FOCUS... 27

E. STANDARDS OF ACHIEVEMENT AND RESULTS... 27

F. ANALYSIS... 28

G. CLOSING THE LOOP... 30

VII. LEARNING GOAL 4: GRADUATES DEMONSTRATE PROFESSIONAL DEPORTMENT, SELF-AWARENESS, LEADERSHIP, POLISH AND EFFECTIVE COMMUNICATION SKILLS31 A. FULL-TIME AND WORKING PROFESSIONAL... 31

1. Introduction ... 31

2. Learning Objective A: Graduates demonstrative effective communication skills... 31

3. Learning Objective B: International students demonstrate effective oral communication skills in English. ... 34

4. LEARNING OBJECTIVE C: GRADUATES DEMONSTRATE LEADERSHIP SKILLS... 35

B. EMBA ... 38 a. Introduction ... 38 b. Assessment ... 38 c. Standard of achievement ... 38 d. Results... 38 e. Discussion/analysis ... 40

f. Closing the loop ... 40

VIII. GOING FORWARD: CLOSING THE LOOP FOR THE MBA AT FISHER ... 41

IX. INDEX OF APPENDICES... 42

APPENDIX 8-LEARNING GOAL 4, OBJECTIVE D: STUDENTS EXHIBIT PROFESSIONAL DEPORTMENT AND COMMUNICATION SKILLS APPENDIX 1-MEMBERS OF MBA PROGRAM COMMITTEE... 42

APPENDIX 1-MEMBERS OF MBA PROGRAM COMMITTEE... 43

APPENDIX 2-FT MBA, LEARNING GOAL 1 CASE COMPETITION RUBRIC... 44

APPENDIX 3-LEARNING GOAL 1, OBJECTIVE B WPMBA PRIOR WORK EXPERIENCE... 45

APPENDIX 4-EMBA FOR LEARNING GOAL 1, LEARNING OBJECTIVE B: STUDENTS DEVELOP A BUSINESS PLAN WINTER 2008 DATA, CLASS OF 2008... 46

APPENDIX 5-LEARNING GOAL 2: EMBA INTERNATIONAL BUSINESS COURSE RUBRIC... 48

APPENDIX 6-EMBA LEARNING GOAL 2, LEARNING OBJECTIVE 2 SCORES ON TEAM PAPERS FOR 895X COURSE... 49

APPENDIX 7-RUBRIC FOR LEARNING GOAL 4, LEARNING OBJECTIVE A: GRADUATES DEMONSTRATE EFFECTIVE COMMUNICATION SKILLS... 50

APPENDIX 8-LEARNING GOAL 4, OBJECTIVE D: STUDENTS EXHIBIT PROFESSIONAL DEPORTMENT AND COMMUNICATION SKILLS... 52

(4)

TABLE OF TABLES AND FIGURES

Table 1-Location of Program Goals within MBA Curriculum Full Time and Working

Professionals Tracks ... 6

Table 2-Location of Program Goals within MBA Curriculum- EMBA Track ... 6

Table 3-Relationship of Learning Goals to MBA program mission... 7

Table 4-The Ohio State University- Fisher College of Business Full Time MBA Assessment Plan... 8

Table 5-The Ohio State University- Fisher College of Business MBA Assessment Plan for Working Professionals ... 9

Table 6-The Ohio State University- Fisher College of Business EMBA Assessment Plan ... 10

Table 7-Tally of 2008 First Year MBA Case Competition, WI08, Fulltime MBA Class of 2009 a ... 12

Table 8-Pilot Data Collection-Tally of 2008 First Year MBA Case Competition, Winter 2008 Working Professionals MBA Class of 2009 a ... 16

Table 9-Tally of Work Experience Assessment Scores at Admission ... 17

Table 10-Entering EMBA Students with 7 or more years of significant managerial experience ... 18

Table 11-Average Years of Work Experience of 3 most recent cohorts of entering EMBA Students... 19

Table 12-Estimated Data on International-related activities ... 22

Table 13-EMBA Global Integration Course Tally Sheet for Cohort EMBA 2007 Winter Quarter 2008 ... 24

Table 14-Full-Time MBA Class of 2009 N=141... 28

Table 15-EMBA Class of 2008 N=53 ... 28

Table 16-EMBA Class of 2007 N=44 ... 28

Table 17-WP Class of 2009 N=71 ... 28

Table 18-Working professionals enrolled in class fall of 2006 and fall 2007 ... 32

Table 19-Full Time MBA Students from classes of 2006, 2007 and 2008 ... 32

Table 20-Graduates demonstrate effective communication skills in an oral presentation Pilot Tally for 7 MBA teams ... 33

Table 21-Extraordinary Leaders Assessment Data for Full-Time MBA Class of 2009... 37

Table 22-Professional Deportment (Self-Presentation) for EMBA Class of 2010... 39

Figure23-EMBA Learning Goal 4: Number of Students Meeting and Exceeding Standards... 39

(5)

I. Mission Statement

The Masters of Business Administration (MBA) Programs at the Fisher College of Business, The Ohio State University are designed to prepare students to achieve

excellence in their management careers and to assume leadership roles in a complex and diverse global environment. Our MBA programs are nationally and internationally recognized as offering a comprehensive learning environment, informed by the latest research and designed to prepare the next generation of managers for excellence and responsible leadership. Investment in a Fisher MBA enhances students’ market value, expands their career prospects and accelerates their career advancement.

Each of our MBA programs or tracks (currently Full time, Executive and Working Professional tracks) is designed to address the needs of students of different types, providing them with customized opportunities to hone executive leadership and teamwork skills, to develop appropriately sophisticated perspectives on management practices and processes, and to acquire skills for career specialization. In addition, each of our MBA programs has a format designed to meet the needs of its target student audience and uses different mixes of on-site and distributed technology-based learning. These formats can be used to create a core knowledge base for other programs geared toward professionals in specialized market segments.

II. Learning Goals

1. Graduates are able to apply their knowledge & skills to solve business problems 2. Graduates have a global perspective and an awareness of how cultural differences

impact business

3. Graduates are able to work and lead effectively in a team-based environment

4. Graduates demonstrate professional deportment, self-awareness, leadership, polish and effective communication skills

(6)

A. Development of learning goals

The process for developing learning goals for our MBA programs began in December of 2005, when Steve Mangum, our senior associate dean for academic programs, assembled a meeting of all program directors (Associate Deans and Program Heads who are all faculty members) to discuss the AACSB process and begin the process of educating ourselves as what was required from each degree granting program. The program directors group has met approximately monthly since that time to report on progress toward the development of our learning goals and measures, to assess progress toward those goals, and to discuss challenges and share experiences across programs.

For the full time (FT) and working professional (WP) MBA tracks, the MBA program committee (MBAPC), comprised of eight to ten full time tenure track faculty

representing each department in the college, engaged in the process of developing

learning goals for these tracks (See Appendix 1). This committee meets monthly and has been actively engaged in the AACSB assurance of learning development process since March 2006. At that time, the Associate Dean for MBA programs, who chairs this committee, provided materials to the MBAPC and the process of discussion and

assessment of appropriate learning goals for the FT and WP MBA programs began. At the same time, the EMBA Academic Director, in consultation with EMBA faculty and the Associate Dean for MBA Programs, determined that the learning goals for EMBA students should mirror those of the full time and working professional tracks. The Associate Dean for MBA programs and the Academic Director of the EMBA program communicated regularly on progress so that goals for all MBA programs would be consistent and coordinated.

As part of the discussion and research that took place in the development of learning goals, a curriculum benchmarking study was undertaken and attention was given to feedback from students and recruiters on their assessment of the strengths and weaknesses of our students. The curriculum benchmarking study was conducted internally and assessed our program relative to other programs ranked in the top 20 by

Business Week and U.S. News and World Report. The student feedback was based on

data provided by our student services staff and also on the results of the EBI (Educational Benchmarking Survey) in which we participated. The recruiter feedback was provided by our Career Services office and also from recruiter assessments in rankings surveys. To summarize briefly, we learned that our program would benefit from a more global approach to the curriculum and more global opportunities for students, and that our students would benefit from increased emphasis on applied learning opportunities, leadership and communications skills.

In October 2006, full time and working professional MBA program learning goals were sufficiently well defined that our focus turned to refining the learning goals and the development of learning outcomes measures. Toward this end, four faculty

subcommittees were formed to focus on each of the four learning goals. The

subcommittees consisted of members of the MBAPC and additional faculty members with expertise relevant to the learning goal at hand (for example, international business,

(7)

leadership development and professional communications skills). The subcommittee’s task was to finalize learning goals and develop measures to recommend to the full MBAPC for final approval. This process was completed in December 2006.

1. Learning Goal 1: Graduates are able to apply their knowledge & skills to solve business problems

Employers value the ability of students to apply the knowledge and skills gained in our MBA programs to business problems while on the job. Feedback from recruiters and from students and graduates suggested that increased emphasis on this goal would benefit our students and subsequently their employers. Thus, the decision was made to place on-going emphasis on this learning goal and to formally track our progress along this dimension.

As part of our efforts in this area, the MBA program has placed increased emphasis on case competitions, business plan competitions and access to other opportunities outside the classroom that provide students with opportunities to apply what they have learned and showcase their abilities to practicing managers, and to gain feedback from them. In addition, we have worked to strengthen our project-based course offerings and launched a student-led consulting organization called Fisher Professional Services (FPS). FPS

provides students with the opportunity to work on projects for companies and for units within the university, and thus gain practical experience and prepare for careers in consulting or project management. On the non-profit side, we have supported the launch of a student-led organization called Fisher Board Fellows, in which MBA students serve on the board of non-profit organizations in the community and help them with the development of strategic plans and in the solving of business problems they face.

2. Learning Goal 2: Graduates have a global perspective and an awareness of how cultural differences impact business

Curriculum benchmarking, recruiter survey feedback as well as feedback from a CIBER1 survey suggested that our MBA program would benefit from increased emphasis on global business issues in our curriculum as well as enhanced global opportunities for study and project work for our students. Thus, the decision was made to place strong emphasis on this learning goal and to formally track our progress along this dimension. On the curricular side, we have recently expanded our faculty capability in the area of International Business and this has allowed us to significantly expand our global

curriculum. We have put into place a required international business course that will be part of our MBA core curriculum beginning in fall of 2008 for FT and WP MBA. Such a course, which includes a trip abroad, is already a part of the EMBA core. In addition, we have created an International Business (IB) Major for our full time MBA students. The IB major is designed to complement a traditional functional major and includes a

1

The Center for International Business Education & Research (CIBER) is located within the International Programs Office in the Fisher College of Business.

(8)

mandatory study abroad requirement and the opportunity for certification in competency in a foreign language at the students’ option.

To expand the experiential global opportunities available to our students, we have created a new format for international travel called an “elective trek.” Elective treks are two- week not-for-credit travel experiences that focus on an area of study, such as finance or logistics. We have also added additional sections of our emerging market field studies class, in which students study an emerging economy for a quarter, then travel to that country for approximately two weeks. We have worked to forge new, more effective partnerships with foreign universities to provide better study abroad experiences. In addition, the office of Career Management is working to enhance international internship and career opportunities available to our students and we have added opportunities for students to study foreign languages during their time in our MBA program.

3. Learning Goal 3: Graduates are able to work and lead effectively in a team-based environment

Employers value the ability of students to work and lead effectively in a team-based environment. Feedback from recruiters and from students and graduates suggested that increased emphasis on this goal would benefit our students and subsequently their employers. Thus, the decision was made to place on-going emphasis on this learning goal and to formally track our progress along this dimension.

In our full time and executive MBA programs, students are assigned to teams and work closely with these teams on projects throughout the core of their MBA program. Team assignments are made with diversity in mind so that students are able to work with and learn from individuals from different professional backgrounds and from different ethnicities and cultural backgrounds. In our working professional MBA program, students also do a great deal of teamwork, but they select their own teams to accommodate personal work schedules.

To leverage the value of the team experience and to help students hone their leadership capabilities, we formally launched a Leadership and Professional Development (LPD) Program in the fall of 2006 for our full time MBA students. LPD is a required series of individual assessments, team skills assessments, workshops, speakers and outside the classroom experiences designed to prepare our students for leadership in a team-based environment. More recently, we have rolled out some parts of this program to our working professional MBA students, with plans to do more over time. As part of the LPD program, we have adopted and developed assessment tools that both provide

valuable developmental feedback to students and also help us gauge their progress toward learning goal 3. Our EMBA program requires a class in Professional Development. In that class, students take part in the Team Member Effectiveness Survey (Larson & LaFasto) that is also used in the FT and WP tracks. Students discuss results with each other and with the instructor, focusing on areas for improvement. The Executive MBA students complete this instrument after they have been working in assigned project teams for one year. FT students complete the instruments after nine months on assigned teams. WP students complete the survey with feedback from teammates at school and from

(9)

co-workers. This survey is administered such that the results are quite useful for students in all three MBA tracks.

4. Learning Goal 4: Graduates demonstrate professional deportment, self-awareness, polish and effective communication skills

Faculty concerns and input from business people who attended various student

presentations or judged case competitions and from recruiters suggested an improvement in students’ oral and written communication skills would be beneficial. As a result, a communications course was added to the core of our full time MBA program. This course provides our students with the opportunity to further develop the skills identified in learning goal 4. Faculty working in this area developed assessment tools that both provide valuable developmental feedback to students and also help us gauge their

progress toward learning goal 4. The communications course will be added to the core of our working professional MBA program beginning in fall of 2008. In the Executive MBA track, polish, self-awareness, deportment and effective communication are assessed during the application and interview process. Because candidates are required to have several years of significant managerial experience, they tend to exhibit strong

professional skills. In addition, most candidates attend an on-site information session, where their personal interactions can be observed. Finally, each candidate must visit campus for a face to face interview. At this interview, the EMBA Academic Director completes a checklist rubric to assess the relevant issues.

(10)

B. Location of Goals within Curriculum

The following shows the location of MBA program goals within the MBA curriculum for the Full Time and Working Professionals Tracks as well as the EMBA track.

Table 1-Location of Program Goals within MBA Curriculum Full Time and Working Professionals Tracks

Goal Location within Curriculum 1. Graduates are able to apply their

knowledge & skills to solve business problems

Throughout core curriculum, in project and field based elective courses, in student oriented consulting opportunities; assessed through standalone testing in the Fisher Case Competition

2. Graduates have a global perspective and an awareness of how cultural differences impact business

 MBA Business in the Global Macro economy (core course)

 International Business Major option in FT MBA

 Internationally focused elective courses

 DISS lecture series

 Required assessments and workshops in the Leadership and Professional Development Program.

 MBA 860 Organizational Behavior & Teamwork (core course)

4. Graduates demonstrate professional deportment, self-awareness, polish and effective communication skills

 MBA 825 Enhancing Professional Communication (core course)

Table 2-Location of Program Goals within MBA Curriculum- EMBA Track

Goal Location within Curriculum 1. Graduates are able to apply their

knowledge & skills to solve business problems

Throughout core curriculum; assessed through selection

2. Graduates have a global perspective and an awareness of how cultural differences impact business

International Field Study: Emerging Markets course

 Organizational Behavior & Teamwork course (core course)

 Team Contract

 Communication is assessed at selection  Professional Development course addresses

individual assessment, personal

(11)

C. Relationship of Learning Goals to MBA program mission

The relationship of Learning Goals to the MBA program mission is listed in table 3 below.

Table 3-Relationship of Learning Goals to MBA program mission

Learning Goal Relationship to Fisher College & MBA program mission

1. Graduates are able to apply their knowledge & skills to solve business problems

Our mission statement specifies that students will:

1. Achieve excellence in their management careers

2. Assume leadership roles in a complex and diverse global environment To achieve these goals, students must be able to apply knowledge and skills in practice 2. Graduates have a global perspective and an

awareness of how cultural differences impact business

Our mission states that we prepare students to assume leadership roles in a complex and diverse global environment. A global perspective and awareness of cultural differences is critical to effectiveness in such leadership positions.

Our mission statement states that we will provide our students customized opportunities to hone executive leadership and teamwork skills.

The mission statement states that our students will achieve excellence in their management careers, and such excellence is not possible without the appropriate degree of professional deportment, self-awareness, polish and effective communication skills. Effective leadership is also not possible without effective communication skills.

(12)

III. Assessment Plan

A. The Ohio State University- Fisher College of Business Full Time MBA Assessment Plan

Learning Goals 1. Graduates are able to apply their knowledge & skills to solve business problems

2. Graduates have a global perspective and an awareness of how cultural differences impact business 3. Graduates are able to work and lead effectively in a team-based environment

4. Graduates demonstrate professional deportment, self-awareness, leadership, polish and effective communication skills Table 4-The Ohio State University- Fisher College of Business Full Time MBA Assessment Plan

Data Learning Goals Where Measured How

Measured

Administration (X), Analysis (Y) and Implementation (Z) Au 06 Au 07 Win 08 Spr 08 Sum 08 Au 08 Direct 1. Apply knowledge &

skills

Demonstration through stand-alone testing in Fisher Case Competition

Rubric X Y Y Z

3. Work in teams Demonstration through stand alone testing through our Team Effectiveness Surveyc

Web Survey X,Y Y Z

4. Deportment,

Leadership & Communication

Learning Objective A: Students will demonstrate effective communication skills

Course-embedded measurement: 825 Enhancing Professional Interchange mid-term and final

Rubric Xb Xb Y X,

Z

4. Deportment,

Learning Objective B: Students will demonstrate leadership skills.

Demonstration through stand-alone testing Extraordinary Leader (or equivalent) survey

Web Survey X, Y Z

Indirect 2. Global Awareness Distinguished international speaker series, Emerging Market Field Study Courses, Elective Trek, Study Abroad, International Case

Competitions, etc... Track participation X, Y X, Y X, Y X, Y Y Z

(13)

Assessment Plan

4. Graduates demonstrate professional deportment, self-awareness, leadership, polish and effective communication skills Table 5-The Ohio State University- Fisher College of Business MBA Assessment Plan for Working Professionals

Data Learning Goals Where Measured How Measured Administration (X), Analysis (Y) and Implementation (Z) Au 06 Au 07 Win 08 Spr 08 Sum 08 Au 08 Win 09 Direct 1. Apply knowledge

& skills

Demonstration through stand-alone testing in Fisher Case Competition

Rubric Xe Y,Z X

1. Apply knowledge & skills

Selection Work experience

at admission

X, Y Z

3. Work in teams Demonstration through stand-alone testing through our Team Effectiveness Surveyd

Web Survey X,Y Z

4. Deportment,

Learning Objective A: Students will demonstrate effective communication skills

Course-embedded measurement: 825 Enhancing Professional Interchange mid-term and final

X

4. Communication Selection Résumé and

Admissions Essay Rubric 2. Global Awareness Distinguished international speaker

series, Emerging Market Field Study Courses, Elective Trek, Study Abroad, International Case Competitions, etc...

Track participation X, Y X, Y X, Y X, Y Y Z 2. Global Awareness Indirect 4. Deportment & Communication Course-embedded measurement: 825 Enhancing Professional Interchange mid-term & final

Rubric Xb Xb X Y,Z X

a _{Not sure where & when post-test will be administered. Do not have pre-test data available.} b_{A rubric was not used.}

c _{Debrief sessions with survey data}

d_{To be administered to all students who enroll in Summer 08 quarter} e _{Pilot Data collection}

(14)

C. The Ohio State University- Fisher College of Business EMBA Assessment Plan

4. Graduates demonstrate professional deportment, self-awareness, leadership, polish and effective communication skills Table 6-The Ohio State University- Fisher College of Business EMBA Assessment Plan

Data Learning Goals Where Measured How Measured Administration (X), Analysis (Y) and Implementation (Z) Win 07 Sum 07 Fall 07 Win 08 Sum 08 Fall 08 Direct 1. Apply knowledge &

skills

Selection-Admissions Prior work

experience

Data collected year-round X, Y Z 3. Work in teams Demonstration through stand-alone

testing : Team Effectiveness Survey a

Web Survey X, Y Z XYZ

4. Deportment & Communication

Selection-Admissions Admissions Interview Rubric

Data collected year-round X, Y Z Indirect 1. Apply knowledge &

skills Course-embedded measurement 895X Business Plan Scales of team performance X Y Z

2. Global Awareness Course-embedded measurement Case from Global Integration Course

Rubric for team papers

X Y Z

2. Global Awareness International Experiences Track participation at events

X X Y Z

3. Work in Team Demonstration through standalone performance

Team contract acceptance form

X X Y Z

(15)

IV. Learning Goal 1: Graduates are able to apply their

knowledge & skills to live business problems.

A. Full-Time Track

1. Background

Learning Goal 1 is currently assessed in the Fisher Internal MBA Case Competition (standalone assessment). This competition takes place annually with first year MBAs. The internal competition is used to select a number of students who are then be put into a pool that generates “all star” teams for case competitions throughout the year. Faculty and external judges use rubrics that assist them in the decision of Best Speaker, Best Question and Answer, and Best (4 person) team in each room. Over 85% of the FT MBA students participate in this event.

2. Assessment

Pilot data collection took place in 2006-2007. However, these data cannot be used for Assurance of Learning purposes because formal rubrics2 were not used and data were not collected at the individual student level.

Final data collection was from the Winter 2008 Fisher Internal MBA Case Competition. Individual level data were collected for over 85% of the Class of 2009 cohort. The

individual contributions of team members were rated with a rubric (appendix 2). The tally is in table 7 below.

It should be noted that this event is an internal competition and some of the data becomes normative because the judges must select a winning team from each room.

3. Standard for demonstrating achievement

Students were rated on a scale of 1 (=Good) to 3 (=Best) on eight Content and seven

Delivery traits. The standard for demonstrating achievement is as follows:

 66% of student scores will be 2 (=Better) or higher on at least 5 of 8 Content traits.  66% of student scores will be 2 (=Better) or higher on at least 4 of 7 Delivery traits.

4. Results

Full results can be seen in table 7.

 Only 3 of 8 Content traits had 66%+ of scores that were equal to or higher than “Better”. This standard is not met.

 6 of 7 Delivery traits had 66%+ of scores that were equal to or higher than “Better”. This standard is met.

2

(16)

The traits students had the most difficulty with are analysis of internal firm factors, feasibility of recommendation, analysis supports recommendation and conclusion is persuasive and sound. Students were most successful with eye contact, enthusiasm and voice.

Table 7-Tallya of 2008 First Year MBA Case Competition, WI08, Fulltime MBA Class of 2009

Content Traits Good (1) Better (2) Best (3) Mean Clear Statement of Problem (C1) 144 (32%) 220 (48%) 92 (20%) 1.89 Analysis of external/market factors (C2) 128 (29%) 220 (50%) 96 (22%) 1.93 Analysis of internal/firm/client factors (C3) 192 (46%) 188 (45%) 36 (9%) 1.63 Feasibility of Recommendation (C4) 196 (44%) 188 (42%) 60 (14%) 1.70 Creativity of Recommendation (C5) 160 (35%) 180 (39%) 116 (25%) 1.90 Qualitative and quantitative analysis supports recommendation (C6) 204 (48%) 132 (31%) 92 (21%) 1.74 Appropriate use of concepts/frameworks (C7) 132 (31%) 228 (53%) 68 (16%) 1.86 Conclusion is

persuasive and sound (C8)

216 (50%) 176 (40%) 44 (10%)

1.61 Delivery Traits Good (1) Better (2) Best (3) Mean Exhibited confidence

and poise (D1)

100 (23%) 244 (56%) 92 (21%)

1.98 Eye contact with

audience (D2)

52 (12%) 256 (58%) 133 (30%)

2.19 Exhibited energy and

enthusiasm (D3) 100 (23%) 204 (47%) 132 (30%) 2.08 Voice: sufficiently projected, clear, pleasant tone/inflection (D4) 84 (19%) 212 (49%) 136 (31%) 2.12 Appropriate use of body

language (D5) 96 (22%) 236 (55%) 100 (23%) 2.01 Effectively responded to questions (D6) 120 (27%) 220 (50%) 104 (23%) 1.97 Visual Aids: impact,

appeal and creativity (D7)

185 (42%) 160 (37%) 92 (21%)

1.79 a

Because each student is assessed by between three and five judges, this is a tally of scores rather than of students

(17)

5. Analysis

 The 3 traits with the lowest average scores were

o Conclusion is persuasive & sound (average score=1.61) o Analysis of internal factors (1.63)

o Feasibility of Recommendation (1.70) o These are all Content traits.

 The 3 traits with the highest average scores were o Eye contact (2.19)

o Voice projection (2.12) o Energy & enthusiasm (2.08) o These are all Delivery traits.

Bloom’s Taxonomy3 provides a useful framework for analysis. Cognitive domain skills can be classified as being lower or higher order processes. In this learning goal, the traits students did least well in are the highest order skills in Bloom’s taxonomy, namely analyzing, evaluating and creating.

6. Closing the loop

Based on the analysis of the data, we will work to enhance our communications and related curriculum to emphasize students’ ability to present sound and persuasive conclusions, and to focus more closely on the feasibility of their recommendations. This information will be tracked.

We will also work to refine the rubric so that it can be used in other contexts in which students conduct applied analysis and make presentations, such as other case

competitions, project-based course work and business plan competitions. This will enable us to gather more data and to observe student capability in a broader variety of contexts. It will also allow us to assess capabilities in this area later in the program as well, as the case competition comes only four months into our 21 month FT MBA program.

We also observed relative weaknesses in performance by some international students and by teams of students in our working professionals program. For example, no team made up primarily of international or working professional students was selected for the second round of the competition.

To address the issue of skills among international students, we have taken a variety of steps. First, we have increased our standards for English language competency among international students at the time of admission. For students that we admit who are

3

Taxonomy of Educational Objectives: The Classification of Educational Goals; pp. 201-207; B. S. Bloom (Ed.) Susan Fauer Company, Inc. 1956. A Taxonomy for Learning, Teaching, and Assessing — A

Revision of Bloom's Taxonomy of Educational Objectives; Lorin W. Anderson, David R. Krathwohl, Peter W. Airasian, Kathleen A. Cruikshank, Richard E. Mayer, Paul R. Pintrich, James Raths and Merlin C. Wittrock (Eds.) Addison Wesley Longman, Inc. 2001

(18)

identified as having relatively weak language skills, we require that they come to campus several months early and participate in an English language immersion course offered at the university. In addition, each international student takes a spoken English test during our orientation program and based on that assessment is assigned to a communications curriculum track. For students with insufficient language competency, we have added a focused communications skills course that prepares them for the MBA level required communications course. In addition, we offer a special section of this communications course for international students that they can take following the more basic course. Preliminary findings (the course is still in session as of this writing) are favorable, with improvement in overall skills and in the content traits from above.

(19)

B. Working Professionals Track

Learning Goal 1 is assessed in the Fisher Internal Case Competition and through selection at admissions.

1. Learning Objective A: The Fisher Case Competition

a. Background

Participation in the case competition is voluntary and has been sporadic. However, following input from employers, students and business leaders who visit campus and judge competitions, the Enhancing Professional Communication class (MBA825) is being introduced as a core requirement for the cohort entering in fall of 2008. As more students take this class, the number of students participating in case competitions is expected to increase. Data will be tallied in winter 2009 using the same rubric as for the full-time students (we will use a 1-6 scale instead of 1-3).

b. Pilot Data Collection

For Winter 2008, data were collected on six students. The individual contributions of team members in the Fisher Case Competition were rated with the same rubric used for the FT MBA. The tally is in Table 8 below.

c. Results and Discussion

Results should be interpreted with caution because they are based on only six students. The Working Professionals MBA students did quite poorly on “Feasibility of

Recommendation”, with 18 of 21 scores in the lowest category, and in “Qualitative and quantitative analysis supports recommendation”, with all scores in the lowest category. Also low was “Conclusion is persuasive and sound”, with 12 ‘Good’ scores, 4 ‘Better’ scores and no ‘Best’ scores. Students did better on the delivery traits, on the statement of the problem and on the internal and external analyses.

(20)

Table 8-Pilot Data Collection-Tallya of 2008 First Year MBA Case Competition, Winter 2008 Working Professionals MBA Class of 2009

Content Traits Good

(1) Better (2) Best (3) Mean

Clear Statement of Problem (C1) 8 12 0 1.6

Analysis of external/market factors (C2) 4 10 2

1.8

Analysis of internal/firm/client factors (C3) 2 12 2 2.0

Feasibility of Recommendation (C4) 18 0 3

1.2

Creativity of Recommendation (C5) 6 8 10 2.2

Qualitative and quantitative analysis supports recommendation (C6)

20 0 0 1.0

Appropriate use of concepts/frameworks (C7) 8 4 2 1.4

Conclusion is persuasive and sound (C8) 12 4 0 1.2

Delivery Traits Good

(1)

Better (2)

Best

(3) Mean

Exhibited confidence and poise (D1) 4 14 2 1.9

Eye contact with audience (D2) 4 14 2 1.9

Exhibited energy and enthusiasm (D3) 6 8 6 2.0

Voice: sufficiently projected, clear, pleasant tone/inflection (D4)

0 12 8 2.4

Appropriate use of body language (D5) 6 12 2 1.8

Effectively responded to questions (D6) 2 18 2 1.9

Visual Aids: impact, appeal and creativity (D7) 8 8 4 1.7

a

Because each student is assessed by up to four judges, this is a tally of scores rather than of students

d. Closing the loop

The Enhancing Professional Communication class (MBA825) is being introduced as a core requirement for the cohort entering in fall of 2008, which will provide additional support and training in communication skills. We will also encourage more working professional students to participate in case competitions, project classes and other venues in which they can receive feedback on their skills. Assessment data will be collected to see if the results improve relative to the pilot data collection.

2. Learning Objective B: Prior work experience

a. Introduction and Assessment

Prior work experience at the time of admission to WP MBA was assessed. A rubric was developed to assess with three traits; years of work experience, career progression and quality of work experience (see appendix 3). Data were collected for all admits Summer 2007 through Summer 2008 (five consecutive quarters).

It is important to keep in mind that although the scale follows the conventional does not

meet standards, meets standards, and exceeds standards format, the admissions process

is more holistic than these standards imply. Rather than the scale representing formal admissions standards, a score of does not meet standards denotes the lower end of an

(21)

acceptable range, which, in the case of an admitted student, would be compensated for by higher performance on other parts of the entire admissions packet and not necessarily another trait within the work experience category. This allows the admissions team to consider and admit students with non-conventional backgrounds.

b. Standard of achievement

The standard of achievement is: At least 90% of the admitted students will have a mean score of 3 or better.

c. Results

Only two of 149 students (1.3%) had average scores below 3. The standard of achievement is met.

There is little variation between individual traits, as would be expected of a selective program. There was a small difference between the average Career Progression and the Quality of Work Experience scores, the latter having a higher mean score (Table 9).

Table 9-Tally of Work Experience Assessment Scores at Admission

Years of Work Experience Score Career Progression Score Quality of Work Experience Score N (%) N (%) N (%)

Does not Meet Standards (1-2) 6 (4%) 2 (1%) 2 (1%)

Meets Standards (3-4) 64 (43%) 99 (66%) 78 (52%) Exceeds Standards (5-6) 79 (53%) 48 (32%) 69 (46%) Mean 4.7 4.0 4.4 Median 5.0 4.0 4.0 Mode 6.0 4.0 4.0 Min 1.0 2.0 2.0 Max 6.0 6.0 6.0 Std. Dev. 1.2 0.9 0.9

d. Analysis and Discussion

Work experience is an important factor in assessing the quality of an MBA candidate. It is generally accepted as fact that candidates who possess some degree of work experience are better able to participate in classroom discussion leading to improved academic performance and a richer learning environment for all students, and more rapid career progression during and following the MBA experience. The existence of a measurement tool that quantifies this experience will enable testing of that long-standing hypothesis. Appropriate evaluation of the rubric in assessing work experience, however, will require that evaluators adopt an approach that results in consistent scoring for each applicant. Clear wording in the guidelines for assessment, coupled with a minimal amount of training for evaluators should enable scorers to consistently assign scores.

(22)

e. Closing the loop

We plan to adopt this rubric more formally as part of our admissions process, rather than simply using it as an assessment tool after the fact. The information will be helpful in the admission process in two ways: 1) the information will help admissions staff to quickly compare candidates, as the evaluation of the duration and quality of work experience will be done in an initial evaluation, before the admission committee reviews the candidate for a final admission decision. 2) After admission, the existence of a single measure of work experience will enable the program to improve the admission process by determining the degree of correlation, if any, between work experience and other desirable outcomes, such as academic performance and/or career success and progression.

C. Executive MBA Track

1. Introduction

Executive MBA students have the same learning goal as the other MBA tracks with different learning objectives. One (learning Objective A) is assessed through student selection and involves all of the three most recent cohorts. The other (learning Objective B) is a course-embedded assessment and involves one of the classes in the current cohort.

2. Learning Objective A: Students have at least seven years of significant managerial experience

a. Introduction

At admissions, typically students with 7 or more years of significant managerial experience are selected for the program. This ensures they have enough experience to contribute to class discussion and team projects.

The standards for demonstrating achievement are:

1. 90% of students entering the EMBA have 7 or more years of significant managerial experience

2. The average years of work experience of each cohort is ten or more.

b. Results & Analysis

Table 10-Entering EMBA Students with 7 or more years of significant managerial experience

Cohort No. of Students

No. of students meeting standard % of students meeting standard Class of 2007 45 37 82 Class of 2008 54 51 94 Class of 2009 54 49 91

89% of students in the three most recent cohorts have 7 or more years of significant managerial experience. The standard has been met for the last two years. In 2007, admissions standards related to prior work received greater scrutiny after feedback from faculty and students that some students lacked sufficient significant managerial

(23)

Table 11-Average Years of Work Experience of 3 most recent cohorts of entering EMBA Students

Cohort Average Years

Class of 2007 12

Class of 2008 11

Class of 2009 15

Each of the three most recent cohorts average more than 10 years of work experience. The standard for demonstrating achievement has been met.

c. Closing the loop

The Executive MBA program will continue to seek candidates with at least seven years of significant managerial experience. Occasionally the program will choose to accept a candidate with fewer years experience when that candidate demonstrates either

exceptional promise (fast track management program at his or her job) or brings a unique perspective to the class (experience in government, international issues, entrepreneurial success, for example).

3. Learning Objective B: Students develop a business plan

a. Introduction

Learning Objective B is assessed in a team project in the 895X Business Development course (data from Class of 2008 cohort). This is an indirect measure. Teams prepare business plans. These are rated on multi-item scales (see appendix 4). The faculty found that this scale provides good feedback in each category of performance for the teams. These assessment data are shared with the teams.

b. Results

See appendix 4 for team-by-team results. The weakest areas were:  Problem/plan-defended with examples

 Market-Differentiation from competition  Customer Service-Description & feasibility  Team-Clear on gaps, clear on roles

 Financials-Capital needs & uses The strongest areas were:

 Solution-feasibility & technology  Customer-description

 Manufacturing/delivery/production-description  Financials-Summary & assumptions

(24)

c

.

Analysis

One weakness apparent in some of the presentations is that the buyer within a company is often not well identified. This feedback should be useful to students preparing for the Deloitte and Touche Business Plan Competition being held in June 2008.

d. Closing the loop

After learning of the results of this analysis, it was determined that many of the weaknesses of students could be addressed by encouraging students to “think ahead” about the business plan during the other core EMBA classes. For example, during the marketing management class, the instructor will emphasize that market-differentiation and customer service issues have been weaknesses of business plans in past classes. In this way, students can begin earlier and more actively use what they are learning to strengthen the weak areas of the business plans.

(25)

V. Learning Goal 2: Graduates have a global perspective

& awareness of how cultural differences impact

business

A. Full-Time and Working Professionals

1. Background

Developing a global perspective and international experiences is consistent with our mission statement and is increasingly important for professional success. Thus, it is receiving increased emphasis in both our FT MBA and our WP MBA.

2. Assessment

Initially, an international perspective survey was administered to all incoming first year students from the Full-Time class of 2009. After this trial survey and feedback from recruiters and faculty, the MBA core was revised to include an International Business course. In addition an International Business major was added to the FT MBA

curriculum. At this time, we are tracking the international experiences to which our students are exposed. This is an indirect measure. Our international business faculty are currently working on an assessment tool to be used in the future, which will provide a direct measure of our students’ international capabilities and cultural awareness. The following international experiences are being tracked:

 Participation in the Emerging Markets Field Study course (MBA 808). The class studies and then visits companies in Columbus area and in the selected cities and countries. Language and cross cultural training is a component of this course.  Participation in international case competitions, including the CIBER case

competition.

 Participation in the MBA program by international students  Distinguished International Speakers Series attendance

 ICE (InterCultural Edge) Survey and debrief (or equivalent survey)  Fisher Student Exchange Programs

 Participation in Elective Treks program. Student travel with a faculty member to a country where they can see classroom theories in practice for a particular functional area, such as finance, logistics and so forth.

3. Standard of achievement

Our goal is to increase the percentage of students who participate in international business coursework and in international experiences during their time in the program. With the launch of an international business core class, we ensure that 100% of our FT MBA and WP MBA students take at least one class in international business. The EMBA program has required such a course for a number of years. That leaves us to focus on increasing the percentage of students who participate in international

experiences as outlined above. Because this is only the second year of our initiative to increase international experiences, it is too early for us to set a participation goal. We will

(26)

monitor student participation over the next two years and then adjust program goals and resources accordingly.

3. Results

The data are broken down into several categories.

 International students are counted as having an international experience by virtue of their presence at Fisher.

 For domestic students, international experiences are divided into experiences involving travel abroad and other experiences.

 All full-time students from the class of 2009 participated in the ICE survey debrief.

Table 12-Estimated Data on International-related activities of MBA Students, 2006-2008

Cohort: 2008 FT 2009 FT 2007 WP

Students in cohort 125 139 147

International students 46 44 18

Academic Year: 2006-07 Students who have participated

in campus-based international activities: DISS, CIBER Case Competition

34 N/A 0

Students who have participated in International activities involving foreign trips:

EMFS, Elective Treks

9 N/A 0

Student Exchange 0 N/A 0

Academic Year: 2007-08

ICE (involves entire cohort) No Yes No

Students who have participated

in campus-based international activities: DISS, CIBER Case Competition

27 58 3

Students who have participated in International activities involving foreign trips:

EMFS, Elective Treks

16 9 10

Student Exchange 5 4 4

5. Closing the loop

As stated earlier, our goal is to increase the number of students who have international experiences. It is too early to set a specific goal in terms of both annual increases and total participation. Our international business faculty are working on an international awareness assessment which will provide a direct measure of our students’ capabilities in this area. We would also like to emphasize participation in international experiences that involve experiences outside of the U.S. Such experiences, however, are expensive for students, so while we will do our best to make these experiences affordable, financial constraints may play inhibit our ability to attain the same percentage for these

(27)

Also, as mentioned earlier, a new International Business course is being implemented for the first time as a mandatory core course for the FT MBA and WP MBA. It will be offered in spring of 2009 for FT and summer of 2009 for WP MBA. Fisher College has also designed a new International Business major designed to complement existing majors and enable students to specialize in the area of international business. This major will be available to students in the Full-Time MBA class of 2010. We will monitor the effectiveness of this new core course and the popularity of the international business major, making adjustments as necessary to ensure their effectiveness.

B. EMBA

1. Learning Objective A – Students will have international experiences as part of their program

The International Field Study: Emerging Markets 856X course is mandatory for all EMBA students, including the portion that involves travel outside the U.S. The standard for demonstrating achievement is that at least 95% of students in each cohort will participate in the international trip. The results are as follows:

 98% of the Class of 2007 participated (all but one).  100% of the Class of 2008 participated.

 The standard for demonstrating achievement has been met.

2. Learning Objective B- Students integrate a diverse body of knowledge to address the issues faced by the multinational firm.

a. Assessment

An indirect measure is used to assess global perspective and the integration of a diverse body of knowledge in the EMBA International Field Study: Emerging Markets 856X course. A rubric was used to evaluate team papers. The tally is below and the rubric is in appendix 5. This is a course-embedded assessment. The rubric was designed by that course instructor. The papers were assessed by a graduate student under the guidance of that professor.

b. Standard for demonstrating achievement

1. 100% of teams will average at least 3 (Meets Standards) on all traits. 2. The Average for each trait will be at least 3 (Meets Standards).

3. Less than 5% of all scores will be 2 or below (Does Not Meet Standard).

c. Results

1. All teams averaged at least 3 on all traits, with 8 of 10 teams averaging at least 3.5 (out of 6). The standard was met.

2. All traits averaged at least 3 (Meets Standards), with 5 of 6 traits at or above 3.5. The standard was met.

(28)

There was little variation across traits. Only one trait had scores (3 of 10) that did not meet standards; defends decision. For most of the other traits, most teams scored within the “meets Standards” range.

Table 13-EMBA Global Integration Course Tally Sheet for Cohort EMBA 2007 Winter Quarter 2008

Criteria Trait (mean score) Does not meet

standards=1-2

Meets Standards=3-4 Exceeds Standards= 5-6

Content

# of teams (percent) # of teams (percent) # of teams (percent) Applies functional

concepts (3.6)

10 (100%)

Integrates what has been learned in the curriculum (3.9)

8 (80%) 2 (20%)

Integrates across disciplines & outside the curriculum (3.9) 8 (80%) 2 (20%) Integrates from personal experience (3.5) 9 (90%) 1 (10%) Understands & articulates pertinent cultural issues & concepts (3.2)

9 (90%) 1 (10%)

Defends Decision 3 (30%) 5 (50%) 2 (20%)

d. Analysis

These results show that our students are competent in their ability to deal with global issues. This is not surprising given the overall experience of these students. Many of them routinely travel internationally for work or interact with clients or suppliers in other countries. Therefore, the overall results certainly demonstrate face validity. The final results seem to indicate that the primary weakness is the ability to defend a decision made regarding a global business issue.

e. Closing the loop

Since students are competent in other areas, it seems that the curriculum needs more focus on students’ ability to defend a decision made as part of a global business issue. This might be a writing/presentation issue rather than a content issue. However, one would expect the course to place greater emphasis on this variable in the future.

(29)

VI. Learning Goal 3: Graduates are able to work and lead

effectively in a team-based environment

A. Background

The assessment of this learning goal is the same for all three MBA tracks. National recruiting surveys with employers have identified that “team play” and “team

effectiveness” are crucial and increasingly important criteria for MBA employment and career advancement. Because of this, our MBA program has determined to move beyond simple pedagogical inclusion of required teamwork in courses to a more introspective and experiential approach to improving individual team member effectiveness. Over the past two years we have begun to include our MBA students (Full time, working professional and executive MBA’s) in an individually based assessment on teamwork and team effectiveness skills. The process is intended to:

a. Gain peer-level team colleagues’ input and assessment of each student’s effectiveness, productivity and contribution to their MBA teams.

b. Provide each student with highly personalized, behaviorally specific feedback about their team strengths and areas in need of improvement.

c. Encourage and require each student to formulate and report their development plans for improving their contribution to their respective teams.

B. Theoretical and research basis for surveys used

The instrument we are using is the Team Effectiveness Survey and Team Member

Effectiveness Survey, developed by Larson & LaFasto (1989; 2002). The instrument was developed based on an investigation of thirty “high performance” teams across a wide array of circumstances: presidential cabinets, business executives, expedition teams, entrepreneurial ventures, heart transplant teams, sports teams, etc.4

The important theoretical underpinning of the work and resulting team assessment survey instruments is that team “effectiveness” is clearly and simply defined as “successful achievement of the team goal.” In that respect, the entire body of work is focused on outcomes and behaviorally-based feedback that can be improved. The research work has yielded three different survey instruments that have been normed and psychometrically evaluated for internal reliability:

1. A Team Effectiveness survey. Measures a team’s self-assessment of overall team effectiveness at achieving the goal. This survey yields scores on eight dimensions of team effectiveness.

4

Among the thirty teams analyzed and interviewed were: Mt. Everest Expedition Team, Presidential Commission on the Challenger Space Shuttle Disaster, Drs. DeBakey/Cooley Cardiac Surgery Team, IBM PC Design and Development Team, Boeing 747 Airplane Design Team, U.S. Navy Strike Warfare Center, National Championship Football Team and Center for Disease Control Epidemiology Teams.

(30)

2. A Team Leader survey. Measures the team’s assessment of the team leader’s personal leadership on six dimensions of leadership effectiveness.

3. A Team Member survey. Measures the team’s assessment of each individual team member’s productivity and contribution to the team.

The survey instrument used with our MBA student teams is number three above, the Team Member survey. It is based on the normative results of Larson and LaFasto’s work, which has now been extended to over 600 teams and 15,000 participants. The survey includes the original 34 LaFasto and Larson items, and is supplemented by a final rated question (#35. Overall, this person’s contribution to the team’s effectiveness is:

“unacceptable” to “exceptional”), as well as two open-ended narrative questions (#36. “What is this person’s most positive contribution to the team?”; #37. “What could this person do to improve their contribution ?”).

The survey yields scores on six dimensions of team member effectiveness: 1. Experience 2. Problem-solving 3. Openness 4. Supportiveness 5. Action Orientation 6. Personal Style

In addition, the student receives a score on a summary question and verbatim comments received on narrative questions.

C. Survey Administration

All MBA students are assessed on their productivity and team contribution by their MBA work teams/teammates. WP MBA students also have the option of asking co-workers to provide feedback as well. Raters are assured that their ratings and written comments are anonymous.5 Typical sample sizes for each student’s report are based on 4-7 raters. Data are collected via a Qualtrix web survey administration process, developed in-house, generally during the quarter in which students take the related coursework in leadership, team effectiveness and organizational behavior. During this course the Larson and LaFasto work is covered in detail in the course syllabus and lectures.

Within two weeks of the administration of the survey, each student receives a

personalized, individual feedback report. Those reports contain that student’s: 1) scores on the six dimensions along with total class “normative” scores for comparison; 2) scores on each of the individual survey items; and 3) the verbatim transcription of any written

5

The individual feedback reports are completely confidential between the student and the course instructors. That is, the reports are made available only to the individual student, and only the course instructors and scoring administrators have access to the individual data. MBA program leadership and faculty (other than the course instructors) are not given access to any individual’s data or feedback reports. We believe this is a critical requirement to protect the integrity of the survey feedback process, as well as

(31)

comments provided by the raters. The written comments are viewed as particularly valuable in students’ understanding of the ‘qualitative’ texture of their feedback report. These written comments are routinely a rich source of behavioral feedback, and can greatly aid an individual’s efforts to improve. All of the written comments are pre-reviewed by the course instructors in advance of any feedback reports being shared to ensure that the comments are professional and appropriate to the context.

Once all student surveys have been scored, summary statistics are prepared, and a group feedback session with all students is conducted to share the overall results with the class, as well as to personally hand out the individual feedback reports. Importantly, this group

feedback session (as well as the feedback report itself) instructs students on the proper reading and interpretation of their personal feedback results. We believe this is a critical

step in the feedback process to ensure that students do not ‘over-interpret’ their results. Each student is also invited to schedule an individual feedback appointment with one of the trained instructors to further discuss and interpret their feedback report. The goal is to ensure that students maintain an appropriate and constructive mind-set regarding their feedback results.

D. Development focus

Once the feedback reports have been received, and the total class results have been reviewed and discussed by the instructors, students are encouraged to share a summary of their results (not the actual report) with their teammates who generated the feedback. The purpose of this recommended step is to encourage dialogue among the team members and to aid students in clarifying their feedback and in developing improvement plans.

Finally, students are required to include a summary of their feedback results and action plans for improvement in their final papers submitted as part of their coursework for that term.

E. Standards of Achievement and Results

Because this is the first time we used this instrument as an assessment in the MBA program, we did not establish an initial standard of achievement. Now that we have had an opportunity to analyze data from several cohorts and MBA tracks, we have decided to adopt the following standard of achievement: at least 75% of the entire MBA cohort score 4.0 or higher on at least four of the six dimensions of team member effectiveness before leaving the MBA program. If a specific cohort does not meet this standard, we will offer this group additional instructional and developmental activities (e.g., content elements from professional speakers).

To date, the survey has been conducted with three MBA classes (two EMBA class cohorts and the entire FT MBA class of 2009). In Summer 2008, WP MBAs were given the opportunity to participate on a voluntary basis; 71 students chose to participate. Thus far, our survey has included a total of over 300 MBA students, and has involved ~1,300 raters. Appendices 3, 4 and 5 show the dimension score results (anonymously) for the

(32)

three different MBA cohorts who have completed the survey thus far. The tables below show summary results from the three classes who have participated:

Table 14-Full-Time MBA Class of 2009 N=141

Dimension Class Mean Std. Dev. % at 4.0 or higher Experience 4.1 .66 65% Problem-solving 4.2 .67 72% Openness 4.1 .69 67% Supportiveness 4.2 .64 77% Action Orientation 4.1 .71 64% Personal Style 4.2 .63 70%

Table 15-EMBA Class of 2008 N=53

Dimension Class Mean Std. Dev. % at 4.0 or higher Experience 4.0 .57 55% Problem-solving 4.0 .53 60% Openness 3.9 .53 60% Supportiveness 4.1 .59 58% Action Orientation 3.9 .64 55% Personal Style 3.8 .58 51%

Table 16-EMBA Class of 2007 N=44

Dimension Class Mean Std. Dev. % at 4.0 or higher Experience 4.4 .32 88% Problem-solving 4.4 .27 88% Openness 4.4 .39 86% Supportiveness 4.5 .39 84% Action Orientation 4.2 .41 70% Personal Style 4.4 .34 84% Table 17-WP Class of 2009 N=71 Dimension Class Mean Std. Dev. % at 4.0 or higher Experience 4.3 .57 89% Problem-solving 4.3 .61 86% Openness 4.3 .65 85% Supportiveness 4.3 .67 86% Action Orientation 4.3 .62 87% Personal Style 4.2 .66 76%

F. Analysis

A reasonable amount of variation in team-based skills is expected upon program entry, with the instructional emphasis being on improving team effectiveness through the 360 assessment and other class activities. This assumption is reflected in the survey results of