Industrial Engineering and Systems Engineering. Faculty of Technology, Policy and Management Delft University of Technology

(1)

July 2010

Industrial Engineering and

Systems Engineering

Faculty of Technology, Policy and

Management

(2)

Quality Assurance Netherlands Universities (QANU) Catharijnesingel 56 P.O Box 8035 3503 RA Utrecht The Netherlands Phone: 030 230 3100 Fax: 030 230 3129 E-mail: info@qanu.nl Internet: www.qanu.nl © 2010 QANU

Text and numerical material from this publication may be reproduced in print, by photocopying or by any other means with the permission of QANU if the source is mentioned.

(3)

Foreword

5 Preface

7 Part I

General Part

9

1. Structure of the report 11

2. Task and composition of the committee 13

3. Working method of the committee 15

4. Domain-specific framework of reference for Industrial Engineering and Systems

Engineering 17

Part II

Programme Report

23

5. Report on the bachelor’s programme Technische Bestuurskunde, and the master’s programmes Systems Engineering, Policy Analysis and Management, Engineering

and Policy Analysis, and Management of Technology 25

Appendices

93

Appendix A: Learning outcomes Technische Bestuurskunde 95

Appendix B: Learning outcomes Systems Engineering, Policy Analysis and Management 97 Appendix C: Learning outcomes Engineering and Policy Analysis 99

Appendix D: Learning outcomes Management of Technology 101

Appendix E: Curricula Vitae of the committee members 103

(4)

(5)

Foreword

This report describes the findings of the Industrial Engineering and Systems Engineering assessment committee for the bachelor’s programme Technische Bestuurskunde and the master’s programmes Systems Engineering, Policy Analysis and Management, Engineering and Policy Analysis, and Management of Technology. The report is part of the quality assessment of university bachelor’s and master’s programmes in the Netherlands. The purpose of this report is to present a reliable picture of the results of the degree programmes, to give feedback to the internal quality assurance of the programmes, and to serve as the basis for accreditation of these programmes by the Accreditation Organisation of the Netherlands and Flanders (NVAO).

Quality Assurance Netherlands Universities (QANU) aims to ensure independent, unbiased, critically constructive assessments using standardised quality criteria, while taking specific circumstances into account.

The QANU Industrial Engineering and Systems Engineering assessment committee has fulfilled its task with great dedication. The programmes have been evaluated in a thorough and careful manner. We expect that the judgements and recommendations will be carefully considered by the programme organisation and the Board of the University.

We thank the chairman and members of the assessment committee for their willingness to participate in this assessment and for the dedication with which they carried out their task. We also thank the staff of the department concerned for their efforts and for their cooperation during the assessment.

Quality Assurance Netherlands Universities

Mr. Chris J. Peels Dr. Jan G.F. Veldhuis

(6)

(7)

Preface

Our committee visited Delft University of Technology on 31 March and 1 April 2010 to assess the bachelor’s programme Technische Bestuurskunde and the master’s programmes Systems Engineering, Policy Analysis and Management, Engineering and Policy Analysis, and Management of Technology.

To evaluate the programmes in Delft, the committee carefully read the programmes’ self-evaluation reports and many supportive documents, and talked to the programme management, staff, students and alumni during the site visit. The committee appreciated the quality and clarity of the documentation provided and the willingness of those involved in the programmes to address the questions we raised. On the basis of this, the committee was able to form a well-reasoned opinion of the various aspects of the programmes, as summarized in this report.

On behalf of the committee members, I would like to thank all of those involved in the preparation and execution of this assessment procedure for their contributions and support, both at the Delft University of Technology and at QANU. Without their effort and their willingness to respond constructively to the many requests from the committee, we could not have carried out the work as smoothly and pleasantly as we did.

Also, I gratefully acknowledge the contributions of the other committee members. In a very pleasant and creative atmosphere, we have been able to work through the whole process to produce this report. In this context, the support of Trees Graas deserves a special note of appreciation. Without her, we would have been nowhere.

Ludo Gelders

(8)

(9)

(10)

(11)

1. Structure of the report

In this document, the Industrial Engineering and Systems Engineering assessment committee (hereafter: the committee) reports its findings. The report consists of two parts: a general part (Part I) and a programme part (Part II).

The general part summarises the tasks, composition, input documentation and working methods of the assessment committee. This part of the report also contains the domain-specific requirements that were used by the assessment committee. The programme part describes the evaluation and assessment of the bachelor’s programme Technische Bestuurskunde and the master’s programmes Systems Engineering, Policy Analysis and Management, Engineering and Policy Analysis, and Management of Technology at Delft University of Technology. This programme part is structured in accordance with the accreditation criteria of NVAO (Accreditation Organisation of the Netherlands and Flanders).

(12)

(13)

2. Task and composition of the assessment committee

2.1. Task of the committee

The task of the Industrial Engineering and Systems Engineering committee is to evaluate and assess nine degree programmes at the three technical universities according to the accreditation criteria set by NVAO. Based on and in accordance with these criteria, the committee is expected to assess different aspects of quality of the programmes, according to the information provided by the programmes in the self-evaluation reports and discussions held during the site visits. The assessment report contains recommendations by the committee, but the emphasis lies on assessing the programmes’ basic quality.

The assessment committee has been requested to assess the following programmes (including CROHO number):

Delft University of Technology:

• Bachelor’s programme Technische Bestuurskunde (56995)

• Master’s programme Systems Engineering, Policy Analysis and Management (60358)

• Master’s programme Engineering and Policy Analysis (60179)

• Master’s programme Management of Technology (66995) Eindhoven University of Technology:

• Bachelor’s programme Technische Bedrijfskunde (56994)

• Master’s programme Innovation Management (60430)

• Master’s programme Operations Management and Logistics (66430) University of Twente:

• Bachelor’s programme Technische Bedrijfskunde (56994)

• Master’s programme Industrial Engineering and Management (60029) 2.2. Constitution of the committee

The committee consists of a chairman and seven members. Appendix E lists short descriptions of the curricula vitae of the committee members.

Chair

• Prof. dr. ir. L.F. (Ludo) Gelders, emeritus professor of Industrial Management, Katholieke Universiteit Leuven (University of Leuven), Belgium.

Members

• Prof. dr. J. (Jan) Kratzer, professor of Entrepreneurship and Innovation Management, Institute of Technology, Berlin, Germany;

• Prof. dr. J. (John) Grin, professor of Policy Science, especially System Innovation, University of Amsterdam;

• Ir. J.R. (Hans) Wierda, self-employed advisor on competence development for Centres of Excellence;

• Dr. C. (Cees) Terlouw, associate professor of Enrollment Management and Educational Transition, Saxion University of Applied Sciences, and director of LICA;

• Drs. N.J. (Nynke Jo) Smit, Academic Registrar and Head of the Office of Educational Affairs, Institute of Social Studies, Erasmus University Rotterdam;

(14)

• F. (Frank) Pijnenborg,MSc student of System Engineering, Policy Analysis and Management, Delft University of Technology;

• R.M. (Richelle) Rijntjes BSc, MSc student of Industrial Engineering and Management, University of Twente.

Prof. Gelders, Prof. Kratzer, Prof. Grin and Ir. Wierda participated in all the site visits of the committee. C. Terlouw was involved in the assessment of the programmes at Delft University of Technology and Eindhoven University of Technology. Drs. N.J. Smit was involved in the assessment of the programmes at the University of Twente. F. Pijnenborg participated in the site visits at Eindhoven University of Technology and the University of Twente. R.M. Rijntjes Bsc was involved in the assessment of the programmes at Delft University of Technology.

All members of the assessment committee signed a declaration of independence, as required by the QANU protocol, to ensure that they judge without bias, personal preference or personal interest, and the judgement is made without undue influence from the institute, the programme or other stakeholders.

The project leader of the assessment was drs. M. (Trees) Graas, QANU staff member. The site visit took place on 31 March and 1 April 2010. The programme of the site visit is included as appendix F.

(15)

3. Working method of the committee

3.1. Introduction

The committee was constituted formally on 8 February 2010. During this inaugural meeting the committee discussed its task and the working methods. Furthermore, it discussed the proposal for domain-specific requirements. This proposal was adjusted and subsequently instituted as the domain-specific framework of reference, provided in chapter 4 of this report.

3.2. Preparatory phase

After receiving the self-evaluation reports, the project leader checked the quality and completeness of the information provided. After approval, the self-evaluation reports were forwarded to the committee. During the initial meeting at the start of the site visit, the committee members discussed their findings.

In addition, the committee members each read at least one thesis for the programmes being assessed. This led to the assessment of at least six theses for each programme. When considered necessary, committee members read additional theses during or after the site visit. Selection of the theses was done at random by the chair of the committee. Specific attention was paid to the scientific level of the theses, the requirements, and carefulness of judgement by the reviewer of the programme and the assessment procedure used. Since the evaluated programmes lead to a scientific degree, the student has to show evidence of the required qualifications to earn a degree in the thesis.

Within the committee a specific allocation of tasks was agreed upon, based on the expertise of its members. It should be emphasized that although specific tasks are assigned, the entire committee remains responsible for the judgements and the final report.

3.3. Site visits

Before each site visit the project leader created a programme for the interviews. The draft programmes were discussed with the chair of the committee and the coordinators of the programmes. During the site visits, interviews were held with representatives of the faculty boards, the programme management, alumni, programme committees, boards of examiners, and study advisors. Furthermore, for each programme selected students and lecturers were interviewed.

During the site visits the committee examined additional information, for example study books and reports from the meetings of the programme committees. A consultation hour was scheduled to give students and staff of the programmes the opportunity to talk to the committee.

The committee used a significant part of the final day of a site visit to discuss the assessment of the programmes and to prepare a preliminary presentation of its findings. Each site visit concluded with a presentation by the chairman, consisting of a general assessment and several specific findings and impressions of the programme.

3.4. Scores of the standards

The committee adopted the standard decision rules provided by QANU. These are:

• Unsatisfactory, which means that the level for this facet is below the basic standard of quality;

(16)

• Satisfactory, which means that the level meets the basic standards of quality;

• Good, which means that a quality level is attained that exceeds the basic standards of quality;

• Excellent, which means that a quality level is attained that is very good in all aspects and meets international benchmarking. It is an example of international best practice.

The default assessment is ‘satisfactory’, i.e. the programme complies adequately with the criteria.

The committee feels that despite critical remarks, the score ‘satisfactory’ can be given to a specific standard. In those situations, the critical remarks will be accompanied by positive observations. Furthermore, the committee is of the opinion that if the programme management deals adequately with the critical remarks, the score ‘satisfactory’ might become ‘good’ at the next site visit.

When the assessment committee observes a good national practice, the judgment will be ‘good’. When both a good practice is observed and a critical remark is made by the committee, a weighed average score is given.

3.5. Reporting

After each site visit the project leader wrote a draft report based on the findings of the committee. The draft report was read and commented upon by the committee members. It was then sent to the faculty involved to check for factual irregularities. Any comments of the faculty were discussed with the chair of the committee and, if necessary, with the other committee members. The final report was endorsed on July 9, 2010.

(17)

4. Domain-specific framework of reference for Industrial

Engineering and Systems Engineering

This chapter contains a short summary of converging views on the field of Industrial Engineering and Systems Engineering (IE&SE). These views have been gathered from the following organizations that focus on the professional development and application of the field:

• the Global Association of Productivity and Efficiency Professionals (IEE);

• the Manufacturing and Service Operations Management Society (MSOM);

• the Institute for Operation Research and the Management Sciences (Informs);

• the International Council on Systems Engineering (INCOSE);

• and the Council of Engineering Systems Universities (an association of more than 50 universities in North America, Europe, Asia, and Australia with the aim to work together to develop engineering systems as a new field of study);

and from the following leading academic programmes in the field of IE&SE:

• Industrial Engineering and Operations Research at the University of California, Berkeley, USA; Industrial and Systems Engineering at the Georgia Institute of Technology, USA;

• Management Science and Engineering at Stanford University, California, USA;

• Engineering and Public Policy at the College of Engineering at Carnegie Mellon University, Pittsburgh, USA;

• Systems Engineering and Operations Research at George Mason University, Washington DC, USA;

• Engineering Systems at Massachusetts Institute of Technology, Cambridge, Massachusetts, USA.

A few excerpts from these views are included in section 4.4.

Although converging views are emerging for IE&SE, different domain names with slightly different emphases are also used for the field. Industrial Engineering is also known as Operations Management, Production Engineering, or Manufacturing Engineering, a distinction that seems to depend on the viewpoint or motives of the user. In healthcare, for example, IE&SE graduates are more commonly known as management engineers, engineering management, or even health systems engineers. Systems Engineering is sometimes known as Engineering Systems Design. Also, the examples in section 4.4 demonstrate the different emphases educational institutes give to the field.

Nevertheless, there are a number of clearly common elements that constitute the shared view of what the field of IE&SE represents in education, application, and research. Below we will first discuss the common elements, and then we will formulate some generic competences as a consequence of these common elements.

4.1. Common elements of the field of IE&SE

These common elements concern: (a) the common basis, (b) the focus: design, installation, and improvement of processes and systems, (c) broad application in private and public domains and within and between organisations, (d) the application of quantitative methods, and (e) complex problem-solving with a scientific and a pragmatic multidisciplinary approach.

(18)

(a) The common basis

Industrial Engineering (IE) and Systems Engineering (SE) are interrelated. IE is concerned with the design, improvement and installation of integrated systems of people, information, materials, equipment and energy. It focuses on the analysis, design and control of operational processes, at the level of both individual organisations and supply networks. SE mainly focuses on inter-organisational questions that involve the use of technology and the interests of multiple stakeholders, typically linking public and private organisations. As a consequence, the common basis of IE and SE draws upon specialised knowledge and skills in the mathematical, physical and social sciences together with the principles and methods of engineering analysis and design in order to specify, predict, and evaluate the results to be obtained from the systems involved.

(b) The focus: design, installation, and performance improvement of processes and systems IE&SE is concerned with the design and improvement of operational and/or strategic processes and integrated systems. These processes or systems provide products or services to customers or to the society at large. As such, both private and public organisations are concerned. The design and improvement of processes and systems have multiple goals concerning time, money, materials, energy and other resources. Several organizations and multiple stakeholders may be involved (supply chains, alliances, public-private partnerships), and governance structures can form part of design and improvement initiatives. In summary, IE and SE graduates may be considered productivity and efficiency professionals.

(c) Broad application, in both private and public domains and both within and between organizations

IE&SE is used in a variety of fields. It applies to all steps in the product life cycle, from research and development through design, manufacturing, distribution and disposal. And it applies in all phases of the value chain. Whereas the initial applications were mainly limited to industrial settings, we now witness more and more applications in the service industry. Its principles apply as well in all fields of the private and in the public sector. Nowadays, there is a fast growth of applications in banking, healthcare, transportation, and the like.

Therefore, the term ‘industrial’ can be misleading; this does not mean just manufacturing. It encompasses the service industries as well. It has long been known that industrial engineers have the technical training to make improvements in a manufacturing setting. However, many of the same techniques can be used to evaluate and improve productivity and quality in a wide variety of service industries, as well as in the public sector. The term ‘Systems Engineering’ emphasizes this broader scope for design, improvement, and problem-solving.

(d) The application of quantitative methods

IE&SE is a field of engineering, and one important element of its approach to the design and improvement of processes and systems is the use of quantitative methods. These are derived from fields such as operations research, management science, mathematics, economics, data analysis and statistics, information systems, and engineering.

(e) Complex problem-solving with a scientific and a pragmatic multidisciplinary approach Complex problems are central to IE&SE. In order to be able to solve these kinds of problems, it is necessary to synthesize knowledge from different disciplines (e.g. engineering, economics, mathematics, organizational behaviour, and psychology, although not all disciplines are equally important in all problem domains). IE&SE draws upon specialized knowledge and analytical skills in the mathematical, physical, and social sciences, together with the principles and methods of engineering analysis and design. Unlike traditional

(19)

disciplines in engineering, IE&SE addresses the role of human decision-makers and other stakeholders as key contributors to the inherent complexity of systems. The programmes offer the relevant knowledge and skills from different disciplines and provide a framework for the application and integration of this knowledge in analysing a problem situation and in designing and implementing solutions. In brief, IE and SE graduates support scientific decision-making.

In addition, IE&SE graduates ought to be pragmatic people. They work to understand and resolve real problems from society and hence – as stated above – need to combine knowledge and experience from many disciplines to develop project and process management expertise and communication skills. They choose a method to fit the problem, which means that they combine the quantitative and problem-solving approach of engineers with research methods and qualitative insights from the social sciences.

4.2. Generic competences

Taking into account the above-mentioned common elements of the field, the generic competencies for industrial and systems engineering are listed below:

• Sufficient understanding of technology and technological innovation;

• Keen analytic mindset combined with a drive to synthesize towards a solution;

• Competent in translating complex issues into workable models and designing and executing appropriate research programmes;

• Adequate mathematics skills for modelling and executing research activities;

• Adequate understanding and competencies in a number of technical, economic and social disciplines to underpin research programmes;

• An adequate understanding of the drivers of socio-economic and political organizations in society;

• Able to organize and aim for efficiency and effectiveness;

• Resourcefulness and creative problem-solving;

• Excellent communication, listening, and negotiation skills;

• Ability to adapt to many environments, interact with a diverse group of individuals and understand the roles of various stakeholders in the processes.

4.3. Bachelor and master level

The specific blend of competencies varies per programme and is laid down more specifically in the final qualifications of each programme. Although the emphasis differs among the programmes, there is a distinction between the bachelor and master levels regarding:

• Complexity of the problem situations (in terms of technical and/or stakeholder complexity and/or the number of disciplines involved);

• Amount of information necessary, known, and available from the practical problem situation;

• Level of autonomy.

Bachelor’s students receive a sound general education in all basic fields of IE&SE (technology, engineering, optimisation, engineering economy, business economy, organisational theory, social sciences, et cetera). They should be able to continue their studies in a more in-depth and specialised master’s programme or fill appropriate positions in business.

(20)

Master’s programmes in IE&SE generally offer different fields of study in which students can specialise. Examples of such fields are Operations Management, Operations Research and Management Science, Communication and Information Technology, Product Design and Logistics, Policy Analysis, Man-Machine Systems, Performance Analysis, and Supply Chain Management.

Whereas bachelor’s students are mainly involved in analysis, master’s students typically deal with design questions. In addition, they should also be exposed to research questions. Master’s graduates should be able to formulate and carry out independent research projects.

The IE&SE bachelor’s programmes provide an excellent basis for one of the IE & SE master’s programmes, but students in IE&SE master’s programmes can also come from a variety of undergraduate backgrounds in engineering and other quantitative fields.

Graduates of a master’s programme will typically start their career as project or planning managers, functional managers, policy analysts/advisers, engineering consultants, et cetera. But they could also start an academic track through further involvement in research (e.g. PhD and academic positions). They should be able to move on later to managerial positions (e.g. as CTO). Some may prefer to become private entrepreneurs.

4.4. Excerpts

Institute of Industrial Engineers (IIE) Definition of Industrial Engineering: ‘IE is concerned with the design, improvement and installation of integrated systems of people, materials, information, equipment and energy. It draws upon specialised knowledge and skill in mathematical, physical and social sciences together with the principles and methods of engineering analysis and design, to specify, predict and evaluate the results to be obtained from such systems’. (www.iienet.org/Details.aspx?id=282)

Stanford Engineering established the Department of Management Science and Engineering (MS&E) five years ago: ‘Engineers know how to analyze and solve problems, and they thoroughly understand technology. With this quantitative background and additional training, for example in social sciences or finance, engineers should therefore be leaders in management and public policy. The department’s eight research areas [are]: Organizations, Technology Management and Entrepreneurship; Production and Operations Management; Decision Analysis and Risk Analysis; Economics and Finance; Optimization and the Analytical Tools of Systems Analysis; Probability and Stochastic Systems; Information Science and Technology; and Strategy and Policy. MS&E also includes several centres and programs such as the Energy Modelling Forum and the Centre for Work, Technology and Organization. In addition, it hosts the Stanford Technology Ventures Program.

The department’s strengths are also manifest in the talents of students and alumni who work in Investment Banking, Management Consulting, and other fields that have not been closely associated with engineering in the past. These fields will be in the future because a deep understanding of technology has become critical to their operations. “For example, a growing number of people address finance problems using methods that have been traditionally associated with engineering systems analysis,” says Paté-Cornell, referring to the fast-growing specialty of financial engineering. Paté-Cornell’s hope is that more engineers will also join the ranks of government and use their skills to shape and implement policies. MS&E students gain the training that they need to be leaders in finance, industry, policy, or other specialties by completing a core engineering curriculum, followed by a concentration in an area such as

(21)

finance, operations research, production, or public policy.’ (www.stanford.edu/dept/MsandE /about/MSandE-5yr.pdf)

Georgia Tech: ‘Industrial engineering (IE), operations research (OR), and systems engineering (SE) are fields of study intended for individuals who are interested in analyzing and formulating abstract models of complex systems with the intention of improving system performance. Unlike traditional disciplines in engineering and the mathematical sciences, the fields address the role of the human decision-maker as key contributor to the inherent complexity of systems and primary benefactor of the analyses. In short, as practitioners and researchers in IE/OR/SE, we consider ourselves to be technical problem solvers. We are typically motivated by problems arising in virtually any setting where outcomes are influenced by often complicated and uncertain interactions, involving a variety of attributes that affect system performance. Against this backdrop, students have historically been attracted to our academic programmes with a variety of career objectives and from a host of disciplines and academic interests.’ (www.isye.gatech.edu)

(22)

(23)

(24)

(25)

1. Report on the bachelor’s programme Technische

Bestuurskunde and the master’s programmes Systems

Engineering, Policy Analysis and Management, Engineering

and Policy Analysis, and Management of Technology offered by

Delft University of Technology

Administrative data

Bachelor’s programme Technische Bestuurskunde: Name of the programme: Technische Bestuurskunde

CROHO number: 56995

Level: bachelor

Orientation: academic

Number of credits: 180 EC

Degree: Bachelor of Science

Mode(s) of study: full-time

Location(s): Delft

Expiration of accreditation: 27 Sept 2011

Master’s programme Systems Engineering, Policy Analysis and Management: Name of the programme: Systems Engineering, Policy Analysis and Management

CROHO number: 60358

Level: master

Degree: Master of Science

Location(s): Delft

Master’s programme Engineering and Policy Analysis: Name of the programme: Engineering and Policy Analysis

CROHO number: 60179

Level: master

Location(s): Delft

Master’s programme Management of Technology: Name of the programme: Management of Technology

CROHO number: 66995

(26)

Orientation: academic Number of credits: 120 EC

Location(s): Delft

The site visit of the Industrial Engineering and Systems Engineering assessment committee to the Faculty of Technology, Policy and Management of Delft University of Technology took place on 31 March and 1 April 2010.

1.0. Structure and organization of the faculty

The Faculty of Technology, Policy and Management (TPM) of Delft University of Technology (TU Delft) was founded in 1997. TPM offers a wide range of educational products for many different target groups. The spectrum stretches from subjects and subject clusters to complete programmes and from initial degree programmes to post-doctoral courses. The content provided focuses on:

• methods for analysis, design and management of multi-actor systems,

• domains (technological application domains, e.g. transport, energy),

• aspects (safety, economic, sustainability, ethical, legal, etc.).

The Dean of TPM is responsible for the programmes. Since 2004, the Dean of TPM has further delegated some of his responsibilities to a director of education. The Director of Education at TPM develops and manages the faculty education portfolio. She coordinates the team of directors of studies, one for each degree programme and one for interfaculty education, and is responsible for the coherence between the different programmes. She oversees the implementation of university-wide agreements and acts as a representative of the faculty in various internal and external education-related bodies. The Director of Education is supported by the Head of Education and Student Affairs of TPM. The Head of Education and Student Affairs coordinates the education support processes, which include quality monitoring, study counselling and internationalization.

The directors of studies are responsible for the organisation of the education within their degree programmes, including policy formulation and preparation. They are responsible for the quality of the curriculum and facilitate all aspects of the development of their programmes. The Director of Education and the directors of studies are appointed for three years, with extension being optional.

The Director of Education and the directors of studies make up the Faculty Educational Management Team (OMT) together with the Head of Education and Student Affairs, the Commissioner for Bachelor’s Education and the Commissioner for Master’s Education of the Curius study association. The OMT members contribute to the strategic educational policy of the faculty. The Director of Education submits the policy proposals of the OMT to the Dean and the Faculty Management Team (MT). The MT, chaired by the Dean, consists of the four heads of research department, the Director of Research, the Director of Education, the Director of the Institute for Technology and Communication, the faculty secretary, the Head

(27)

of Finance & Control and the Head of Human Resource Management. One student sits on the MT in an advisory capacity.

1.1. The assessment framework 1.1.1. Aims and objectives

S1: Subject-/discipline-specific requirements

The intended learning outcomes of the programme correspond with the requirements set by professional colleagues, both nationally and internationally and the relevant domain concerned (subject/discipline and/or professional practice).

Description

According to the self-evaluation report, the bachelor’s programme Technische Bestuurskunde (TB) and the master’s programmes Systems Engineering, Policy Analysis and Management (SEPAM), Engineering and Policy Analysis (EPA) and Management of Technology belong to the broad field of Industrial Engineering and Systems Engineering (IE and SE).

The TB bachelor’s programme and SEPAM master’s programme are SE-oriented. The systems which are considered are socio-technical systems, and the programmes are aimed at inter-organisational issues at the interface of public and private organisations. The self-evaluation report points out that the TB bachelor’s programme concentrates on the analysis of problems related to socio-technical systems, while the SEPAM master’s programme focuses on the design of such systems. The self-evaluation report provides a comparison of the learning outcomes of both programmes with those of other, comparable programmes within the same discipline, nationally and internationally. This benchmark study demonstrates that the combination of technology, policy and management within both programmes is unique.

The EPA master’s programme is also a SE-oriented programme focusing on the analysis and modelling of socio-technical systems. The self-evaluation report points out that in contrast to the SEPAM programme, students require a bachelor’s degree in a mono-disciplinary technical or natural sciences field to enter the programme. In addition, the programme is geared more to problems in an intercultural context with an orientation towards sustainable development.

The self-evaluation report explains that the MoT master’s programme is IE-oriented, but is aimed specifically at innovation and research & development management in high-tech industries. In order to compare the MoT programme, a benchmark study was carried out. The self-evaluation report states that some comparable programmes focus more on scientific work, while others are more specialized in entrepreneurship. MoT enables specialization in both directions, which is considered a strong point. In comparison with similar programmes, the Finance and Marketing fields are covered less by the MoT programme.

The objective of the TB bachelor’s programme is to enable students to contribute to the design and management processes of complex socio-technical systems in today’s society. These systems evolve in the power game between providers and users in which the government bears a large degree of responsibility, given the social import of such systems.

According to the self-evaluation report, TB specialists make their contributions in the field of integrated systems analysis and decision-making process management. Their core quality is the ability to structure and analyse technically and administratively complex issues, and make them more manageable. The programme aims to provide students with a sound knowledge of the fields of technology, policy and management, with a stress on technical application. Graduates have the skills and techniques needed for analysing multi-actor problems. They

(28)

have command of the principles of mathematical modelling, economics, law and organisation theory.

In addition, TB graduates have studied specific technical areas of application in greater depth. They have deployed specific methods and techniques from these particular specialism’s and also have extensive experience in applying methods and techniques from the more generic subjects to their specialist field. A list of intended learning outcomes of the programme is attached as appendix A.

The objective of the SEPAM master’s programme is to educate students as designers and managers of complex multi-actor systems, and of their policy and decision-making processes. According to the intended learning outcomes of the programme, SEPAM graduates are familiar with existing scientific knowledge related to the analysis, design and management of multi-actor systems and have the competence to increase and develop this knowledge through study. In addition, they have learned to produce a research plan and conduct research independently in order to acquire new scientific knowledge. To be able to conduct research properly, the graduate is able to critically examine existing theories, models and interpretations in the field of multi-actor design and to use, develop and validate models for designing new solutions. Furthermore, a SEPAM graduate is competent in reasoning, critical reflection, and forming a well-reasoned opinion, and is able to perform and lead project-based team work in national or international settings.

SEPAM teaches students to reformulate ill-structured design problems and to make sound judgments in the absence of complete information. Graduates are able to communicate their findings in English, orally as well as in writing, to specialist and non-specialist audiences, both academic and professional. Students are taught self-direction and originality in tackling and solving problems, collaboration in inter-disciplinary teams and autonomous action in planning and implementing tasks at a professional or equivalent level.

Finally, SEPAM teaches students to analyse and discuss the social consequences of new developments, scientific thinking and acting, and the related ethical and normative aspects, and to integrate them into their scientific work. Graduates are taught to assign equal weight to technical as well as to social factors in the design process of large-scale socio-technical systems, and to understand and conceptualize systems as products of interacting technical and social aspects. For an elaborate list of intended learning outcomes of the SEPAM programme, see appendix B.

The EPA master’s programme intends to educate students as policy analysts for a range of technological sectors. The programme focuses on decision-making processes for large-scale systems, in particular infrastructures for transport, telecommunication, energy, water, waste, industrial production and innovation. The programme has been designed to transfer multidisciplinary knowledge and practical skills in the areas of problem structuring, systems analysis, policy analysis, systems modelling and design, decision support, socio-economics and intercultural management to candidates with a bachelor’s degree in a relevant technical or engineering discipline.

The EPA approach starts from the viewpoint that modern technology is deeply embedded and intertwined in society; that sound analysis and problem structuring are essential for good problem-solving; that dealing with uncertainty is essential for good problem analysis; that there is no single best solution to complex multi-actor problems; and that models can help to learn about systems behaviour. An international benchmark study showed that there are a

(29)

limited number of peer programmes, most of which are located in the United States. However, none of these programmes offer the same combination of analytical, managerial and modelling techniques focusing on the same domain.

The objective of the MoT master’s programme is to educate students as technology managers, analysts of technological markets (either as scientists or consultants), and entrepreneurs in highly technology-based, internationally oriented and competitive environments for a variety of industrial sectors. The MoT programme focuses on decision-making in a technological context, technology and strategy, knowledge management, research and development management and innovation processes.

According to the intended learning outcomes, both EPA a MoT graduates are familiar with existing scientific knowledge. For EPA the main disciplines are policy analysis, systems modelling, intercultural management and economics, for MoT it is management sciences. Graduates of both programmes are able to increase and develop their knowledge. For example, MoT graduates are able to adapt and apply the models of management sciences in a high-tech engineering environment.

Furthermore, EPA and MoT graduates can acquire new scientific knowledge through research. To carry out research, graduates have to acquire a systematic approach characterised by the development and use of theories, models and coherent interpretations. Moreover, they develop a critical attitude and gain insight into the nature of science and technology. The intended learning outcomes of both programmes also include the ability to design conceptual and quantitative models as a synthesising activity aimed at analysing systems and systems behaviour.

EPA and MoT graduates are competent in reasoning, reflecting, and forming a judgment. They are also capable of adequate interaction, have a sense of responsibility and leadership, and possess good intercultural and interdisciplinary communication skills that allow them to participate effectively in a scientific or public debate.

The self-evaluation report emphasizes that science and technology always have a temporal, market and social context. Beliefs and methods have their origins; decisions have social consequences in time. EPA and MoT graduates are aware of this and can integrate these insights into their own scientific work or - for MoT graduates - managerial work.

Comprehensive lists of the intended learning outcomes of the EPA and MoT programmes are attached as appendices C and D, respectively.

Assessment

The committee studied the intended learning outcomes of the programmes in order to investigate whether the learning outcomes meet the objectives as stated in the domain-specific framework of reference.

The committee concluded that the general aims of the TB bachelor’s programme, to provide students with a sound knowledge of the field of technology, policy and management based on technical application and the skills and techniques needed for analysing multi-actor problems, correspond with the domain-specific requirements - set by (inter)national professional colleagues - as formulated for the assessment of programmes in Industrial Engineering and Systems Engineering.

(30)

The committee appreciates the deliberate decision to focus on the design and management of socio-technical systems and, alongside this, on the skills to analyse complex multi-actor problems. However, the committee observed that the intended learning outcomes of the bachelor’s programme are formulated in rather general terms. The committee advises to reformulate the learning outcomes and make them more specific for TB. Moreover, the committee feels it would be beneficial for the consistency with the overall programme objectives as well as for the external identity of the programme to consider the addition of another learning outcome, in which the multidisciplinary character is explicitly expressed.

The committee also established that the competencies of the programme refer to skills which graduates need in professional practice. For instance, the learning outcomes which express the ability to work in multi-disciplinary teams, to be aware of the various roles in collaborative partnerships, and to carry out project-based work within the allocated time and resources are considered relevant professional skills.

The committee studied the intended learning outcomes of the master’s programmes and established that the intended learning outcomes of both the SEPAM and EPA programme correspond with the competencies set in its domain-specific framework of reference. The intended learning outcomes of the MoT programme correspond less with the demands of the scientific discipline as laid down in this framework. However, the committee believes that with the focus on management sciences in a technological context and the emphasis on the ability to conduct research and on the competencies needed for communicating and co-operating in intercultural and multi-disciplinary environments, the learning outcomes sufficiently match the generic competencies as formulated by the domain and obviously have a clear link with the (inter)national professional field.

The committee further states that with the TB bachelor’s programme and the SEPAM master’s programme, the TPM faculty was able to find a unique niche. The committee considers the mixture of technology and management with a focus on analysing multi-actor problems a welcome addition to the mono-disciplinary technology and management programmes in the Netherlands. However, the committee feels that the branding of this unique and valuable profile leaves room for improvement. The committee advises making this profile reflect the intended learning outcomes more explicitly in order to do justice to the identity and value of these programmes. In this framework one can wonder if the separation made between analysis and design as it is programmed in the bachelor’s and master’s programmes is in accordance with the professional practice, also taking into account that ‘conceptual design’ is a topic in the current bachelor’s programme.

Moreover, the committee would like to draw attention to the importance of insight into finance and marketing for a programme in MoT. The committee believes that knowledge of finance and marketing is essential for future technology managers particularly in a multi actor environment. During the site visit, it became clear that contrary to earlier positions the programme management is planning to introduce more corporate finance and marketing into the curriculum. The committee applauds this decision as recognition to offer a more comprehensive programme to students and strongly recommends including this aspect in the intended learning outcomes.

Finally, the committee addresses the professional relevance of the programmes. According to the self-evaluation report, the programme management uses alumni surveys to establish which knowledge and skills alumni consider particularly relevant for their careers. In addition, it mentions that a few modules are taught by guest lectures from relevant organisations and

(31)

companies and that many graduation projects are carried out in external organisations. However, the employers themselves are not asked for their opinion on the intended learning outcomes, and therefore have no direct influence on the identification of learning objectives, attainment levels or the design of the curricula. The committee feels this is an omission given that the future employers who receive the majority of the graduates are important actors and considering the specific multi-disciplinary field in both the public and private domain at which the programme is aiming. The committee is convinced that the programme management is aware of this problem and supports the recent initiative of the TPM faculty to install an advisory board to give existing links with the professional practice a more formal structure. The committee strongly supports the intention to introduce advisory boards with representatives of both public and semi-public authorities and private industry to strengthen the correspondence of the intended learning outcomes with the (inter)national professional practice.

Taking the preceding considerations into account, the committee concludes that the intended learning outcomes of all programmes correspond sufficiently with the requirements set by professional colleagues.

Bachelor’s programme Technische Bestuurskunde: the committee assesses this standard as satisfactory.

Master’s programme Systems Engineering, Policy Analysis and Management: the committee assesses this standard as satisfactory.

Master’s programme Engineering and Policy Analysis: the committee assesses this standard as satisfactory.

Master’s programme Management of Technology: the committee assesses this standard as satisfactory.

S2: Bachelor and master level

The intended learning outcomes of the programme correspond with the general, internationally accepted descriptions of a Bachelor’s qualification or a Master’s qualification.

Description

The self-evaluation report contains a table which links the intended learning outcomes of the TB bachelor’s programme (see appendix A) to the Dublin descriptors:

Knowledge and understanding: 1, 5, 8, 10 and 11.

Applying knowledge and understanding: 2, 3, 4, 5, 6, 7, 8, 9, 12, 13, 14, 15 and 16. Making judgements: 2, 3, 4, 6, 9, 12, 13 and 15.

Communication: 4, 5, 6, 15 and 16. Learning skills: 1 and 5.

In addition, the self-evaluation report explains that the objectives of the programme focus on the analysis of complex socio-technical systems, with a view to students entering professional practice or continuing with a master’s programme. For example, the methods and techniques are aimed at the actors and factors that play a role in decision-making processes, and on the driving forces, options, and limitations involved. This makes it possible to identify problems and areas where solutions may be found. The process of analysing is therefore a preparation for the process of designing this type of system. Students who continue with the SEPAM master’s programme will take on the design process as the main focus – here, the design space is determined, and the best possible design options are established on the basis of certain criteria.

(32)

The intended learning outcomes of the three master’s programmes have been derived from the objectives of each programme and formulated in line with the criteria for academic bachelor’s and master’s curricula, also known as the ‘Meijers criteria’ for academic competencies. According to the self-evaluation report, these academic competencies can be regarded as a translation of the more broadly defined Dublin descriptors into operational terms for technical universities. In accordance with the general profile, a division into seven competence categories is made. The first three competences focus on the domain, followed by three referring to the methods, while the last competence focuses on the context. Like the Dublin descriptors, the Meijers criteria distinguish between bachelor’s level and master’s level. The Meijers criteria have been jointly accepted by Delft University of Technology, Eindhoven University of Technology and the University of Twente (3TU).

The self-evaluation report provides a table which shows how the Dublin descriptors match with the generic Meijers criteria for a technical curriculum:

Dublin descriptor Generic criteria for a technical curriculum

Knowledge and understanding 1. Competent in one or more scientific disciplines Application of knowledge

and understanding

2. Competent in research 3. Competent in design

Making judgements 4. Able to apply basic intellectual skills: reasoning, reflecting and judging

Communication 5. Competent in cooperating and communicating

Learning skills 6. Able to follow a scientific approach

7. Considering the temporal and societal context

The first criterion ensures the knowledge and application of relevant scientific disciplines. The second and third requirements ensure a proper focus on the graduates’ ability to independently plan and perform research activities. Competencies 4, 5, 6 and 7 ensure that students can solve multi-disciplinary problems in practice and take a leading role in the reflection, decision-making, strategy formation and implementing and acting.

Assessment

The committee studied the intended learning outcomes of the four programmes from the perspective of their level. The committee established that the intended learning outcomes of the TB bachelor’s programme reveal that graduates acquire skills and attitudes at a basic level that is characteristic of a bachelor’s programme. For instance, TB students are familiar with the features of the technical systems and components of their chosen field of application and with the structure of the industrial sectors, legislation and policy frameworks that are relevant to it (Dublin descriptor 1). In addition, they develop the ability to use basic notions and theories effectively when formulating and structuring problems (Dublin descriptor 2). Dublin Descriptor 3 - making judgements - is most distinctively shown in the ability to reflect on the validity of information, analytical methods, results obtained and the conclusions that may be made in consequence. TB graduates acquire the ability to communicate effectively both verbally and in writing with professionals in various disciplines (Dublin descriptor 4) and have the ability to acquire new knowledge independently (Dublin descriptor 5).

The committee took note of the decision by the programme management to use the Meijers criteria as the point of departure for the description of the intended learning outcomes of the master’s programmes. The committee considers this an appropriate choice.

The committee established that the intended learning outcomes of the master’s programmes show that students acquire knowledge, understanding, skills and attitudes at an advanced level that is typical of a master’s programme. For instance, SEPAM students have a thorough mastery of multi-disciplinary knowledge and practical skills relevant to the analysis, design and

(33)

management of multi-actor systems. SEPAM graduates are able to apply their knowledge to multi-actor engineering and management problems and are able to reformulate ill-structured research and design problems. In addition to this, they are able to make sound judgements in the absence of complete data.

Similarly, EPA and MoT graduates attain thorough mastery of the latest theories, methods and techniques within the relevant fields and can apply them independently in the context of more advanced ideas or new fields of application. In addition to this, EPA and MoT graduates are able to analyse and discuss the ethical and normative aspects of the consequences and assumptions of scientific thinking and acting and to integrate them into work. Finally, SEPAM, EPA and MoT graduates are able to discuss the field and the place of the field in society and acquire the ability to critically reflect on their own thinking, decision-making and acting and adjust them on the basis of this reflection.

The committee concludes that the level of the four programmes sufficiently corresponds to the Dublin descriptors, which are considered general, internationally accepted descriptions of academic qualifications. The committee would like to add to this conclusion that in view of the connection between the TB bachelor’s programme and the SEPAM master’s programme and a clear differentiation between the levels of both programmes, it would be desirable to consider to also use the Meijers criteria as starting point for the qualifications of the bachelor’s programme.

S3: Academic orientation

The intended learning outcomes of the programme correspond with the following descriptions of a Bachelor’s and a Master’s qualification:

• The intended learning outcomes are derived from requirements set by the scientific discipline, the international scientific practice and, for programmes to which this applies, the practice in the relevant professional field.

• An academic bachelor (WO-bachelor) has the qualifications that allow access to at least one further programme at academic master's level (WO-master) and the option to enter the labour market.

• An academic master (WO-master) has the qualifications to conduct independent research or to solve multidisciplinary and interdisciplinary questions in a professional field for which academic higher education is required or useful.

Description

According to the self-evaluation report, the relationship of the TB bachelor’s programme to the academic discipline is highlighted through several learning outcomes. For example, graduates adopt an academic approach when reflecting on data, methods, results and conclusions. They can see and understand the limitations of methods and deal with them in an appropriate fashion. Moreover, they can discuss problems and ways of analysing them. In addition, TB graduates focus on how to structure and analyse complex multi-actor problems, which require a scientific approach by their very nature.

Students who have completed the TB bachelor’s programme qualify for direct access to the master’s programme Systems Engineering, Policy Analysis and Management (SEPAM), the

(34)

most obvious follow-up master’s programme. Along with SEPAM, TB graduates are also entitled to enter the following 3TU master’s programmes:

• Construction Management and Engineering (TU Delft);

• Geomatics (TU Delft);

• Transport, Infrastructure & Logistics (TU Delft);

• Business Information Technology (University of Twente);

• Philosophy of Science, Technology and Society (University of Twente).

As noted under standard 1, the entrance requirements for the EPA master’s programme specify a bachelor’s degree in a mono-disciplinary technical or natural sciences field. This also applies to the MoT master’s programme. Therefore, TB graduates do not qualify for access to these two master’s programmes.

The self-evaluation report claims that students who have completed the TB bachelor’s programme are able to take responsibility in positions such as junior analyst or junior policy adviser. According to the learning outcomes of the programme, TB graduates are able to make well-founded conclusions from analyses, consider their implications and evaluate the significance of their limitations. They can communicate this information effectively, both verbally and in writing, to experts from a wide range of fields. They learn how to work in a project and as part of a team. They learn how to apply the methods for analysing multi-actor systems to a specialist field of their own choosing, with regard to technology, legislation and policy. They are able to analyse and interpret both quantitative and qualitative data, make a conceptual model, and evaluate existing designs, suggesting areas for improvement, where possible. Although TB graduates are consequently able to enter the employment market, the decision to continue with a master’s programme is, for most students, still a matter of course.

According to the intended learning outcomes of the programme, SEPAM graduates are able to produce practical solutions for complex multi-actor problems, provide a sound scientific justification for these solutions, determine the implications, and communicate and explain the solutions and their implications clearly. The creation of these solutions requires knowledge from various disciplines within the systems engineering and actor systems field. Solutions need to be evaluated from various disciplines and views, which require full awareness of the ethical, normative, social and economic consequences.

The self-evaluation report states that professional requirements are an important part of SEPAM. The learning outcomes assure that SEPAM graduates are equally comfortable in the company of technical experts as they are with management staff. They often work in interdisciplinary environments and are trained as skilled communicators. They take a systematic approach to problem-solving, think analytically, link the technical and social aspects of a situation, quickly recognize the essentials of a situation and recognize general patterns in different issues.

According to the intended learning outcomes of the programme, EPA graduates are able to analyze, structure and model complex socio-technical problems using rational scientific analytical methods and mathematical modelling techniques in combination with their economic and managerial insights. They have learned how to apply a number of analytical and modelling techniques in a scientifically sound way.

EPA students typically have different disciplinary backgrounds and receive an inter-disciplinary training. The self-evaluation report claims that EPA students know from personal experience how to build bridges between disciplines and how to translate scientific insights to

(35)

different social and professional settings. EPA graduates are therefore able to work in interdisciplinary teams. They have experienced the challenges of working in international and intercultural teams and learned how to overcome these barriers in practice.

The self-evaluation report points out that due to their analytical and problem-structuring skills, most EPA graduates enjoy good career development trajectories. It claims that EPA students are also well prepared for a scientific career because of their analytical training and research skills. At least 10% of the graduates of each of the last three cohorts continued in academia either as a researcher or as PhD student.

According to the self-evaluation report, MoT is both an academic and a professional programme. Academically, this implies that by the end of the programme, students must be aware of and able to apply the general insights and skills of the various sub-disciplines relevant within the MoT programme and to integrate them in general explanatory frameworks. Moreover, they must be able to generate new or more up-to-date knowledge on technology management and its various components based on underlying theoretical frameworks, and its application in generally accepted methodologies, concomitant research methods and techniques.

Professionally, this implies that students are well-versed in applying the same analytical and synthetic skills in real-life situations in organisations. The skills regarding the social context in which engineers must operate, both internally in their own organisation and externally in relation to understanding what the current and future technological, economic and social environments require technological firms to do, are developed up to a higher level than those of most other engineers. This is due to the fact that they must be able to analyse and anticipate wider societal trends in which new technological production is to take shape and understand the market at which the resulting products and services are to be sold.

Assessment

The committee studied the intended learning outcomes of the four programmes from the perspective of its orientation. The committee concluded that the intended learning outcomes of the TB bachelor’s programme refer sufficiently to the demands of the scientific discipline. TB students, for instance, acquire the ability to use effectively basic notions and models of commonly accepted organisational theories, actor network theories and decision-making process theories when formulating and structuring problems. In addition, they acquire the ability to see and understand the limitations of research methods and to act accordingly. In the assessment of standard 1, the committee concluded that the intended learning outcomes of the bachelor’s programme also correspond sufficiently to professional requirements, most explicitly in the ability to work in interdisciplinary teams and to carry out project-based work within the allocated time and resources.

In addition to this, the committee noted that graduates of the TB bachelor’s programme have unconditional access to the SEPAM master’s programme and at least five other 3TU master’s programmes.

As described under the preceding standard, the intended learning outcomes of the master’s programmes are formulated in line with the Meijers criteria for academic competencies. These competences explicitly reflect both scientific and professional requirements. The Meijers criteria 2 and 6 (competence in doing research and the development of a scientific approach) are clearly derived from requirements set by the academic discipline, while criteria 3 and 5 (competence in design, cooperation and communication) most explicitly reflect requirements

(36)

derived from the practice of the relevant professional field. In the learning outcomes which describe the awareness of the temporal, market and social context (Meijers criterion 7), requirements from the scientific discipline and from the professional practice are combined. Illustrative in this respect is the learning outcome which refers to the ability to analyse and discuss the ethical and normative aspects of the consequences and assumptions of scientific thinking and acting and to integrate these ethical and normative aspects into scientific work.

The committee therefore concludes that both the bachelor’s programme and the three master’s programmes clearly aim at educating academic researchers as well as professionals. They focus on analysis and design of technology-intensive multi-actor systems, as well as research on these issues. Analysis includes problem definition as well as (ex ante or ex post) analysis of the costs, effects and side-effects of solutions designed to remedy these problems. It is undertaken with an eye to design, and from the idea that there is an interactive relation between analysis and design. In the bachelor’s programme, the emphasis is on skills, knowledge and insights on doing analysis, and some knowledge of designing; in the master’s curricula, the knowledge, insight and skills on design are further developed and deepened. At both levels, there is due attention to research into how to do analysis and design, including the development of theory and methodology. In the committee’s view, the different uses of the key terms ‘analysis’, ‘design’ and ‘research’ and their interrelationships in the bachelor’s and master’s programmes should be clarified. More specifically, it should become clear how analysis relates to conceptual design in the bachelor’s programme.

Taking the preceding considerations into account, the committee concludes that the objectives of all programmes correspond sufficiently with the requirements regarding orientation.

Assessment of the theme Aims and objectives

The committee comes to an overall assessment of the theme Aims and objectives on the basis of its assessments of the separate standards. In the case of the bachelor’s programme Technische Bestuurskunde, it assesses this theme as satisfactory. In the case of the master’s programme Systems Engineering, Policy Analysis and Management, it assesses this theme as satisfactory. In the case of the master’s programme Engineering and Policy Analysis, it assesses this theme as satisfactory. In the case of the master’s programme Management of Technology, it assesses this theme as satisfactory.

Industrial Engineering and Systems Engineering. Faculty of Technology, Policy and Management Delft University of Technology

July 2010