KEY PERFORMANCE INDICATORS TO ANALYZE AND IMPROVE MANAGEMENT OF INFORMATION FLOW IN THE BIM DESIGN PROCESS

Proceedings of the CIB W78-W102 2011: International Conference –Sophia Antipolis, France, 26-28 October

KEY PERFORMANCE INDICATORS TO ANALYZE AND

IMPROVE MANAGEMENT OF INFORMATION FLOW IN THE

BIM DESIGN PROCESS

Leonardo Manzione, PhD Candidate,

[email protected]

Mariana Wyse, MSc Candidate,

[email protected]

Rafael Sacks, Associate Professor,

[email protected]

Léon van Berlo, PhD Candidate,

[email protected]

Netherlands Organisation of Applied Scientific Research TNO, Delft, The Netherlands

Silvio B. Melhado, Associate Professor,

[email protected]

ABSTRACT

Thanks to the interoperability provided by the IFC standard, BIM technologies and IFC model servers are beginning to enable a design environment where the exchange of information among the actors can be synchronous and continuous using a single and central data model. Although this new set of technologies enables concurrent design, the problems associated with managing the flow of information itself in a concurrent design environment requires explicit management of editing rights and version control at the level of individual objects, rather than at the file level. However, while these are technical issues that have standard solutions, managing designers’ involvement in the process also becomes more challenging, requiring the development of new management methods suitable for the BIM collaborative environment.

Common problems such as information ‘overflow’, incomplete modelling solutions or incorrectly matched technical solutions, and inventories of work in progress due to inattentive designers, if not treated methodically in the BIM platform, can quickly cause bottlenecks for the advancement of the process. The bottlenecks result in process waste (such as time spent waiting, large inventories of design information, processing sequences that cause unnecessary iterations, long cycle times and schedule overruns, etc.). Application of concepts that allow structuring and measuring of the information flow can improve the process and reduce the waste of resources, but there is no specific methodology for measuring information flow in a BIM environment. Taking a previous study, in which seven key performance indicators were developed and validated for application with conventional technology, as a starting point, this work has developed the methodology for using these indicators in a BIM project.

Keywords: information flow measure, collaborative design, model server, design management

1.

INTRODUCTION

When talking about building construction, developers’ claims about design delivery being late are common. The design process is informally managed, with inappropriate planning techniques and low utilization of IT resources. There are also claims from the design professionals because of the need for rework, which is responsible for the loss of profits (MANZIONE, 2006).

Due to the interoperability provided by the IFC standard, BIM technologies and IFC model servers are beginning to enable a design environment where the exchange of information among the actors can

be made synchronously and continuously using a single and central data model (KERRYLONDON, 2010). Although this new set of technologies enables concurrent design, the problems associated with managing the information flow itself in a concurrent design environment require explicit management of editing rights and version control at the level of individual objects, rather than at the file level. However, while these are technical issues that have standard solutions, managing designers’ involvement in the process also becomes more challenging, requiring the development of new management methods suitable for the BIM collaborative environment.

The purpose of this paper is to propose a new methodology for the organization, planning and control of design processes within BIM technology, focusing on defining key performance indicators (KPIs) to measure process efficiency.

Information is the fuel for the design process and the proper organization of the flow of information constitutes one of the key actions to ensure that a project’s goals are achieved.

The design process has a variable nature. In the first development stages, the information content has high impact on the final solution, and as the process follows to more advanced stages the impact is reduced, corresponding to the passage from a less structured status with more uncertainty where negotiations are required, to a more structured status, where information flow complexity grows with the larger number of actors involved (Austin et. al., 2002).

Traditionally, the design process has been planned with the same techniques used for construction planning. But these techniques don’t allow representation of the cycles and iteration existing in a process as complex as the design process. They only monitor design’s progress based on the conclusion and delivery of 2D designs instead of analyzing the increment and advance of the design through key information that aggregates along the process. BIM technology potentially increases the rise in information content impact as it enables all actors to work together from the start of the process due to its semantic richness.

This semantic richness of BIM technology generates an additional obstacle to the process as it facilitates integration and exchange between actors and demands a structured methodology for managing the information flow and the design process as a whole. The literature on this subject consists of several published papers defining concepts and methodologies developed towards directing the implementation of BIM as a process (SCIENCES, 2007), (UNIVERSITY, 2010), (KRISTINEK. FALLON, 2007). Other structured methodologies, like ADePT (Austin 1999), (Austin 2002), were developed in an attempt to structure the design process in a generic way, but without focusing on BIM (Austin 2001). More recently, the Inpro Project (Benning et al. 2010(PIERREBENNING, 2010) can be considered as a broader initiative, as it organizes the initial stages of the BIM design process, aligning the management and information technology demands in IFC language within a collaborative Hub.

In the BIM universe, the preparation of Information Delivery Manuals (IDM) (WIX, 2006) has been used as a process structuring methodology. The main goal of developing an IDM and Model View Definitions (MVD) is to define specifications to map information exchange between object models in IFC, for implementation in software interfacing. Later, rules for its use in process models were developed through Business Process Modeling Notation (BPMN) (WHITE, 2004), (OUYANG, 2009)(Ouyang 2009). BPMN was adopted in the IDM methodology as a process mapping tool and has been efficient, for it creates a bridge linking the design process conception and its implementation, establishing stages, defining responsibilities for each involved actor and creating models of information exchange through Exchange Requirements (RAFAEL SACKS, 2010a). Although the developed methodology has consistency and serves its specific purposes, a systemic approach that combines these methods and enables process planning and control in an integrated IFC platform is still lacking.

2.

TOWARDS A BIM INTEGRATED MANAGEMENT MODEL (B.I.M.M)

Applying advanced knowledge in this area, the B.I.M.M. (BIM Integrated Management Model)

methodology, as shown in Figure 1, was proposed to manage the information flow in an integrated way, combining several methodologies of planning and design management. This work is the central theme of the first author’s PhD research. His research is being developed and as its final result it will

establish the necessary procedures for each stage defined from 1 to 11, as well as the proposition of a collaborative Hub for its support.

Figure 1: BIM Integrated Management Model (B.I.M.M.)

The methodology’s steps will be briefly presented, followed by explanation of the detailed procedure to measure the process efficiency through its KPIs. The methodology consists of four main interconnected loops. Their goals, methods and IT tools used are listed in table 1.

A. Process Model Loop

Organizing the design process requires studying all activities predecessors through the information flow and for this purpose the use of BPMN methodology may lead to flaws as it doesn’t allow the optimization of information flow and the entrance of information may occur in advanced moments where they would result in rework cycles. The Design Structure Matrix (DSM) is a planning tool that allows optimization of the information flow and has been intensively and thoroughly studied (STEWARD, 1981). We suggest the use of the DSM combined with the BPMN as a process improvement. In this case the IDM is used as a guide.

B. Planning Loop

In the planning loop, we need to be careful to distinguish between the different levels of resolution of planning. We think the DSM and the Critical Path Method (CPM) should be used for the Master Planning, which sets the major design project milestones, and only the milestones. The planning cycles review not only the completion of each detailed phase, but also within each phase from time to time. After that, in the planning stage, one can use the DSM again together with process mapping or with the detailed IDM process maps to prepare phase schedules. These could be good for say 1-3 month time slots.

At this level of resolution, one can define actual information transfers that can be monitored using the control cycle. The planning for each phase should be a team collaboration effort, where the different actors meet to prepare the process map and define the interchanges of information among themselves. This should produce a practical and doable plan, in a similar way to doing weekly work planning for production activities using the Last Planner System (LPS).

C. Control Loop

Model and workbreakdown

Design Structure Matrix (DSM) CPM/GANTT PLANNING LOOP CONTROL LOOP MODELING IFC ANALYSIS DASHBOARD KPI Measures PPC Last Planner Disciplines Swinlanes 1 4 5 6 7 8 9 MODEL CONCLUSION EDITING MODELLING LOOP Architecture Structure COORDINATION Design Exchanges and Central Model Updates 10 Architecture Structure HVAC Coordination HVAC Exchanges Requirements

B.I.M.M. Collaboration Hub CORRECTIVE ACTIONS PROCESS MODEL LOOP 2 3 2 3 IDM/BPMN 10

Information Delivery Manual (IDM) Business Process

Modelling Notation (BPMN)

11 MODEL EVOLUTION

The Control Loop uses the BIMserver (LÉON VANBERLO, 2010) as a BIM model repository and IFC analyzer (LIPMAN, 2010) as a tool to develop coverage analysis for IFC data. Analyzing the content of the IFC model with IFC analyzer allow us to measure the efficiency of the design through the measures of the information flow with KPIs (RAFAEL SACKS, 2010b). Then the control loop goes in progression from Modeling to BIMserver to IFC analyzer to KPIs and the dashboard.

D. Modeling Loop

The Modeling Loop occurs across all processes, from the start and incrementally as the project progresses. The Planning and Control Loops give feedback to the modeling process.

LOOP STEP NAME GOALS METHODOLOGIES

AND IT TOOLS P RO CE SS M O DE L

_{1 WBS / MBS Define roles and disciplines}

swimlanes

Break the work and the model into manageable parts

Work Breakdown Structure Model Breakdown Structure

2 IDM /

BPMN Process Model Exchange Requirements

Exchange Models

Information Delivery Manual Business Process Modeling Notation (BPMN)

3 DSM Optimize the BPMN information flow Design Structure Matrix

P

L

ANNI

NG

4 CPM Master Plan

Sequencing phases, scheduling tasks and information milestones

Critical Path Method (CPM) MSProject, Primavera, etc.

5 LPS Last Planner, short term tasks and

information delivery

Last Planner System (LPS) and Percent Planned Completed (PPC) CO NT RO L

6 Modeling Modeling in incremental stages BIM software tools

7 Coordination Model Merge and IFC Integration

Clash detection

BIMserver

8 IFC

Analysis Developing Coverage Analysis for IFC Files IFC File Analyzer - NIST

9 Dashboard

KPIs: Key Performance Indicators measure

AR Action Rate BS Batch Size

PS Package Size DV Development Velocity

WIP Work in Process BN Bottleneck

RW Rework

MOD

ELI

NG

_{10 Action From the KPIs the necessary correcting actions are raised to define the next}

steps, process feedback and modeling.

11 Evolution Design model will develop through coordinated evolution cycles until it

reaches its final form.

Table 1: BIM Model Management Procedure: Phases and Steps

3.

KEY PERFORMANCE INDICATORS METHODOLOGY

Structuring the proposed methodology will require several steps, with the measure of process performance being one of the most important steps. For that, this paper’s main goal is to specifically define the KPIs for a BIM environment, and to establish how information flow evaluation will be measured in the management model previously presented.

Taking a previous study of Tribelsky and Sacks (2010b), in which seven key performance indicators were developed and validated for application with conventional technology, as a starting point, this work has developed the methodology for using these indicators in a BIM project. A BIM model of a simple housing building with only the architectural discipline (Figure 2) was used for testing. Five stages of evolution of this model, in which the degree of detail steadily increases, were simulated and for each one of the stages an IFC data was exported and sent to the BIMserver using a

hypothetical user. Later, the IFC data were processed in the IFC analyzer and generated the respective worksheets.

Figure 2: Stages of model’s sample evolution

For KPI calculation some preliminary definitions are needed. These concepts were defined in a previous paper by Tribelsky and Sacks (2010) and are reproduced using the IFC file analyzer generated screens. For better conceptual understanding of the IFC file analyzer, Figure 3 shows the correspondence between the Information Objects definitions and Information attributes crossing and corresponding with the IFC Schema.

Figure 3: Correspondence between IFC analyzer and IFC Schema

Basic definitions:

The KPIs are based on lean concepts, such as batch sizes, bottlenecks and work in process and they measure patterns and rates of design information. To enable their calculation, a number of concepts were defined below, (Table 2).

INFORMATION PACKAGE

Definition Represents an entire model or a subset of a model in an exchange, (Figure 4)

Figure 4: An Information Package

INFORMATION OBJECT

Definition A distinct component of a building or facility with technical and engineering attributes and

characteristics, (Figure 5).

INFORMATION ITEM

Definition A single piece of information. It may be textual or graphic.An information package represents a

set of information items, (Figure 5).

INFORMATION OBJECTS INFORMATION ATTRIBUTES Information Package A entire model or a subset of a model in an exchange. IFA IFC file analyser spreadsheet representing the whole content of the package information

Figure 5: An Information Object and an Information Item

INFORMATION ATTRIBUTE

Definition A technical, engineering or management attribute of an information object such as its dimensions,

material type, supplier name, colour, price, etc. The value of an information attribute may appear in any number of information packages, (Figure 6).

Figure 6: An Information Attribute

ACTION

Definition Performed by a team member to communicate information. In our case: upload or download an

IFC data to the central model at BIMserver and vice-versa.

INFORMATION BATCH

Definition A set of information packages transferred together.

Table 2: Basic Definitions

4.

KPI

DEFINITION AND PROOF OF CONCEPTS

With the definitions and model data example from Figure 3, the KPIs were calculated for the sample models using the IFC analyzer and the BIMserver data indicated in tables 3 and 4, and represented graphically through a dashboard as shown in Figure 8. From Tribelsky and Sacks’ definitions of KPIs for traditional projects, we are able to extend and prove their utilization in a BIM environment, as shown in the following table:

AR: ACTION RATE Definition: The rate at which information is transferred, (Figure 7).

Goal This index goal is to measure design team members’ actions

Formula This KPI is calculated from the measurement of users' actions directly from a BIMserver log file.

The average over any given time period t1 to t2 for a number of team members nTM is defined as:

Information Iten A single piece of information Information Object Information Attribute A technical or management attibute of an information object

IfcArbitraryClosedProfileDef IfcArbitraryProfileDefWithVoids IfcCircleProfileDef IfcExtrudedAreaSolid IfcRectangleProfileDef IfcDoorLiningProperties IfcDoorStyle IfcPropertySet IfcPropertySingleValue IfcRelDefinesByProperties IfcWindowLiningProperties IfcWindowStyle IfcMaterial IfcMaterialLayer IfcMaterialLayerSet IfcMaterialLayerSetUsage IfcRelAssociatesMaterial IfcGeometricRepresentationContext IfcGeometricRepresentationSubContext IfcGeometricSet IfcMappedItem IfcMaterialDefinitionRepresentation IfcProductDefinitionShape IfcRepresentationMap IfcShapeRepresentation IfcStyledRepresentation Actions at BIMServer Revisions, Checkouts

Figure 7: Action Rate = Actions of a user at BIMserver per unit time

PS: PACKAGE SIZE Definition: Quantifies the level of detail of information packages

Goal This KPI measures the level of detail of the information contained in the packages and enables the

evaluation of the rate of accumulation of these details, allowing the deduction of the level of design conclusion. A change in the package information size doesn’t necessarily reflect an increase or a reduction of transferred information content, which is why this KPI is, based on units of information counted.

Formula nIAi v is the number of attributes belonging to an information object that have values assigned to

them (at time t), and nIO is the total number of information objects in the package. In the IFC analyzer worksheet, the Package Size is calculated directly by adding the number of entities

corresponding to the IFC classes of the IFC Shared Building Elements, (Table 4).

Obsv. An important point is to measure how complete the package is and it will be necessary to first

determine what size it should have. The solution to this problem is not simple because it sets the amount of information that should be contained in a project, and specifying it will require planning of packages to be produced and referencing them to specific stages of development. By estimating

the total PS we can define the PSMaturity by the following formula. In the example, the PSM was

calculated by dividing each PS value from each model version by the total PS of the last version, for in this case this value was already known; in other cases it could have been estimated.

Formula

WIP: WORK IN PROGRESS Definition: The number of available but unused information packages

Goal As this KPI measures the delay between information upload and download, it indicates possible

bottlenecks in team members that build up a large volume of information before initiating their work.

Formula This KPI is based on calculating the interval time that a package of information is available on a

server waiting to be downloaded or viewed by the interested user. Working with a model in the central BIMserver the calculation of this KPI is simple and uses the BIMserver log file with the actions date of the team member k. Tupj is the day on which package j was uploaded, package size (PS) is as defined earlier and Uj = 1, if package j has been viewed or downloaded by team member k, Uj = 0 if not. In the example, the WIP was calculated from normalized distributions of the number of IFC entities counted in the IFC analyses in each model version.

BS: BATCH SIZE Definition: The batch volume of information transferred

Goal This KPI reflects the quantity of information built up by a specific team member since his last

information input. Professionals have a frequently observed tendency to transmit information in large packages, usually right before meetings or on the eve of a payment milestone date. This procedure is adopted for the optimization of office production, but it harms their team partners’ workflows, which are frozen while waiting for information. Instead, it is recommended that information should be transferred in many small packages.

Formula nIP is the number of information packages in the batch. In the example, the number of information

DV: DEVELOPMENT VELOCITY

Definition: Represents the speed with which information is transferred to the Project team.

Goal Allows the identification of bottlenecks in the process as low speed can point out delays and big

packages of information being transferred.

Formula Easily measured because it is calculated directly from measures of Package Sizes or Batch Sizes.

BN: BOTTLENECKS

Definition: Identifies possible bottleneck partners in the process at any given time.

Goal Allow the identification of the nodes where information gets obstructed in its flow.

Formula Easily measured because it correlates the indices DV and WIP. With the IFC analyzer this KPI was

graphically calculated by the correlation of DV and WIP indexes; the linear correlation index was also calculated, and in the example we’ve obtained -0.79, which shows strong relation to these indices.

RW: REWORK Definition: Quantify the rework included in information packages

Goal The goal is to identify rework resultant from negative iterations, and the KPI calculation must be

supplemented by the direct analysis of the design and the observations of the changes occurred so negative and positive iterations can be separated.

Formula nIAt is obtained through the counting of Information Attributes, whose values were modified in the

time gap between T and T-1. This KPI is obtained by using the IFC analyzer to process simultaneously two different model versions (T and T-1), and compare IFC entities existent in both versions, and the count of attributes that have suffered any modifications (Table 4).

Table 3: KPIs Definition

5.

KPIS CALCULATION AND DASHBOARD

With the data example (Figure 3) and the definitions the KPIs were calculated. The sample model data are calculated with the metadata of BIMServer and the results of IFC analyser. The results are indicated in Table 4 and the KPIs are represented, as an illustration, through a dashboard as show in Figure 8 below.

Action Rate Package Size Maturity

0,00 0,50 1,00 1,50 2,00 2,50 19/04/11 20/04/11 21/04/11 22/04/11 23/04/11 Action Rate -0,2000 0,4000 0,6000 0,8000 1,0000 1,2000 1 2 3 4 5

Work in Process Batch Size

Development Velocity Bottlenecks

Rework

Figure 8: KPIs Dashboard -1,50 -1,00 -0,50 0,00 0,50 1,00 1,50 1 2 3 4 5

Work in Process Normalized

-5.000 10.000 15.000 20.000 25.000 30.000 35.000 1 2 3 4 5 Batch Size -1,5 -1 -0,5 0 0,5 1 1,5 1 2 3 4 5

Development Velocity Normalized

-1,50 -1,00 -0,50 0,00 0,50 1,00 1,50 -1,5DV -1 -0,5 0 0,5 1 1,5 WIP Linear (DV) 0 0 0 0 -63,50 1 2 3 4 5 Rework

REFERENCES

Austin, S. (1999). "Analytical design planning technique: a model of the detailed building design process." Design Studies 20(3): 279-296.

Austin, S. (2002). "Modeling and managing project complexity." International Journal of Project

Management 20(3): 191-198.

Austin, S. et al. (2001). "Design Chains: a Handbook for Integrated and Collaborative Design." Loughborough, Loughborough University.

London, K., et al. (2010). "Towards the Development of a Project Decision Support Framework for Adoption of an Integrated Building Information Model using a Model Server. Building Information Modeling and Construction Informatics". J. U. U. Isikdag. New York, Information Science Reference:270-300.

Fallon, K., et. al (2007). General Buildings Information: Handover Guide: Principles, Methodology and Case Studies, National Institute Of Standards and Technology:99.

Berlo, L. et. al (2010). "BIMSERVER.ORG – an Open Source IFC Model Server". 27th International

Conference - Applications of IT in the AEC Industry & Accelerating BIM Research Workshop,

Cairo, Egypt, CIB.

Lipman, R. (2010). "Developing Coverage Analysis for IFC Files". 27th International Conference -

Applications of IT in the AEC Industry & Accelerating BIM Research Workshop, Cairo,

Egypt, CIB.

Manzione, L. (2006)." Estudo de Métodos de Planejamento do Processo de Projeto de Edifícios". Master Dissertation, Universidade de São Paulo.

Ouyang, C., et al. (2009). "From Business Process Models to Process-oriented Software Systems: The BPMN to BPEL Way." ACM Transactions on Software Engineering and Methodology. Benning, P., et al. (2010). "Collaboration processes. Framework for Collaboration". Inpro Report,

Inpro Consortium:43.

Sacks, R., et al. (2010). "Introducing a new Methodology to Develop the Information Delivery Manual for AEC Projects". 27th International Conference - Applications of IT in the AEC

Industry & Accelerating BIM Research Workshop, Cairo, Egypt, CIB.

Sciences, National Institute of Building (2007). National Building Information Model Standard. "Overview, Principles and Methodologies".

Steward, D. (1981). "The Design Structure System: a Method for Managing the Design of Complex Systems." IEEE Transactions on Engineering Management 28: 71-74.

Tribelsky, E., and Sacks, R., (2011). "An empirical study of information flows in multi-disciplinary civil engineering design teams using lean measures", Architectural Engineering and Design Management (in press).

University, The Pennsylvania State (2010). "BIM Project Execution Planning Guide – Version 2.0". University Park, PA, USA., The Pennsylvania State University:127.

White, S. A. (2004). "Introduction to BPMN", IBM Corporation.

Wix, J. (2006). "Information Delivery Manual Guide to Components and Development Methods", buildingSMART, Oslo, Norway:82.