Benchmarking methodologies

This section presents the most relevant benchmarking methodologies from the business management community. The first methodology was proposed by Robert Camp [1989], who initiated the Xerox’s benchmarking program and is generally regarded as the guru of the benchmarking movement. On the other hand, Xerox was the pioneer in American companies in the practice of bench-marking. The second methodology is the benchmarking wheel designed by An-dersen and Pettersen [1996] after analysing about sixty different benchmarking processes in companies. Finally, the third methodology is the one proposed by the American Productivity and Quality Centre [Gee et al., 2001], which is one of the premier benchmarking methodologies in the world.

These methodologies view benchmarking as a mechanism to improve busi-ness processes. All of them have similar elements, consider that benchmarking is mainly performed by a single company, and coincide in that benchmarking is a continuous process. Therefore, the steps proposed are just an iteration of the benchmarking cycle. As these methodologies are quite general, they can be easily adapted for software benchmarking.

The methodology proposed by Camp [1989] includes the following four phases:

Planning phase. Its objective is to schedule the benchmarking investiga-tions. The essential steps of this phase are

• To identify what is to be benchmarked.

• To identify comparative companies.

• To determine the data collection method and collect data.

Analysis phase. This phase involves a careful understanding of the current process practices as well as of those practices of benchmarking partners.

The steps to follow in this phase are

• To determine the current performance gap between practices.

• To project the future performance levels.

Integration phase. This phase involves planning to incorporate the new practices obtained from benchmark findings in the organization. The main step of this phase is

• To communicate benchmark findings and to gain acceptance.

Action phase. In this phase, benchmarking findings and operational prin-ciples based on the findings are converted into actions. The steps recom-mended are

• To establish functional goals.

• To develop action plans.

• To implement specific actions and monitor progress.

• To recalibrate benchmarks.

Camp also identifies a maturity state that will be reached when best industry practices are incorporated into all business processes and benchmarking becomes institutionalised.

The benchmarking process proposed by Andersen and Pettersen [1996]

is called the benchmarking wheel. Such process is composed of five phases that have a degree of overlap:

Plan phase. This is the most important phase since thorough planning is key to create the foundation for an effective benchmarking that produces good results. The steps to follow in this phase are

• To select the process to be benchmarked, based on the company’s strategy.

• To form the benchmarking team.

• To understand and document the process.

• To establish performance measures for the process (quality, time, and cost).

Search phase. This phase is devoted to searching and selecting suitable benchmarking partners. The steps to follow in this phase are

• To design a list of criteria that an ideal benchmarking partner should satisfy.

• To search for potential benchmarking partners, i.e., who performs the process in question better than the company.

• To compare the candidates and select benchmarking partner(s).

• To establish contact with the selected benchmarking partner(s) and gain acceptance for participating in the benchmarking study.

Observe phase. The goal of this phase is to observe the benchmarking partners and obtain their performance levels, the practices or methods that make it possible to achieve these levels, and the enablers that permit performing the process according to these practices or methods. The steps to follow in this phase are

• To assess information needs and sources.

• To select methods and tools for collecting information and data.

• To observe and debrief.

Analyse phase. The main purpose of this phase is to find out gaps in performance levels between own and partners’ process, the root causes of the gaps, and the enablers that particularly contribute to the gaps. The steps to follow in this phase are

• To sort the collected information and data.

• To quality control and normalise the information and data.

• To identify gaps in performance levels.

• To identify the causes for the gaps.

Adapt phase. In this phase, the findings from the Analysis phase must be adapted to the organization’s own conditions and implemented in the company. The steps to follow in this phase are

• To communicate the findings from the analysis and gain acceptance through participation and information.

• To establish functional goals for the improvements that match the other improvement plans of the company.

• To design an implementation plan for the improvements.

• To put the implementation plan into action.

• To monitor the progress and adjust deviations.

• To close the benchmarking study with a final report.

Another methodology is the one proposed by the American Productivity and Quality Centre. This methodology has been broken down by Gee et al.

[2001] into the following four phases:

Plan phase. Its goal is to prepare the benchmarking study plan, select the team and partners, and analyse the organizational process. The steps to follow are

• To form (and train, if needed) the benchmarking team.

• To analyse and document the current process.

• To identify the area of study on which the team will focus.

• To identify the most important customer.

• To identify the smaller subprocesses, especially problem areas.

• To identify the critical success factors for the area and develop mea-sures for them.

• To establish the scope of the benchmarking study.

• To develop a purpose statement.

• To develop criteria for determining and evaluating prospective bench-marking partners.

• To identify target benchmarking partners.

• To define a data collection plan and determine how the data will be used, managed, and distributed.

• To identify how implementation of improvements will be accom-plished.

Collect phase. The goals of the data collection phase are to prepare and administer questions, capture the results, and follow-up partners. The steps to follow here are

• To collect secondary benchmarking information in order to determine whom to target as benchmarking partners.

• To collect primary benchmarking data from the benchmarking part-ners.

Analyse phase. The goals of this phase are to analyse performance gaps and identify best practices, methods, and enablers. The steps to follow here are

• To compare one’s current performance data with the partner’s data.

• To identify any operational best practices observed and the factors and practices that facilitate superior performance.

• To formulate a strategy to close any identified gaps.

• To develop an implementation plan.

Adapt phase. The goals of this phase are to publish findings, create an improvement plan, and execute the plan. The steps to follow here are

• To implement the plan.

• To monitor and report progress.

• To document and communicate the study results.

• To plan for continuous improvement.

Gee et al. [2001] also identify the final steps that should be carried out after the Adapt phase. These steps are

To document the benchmarking in a final report and capture any lessons learned that can be of future value, capturing also a variety of process information.

To communicate the results of the benchmarking effort to the management and staff.

To send a copy of the final report to the benchmarking partners.

To routinely review the performance of the benchmarked processes and ensure that goals are being met.

To move on to what is next by identifying other candidate processes for benchmarking.

It can be observed that the previous methodologies contain some similar tasks. Table 2.2 shows a comparison of the tasks that each of these method-ologies deal with, whereas table 2.3 identifies the set of common tasks that these methodologies treat. These common tasks have been grouped into phases according to the phases used in the methodologies.

[Camp,1989][AndersenandPettersen,1996][Geeetal.,2001] IdentifywhatistobebenchmarkedSelecttheprocesstobebenchmarked FormthebenchmarkingteamForm(andtrain,ifneeded)benchmarkingteam UnderstandanddocumenttheprocessAnalyseanddocumentthecurrentprocess Establishscopeofbenchmarkingstudy Developpurposestatement Identifytargetbenchmarkingpartners Definedatacollectionplanandmethods Identifyimprovementsimplementationmethods Establishprocessperformancemeasures DesignbenchmarkingpartnercriteriaDevelopcriteriaforbenchmarkingpartners Secondaryresearchbasedonselect/sortcriteria Searchforpotentialbenchmarkingpartners Evaluateresultsand... IdentifycomparativepartnersCompareandselectbenchmarkingpartners...identifypotentialpartners Developdatacollectioninstruments Pilotdatacollectioninstrumentsinternally Reviewthesecondaryinformation ContactpartnersandgainparticipationIdentify,contactandenlistbestpracticepartners Screenpartnersandevaluateforbest“fit” Assessinformationneedsandsources Determinedatacollectionmethodand...SelectmethodandtoolfordatacollectionDevelopdetailedquestionnaire ...collectdataObserveanddebriefConductdetailedinvestigation Sortthecollecteddata Qualitycontrolthedata DeterminecurrentperformancegapIdentifygapsinperformancelevelsCompareyourdatatoyourpartners’data IdentifythecausesforthegapsIdentifyoperationalbestpracticesandenablers Projectfutureperformancelevels CommunicatefindingsandgainacceptanceCommunicatefindingsandgainacceptance EstablishfunctionalgoalsEstablishfunctionalgoalsFormulatestrategytoclosethegaps DevelopactionplansDesignanimplementationplanDevelopimplementationplan ImplementactionsandmonitorprogressPuttheimplementationplanintoactionImplementtheplan MonitorprogressandadjustdeviationsMonitorandreportprogress WriteafinalreportDocumentthestudy Recalibratebenchmarks Planforcontinuousimprovement Table2.2:Comparisonoftasksinbenchmarkingmethodologies.

Phase Task

Plan Identify the process to be benchmarked Form and train the benchmarking team Analyse and document the current process Data collection Define criteria for benchmarking partners

Identify potential partners

Contact the potential partners and enlist Define method and tools for data collection Collect data

Analysis Determine the current performance gaps Identify the causes for the gaps

Change/adapt Communicate findings and gain acceptance Formulate strategy to close the gaps Develop the implementation plan Implement the plan

Monitor and report progress Document the study

Table 2.3: Common tasks in benchmarking methodologies.