Cost of Poor Quality:

(1)

_____________________________________________________

Daniel Zrymiak (Accenture): ASQ Fellow Member, CSQE, CSSBB, CMQ/OE; Member-Leader ISE 2012: Anaheim, California: Cost of Poor Quality: Analysis for IT Service Management Software

Cost of Poor Quality:

Analysis for IT Service Management Software

Software Concurrent Session: ISE 09

Wed. May 23, 8:00 AM – 9:00 AM

Presenter: Daniel Zrymiak

(2)

_____________________________________________________

Key Points:

• Use the Cost of Poor Quality: Failure Dimensions to address and resolve problems originating from an inadequate IT Service

Management application.

• Define Factors and Dimensions of Failure by Type • Define Factors and Dimensions by Correction Type • Define Failures by Phase

• Track specified outcomes for Continual Improvement • Show the Cost-Benefit advantages of improvements

(3)

_____________________________________________________

Presentation Benefits (1 of 2):

Learning Objectives:

Demonstrate how a detailed classification of Failure Costs and

Resolution Methods can be applied to support the development and maintenance of IT Service Management software.

Can you think of one example of a Failure Cost or Cost of Poor Quality related to the development , deployment, and maintenance of a

(4)

_____________________________________________________

Presentation Benefits (2 of 2):

Career Benefits:

• Align quality initiatives to financial realities and priorities • Expand analysis to include financial tracking of failures and corrections for future reference

Technical Content:

• Software Quality Engineering/Management, Lean Six Sigma

Professional Development:

• Cost of Poor Quality is pertinent for Managers of Quality and Organizational Excellence, also Lean Six Sigma Black Belts. • Automation software is relevant for Testing, IT Governance.

(5)

_____________________________________________________

Scenario: Deficient IT application

Canadian-based Energy company used an IT Service Management application from an external vendor.

Development, training, and deployment did not occur as scheduled causing schedule delays and cost overruns.

Eventual implementation and go-live was characterized by frequent outages, user dissatisfaction, and interruption of regular business processes.

(6)

_____________________________________________________

Factors and Dimensions

Factors: Broad categories or segments to address the failures at a more granular level

Dimensions: Elements of analysis within a factor to determine appropriate resolution and corrective action

(7)

_____________________________________________________

Factors of Failure Costs

Failure Type:

People, Product, Process, Transaction, General Correction Type:

Resolution, Restoration Failure Phase:

Expectations, Commitment, Realization, Delivery, Support

(8)

_____________________________________________________

People (Capability vs. Engagement)

For failures related to people performing the work, two dimensions should be considered.

Capability refers to the qualifications and general competency.

Engagement considers the overall motivation and willingness to perform the work to the required standard.

Due to inadequate training and communication, the individuals using the tool were not adequately capable nor engaged to use the tool correctly.

(9)

_____________________________________________________

Product (Meeting Specs vs. Fitness For Use)

For failures related to products, two dimensions should be considered. Meeting Specifications can be demonstrated by objective verifications. Fitness For Use can be observed through subjective validations (i.e. look and feel)

The tool was ITIL compliant and demonstrated correct functionality in controlled tests, but was not robust to handle normal volumes, nor could it be adapted to fit the client’s normal business processes or expected workflows.

(10)

_____________________________________________________

Transaction (Accuracy vs. Timeliness)

For failures related to transactions, two dimensions should be

considered. Transactions can fail at the point of interaction or discovery can occur later through reporting.

Accuracy can relate to the correctness of quantity, suitability, and thoroughness.

Timeliness refers to tracking delays and lateness of the deliverable. The tool was difficult to configure, and configuration problems affected accuracy. Timeliness was impaired by performance problems, causing delays to handle requests.

(11)

_____________________________________________________

Process (Effectiveness vs. Adherence)

For failures related to processes, two dimensions should be considered. Process failures can lead to subsequent failures in other areas.

Effectiveness determines whether the process was suitable and applicable.

Adherence analysis reveals points where a valid process is being disregarded in favor of ad-hoc or tailored solutions.

The tool had a distinct workflow which was different from the client’s prior patterns, requiring adjustment. To bypass restrictions, tailoring and

(12)

_____________________________________________________

General (Explicit vs. Implicit)

For failures that do not align with a single category, a general analysis should occur.

Explicit failures reveal shortcomings relative to an approved standard or benchmark.

Implicit failures diverge from expected outcomes and contribute more significantly to overall dissatisfaction.

There was a subjective sense of frustration and disappointment with the tool, which diminished the overall effectiveness of the transition.

(13)

_____________________________________________________

Cost of Poor Quality Measurements

People: Losses are measured in hours and days delayed per person, and Escalation/Specialist resource time and materials to resolve.

Product: Losses are measured in the variable costs of system impairments and downtime arising from the deficiencies.

Process: Losses are measured in the frequency of redundant or abandoned processes needed to complete transactions.

General: There were indirect outcomes assignable to the low quality tools including team member attrition and project deferral due to lack of confidence with the tool.

(14)

_____________________________________________________

Factors of Failure Costs

Failure Type:

People, Product, Process, Transaction, General Correction Type:

Resolution, Restoration Failure Phase:

Expectations, Commitment, Realization, Delivery, Support

(15)

_____________________________________________________

Resolution (Contingency vs. Correction)

All failures must be resolved to move from a failed error state to a stabilized recovery.

Contingency programs involve different initiatives to compensate for the error condition.

Corrective actions require the root causes to be identified and resolved. As a contingency, departments revert to tracking information on their MS Office applications and updating by email. To correct the problems,

technical teams investigated based on priorities reported by the post-Go Live Command Center.

(16)

_____________________________________________________

Restoration (Mitigation vs. Accommodation)

Following the interruption and loss arising from failures, efforts are needed to restore the trust and confidence in the relationship.

Mitigation includes collaboration through awareness and vigilance, and applied controls and standards to entities.

Accommodation refers to the concessions and compensations to address losses and penalties.

Mitigation for future releases could be established through a more

rigorous approval and release process (on next slide). Accommodation was made through financial concessions.

(17)

_____________________________________________________

Establish Mandatory Release Criteria

Given the error conditions realized from a poor IT Service Management application, releases to client production systems should be restricted until objectively proven adequate in the following categories of system assurance:

PRODUCT SYSTEM COMPLIANCE

Functionality Availability Security

Reliability Configurability Environmental Controls (i.e. SOX) Usability Performance Verification and Validation Assurance Maintainability Efficiency Data Privacy

(18)

_____________________________________________________

Cost of Correction Measurements

Contingency: These costs are measured in hours and days of “just in case” plans to continue in the event of system failure or data corruption. Corrective Actions: These costs reflected the rework, redesign, and

incremental variable costs associated with maintenance and support releases and retraining of personnel.

Mitigation: These costs reflected the additional design and process steps needed in response to identified risks and hazards of software failure. Accommodations: These costs include the various penalties and

concessions which were needed to comply with contractual service level expectations and preserve the business relationship.

(19)

_____________________________________________________

Factors of Failure Costs

Failure Type:

People, Product, Process, Transaction, General Correction Type:

Resolution, Restoration Failure Phase:

Expectations, Commitment, Realization, Delivery, Maintenance and Support

(20)

_____________________________________________________

Need to Distinguish Failures by Phase

CoPQ programs experience a disproportionate clustering of failures within the client-facing areas of the organization.

The true source and timing of failures actually occurred earlier in the business cycle.

Distinguishing failures by phase increases transparency and accountability across the entire enterprise.

(21)

_____________________________________________________

Phase 1: Expectations

Prior to engagement, the organization establishes expectations as the basis for comparisons and standards (i.e. boutique shop vs. factory outlet merchandise).

Failure points are established by the overall impression of quality as communicated by marketing and positioning strategies.

The vendor promoted itself as a reliable provider of IT Service

Management applications, and sold itself being capable of delivering a quality product.

(22)

_____________________________________________________

Phase 2: Commitment and Planning

In this phase, specific promises and delivery terms are established. Based on these agreements, the details for fulfillment are created.

“Fine print” needs to be carefully reviewed to avoid potential problems. Risks and contingency scenarios should be considered in anticipation of probable potential failures.

When committing and planning for the development and deployment of the IT Service Management tool, there were inadequate resources

applied to testing, training, and deployment, creating opportunities for failures

.

(23)

_____________________________________________________

Phase 3: Realization and Fulfillment

In this phase, actual work is performed to bring the solution from concept to readiness. Depending on the complexity, this is the phase where most failures are injected.

Without the necessary controls, failures from this phase can propagate and taint the overall solution.

With respect to the IT Service Management tool, inadequate

configurations were pushed to the client system by the vendor, without having had the key assurance attributes (i.e. reliability, usability, system performance, etc.) properly verified and validated. False “Green”

(24)

_____________________________________________________

Phase 4: Delivery

In this phase, the product, service, or solution is provided to the customer or stakeholder. This is where evident failures could be rejected, causing delays and dissatisfaction.

This is the “go-live” portion where failures combine technical “internal failure” characteristics and supplemental “external failure” business losses.

With the vendor, there was inadequate preparation for the go-live, and the resulting change to the business was met with surprise and

frustration. Damage control was needed with the involvement of senior technical expertise.

(25)

_____________________________________________________

Phase 5: Maintenance and Support

In this final phase, constant interaction is performed to address the correction, adaptation, or prevention of identified or potential

shortcomings.

Support can be scheduled or immediate, depending on the scope and importance of the failure. After applying the corrections, organizations should communicate causes and actions through a Failure Tracking System to reduce overall failures.

Maintenance and Support are provided by the vendor through a responsive “Command Center” to expedite fixes.

(26)

_____________________________________________________

Continual Improvement through Tracking

Failure Analysis includes the categorization and discovery of the modes and effects of different failure types. Failure Tracking links this

information to specific actions and an ongoing information system. SLACK model: • Summary, • Lessons Learned, • Action, • Commitment, • Knowledge Base

(27)

_____________________________________________________

Summary

Failures can be summarized and categorized by factor and dimension (i.e. People-based, Capability).

From a simple description, quick analysis and rapid responses can be consistently supported across the organization.

Common failure identifications can support trend analysis and statistical process controls.

The IT Service Management tool failures reflected failures across all

categories (i.e. people, product, process) and in almost every dimension (i.e. capability, engagement).

(28)

_____________________________________________________

Lessons Learned

This instructive portion captures the discovery of the information while it is current and available. Insights on the impacts and downstream effects of particular failures will prompt the establishment and application of

appropriate controls and standards.

Real-life examples will reinforce training and awareness, and leverage common experiences across the enterprise.

The lessons learned from the deficient IT Service Management tools are shown on the next slide.

(29)

_____________________________________________________

Lessons Learned

People: Ensure intended users are proficient and engaged with the product prior to release, and are positive advocates.

Product: Restrict releases until they have met the mandatory assurance criteria for Product, System and Compliance.

Process: Align the IT Service Management tools with the established processes agreed by client, business, and stakeholders; preventing workarounds and ad hoc solutions.

General: Forcing a deficient application upon a client and delivery team will deplete time and resources, causing losses. It is better to stabilize the system prior to release.

(30)

_____________________________________________________

Action

The correction is outlined by factor and dimension (i.e.: Resolution,

Contingency). By recording the actions, this provides a ready reference to support recurrence and rapid turnaround of corrections.

There are many actions required to correct and mitigate the risks arising from the deployment of a deficient IT Service Management System.

These should be noted in order to be consistently leveraged and replicated across different engagements.

(31)

_____________________________________________________

Commitment

Every action requires a commitment which includes the resource owning the solution and the completion date. Tracking will increase

transparency and accurate computation of costs for failure response. CoPQ can be approximated and used as a baseline for cost reduction initiatives including Variance Analysis, Activity Based Costing, and Lean Six Sigma programs.

Compressing transition costs comes from increasing preparations and preventive measures, increasing levels of assurance, and controlling the deployment of applications.

(32)

_____________________________________________________

Knowledge Base

All failure information should be aggregated into a common knowledge base. Objective information can be used as inputs for verification,

validation and future planning activities.

Build a continual learning organization which updates knowledge for continual improvement.

Knowledge of IT Service Management applications currently in use can be captured and made available for future reference.

(33)

_____________________________________________________

Benefits to CoPQ Programs

Increased accuracy of failure tracking and diagnosis.

Improved application of corrective methods to target nature and timing of failure type.

Diligent tracking of failures and response activities will generate more accurate cost information.

Combination of approaches will objectively reduce Cost of Poor Quality and reduce turnaround time for solutions.

(34)

_____________________________________________________

Cost-Benefit Rationale for Improvements

False savings to reduce costs by “streamlining” development and deployment of IT Service Management applications.

Actual Experience from Canadian Energy company demonstrates that a deficient solution will create delays and disruptions, requiring rework from technical resources, and deplete transition budgets without client satisfaction.

Incremental improvements to enhance assurance, user preparations, and release controls will result in a more correct and usable solution,

reducing the need for expensive restoration and stabilization, saving time and money.

(35)

_____________________________________________________

Learning Confirmation: Demonstrate

Demonstrate how a detailed classification of Failure

Costs and Resolution Methods can be applied to support

the development and maintenance of IT Service

Management software.

Can you now apply Cost of Poor Quality analysis

methods, using factors and dimensions, to demonstrate

the expected benefits and mitigations resulting from

(36)

_____________________________________________________

Presentation Business Context

What is the business context and how does it improve

upon the existing knowledge base?

This presentation extends the analysis of Failure Costs

by defining categories of failure and resolution, and

(37)

_____________________________________________________

Presentation Practical Applications

What are the practical applications towards the

participants’ jobs, functions, projects, or departments?

This presentation defined the cost descriptions to

support diagnosis and the establishment of an ongoing

knowledge base to support more rapid resolutions.

(38)

_____________________________________________________

Presentation Integration or Connection

How can this knowledge be integrated or connected to

other areas?

This presentation is tied to overall software quality

management and continual improvement. Lean and Six

Sigma practitioners will benefit from obtaining the cost

measures and deriving the value streams for software.

(39)

_____________________________________________________

Presentation Business Context

What are the expected gains, and the risks and

implications of omission?

This presentation provides awareness of potential

software quality problems and a structured approach to

correction, with the risks being expanded losses and

Cost of Poor Quality from undesirable outcomes.

(40)

_____________________________________________________

Comments/ Questions:

Daniel Zrymiak

Accenture: Mobilization/ Outsourcing Excellence

dzss@shaw.ca

daniel.zrymiak@accenture.com

(604) 575-3269 or (604) 862-4808