Engineering Project Data Analysis

Copyright © 2014 Raytheon Company. All rights reserved.

Engineering

Project Data Analysis

By:

George Graw

Stuart Simon

Agenda



Raytheon/IDS Company Overview



High Consequence Engineering



CMMI Opportunity



Project Data Analysis (PDA)



PDA R6s Project



PDA Training Improvements



Standard PDA Measurement



PDA Defect Prevention Team



PDA Workshops

Raytheon Company

Thomas A. Kennedy

Integrated Defense

Systems

Daniel J. Crowley

Intelligence, Information

and Services

Lynn A. Dugle

Missile Systems

Taylor W. Lawrence

Space and Airborne

Systems

Richard R. Yuse

Raytheon Government and Defense Business

Integrated Defense Systems

Serving customers worldwide with air &

missile defense; radars; C4I; air traffic

management; and naval systems

Daniel J. Crowley

President

2013 Net Sales:

$6.5B

Headquarters:

Tewksbury, Mass.

– Integrated air & missile defense

– Battlefield, land and sea-based radars

– C4I for defense & civil markets

– Air traffic management

– Sonar, navigation, integrated bridge &

shipboard combat systems

Integrated Defense Systems

Our Vision

To be the most admired defense and aerospace systems company

through our world-class people, innovation and technology



2013 revenue of $6.5 billion



2,500 contracts



Over 15,000 global employees



25 locations



Global customer base

IDS’ Four Core Capabilities

Large Ground and Naval Radars

– IDS is a world leader in

large scale radar

development, technology

and production.

– AN/TPY-2, AMDR,

Upgraded Early Warning

Radar, SBX

C4I Systems

– IDS has more than 45 years

experience in developing command

and control systems for defense and

civil requirements.

– FAA and global air traffic systems,

U.S. Air Force battle control system,

multiple national sovereignty systems

IDS is a leader and innovator across

four core capability areas

Integrated Air and Missile Defense

– IDS delivers combat-proven

performance against the

complete spectrum of

airborne and ballistic

missile threats.

– Patriot, Hawk XXI, JLENS,

NASAMS, THAAD

Naval Combat Systems and Electronics

– IDS is established in the design,

development, production, integration

and support of innovative and

affordable ship systems.

– DDG 1000 technologies, naval

radars, sonar, torpedoes, ship

combat systems, navigation and

bridge systems

Patriot

SBX

ALFS

High Consequence Engineering



Definition

– High Technology Readiness Levels

– High Reliability

– Long Life span

– Low Volume



Traditional Quality Roles

Role

Driving Forces

Restraining Forces

Compliance

• Auditing and Corrective Action Management Key Competency • Independent – Requirement

• External to Function involvement

Inspection

• Not part of “design” process • Independent

• Need Subject Matter Expertise • Will catch in test

Test

• Key element of Compliance • Complex Process

• Another independent org completes test

RC/CA

• Subject Matter Experts • Required by Audits/CMMI NC

• Not sought after

• Usually just fixes problems

Product

Release

• Required by Policy • Good to have final Check • “Somebody else” will check

• Time to do complete review • Skill to do complete review

Quality’s Role



Provide more than compliance



Be integral to project teams



Build Partnerships



Bring Continuous Improvement into the Development phase

The CMMI Opportunity

Applying Quality Sciences

Ad Hoc

Processes

•

Measurement and Analysis

•

Project Monitoring and

Control

3

Optimizing

Quantitatively

Managed

Initial

Defined

Managed

•Verification

•Validation

•

Quantitative Project Management

•

Causal Analysis and Resolution

Process Areas

Gen

eri

c

Practice

s

•

Objec

ti

v

el

y

Ev

al

uate

Adhe

rence

2

1

5

4

•

Organizational Process Focus

•Process and Product QA

•

Inv

ol

v

e Rel

ev

ant

Stak

ehol

ders

•

Moni

tor

and Control

the

Proc

es

s

•

Rev

iew

Stat

us

w

ith

Manag

ement

Measurement & Analysis



What is Project Data Analysis (PDA)?

– Process of understanding project performance and documenting the analysis

of metrics and measures



Why conduct PDA?

– Manage performance that will result in quality products delivered within cost

and schedule with the project’s defined process

– Meet business objectives

– Identify opportunities for improvement

– Provide insight that leads to informed decisions

– Quantify and take action to minimize risks

Who is involved?

Functional Management

Other Stakeholders

• Cost Account Manager (CAM)

• Shoulder-to-Shoulder(S2S)

Mentor

• Risk Manager

Project Management

Lead Engineer

• Project Status

• Project Issues

• Project Process Directives

• Project Lessons Learned

• Measurement Data

• Project Training Needs

Engineering Process

Process Deployment Engineer

• Tools and Enablers

• Standard Process Definition

• Organization Lessons Learned

• Process Training and Awareness

Functional Management

Department Manager/Section

Manager

• Staffing Resources

• Process Authority

• Process Responsibility

Quality

Quality Engineer

• Process Compliance

• Audit Information

• Objective Assessment

• Causal Analysis

• Root Cause Expertise/Tools

• R6s/CAR Projects

4 Tenets of Good Analysis



Observation:



Root Cause:



Impact:

Why is PDA Important?



Aids in the identification of processes

that are not meeting project goals



Leads to continuous improvement

opportunities and/or Causal Analysis

and Resolution (CAR) projects



Emphasizes the Quality Sciences of

Root Cause and Corrective Action

(RCCA) and Six Sigma as well as

using Statistical Process Control on

statistically managed measures

PDA R6s Project



Strategic Business Case Alignment

-– Without robust project data analysis, it is difficult to determine if any systemic or

organizational issues are occurring. By not being able to identify systemic issues,

we are missing opportunities to become more affordable.



Problem & Visualize (Goal) Statement

– By identifying and correcting project data analysis quality and consistency,

Engineering will be able to improve their analysis of measures and metrics at the

organization level. This will enable better identification of special cause and

common cause variations leading to continuous improvement opportunities.



Key Stakeholders

– Engineering Disciplines, Quality



Risk/Barriers

– Manpower – availability of team to perform necessary tasks

– Management Support – the corrective actions may require funding and staffing

allocations

Root Causes



Inadequate training – all roles except Quality



Breakdown in process – all roles



Lack of empowerment – Process Deployment Engineers

(PDEs) & Quality Engineers (QEs)



Lack of accountability – Department Managers (DMs) &

Leads

Achieve



Training

– Standardized PDA Organizational Training Matrix (OTM) across Engineering

and Quality

– Drove improvement of PDA class to include Applied Stats, SPC, CAR

– Certified Quality instructors



People

– Developed annual performance Engineering/Quality PDA goals for DMs,

Leads, PDEs and QEs for 2013 Goals

– Partnered with Engineering on PDA workshop



Process

– Implement best practices



Centralize PDA Analysis

PDA Training Improvements



Standard Engineering OTM on PDA



Course Material



Certified Quality Instructors



Quality trained major stakeholders

helping Engineering to meet training

requirements

Standardize PDA OTM across Eng. and Quality



Maintenance through agreement across stakeholders



Enforce through Learning Plan, Interim and PDS reviews



Hold focused set of classes to catch up core group in Q3



Required

– R6s Specialist

– Root Cause and Corrective Action Overview

– CMMI Overview

– Project Data Analysis



Optional

– SPC

– CMMI

– Applied Statistics

– EVM Fundamentals

Project Data Analysis Course Material



Issues with existing course

– Outdated

– Too long

– Did not include some of the quality sciences



Applied Statistics



Statistical Process Control



Causal Analysis and Resolution



SME team formed



Course updated



Peer reviewed jointly with Eng and Quality

Certified Quality Instructors



Opportunity to improve

Quality’s value add



Injected Quality with

Engineering top talent



Certified several leaders

as instructors



Instruct majority of PDA

classes

Standard PDA Measurements



Engineering Process Group and

Quality

are responsible to

ensure high quality PDA

– Centralize process for standard, consistent, high quality results

– Analysis of each Directorates PDA performed every quarter

– Provides independent, closed loop process for sustainment of high quality PDA

– Provides independent team by EPG/Quality (EPG, Quality & Org SMEs)



Centralized process provides significant benefits

– Drives efficiency and consistency across Engineering

– Provides closed loop system with regular feedback

– Removes cost from individual Directorates Overhead

– Allows for identification of Engineering systemic issues

2013 PDA Results

PDA Defect Prevention Team (DPT)



2013 Results – variation reduction, consistent measurement

methodology, improved inspection efficiency, defect

reduction and increased management visibility

PDA

Evals

Analysis

Actions

Eval

Discussion

PDA Workshop Objective



Discuss PDA and the four tenets



Review current capability of using measures to manage the

programs in Engineering



Identify any continuous improvement projects to address

program problems or opportunities



Provide hands on workshop where project teams use their

own data and generate continuous improvement (R6s/CAR)

opportunities

PDA Workshop Structure



PDA Refresher

– Good analysis

– Applied Statistics

– Statistical Process Control

– Causal Analysis & Resolution



Brainstorming: PDA Obstacles



Exercise: Continuous Improvement



Exercise: Problem Statement



Exercise: R6s Project Plan

CAR/R6s Work Product

Vision

Commit

Strategic Business Case Alignment

-

(e.g.

“Affordability” )

Problem Statement:

A clear, concise, and specific

definition of a problem that needs improvement which

describes the symptoms in measurable terms and conveys the

business impact of the problem

Vision (Goal) Statement:

Clearly and succinctly articulates the future state. Data based

and as quantitative as possible. Avoid embedding a specific

solution — keep your options open.

R6

s

Project Stakeholders

•

Sponsor – Name

•

Team Lead – Name

•

Team Members

– Name, …

•

Key Stakeholders – Name; Org, Name; Org

Resources Required

Prioritize

Barriers to Completion

Prioritization (Schedule / Milestones)

• Visualize – ECD

• Commit – ECD

• Prioritize – ECD

• Characterize – ECD

• Improve – ECD

• Achieve – ECD

Project Scope

The scope statement should clearly define

the domain within which the problem and solution are

bounded. Describe what is and is not to be addressed by the

project

Risk/Barriers

Describe the risk or barriers , impact and when needed

factors

Key Results – Continuous Improvement



In 2013, Quality either led or was a team member on

29 R6s/CAR Projects



Which had a cost benefit of

Key Results – Improved Partnering



By

leading PDA improvements

instructing classes

applying quality sciences

driving R6s/CAR projects



Engineering is

coming to Quality for help

asking Quality to lead PDA improvements

Which

improved the quality of PDA

delivered improvements with measureable results

improves program affordability

Thank You

Contact Information



George Graw

– Sr Manager, MAQS Process & Enterprise Initiatives

– Work Phone: (978)858-9648

– E-mail:

[email protected]



Stuart Simon

– Sr Manager, Systems/Software Engineering Quality

– Work Phone: (978)858-4494

Speaker’s Bio

George Graw



Mr. Graw is a Senior Quality Manager in IDS’ Mission Assurance, Quality and Six

Sigma Function. He is currently responsible for function wide process, training, cost

estimating and strategic initiatives. During his 25+ years at Raytheon, George has

held various positions within Product Assurance and Applied Technology, including

Principle Reliability Engineer and an assignment within the Advanced Device Center.

George graduated with a Bachelors Degree in Electrical Engineering from Worcester

Polytechnic Institute and a Masters Degree in Engineering Management from the

University of Massachusetts Amherst. George is a Certified Quality Engineer, Senior

Member of the American Society for Quality as well as a Systems Reviewer for the

Academic Quality Improvement Program.

Stuart Simon



Mr. Simon has been with Raytheon Company for 29 years. He is currently a Senior

Quality Manager in Integrated Defense Systems (IDS) and is responsible for all

Systems and Software Quality Engineers. Mr. Simon has had a varied career with

Raytheon starting out working in the Electro-Optics Systems Laboratory and migrating

into Systems Engineering. He held positions of increasing responsibility as a project

lead, section manager, and department manager in Systems Engineering and most

recently as a Department Manager in the IDS Mission Assurance, Quality and Six

Sigma organization. He graduated with a Bachelors and Masters in Electrical

Engineering from Massachusetts Institute of Technology.