Productivity Measurement and Analysis

Productivity Measurement

and Analysis

Best and Worst Practices

Dr. Bill Curtis Director, CISQ

Consortium for IT Software Quality

CISQ

Co-sponsorship

IT Executives Technical

CISQ Objectives and Process

CISQ Exec Forum Automated Function Point Reliability Performance Efficiency Security Maintainability CISQ Work Groups

ISO Fasttrack Defined Measures

OMG

Deployment Workshops

Automated FP Specification

•

OMG approved Automated

Function Point specification

•

Specification developed by

international team led by

David Herron of the David

Consulting Group

•

Mirrors IFPUG counting

guidelines, but automatable

•

Commercial implementations

expected in 2Q13

Productivity Analysis Objectives

Improvement

Estimation Productivity Benchmarking

Analysis

Managing Vendors

Productivity Analysis Measures

Adjustment Measures

Primary Measures

• Instructions • Functions • Requirements • Hours • Roles • Phases • Functional • Structural • Behavioral • Application • Project • Organization

Size

Quality

Effort

Demo-graphics

Productivity

Analysis

Software Productivity

Release

Productivity

=

developed,

Size of software

deleted, or

modified

Total effort expended on the release

{{{{

Software

Productivity

=

Size of software produced

Size Measures

Requirements-based Use Case Points, Story Points

Use Case Points have not become widely used and need more development. Story points are subjective to each team and are subject to several forms of bias.

Functions Function Points

Popular in IT. Several variations of counting schemes (IFPUG, NESSMA, Mark II, COSMIC, etc. Manual counting is expensive and subjective—certified counters can differ by 10%

Computer instructions Lines of Code

Most frequently used. Different definitions of a line can cause counts to vary by 10x. Smaller programs often accomplish the same functionality with higher quality coding.

Effort

Effort Measures—Weakest Link

Unreliable,

Inconsistent

Effort

After the fact

estimates

• Memory lapses • Time-splicing • Inconsistency

Under-reporting

• Contract issues • HR issues • Impressions

Lack of

normalization

• Roles included • Phases included • Hours in P-Year

Lifetime Productivity Decline

Original productivity baseline

Incremental increases in technical debt

Continuing decrease in productivity

Unless you take action !!!

Assumption

: Productivity is a stable number

Carry-forward Rework

Release N+2

Develop N+2

Rework N+2

Rework N

Rework N+1

Develop N

Release N

Rework N

Unfixed defects

release N

Release N+1

Develop N+1

Rework N+1

Rework N

Unfixed defects

release N

Unfixed defects

release N+1

Quality-Adjusted Productivity

Quality-Adjusted

Productivity

=

Release productivity

⊗

⊗

⊗

⊗

f(Technical debt)

Quality-Adjusted

Productivity

=

Effort

(Release N)

+

Effort for ∆ Carry-forward Should-fix Functional Defects

(Release N)

Effort for ∆ Carry-forward Should-fix

Structural Defects (Release N)

+

+

Maintainability feffort (∆ (Release N)) Function Points or Enhancement Function Points

Release Productivity should be adjusted for:

1. Effort shifted forward for fixing functional defects added in this release 2. Effort shifted forward for fixing structural defects added in this release 3. Future effort caused by maintainability problems added in this release

Productivity Baselines

13

Pro

-

duc

-

tiv

-

i

-

ty base

-

line (n.)



a value in a monotonically declining function

that compares the amount of product produced

to the amount of effort required to produce it.

… unless you take action

Release Productivity

Volume of code developed, modified, or deleted Total effort expended on the release

=

Quality-Adjusted

Segment Applications









1

Small

programs

Engineering

programs

Productivity

K

L

O

C

~20

8

Business

programs

Segment Applications









2

0

500

1000

1500

2000

2500

1980

1981

1982

L

in

e

s

o

f

c

o

d

e

/p

e

rs

o

n

-y

e

a

rs

Baselines

Demographic Measures

Level Examples of demographic factors

Application Type of application, languages, frameworks, technology platforms, application age, etc.

Project Development method, lifecycle phases, release frequency, team size, software tool usage, team experience, customer, requirements volatility, in- vs. outsourced, location, etc.

Productivity Analysis Process

11) Structure final results 1) Establish executive leadership 5) Check data validity 2) Form Measure-ment group 3) Automate data collection 4) Seek measure consensus 6) Enforce data management 8) Segment applications 9) Iterate analyses 10) Pre-brief results 7) Inspect data distributions

Best Practices—Initiation

Establish

executive

support

• State objectives • Track progress • Enforce compliance • Achieve insight

Charter

measurement

group

• At least one full time member

• Linked to improvement function

• Measurement & statistical expertise

• Communication skills

Automate

data

collection

• Integrate with development environment

• Automate collection and reporting

• Train data providers

• Assist semi-automated collection

Seek measure

consensus

• Match measures to objectives

• Start with industry standards measures

• Review with staff and management

Best Practices—Gathering Data

Check data

validity

• Audit data submissions

• Make common sense data checks

• Review questionable data with projects

• Correct issues

Enforce data

management

• Clarify application boundaries

• Establish change and version control

• Continually normalize measures

Best Practices—Analyzing Data

Inspect data

distributions

• Evaluate shape and range

• Investigate outliers and extreme values

• Revalidate strange data

• Establish impact on interpretation

Segment

applications

• Identify important demographic factors

• Test differences among factor categories

• Evaluate cross factor relationships

• Establish baseline groupings

Iterate

analyses

• Develop overall and segmented baselines

• Study scatterplots of data

• Ask ‘why’ questions about the results

Best Practices—Reporting Data

Pre-brief

Results

• Discuss results with application managers

• Seek clarification and explanations

• Correct mistakes or misinterpretations

• Prepare app manager for report to execs

Structure

Final Results

• Interview execs about needs from data

• Organize results to support decisions

• Highlight the important messages

www.it-cisq.org

1. Join CISQ

2. Contribute to the blog

3. Use CISQ standards

4. Attend CISQ seminars • Berlin, June 19

• NJ, Sept. 25 • SF, Dec. 11

5. Initiate measurement

6. Improve continually