30 30
Software Engineering Software Engineering
and and
Best Practices Best Practices
Sources: Various.
Sources: Various.
Rational Software Corporation slides, Rational Software Corporation slides,
OOSE textbook slides, Per Kroll talk, How to Fail with OOSE textbook slides, Per Kroll talk, How to Fail with
the RUP article, textbooks the RUP article, textbooks
Most slides have been modified considerably Most slides have been modified considerably
Fundamental Terms / Concepts Fundamental Terms / Concepts
► Science and Engineering Science and Engineering
Discover Discover
►
Relationships that exist but are not found Relationships that exist but are not found
►
Formulas; chemical composition, d=r*t; calories in Formulas; chemical composition, d=r*t; calories in fats, carbohydrates, proteins; experimentation;
fats, carbohydrates, proteins; experimentation;
►
Astrophysics – origins of the universe Astrophysics – origins of the universe
Build Build
►
Apply principles of science and mathematics to real Apply principles of science and mathematics to real needs, commodities, structures, products, etc.
needs, commodities, structures, products, etc.
► Software Engineering; Software Development Software Engineering; Software Development
30 3
Fundamental Concepts / Terms (2) Fundamental Concepts / Terms (2)
► Software Engineering; Software Development Software Engineering; Software Development
► Job positions: Job positions:
Software developer Software developer
Programmer Programmer
Software engineer Software engineer
Analyst / Programmer Analyst / Programmer
Senior … what have you… Senior … what have you…
What is Software Engineering?
What is Software Engineering?
► The process of solving customers’ problems by The process of solving customers’ problems by the systematic development and evolution of the systematic development and evolution of
large, high-quality software systems within large, high-quality software systems within
cost, time and other constraints cost, time and other constraints
► Note: Note:
Process, systematic (not ad hoc), evolutionary… Process, systematic (not ad hoc), evolutionary…
Constraints: high quality, cost, time, meets user Constraints: high quality, cost, time, meets user requirements
requirements
30 5
Analysis of the Definition:
Analysis of the Definition:
► Systematic development and evolutionSystematic development and evolution
An engineering process involves applying An engineering process involves applying well understood techniques in a well understood techniques in a organized
organized and and disciplineddisciplined way way
Many Many well-accepted practices have been formally standardizedwell-accepted practices have been formally standardized
► e.g. by the IEEE or ISOe.g. by the IEEE or ISO
Most development work is evolutionaryMost development work is evolutionary
► Large, high quality software systemsLarge, high quality software systems
Software engineering techniques are needed because large systems Software engineering techniques are needed because large systems cannot be cannot be completely understood
completely understood by one person by one person
Teamwork and co-ordination are requiredTeamwork and co-ordination are required
Key challenge: Dividing up the work and ensuring that the parts of the system Key challenge: Dividing up the work and ensuring that the parts of the system work properly together
work properly together
The end-product that is produced must be of sufficient qualityThe end-product that is produced must be of sufficient quality
► Cost, time and other constraintsCost, time and other constraints
Finite resourcesFinite resources
The benefit must outweigh the costThe benefit must outweigh the cost
Others are competing to do the job cheaper and fasterOthers are competing to do the job cheaper and faster
Inaccurate estimates of cost and time have caused many project failuresInaccurate estimates of cost and time have caused many project failures
30 6
Comments:
Comments:
►
$250 billion annually in US. $250 billion annually in US.
►
Over 175,000 projects! Over 175,000 projects!
►
Complexity, size, distribution, importance push our Complexity, size, distribution, importance push our limits.
limits.
►
Business pushes these limits: Business pushes these limits:
Great demands for Great demands for rapid development and deployment rapid development and deployment
►
Incredible pressure: develop systems that are: Incredible pressure: develop systems that are:
On time, On time,
Within budget, Within budget,
Meets the users’ requirements Meets the users’ requirements
►
Figures in the late 90s indicated that at most Figures in the late 90s indicated that at most
70% of projects completed 70% of projects completed
Over 50% ran over twice the intended budget Over 50% ran over twice the intended budget
$81 billion dollars spent in cancelled projects!! $81 billion dollars spent in cancelled projects!!
►
Getting better, but we need better tools and Getting better, but we need better tools and techniques!
techniques!
30 7
What Happens in Practice What Happens in Practice
Sequential activities: (Traditional
‘Waterfall’ Process)
Requirements Design Code Integration Test
Late Design Breakage
100%
Project Schedule Development Progress (% coded)
Original Target Date
Integration Begins Risk inadequately addressed Process not receptive to Change Problems not really ‘seen’
until near delivery date!
Until then, ‘all is well…’
Big Bang approach – full delivery Long development cycles…
Little user involvement, etc. etc…
Symptoms of Software Symptoms of Software Development Problems Development Problems
► Inaccurate understanding Inaccurate understanding of end-user needs of end-user needs
►
Inability to deal with Inability to deal with changing requirements changing requirements
►
Modules that don’t fit together (integration) Modules that don’t fit together (integration)
►
Software that’s hard to maintain or extend (brittle) Software that’s hard to maintain or extend (brittle)
►
Late discovery of Late discovery of serious project flaws serious project flaws (integration) (integration)
►
Poor software quality Poor software quality (architecture, risks unanticipated…) (architecture, risks unanticipated…)
►
Process Process not responsive to Change (Gantt Charts…) not responsive to Change (Gantt Charts…)
►
Unacceptable software performance Unacceptable software performance
►
Team members in each other’s way, unable to reconstruct Team members in each other’s way, unable to reconstruct
who changed what, when, where, why (software architecture, who changed what, when, where, why (software architecture,
… …
►
… … and we could go on and on… and we could go on and on…
30 9
► We need a We need a process process that that
Will serve as a framework for large scale and small Will serve as a framework for large scale and small projects
projects
Adaptive – embraces ‘change!’ Adaptive – embraces ‘change!’
►Opportunity for Opportunity for improvementimprovement not identification of not identification of failurefailure!!
Iterative (small, incremental ‘deliverables’) Iterative (small, incremental ‘deliverables’)
Risk-driven (identify / resolve risks up front) Risk-driven (identify / resolve risks up front)
Flexible, customizable process (not a burden; adaptive to Flexible, customizable process (not a burden; adaptive to projects)
projects)
Architecture-centric (breaks components into ‘layers’ or Architecture-centric (breaks components into ‘layers’ or common areas of responsibility…)
common areas of responsibility…)
Heavy Heavy user involvement user involvement
► Identify best ways of doing things – a better process Identify best ways of doing things – a better process – acknowledged by world leaders…
– acknowledged by world leaders…
Need a Better Hammer!
Develop Iteratively Develop Iteratively
Control Changes Control Changes Use Use
Component Component Architectures Architectures Manage
Manage Requirements Requirements
Model Model Visually
Visually VerifyVerify Quality Quality
Best Practices of Software Best Practices of Software
Engineering Engineering
Know these!
30 11
Symptom s
end-user needs
changing
requirements modules don’t fit
hard to maintain
late discovery poor quality poor
performance colliding developers build-and- release
Root Causes
insufficient requirements ambiguous
communications brittle
architectures overwhelming complexity
undetected
inconsistencies poor testing subjective assessment waterfall
development uncontrolled change
insufficient automation
Best
Practices
develop iteratively manage
requirements use component architectures model the software visually
verify quality control changes
Addressing Root Causes Addressing Root Causes Eliminates the Symptoms Eliminates the Symptoms
Symptoms of problems can be traced to having Root Causes.
Best Practices are ‘practices’ designed to address the root causes of software problems.
Practice 1: Develop Software Practice 1: Develop Software
Iteratively Iteratively
Develop Iteratively Develop Iteratively
Control Changes Use
Component Architectures Manage
Requirements
Model
Visually Verify Quality
Considered by many practitioners to be the most significant of the six
30 13
Practice 1: Develop Software Iteratively Practice 1: Develop Software Iteratively
► Until recently, developed under assumption - Until recently, developed under assumption - most requirements can be identified up front.
most requirements can be identified up front.
► The research deconstructing this myth includes The research deconstructing this myth includes work by Capers Jones. (See next slide) In this work by Capers Jones. (See next slide) In this
very large study of 6,700 projects,
very large study of 6,700 projects, creeping creeping requirements
requirements — those not anticipated near the — those not anticipated near the start—are a
start—are a very significant fact of software very significant fact of software development life
development life , ranging from around 25% on , ranging from around 25% on average projects up to 50% on larger ones.
average projects up to 50% on larger ones.
Look up a definition of ‘Function Points.’
30 15
Interestingly, Interestingly,
►
An initial design will An initial design will likely be flawed likely be flawed with respect to its key with respect to its key requirements. Requirements rarely
requirements. Requirements rarely fully known fully known up front! up front!
►
Late-phase discovery of design defects Late-phase discovery of design defects results in costly results in costly over-runs and/or project cancellation
over-runs and/or project cancellation
Oftentimes requirements change – even during implementation! Oftentimes requirements change – even during implementation!
►
While large projects are more prone to cost overruns, While large projects are more prone to cost overruns,
medium-size/small projects are vulnerable to cancellation.
medium-size/small projects are vulnerable to cancellation.
►
The key reasons continue to be The key reasons continue to be
poor project planning and management, poor project planning and management,
shortage of technical and project management expertise, shortage of technical and project management expertise,
lack of technology infrastructure, lack of technology infrastructure,
disinterested senior management, and disinterested senior management, and
inappropriate project teams.”inappropriate project teams.”
Waterfall Delays Risks Waterfall Delays Risks
R I S K
T I M E
Integration
System Test Code
Design Requirements
Waterfall risk
Walker Royce, 1995
30 17
Iterative Development Iterative Development
Earliest iterations address greatest risks
• Each iteration produces an executable release
• Each iteration includes integration, test, and assessment!
• Objective Milestones: short-term focus; short term successes!
Iteration 1 Iteration 2 Iteration 3
Accelerate Risk Reduction Accelerate Risk Reduction
Iterative
T I M E
Iteration Iteration Iteration Iteration Iteration
Risk reduction Risk reduction R
I S K
Waterfall risk
Walker Royce, 1995
30 19
Iterative Development Characteristics Iterative Development Characteristics
► Critical risks are resolved before making Critical risks are resolved before making large investments
large investments
► Initial iterations enable early user feedback Initial iterations enable early user feedback
Easy to resolve problems early. Easy to resolve problems early.
Encourages user feedback in meaningful ways Encourages user feedback in meaningful ways
► Testing and integration are continuous Testing and integration are continuous – – assures successful integration (parts all fit) assures successful integration (parts all fit)
Continuous testing. Continuous testing.
► Objective milestones provide short-term focus Objective milestones provide short-term focus
► Progress measured by assessing Progress measured by assessing implementations implementations
► Partial implementations can be deployed Partial implementations can be deployed
Waterfall method – no delivery Waterfall method – no delivery
Incremental development? May be some great Incremental development? May be some great values in delivering key parts of application.
values in delivering key parts of application.
Critical components delivered first?
Critical components delivered first?
► No big-bang approach! No big-bang approach!
30 20
UP Lifecycle Graph – Showing Iterations UP Lifecycle Graph – Showing Iterations
In an In an
iteration, , you may you may walk
walk
through all through all disciplines disciplines
OC NT NE T ST UR CT UR E
T I M E
STUDY THIS!!!
30 21
Executable Releases
Unified Process Iterations and Phases Unified Process Iterations and Phases
An iteration is a distinct sequence of activities with an established plan and evaluation criteria,
resulting in an ‘executable release.’
(There is a lot of very important ‘key’
terminology used here…
(cycle, iteration, phase, milestones, core disciplines, content of iterations, etc….)
Preliminary
IterationArchitect.
IterationArchitect.
Iteration Devel.
Iteration Devel.
Iteration Devel.
IterationTransition
IterationTransition Iteration
Elaboration Construction Transition Inception
Enables and encourages
Enables and encourages user user feedback
feedback Serious
Serious misunderstandingsmisunderstandings evident early in the life cycle evident early in the life cycle Development focuses on
Development focuses on critical critical issues – break it down!
issues – break it down!
Objective assessment thru Objective assessment thru testing and assessment testing and assessment
Inconsistencies detected early Inconsistencies detected early Testing starts earlier –
Testing starts earlier – continuous!
continuous!
Risks identified and addressed Risks identified and addressed early - via
early - via plannedplanned iterations! iterations!
Problems Addressed by Iterative Development Problems Addressed by Iterative Development
Root Causes
Root Causes Solutions Solutions
Insufficient requirementsInsufficient requirements
Ambiguous Ambiguous
communications communications
Brittle architecturesBrittle architectures
Overwhelming Overwhelming complexity
complexity
Subjective assessmentSubjective assessment
Undetected Undetected inconsistencies inconsistencies
Poor testingPoor testing
Waterfall developmentWaterfall development
Uncontrolled changeUncontrolled change
Insufficient automationInsufficient automation
30 23
No Free Lunch
No Free Lunch - Traps Abound… - Traps Abound…
► Major impacts on Project Managers, though…. Major impacts on Project Managers, though….
► Trap: When the initial risks are mitigated, new ones emergeTrap: When the initial risks are mitigated, new ones emerge
Do not do just the easy stuff, to look good.Do not do just the easy stuff, to look good.
Keep re-planning based on all new information.Keep re-planning based on all new information.
► Trap: Remember ‘some’ Trap: Remember ‘some’ Rework enables you to enhance your solutionRework enables you to enhance your solution
Accommodate change Accommodate change earlyearly in the project in the project
► Trap: Iterative development Trap: Iterative development does does notnot mean mean never to commit to a solution never to commit to a solution
► Monitor ‘scrap and rework’ Monitor ‘scrap and rework’
► Trap: Must Control “requirement creep, ” however… SomeTrap: Must Control “requirement creep, ” however… Some
clients will now naturally recognize many ‘musts…’clients will now naturally recognize many ‘musts…’
Many Traps in Iterative Development Many Traps in Iterative Development
Here is another trap: Too long initial iteration Here is another trap: Too long initial iteration
►
Winning is fun. Winning teams work better than loosing teams Winning is fun. Winning teams work better than loosing teams
►
Better Better to have a short initial iteration, than one too long to have a short initial iteration, than one too long
Cut scope if necessary (much more later)Cut scope if necessary (much more later)
►
Avoid ‘analysis-paralysis’ by Avoid ‘analysis-paralysis’ by time-boxing; time-boxing; you can enhance you can enhance in later iterations (more later)
in later iterations (more later)
►
Establish an Establish an even rhythm even rhythm for project (at least w/i a phase) for project (at least w/i a phase)
►
Focus on Focus on results results and and deliverables deliverables , not activities , not activities
30 25
Iterations Are Time-boxed Iterations Are Time-boxed
►
Work is undertaken within an Work is undertaken within an iteration iteration . .
►
The iteration plan The iteration plan defines defines the the artifacts artifacts to be to be delivered,
delivered, roles roles and and activities activities . .
►
An iteration is clearly An iteration is clearly measurable measurable . .
►
Iterations are Iterations are risk-driven risk-driven
►
Iterations are Iterations are planned planned . .
►
Iterations are Iterations are assessed assessed ! !
►
Generally, Generally, initial initial iterations (in Construction) based on iterations (in Construction) based on high risk and core functionalities!
high risk and core functionalities!
The Iteration Plan Defines….
The Iteration Plan Defines….
TheThe
deliverables
deliverables for for that iteration.
that iteration.
The The to do listto do list for the team for the team members
members
artifacts
30 27
Problem:
Problem:
Fixed
Fixed Plans Produced Upfront – Not Plans Produced Upfront – Not Real Practical!
Real Practical!
►
Yet, senior management wants firm, fixed plans! Yet, senior management wants firm, fixed plans!
Part of their culture / upbringing/ experiencePart of their culture / upbringing/ experience
Necessary for ‘planning’ budgeting, etc. of resources, projects…. BUT:Necessary for ‘planning’ budgeting, etc. of resources, projects…. BUT:
►
Trap Trap : Fine-grained planning : Fine-grained planning from start to end? from start to end?
Takes too much timeTakes too much time
Frustrating as change occurs (and it Frustrating as change occurs (and it will)will), if plans too fine-grained., if plans too fine-grained.
►
Know that: Know that: Projects typically have some degree of Projects typically have some degree of uncertainty uncertainty
►
This makes This makes detailed detailed plans for the plans for the entire entire project meaningless project meaningless
►
Does not mean that we should not plan Does not mean that we should not plan
Solution:
Solution:
Plan With Evolving Levels of Detail Plan With Evolving Levels of Detail
Current Iteration
Next Iteration
Phases and major milestones
What and when Iterations for each phase
Number of iterations
Objectives and Duration
One For Entire Project
Fine-grained Plans: Iteration
Plans Coarse-grained Plan:
Software Development Plan
Iterative Development does not mean less work and shorter schedule
It is about greater
predictability
30 29
Progress is made against MILESTONES Progress is made against MILESTONES
► In the Unified Process: In the Unified Process:
Each phase is defined by a Each phase is defined by a milestone milestone . .
Progress is made by passing Progress is made by passing milestones milestones . .
Milestones measure Milestones measure success success
► Phases Phases - NOT TIMEBOXED. - NOT TIMEBOXED.
► Iterations Iterations ARE TIMEBOXED. ARE TIMEBOXED.
Inception Elaboration Construction Transition Major Milestones