Barry Boehm, USC-CSSE
http://csse.usc.edu
ISCAS Boehm 75 Symposium
April 27, 2011
Skating to Where the Puck Is Going:
Future Software Engineering Opportunities
and Challenges
4/27/2011
Outline
•
The Future of Information Technology
–
8 surprise-free trends; 2 wild-card trends
–
Changes since 2006 paper
–
Individual and combined software
engineering opportunities and challenges
•
Conclusions: General SW engineering
implications
The Future of Systems and Software: 2006
•
Eight surprise-free trends
1. Increasing integration of SysE and SwE 2. User/Value focus
3. Software Criticality and Dependability 4. Rapid, Accelerating Change
5. Distribution, Mobility, Interoperability, Globalization 6. Complex Systems of Systems
7. COTS, Open Source, Reuse, Legacy Integration 8. Computational Plenty
•
Two wild-card trends
9. Autonomy Software
2011 Trends Largely Missed in 2006
•
Megasensor-intensive smart systems
•
Search and mining of ultralarge data aggregations
•
Software implications of multicore chips
•
Rapid growth of software as a service
•
Rapid growth of social networking technologies
The Future of Systems and Software: 2011
•
Eight surprise-free trends
1. Rapid, Accelerating Change
2. Software Criticality and Dependability
3. Complexity; Global/Mobile Systems of Systems 4. COTS, Open Source, Services, Legacy Integration 5. Smart Systems; Mining huge volumes of data
6. User Evolution and End Value Focus
7. Computational Plenty and Multicore Chips 8. Increasing integration of SysE and SwE
•
Two wild-card trends
9. Autonomy Software
4/27/2011
1. Rapid Change Trends
•
Global connectivity and competition accelerate
change
– More ripple effects of technology, marketplace changes
•
Increased need for agility, continuous learning
– Need to balance agility and plan-driven dependability
– Decline of THWADI (That’s how we’ve always done it)
– Avoid technical agility, administrative THWADI
•
Hybrid agile/plan-driven processes needed for
larger systems
•
Need for incremental processes, methods, tools,
skills
•
Need for pro-active technology, marketplace
monitoring
Architected Agile Approach
•
Uses Scrum of Scrums approach
– Up to 10 Scrum teams of 10 people each
– Has worked for distributed international teams
– Going to three levels generally infeasible
•
General approach shown below
4/27/2011
2. Criticality and Dependability Trends
•
Software increasingly success-critical to product and
services
– Provides competitive differentiation, adaptability to change
•
Dependability is generally not vendors’ top-priority
– “The IT industry spends the bulk of its resources… on rapidly bringing products to market.” – US PITAC Report
•
By 2025, there will be a “9/11” – magnitude software
failure
– Major loss of life or collapse of world financial system
•
This will raise dependability to vendors’ top priority
– Market demand; stronger warranties and accountability
– Value-based dependability processes and tools
• Avoid bureaucratic solutions
Achieving Agility and High Assurance -I
Using timeboxed or time-certain development
Precise costing unnecessary; feature set as dependent variable
Rapid
Change
High
Assurance
Short, Stabilized
Development
Of Increment N
Increment N Transition/O&M Increment N Baseline Short Development Increments Foreseeable Change (Plan) Stable Development IncrementsEvolutionary Concurrent Engineering:
Incremental Commitment Spiral Model
Agile Rebaselining for Future Increments Short, Stabilized Development of Increment N Verification and Validation (V&V) of Increment N Deferrals Artifacts Concerns Rapid Change High Assurance
Future Increment Baselines
Increment N Transition/ Operations and Maintenance
Future V&V Resources Increment N Baseline
Current V&V Resources
Unforeseeable Change (Adapt)
Short Development Increments Foreseeable Change (Plan) Stable Development Increments Continuous V&V 4/27/2011
3. Complexity and Global Software-Intensive
Systems of Systems (SISOS)
•
Lack of integration among stovepiped
systems causes
– Unacceptable delays in service
– Uncoordinated and conflicting plans
– Ineffective or dangerous decisions
– Inability to cope with fast-moving events
•
Increasing SISOS benefits
– See first; understand first; act first
– Network-centric operations coordination
– Transformation of business/mission potential
4/27/2011
Complexity of Solution Spaces
• Size: 10-100 MLOC
• Number of external interfaces: 30-300
• Number of “Coopetitive” suppliers: 20-200
– Even more separate work locations
• Depth of supplier hierarchy: 6-12 levels
• Number of coordination groups: 20-200
– Reviews, changes, risks, requirements, architecture,
standards, procedures, technologies, -ilities, integration, test, deployment, personnel, infrastructure, COTS,…
– Key personnel spend 60 hours/week in meetings
• Unprecedentedness
• Emergence
The Future of Systems and Software: 2011
•
Eight surprise-free trends
1. Rapid, Accelerating Change
2. Software Criticality and Dependability
3. Complexity; Global/Mobile Systems of Systems 4. COTS, Open Source, Services, Legacy Integration 5. Mining huge volumes of data
6. User Evolution and End Value focus
7. Computational Plenty and Multicore Chips 8. Increasing integration of SysE and SwE
•
Two wild-card trends
9. Autonomy Software
4/27/2011
4. COTS: The Future Came and Is Leaving
•
Escalate COTS priorities for research, staffing,
education
– It’s not “all about programming” anymore
– New processes required
CBA Growth in USC E-Service Projects
* Standish Group CHAOS 2000
CBA Growth Trend in USC e-Services Projects
0 10 20 30 40 50 60 70 80 1997 1998 1999 2000 2001 2002 Year P e rc e n ta g e *
Purchased Services (Cloud Computing) Growth
in USC e-Services Projects
4/27/2011
Persistence of Legacy Systems
•
Before establishing new-system increments
– Determine how to undo legacy system
5. Megasensor- Empowered Smart Systems
Smart power grids, buildings, companies, cities
•
Ubiquitously-instrumented artifacts and processes
•
Complementary growth in data storage and analysis
•
EU Digital Agenda “Internet of Things”
•
Commitments: Singapore, Abu Dhabi, S. Korea, Portugal
– Industry: IBM, HP, Cisco, Siemens, GE
Mining huge volumes of data
•
Google example: billions (B) of search hits
– All in about 0.2 seconds (9/25/10; fewer, faster 11/17/10)
– Video, 16.1B; TV, 9.6B; Star, 6.1B; Time, 5.4B; Movie, 4.4B; News, 2.8B; Music, 2.7B; Life, 2.3B; Play, 2.1B; Book, 1.7B
– What to show first?
– How to narrow search to what you want?
•
Recommender systems
– Based on preference data or past activity
– Amazon.com; Pandora; Netflix
•
Service-provider data warehousing
– Better services, but service provider has your data
•
General concerns with privacy, controls
6. User/Value Focus Trends
•
Computerworld panel: More focus on user/ownership
costs and benefits; less focus on features and license
costs
– Technology should adapt to people, not vice versa
– Tension between usability and feature creep
•
User-orientation has many challenges
– Emergent needs and priorities: IKIWISI, Maslow
– Diversity of people and cultures: no OSFA solutions
– Group vs. individual performance
– Engineer focus on engineer-usability
Golden Rule: Do unto others as you would have others do unto you
Platinum Rule: Do unto others as they would be done unto
4/27/2011
Value-Based Testing: Empirical Data and ROI
—
LiGuo Huang, ISESE 2005-1.5 -1 -0.5 0 0.5 1 1.5 2 0 10 20 30 40 50 60 70 80 90 100 % Tests Run R e tu rn O n I n v e s tm e n t (R O I)
Value-Neutral ATG Testing Value-Based Pareto Testing
% of Value for Correct Customer Billing Customer Type 100 80 60 40 20 5 10 15 Automated test generation (ATG) tool
- all tests have equal value
Bullock data – Pareto distribution % of Value for Correct Customer Billing Customer Type 100 80 60 40 20 5 10 15 Automated test generation (ATG) tool
- all tests have equal value
% of Value for Correct Customer Billing Customer Type 100 80 60 40 20 5 10 15 Automated test generation (ATG) tool
- all tests have equal value
Bullock data – Pareto distribution
(a)
The Future of Systems and Software: 2010
•
Eight surprise-free trends
1. Rapid, Accelerating Change
2. Software Criticality and Dependability
3. Complexity; Global/Mobile Systems of Systems 4. COTS, Open Source, Services, Legacy Integration 5. Mining huge volumes of data
6. User Patterns and End Value Focus
7. Computational Plenty and Multicore Chips 8. Increasing integration of SysE and SwE
•
Two wild-card trends
9. Autonomy Software
7. Computational Plenty and Multicore Chips
•
Moore’s Law stymied by heat dissipation problems
– 2x circuit speed, density every 18 months
•
Keep growth by developing multi-CPU chips
– Lower circuit speed, but lower power consumption
– Growth in #CPUs keeps up processing power growth
– But only if programs can be parallelized
– Otherwise, legacy software will run more slowly
– Amdahl’s Law: Speed limited by speed of slowest part on critical computation path
•
But can also use CPUs for other purposes
– Assertion checking, intrusion detection, trend analysis, option analysis, performance monitoring, fault tolerance
8. Increasing SysE/SwE Integration
•
Can’t do good SwE by neglecting SysE
–
Weak SysE the root cause of most SW project
failures
•
Can’t do good SysE by neglecting critical
success factors
–
Software an increasing system critical success
factor
• Provides most of competitive differentiation
• Provides most of adaptability to change
The Incremental Commitment Spiral Model
4/27/2011 1 2 3 4 5 6 RISK-BASED STAKEHOLDER COMMITMENT REVIEW POINTS: Opportunities to proceed, skip phases backtrack, or terminateExploration Commitment Review Valuation Commitment Review Foundations Commitment Review Development Commitment Review Operations1and Development2 Commitment Review
Operations2and Development3
Commitment Review
Cumulative Level of Understanding, Product and Process Detail (Risk-Driven) Concurrent Engineering of Products and Processes 2 3 4 5 EXPLORATION VALUATION FOUNDATIONS DEVELOPMENT1 FOUNDATIONS2 OPERATION2 DEVELOPMENT3 FOUNDATIONS4 1 6
Evidence-Based Review Content - A first-class deliverable - Independent expert review
- Shortfalls are uncertainties and risks
OPERATION1 DEVELOPMENT2 FOUNDATIONS3 Risk Risk-Based Decisions Acceptable Negligible High, but Addressable Too High, Unaddressable
Concurrent
Engineering:
ICSM Activity
Levels for
Complex
Systems
|
Creates Need to
Synchronize
and Stabilize
the
Concurrency
4/27/2011
ICSM Loop Invariant: Feasibility Evidence and Risk
• Evidence
provided by developer and validated by independent experts
that:
If the system is built to the specified architecture, it will
– Satisfy the requirements: capability, interfaces, level of service, and evolution – Support the operational concept
– Be buildable within the budgets and schedules in the plan – Generate a viable return on investment
– Generate satisfactory outcomes for all of the success-critical stakeholders
• All major risks resolved or covered by risk management plans
– Shortfalls in evidence are uncertainties or probabilities of loss – Risk Exposure = Prob (Loss) * Size (Loss)
• Serves as basis for stakeholders’ commitment to proceed
4/27/2011
9, 10. Wild Cards: Autonomy and Bio-Computing
•
Great potential for good
– Robot labor; human shortfall compensation
5 Senses, healing, life span, self-actualization
– Adaptive control of the environment
– Redesigning the world for higher quality of life
Physically, biologically, informationally
•
Great potential for harm
– Loss of human primacy: computers propose, humans decide
– Overempowerment of humans
Accidents, terrorism, 1984 revisited
– New failure modes: adaptive control instability, self-modifying software, commonsense reasoning, bio-computer mismatches
– V&V difficulties: cooperating autonomous agents, biocomputing
Software Engineering Education Implications
• Current software engineering students will be practicing into the 2050s. Their education should consider the following:
– Anticipating future trends and preparing students to deal with them;
– Capitalizing on information technology to enable the delivery of just-in-time and web-based education;
– Monitoring current principles and practices and separating timeless principles from outdated practices;
– Participating in leading-edge software engineering research and practice and incorporating the results into the curriculum;
– Packaging smaller-scale educational experiences in ways that apply to large-scale projects;
– Helping students learn how to learn, through state-of-the-art analyses, future-oriented educational games and exercises, and participation in research; and – Offering lifelong learning opportunities for systems engineers who must
4/27/2011
Backup Charts
•
Limitations to software process
perfectability
•
Value-based systems and software
Limitations: Brooks’ Four Essentials Plus Two
•
Complexity: larger components, systems of systems,
attribute tradeoffs
•
Conformity: evolving standards, external
system/COTS constraints
•
Changeability: solution half-life, unpredictable
certainties
•
Conceptuality (invisibility): COTS opacity, multi-view
consistency
•
Community: stakeholder proliferation, distribution,
diversity
•
Centrality: software failure risks, rice-bowl
implications
4/27/2011
Limitations: Lampson’s Continuing SW Crisis
•
Moore’s Law enables the feasibility of new
applications
– Requiring new and often more complex software
•
Easier to handle complexity in software than
elsewhere
– Good engineering practice to address via software
•
Few physical limits on software applications
Limitations: Converse of Conway’s Law
•
Convay’s Law (extended to user organizations)
–
The structure of a computer program
–
Reflects the structure of
4/27/2011
Limitations: Converse of Conway’s Law
•
Conway’s Law (extended to user organizations)
–
The structure of a computer program
–
Reflects the structure of
–
The organizations that build and use it
•
Converse of Conway’s Law
–
We will learn how to build perfectly functioning
software
–
As soon as
–
We learn how to build perfectly functioning
organizations
Initial VBSE Theory: 4+1
- with Apurva Jain
•
Engine: Theory W (stakeholder win-win): What values
are important?
– Enterprise Success Theorem – Theory of Justice
– Win-Win Equilibrium and Negotiation
•
Four Supporting Theories
– Utility Theory: How important are the values?
– Multi-attribute utility; Maslow need hierarchy
– Decision Theory: How do values determine decisions?
– Investment theory; game theory; statistical decision theory
– Dependency Theory: How do dependencies affect value realization?
– Results chains; value chains; cost/schedule/performance tradeoffs
– Control Theory: How to monitor and control value realization
4/27/2011
Theory W: Enterprise Success Theorem
– And informal proof
Theorem: Your enterprise will succeed
if and only if
it makes winners of your success-critical stakeholders
•
Proof of “if”:
Everyone that counts is a winner.
Nobody significant is left to complain.
•
Proof of “only if”:
Nobody wants to lose.
Prospective losers will refuse to participate, or will
counterattack.
Initial VBSE Theory: 4+1 Process
–
With a great deal of concurrency and backtracking
Utility Theory Theory W: SCS Win-Win Decision Theory Dependency Theory Control Theory
6a, 7c. State measurement, prediction, correction; 5a. Investment analysis,
Risk analysis
1. Protagonist goals 3a. Solution exploration 7. Risk, opportunity, change management
5a, 7b. Prototyping
2a. Results Chains
3b, 5a, 7b. Cost/schedule/ performance tradeoffs 2. Identify SCSs
3b, 7a. Solution Analysis 5a, 7b. Option, solution
development & analysis
4. SCS expectations management 3. SCS Value Propositions (Win conditions) 6, 7c. Refine, Execute, Monitor & Control Plans 5. SCS Win-Win
4/27/2011
By Number P-value % Gr A
higher
By Impact P-value % Gr A
higher
Average of Concerns 0.202 34 Average Impact
of Concerns
0.049 65
Average of Problems 0.056 51 Average Impact
of Problems 0.012 89 Average of Concerns per hour 0.026 55 Average Cost Effectiveness of Concerns 0.004 105 Average of Problems per hour 0.023 61 Average Cost Effectiveness of Problems 0.007 108
• Group A: 15 IV&V personnel using VBR procedures and checklists • Group B 13 IV&V personnel using previous value-neutral checklists
– Significantly higher numbers of trivial typo and grammar faults
Experiment Experiment
Value-Based Reading (VBR) Experiment
—
Keun Lee, ISESE 2005
Adaptation Challenges: A Dual Cone of Uncertainty
– Need early systems engineering, evolutionary development
Uncertainties in competition, technology, organizations,
mission priorities
Uncertainties in scope, COTS, reuse, services
4/27/2011
Agile and Plan-Driven Home Grounds:
Five Critical Decision Factors
• Size, Criticality, Dynamism, Personnel, Culture Personnel
Dynamism
(% Requirements -change/month)
Culture
(% thriving on chaos vs. order)
Size
(# of personnel)
Criticality
(Loss due to impact of defects)
30 10 3.0 1.0 0.3 90 70 50 30 10 3 10 30 100 300 35 30 25 20 15 Essential Funds Discretionary Funds Comfort Single Life Many Lives
(% Level 1B) (% Level 2&3)
0 10 20 30 40 Agile Plan-driven Personnel Dynamism (% Requirements -change/month) Culture
(% thriving on chaos vs. order)
Size
(# of personnel)
Criticality
(Loss due to impact of defects)
90 70 50 30 10 3 10 30 100 300 35 30 25 20 15 Essential Funds Discretionary Funds Comfort Single Life Many Lives
(% Level 1B) (% Level 2&3)
0 10 20 30 40 Agile Plan-driven
Added Cost of Weak Architecting
Calibration of COCOMO II Architecture and Risk Resolution factor to 161 project data points
Effect of Size on Software Effort Sweet Spots
Software Process Management Implications
•
Increasing uncertainty requires risk/value-based
processes
– Concurrent engineering of system goals, solutions, plans
Integration of systems engineering and software engineering
– Thoroughly validated for consistency and feasibility
Via prototypes, benchmarks, models, role-playing
Addressing both quantitative and qualitative value factors
Validation progress becomes a key management metric
Validation shortfalls become risks to be managed
•
Criticality and rapid change require balance of agile,
plan-driven processes
– Plan-driven for foreseeable change, high criticality
Parnas encapsulation of sources of change
– Agile for unforeseeable change
4/27/2011
Computational Plenty: Other Implications
•
New platforms: smart dust, human prosthetics
(physical, mental)
– New applications: sensor networks, nanotechnology
•
Enable higher levels of abstraction
– Pattern programming, programming by example with dialogue
– Simpler brute-force solutions: exhaustive case analysis
•
Enable more powerful software tools
– Based on domain, programming, management knowledge
– Show-and-tell documentation
4/27/2011
Outline
•
The Future of Information Technology
–
8 surprise-free trends; 2 wild-card trends
–
Changes since 2005 paper
–
Individual and combined software
engineering opportunities and challenges
•
Conclusions: General SW engineering
implications
Software Process Research Implications – I
•
Empirically-evolved process technology
– Languages, methods, metrics, models, and tools
– Incremental and ambiguity-tolerant
– Accommodating incomplete, informal, and partial specification
– Bridging formality, life-cycle, and culture gaps
– Empirical testbed-based maturity/transition acceleration
•
Virtual process collaboration support
– Distributed, multi-stakeholder, multi-cultural
•
Game technology for process education and training
– Acquire and develop the way you train
4/27/2011
Software Process Research Implications – II
•
Lean, value-based processes for balancing
dependability and agility
– Plus scalability, incrementality for systems of systems
– General techniques for multi-attribute tradeoff analysis
– Associated progress metrics, review criteria, early warning indicators
•
Integrated technical and acquisition processes
– Supporting balance of dependability and agility
– Applicable to globally-distributed, multi-cultural collaboration
•
Process capitalization on computational plenty
– Self-monitoring software, higher levels of abstraction, knowledge-based tools
•
Integration and risk assessment of wild-card
technologies
Source Selection ● ●● ● ● ● ● ● ● ●● ● Valuation
Exploration Architecting Develop Operation
Valuation
Exploration Architecting Develop Operation
Operation
Develop Operation Operation Operation
System B System C System x LCO-type Proposal & Feasibility Info Candidate Supplier/ Strategic Partner ●●●● n ● ●● ● ● ● ● ● Candidate Supplier/ Strategic Partner 1
SoS-Level Exploration Valuation Architecting Develop
FCR1 DCR1 Operation OCR1 Rebaseline/ Adjustment FCR1 OCR2 • • • • •••• •••• • •• • ••• •• ••• • • • • •••• •••• • • • • •••• •••• • • • • •••• •••• • • • • •••• •••• • • • • •••• •••• OCRx1 FCRB DCRB OCRB1 FCRA DCRA FCRC DCRC OCRC1
OCRx2 OCRx3 OCRx4 OCRx5
OCRC2
OCRB2
OCRA1
Some Leading Brownfield Approaches
Re-engineering Legacy Software to be Service-Oriented
• IBM Brownfield VITA Approach
– Views: Formal descriptions of enabling business systems or processes
– Inventory: Repository that stores Views information
– Transforms: Define relationships between as-is and to-be states
– Artifacts: Results of Transforms generated from the Inventory
• CMU-SEI Service Migration and Reuse Technique (SMART)
– SMART Process from as-is to to-be state
– Service Migration Interview Guide: over 60 questions about migration context, nature, and feasibility
– SMART Tool: Helps gather data, identify risks
– Artifact Templates: For capturing info about stakeholders,
components, migration issues, legacy components, creating service components, etc.
Motivation for Value-Based SE
•
Current SE methods are basically value-neutral
– Every requirement, use case, object, test case, and defect is equally important
– Object oriented development is a logic exercise
– “Earned Value” Systems don’t track business value
– Separation of concerns: SE’s job is to turn requirements into verified code
– Ethical concerns separated from daily practices
•
Value–neutral SE methods are increasingly risky
– Software decisions increasingly drive system value
– Corporate adaptability to change achieved via software decisions
– System value-domain problems are the chief sources of software project failures
4/27/2011
Motivation for Value-Based SE
•
Current SE methods are basically value-neutral
– Every requirement, use case, object, test case, and defect is equally important
– Object oriented development is a logic exercise
– “Earned Value” Systems don’t track business value
– Separation of concerns: SE’s job is to turn requirements into verified code
– Ethical concerns separated from daily practices
•
Value–neutral SE methods are increasingly risky
– Software decisions increasingly drive system value
– Corporate adaptability to change achieved via software decisions
– System value-domain problems are the chief sources of software project failures
