The Software Development
Life Cycle: An Overview
Presented by
Maxwell Drew
and
Dan Kaiser
Southwest State University
Computer Science Program
Last Time
n
Brief review of the testing process
n
Dynamic Testing Methods
n
Static Testing Methods
n
Deployment in MSF
n
Deployment in RUP
Session 8:
Security and Evaluation
n
General Systems Engineering Concepts
n
Information Systems Security
Engineering Process
n
Relation of ISSE Process to other
Processes
n
Product, Process & Resource
Evaluation
n
Course Evaluations
Information Systems
Security Engineering
n
General Systems Engineering Concepts
n
Information Systems Security
Engineering Process
n
Relation of ISSE Process to other
Processes
Systems Engineering Process
Users / Users’ Representatives Assess Effectiveness Discover Needs Define System Design System Implement System
Discover Needs
nMission/Business Description
nPolicy Consideration
n
Mission Needs Statement (MNS)
Define System Functionality
nObjectives - MoE
nSystem Context/Environment
nRequirements - RTM
nFunctional Analysis
Define System
nFunctional Allocation - CM
nPreliminary Design–Baseline
Configuration
nDetailed Design - CI
Implement System
nProcurement
nBuild
nTest
Assess Effectiveness
nInteroperability
nAvailability
nTraining
nHuman/Machine Interface
nCost
ISSE Activities
n Describing information protection needs
n Generating information protection requirements based on needs
early in the systems engineering process
n Satisfying the requirements at an acceptable level of information
protection risk
n Building a functional information protection architecture based on
requirements
n Allocating information protection functions to a physical and logical
architecture
n Designing the system to implement the information protection
architecture
n Balancing information protection risk management and other ISSE
considerations within the overall system context of cost, schedule, and operational suitability and effectiveness
ISSE Activities - Continued
n Participating in trade-off studies with otherinformation protection and system engineering disciplines
n Integrating the ISSE process with the systems engineering and acquisition processes
n Testing the system to verify information protection design and validate information protection requirements
n Supporting the customers after deployment and
Discover Information Protection Needs
Mission Information Mission Information
System Requirements System Requirements Information Protection Requirements Threat Analysis
Threat Analysis PolicyPolicy
Information Protection Policy
Information Protection Policy
Figure 3-2 Impact of Mission, Threats, and Policies on Information Protection Requirements
Layered Requirements Hierarchy
MISSION/BUSINESS ARCHITECTURE DESIGN IMPLEMENTATION Functions Components Specifications More Specific More Specific More Abstract More Abstract
Mission Information Protection Needs
n What kind of information records are being viewed,updated, deleted, initiated, or processed (classified, financial, proprietary, personal private, etc.)?
n Who or what is authorized to view, update, delete, initiate, or process information records?
n How do authorized users use the information to perform
their duties?
n What tools (paper, hardware, software, firmware, and procedures) are authorized users using to perform their duties?
n How important is it to know with certainty that a particular individual sent or received a message or file?
Threats to Information Management
n
Types of Information
n
Legitimate users and uses of information
n
Threat agent considerations
- Capability
- Intent
- Willingness
- Motivation
- Damage to mission
Information Protection Policy
Considerations
n
Why protection is needed
n
What protection is needed
n
How protection is achieved not
considered at this stage
Information Protection Policy Issues
n
The resources/assets the organization has
determined are critical or need protection
n
The roles and responsibilities of individuals that
will need to interface with those assets (as part
of their operational mission needs definition)
n
The appropriate way (authorizations) authorized
individuals may use those assets (security
requirements).
Define Information Protection System
n
Information Protection Objectives – MoE
n
System Context/Environment
n
Information Protection Requirements –
RTM
n
Functional Analysis
Information Protection
Objectives Should Explain:
n
The mission objectives supported by
information protection objective
n
The mission-related threat driving the
information protection objective
n
The consequences of not implementing
the objective
n
Information protection guidance or
policy supporting the objective
Design Information Protection System
n
Functional Allocation
n
Preliminary Information Protection Design
n
Detailed Information Protection Design
Preliminary Information Protection
Design Activities
n Reviewing and refining Discover Needs and Define System
activities' work products, especially definition of the CI-level and interface specifications
n Surveying existing solutions for a match to CI-level requirements
n Examining rationales for proposed PDR-level (of abstraction)
solutions
n Verification that CI specifications meet higher-level information
protection requirements
n Supporting the certification and accreditation processes
n Supporting information protection operations development and
life-cycle management decisions
n Participating in the system engineering process
Detailed Information Protection Design Activities
n Reviewing and refining previous Preliminary Design work products
n Supporting system- and CI-level design by providing input on feasible
information protection solutions and/or review of detailed design materials
n Examining technical rationales for CDR-level solutions
n Supporting, generating, and verifying information protection test and
evaluation requirements and procedures
n Tracking and applying information protection assurance mechanisms
n Verifying CI designs meet higher level information protection
requirements
n Completing most inputs to the life-cycle security support approach,
including providing information protection inputs to training and emergency training materials
n Reviewing and updating information protection risk and threat
projections as well as any changes to the requirements set
n Supporting the certification and accreditation processes
n Participating in the system engineering process
Implement Information Protection System
n
Procurement
n
Build
Implement Information Protection System
General Activities
n Updates to the system information protection threat assessment, as projected, to the system's operational existence
n Verification of system information protection requirements
and constraints against implemented information protection solutions, and associated system verification and validation mechanisms and findings
n Tracking of, or participation in, application of information protection assurance mechanisms related to system implementation and testing practices
Implement Information Protection System
General Activities (cont.)
n Further inputs to and review of evolving system operational
procedure and life-cycle support plans, including, for example, Communication Security (COMSEC) key distribution or releasability control issues within logistics support and information protection relevant elements within system operational and maintenance training materials
n A formal information protection assessment in preparation for
the Security Verification Review
n Inputs to Certification and Accreditation (C&A) process
activities as required
n Participation in the collective, multidisciplinary examination of
all system issues
Build Information Protection System
n
Physical Integrity.
n Have the components that are used in the production been properly safeguarded against tampering?
n
Personnel Integrity.
n Are the people assigned to construct or assemble the
system knowledgeable in proper assembly procedures, and are they cleared to the proper level necessary to ensure system trustworthiness?
Test Information Protection System
Activities
n Reviewing and refining Design Information Protection System work
products
n Verifying system- and CI-level information protection requirements
and constraints against implemented solutions and associated system verification and validation mechanisms and findings
n Tracking and applying information protection assurance
mechanisms related to system implementation and testing practices
n Providing inputs to and review of the evolving life-cycle security
support plans, including logistics, maintenance, and training
n Continuing risk management activities
n Supporting the certification and accreditation processes
n Participating in the systems engineering process
Assess Effectiveness
n Interoperability.
n Does the system protect information correctly across external interfaces?
n Availability.
n Is the system available to users to protect information and information
assets?
n Training.
n What degree of instruction is required for users to be qualified to operate
and maintain the information protection system?
n Human/Machine Interface.
n Does the human/machine interface contribute to users making mistakes or
compromising information protection mechanisms?
n Cost.
n Is it financially feasible to construct and/or maintain the information
protection system?
Relation to Other Processes
n
System Acquisition Process
n
Risk Management Process
n
DITSCAP
ISSE and System Acquisition Process Flows
• Identify New Concepts • Test Concepts • Formalize Security Concepts • Translate Into System Design • Specify System Components • Purchase Components • Build Production System • Formal Testing • Support Over Lifecycle • User Program Needs • System Req. • Security Req. System Acquisition Production & Operational Support Requirements Identification Phase Concept Exploration Phase Engineering & Implementation Assess EffectivenessUsers / Users’ Representatives Information System Security Engineering (ISSE) Process
Discover Information Protection Needs Define Information Protection System Design Information Protection System Implement Information Protection System
Risk Management Process
Figure 3-5 Risk Management Process
Understand Mission Objectives Risk Management Cycle Understand Information Protection Needs (Services) Characterize Risk Posture Characterize What Can Be Done Implement Decision Actions Decide What Will Be Done
Risk Decision Flow
Countermeasure Identification & Characterization Compare and Contrast Various Courses of Action Mission Critical Parameter Trade-Off Develop Theory of Mission Impact Develop Theory of Adversarial Behavior Compare and Contrast Available Attacks Decide on Courses of Action Mission Impact Identification & Characterization Threat Identification & Characterization Vulnerability and Attack Identification & Characterization System Improvements
Risk Decision Flow
Risk Analysis
Foundation Research and Incidence Analysis
Risk Plane
High Med. Low High Med. Low Y - CONSEQUENCE X - LIKELIHOOD OF SUCCESS A-1 A-2 A-3Figure 3-7 Risk Plane
DITSCAP
Flow
Registration Negotiation AgreementDocument Mission Need SSAA Ready For Certification System Development Activity Certification Analysis Acceptable SSAA Certification Evaluation of The Integrated System Certify System Develop Recommendation Accreditation Granted SSAA System Operation Compliance Validation Required Change Requested Yes No Yes Yes Yes No No Life Cycle Activity (1 to n)
Correct Reanalyze Yes No No Yes No Yes No Yes Phase 1 – Definition Phase 2 – Verification Phase 3 – Validation
Phase 4 – Post Accreditation
Security Concepts & Relationships
Countermeasures Countermeasures Owners Owners Vulnerabilities Vulnerabilities Risk Risk Assets Assets Threat Agents Threat Agents Threats Threats may be aware of value impose to reduce that may possess that may be reduced by
give rise to that increase
wish to abuse and/or may damage that exploit leading to to to wish to minimize
Figure 3-9 Security Concepts and Relationships in the Common Criteria
Protection
Profile
TOE physical environment
TOE physical environment Assets requiring
protection Assets requiring protection TOE purpose TOE purpose Establish security environment Establish security environment Assumptions
Assumptions ThreatsThreats Organizational security problems Organizational security problems Establish security objectives Establish security objectives CC requirements catalog CC requirements catalog Security objectives Security objectives Establish Security Requirements Establish Security Requirements Functional requirements Functional requirements Assurance requirements Assurance requirements Requirements for
the environment Requirements for the environment Establish TOE summary specifications Establish TOE summary specifications TOE summary specification Security Environment material (PP/ST) Security Objectives material (PP/ST) Security Specification material (PP/ST) Security Requirements material (PP/ST)
Evaluation Concepts & Relationships
produce Assurance Techniques Assurance Techniques Confidence Confidence Owners Owners Countermeasures Countermeasures Risk Risk Assets Assets Assurance Assurance Evaluation Evaluation Gives evidence of giving require that minimize to
Use of Evaluation Results
Figure 3-12 Uses of Evaluation Results
Develop & evaluate TOE Develop & evaluate TOE Catalog product Catalog product Accredit system Accredit system Evaluated Products Catalog Evaluated Products Catalog PPs Catalog PPs Catalog Security requirements Security
requirements Evaluationresults Evaluation results Evaluated Product Evaluated Product Accredited system Accredited system System accreditation criteria System accreditation criteria (optional) (optional) (alternatives)
Questions?
Evaluation
nGeneral Techniques
n
Evaluating the Product
n
Evaluating the Process
n
Evaluating Resources
Categories of Evaluation
n
Feature analysis:
n
rate and rank attributes
n
Survey:
n
document relationships
n
Case study
n
sample from variables
n
Formal experiment
Example Feature Analysis
Table 12.1. Design tool ratings
Feature Tool 1: t-OO-l Tool 2: ObjecTool Tool 3: EasyDesign Importance
Good user interface 4 5 4 3
Object-oriented design 5 5 5 5 Consistency checking 5 3 4 3 Use cases 5 4 1 2 Runs on Unix 4 4 5 5 Score 82 77 73
Case Study Types
n
Sister projects:
n
each is typical and has similar values for
the independent variables
n
Baseline:
n
compare single project to organizational
norm
n
Random selection:
n
partition single project into parts
Formal Experiment
n
Controls variables
n
Uses methods to reduce bias and
eliminate confounding factors
n
Often replicated
n
Instances are representative:
n
sample over the variables (whereas case
study samples from the variables)
Evaluation Steps
n
Setting the hypothesis
n the tentative supposition that we think explains the behavior we want to explore
n
Maintaining control over variables
n decide what effects our hypothesis
n
Making investigation meaningful
n determine the degree to which results can be
generalized
Table 12.2. Common pitfalls in evaluation. Adapted with permission from (Liebman 1994)
Pitfall Description
1. Confounding Another factor is causing the effect.
2. Cause or effect? The factor could be a result, not a cause, of the treatment.
3. Chance There is always a small possibility that your result happened by chance.
4. Homogeneity You can find no link because all subjects had the same level of the factor.
5. Misclassification You can find no link because you cannot accurately classify each
subject’s level of the factor.
6. Bias Selection procedures or administration of the study inadvertently bias
the result.
7. Too short The short-term effects are different from the long-term ones.
8. Wrong amount The factor would have had an effect, but not in the amount used in
the study.
9. Wrong situation The factor has the desired effect, but not in the situation studied.
Common Evaluation Pitfalls
Assessment vs. Prediction
n
An assessment system examines an existing
entity by characterizing it numerically
n
Prediction system predicts characteristic of a
future entity; involves a model with associated
prediction procedures
n deterministic prediction (we always get the same
output for an input)
Product Quality Models
n
Boehm’s Model
n
ISO 9126 Model
Boehm’s Model
ISO 9126 Model
Table 12.5. Quantitative targets for managing US defense projects. (NetFocus 1995)
Item Target Malpractice level
Fault removal efficiency > 95% < 70%
Original fault density < 4 per function point > 7 per function point
Slip or cost overrun in excess of risk reserve
0% > 10%
Total requirements creep (function points or equivalent)
< 1% per month average > 50%
Total program documentation
< 3 pages per function point
> 6 pages per function point
Staff turnover 1 to 3% per year > 5% per year
Targeting
Software Reuse
n
Producer reuse:
n creating components for someone else to use
n
Consumer reuse:
n using components developed for some other product
n
Black-box reuse:
n using component without modification
n
Clear- or white-box reuse:
n modifying component before reusing it
Process Evaluation
n
Postmortem Analysis
n
a post-implementation assessment of all
aspects of the project
n
Process Maturity Models
n
development has built in feedback and
Postmortem Analysis
n
Design and promulgate a project survey
to collect relevant data.
n
Collect objective project information.
n
Conduct a debriefing meeting.
n
Conduct a project history day.
n
Publish the results by focusing on
lessons learned.
Table 12.9. When post-implementation evaluation is done.
Time period Percentage of respondents (of 92 organizations)
Just before delivery 27.8%
At delivery 4.20%
One month after delivery 22.20%
Two months after delivery 6.90%
Three months after delivery 18.10%
Four months after delivery 1.40%
Five months after delivery 1.40%
Six months after delivery 13.90%
Twelve months after delivery 4.20%
Capability Maturity Model (CMM)
Table 12.10. Required questions for level 1 of process maturity model. Question number Question
1.1.3 Does the Software Quality Assurance function have a management reporting channel separate from the software development project management?
1.1.6 Is there a software configuration control function for each project that involves software development? 2.1.3 Is a formal process used in the management review of each
software development prior to making contractual commitments?
2.1.14 Is a formal procedure used to make estimates of software size? 2.1.15 Is a formal procedure used to produce software development
schedules?
2.1.16 Are formal procedures applied to estimating software development cost?
2.2.2 Are profiles of software size maintained for each software configuration item over time?
2.2.4 Are statistics on software code and test errors gathered? 2.4.1 Does senior management have a mechanism for the regular
review of the status of software development projects? 2.4.7 Do software development first-line managers sign off on their
schedule and cost estimates?
2.4.9 Is a mechanism used for controlling changes to the software requirements?
2.4.17 Is a mechanism used for controlling changes to the code?
Table 12.11. Key process areas in the CMM (Paulk et. al. 1993)
CMM level Key process areas
Initial none
Repeatable Requirements management
Software project planning Software project tracking and oversight Software subcontract management Software quality assurance Software configuration management
Defined Organization process focus
Organization process definition Training program Integrated software management Software product engineering Inter-group coordination Peer reviews
Managed Quantitative process management
Software quality management
Optimizing Fault prevention
Technology change management Process change management
Evaluating Resources
n
People Maturity Model
Table 12.13. People capability maturity model. (Curtis, Hefley and Miller 1995)
Level Focus Key practices
5: optimizing Continuous knowledge and skills improvement
Continuous workforce innovation Coaching
Personal competency development
4: managed Effectiveness measured and
managed, high performance teams developed Organizational performance alignment Organizational competency management Team-based practices Team-building Mentoring
3: defined Competency-based workforce
practices Participatory culture Competency-based practices Career development Competency development Workforce planning Knowledge and skills analysis 2: repeatable Management takes responsibility
for managing its people
Compensation Training Performance management Staffing Communication Work environment 1: initial
