DANSE Software Quality Assurance

(1)

DANSE Software Quality

Assurance

Tom Swain

Software Quality Research Laboratory

Software Quality Research Laboratory University of Tennessee

University of Tennessee

Department of Computer Science

(2)

DANSE Objectives

Extensible open source framework

Modular interoperability

Reusable distributed components

Legacy code migration path

Broad scientific applicability

HPC compatibility

Broad user expertise range

(3)

Quality Challenges

Large, distributed development team

Wide range of software engineering

experience

Long term maintenance by SNS

Straightforward extensibility for power

users

Reliable, intuitive operation for novice

users

(4)

SQRL Tasks

Software Quality Engineering Workshops

for Each Science Subgroup

Documentation of SNS Interface Documentation of SNS Interface Requirements Requirements SNS Integration Plan SNS Integration Plan

Integration Test Planning

(5)

Essential Process Elements

1)

1) Requirements Capture_{Requirements Capture}

2)

2) Architecture Specification_{Architecture Specification}

3)

3) Behavior SpecificationBehavior Specification

4)

4) Code Generation_{Code Generation}

5)

5) Certification_{Certification}

6)

6) User Guide Publication_{User Guide Publication}

7)

7) Configuration Management_{Configuration Management}

8)

(6)

Primary Work Product

Relationships

(7)

Software Production is

Incremental

? ? ?

An efficient, repeatable process is necessary for expanding, changing requirements.

(8)

Incremental Development Strategy

Organize requirements into small

functional increments

For each increment

–

– Elaborate requirements to expand behavior Elaborate requirements to expand behavior and architecture specifications

and architecture specifications

–

– Code and integrate new functionalityCode and integrate new functionality –

– Certify newly integrated system conformance Certify newly integrated system conformance with the cumulative specification

with the cumulative specification

–

– Update User Guide if increment will be Update User Guide if increment will be released for beta or general use

released for beta or general use

–

(9)

Requirements

Use cases tagged for

traceability

.

Initial requirements assumed to be

incomplete, inconsistent, and possibly

incorrect

(10)

DANSE Architecture Specifications

Framework

Framework –– interfaces, services, and interfaces, services, and rules for well

rules for well--behaved componentsbehaved components Each Component

Each Component –– class diagrams and class diagrams and API design

API design

Each Application

Each Application –– target environment, target environment, participating components, and their

participating components, and their

relationships

(11)

Behavior Specification: The General Case

1)

1) Establish _Establishsystem boundary_{system boundary}._.

2)

2) Define human/software/hardware _{Define human/software/hardware}

interfaces

interfaces..

3)

3) Itemize Itemize stimulistimuli..

4)

4) Itemize _Itemizeresponses_responses._.

5)

5) Map Map every stimulus sequenceevery stimulus sequence to a to a response.

response.

The discovery and documentation of derived requirements is

a natural and desirable part of the specification process.

(12)

Map Code Specified in Behavior

Specification to Architecture

Stimulus Gathering Response Generation State Variable Implementation State Machine Implementation

(13)

Certification:

Operational Definition

Evidence that

Evidence that approved software approved software engineering practices

engineering practices were used to were used to produce the software

produce the software

Results of inspection and tests

Results of inspection and tests provide provide confidence that the as

confidence that the as--built software will built software will perform its intended function

(14)

DANSE Testing Strategy

Dedicated Test Leaders for Framework

and Each Science Team

Statistical Testing to Certify Each

Increment

Increment Release Based on Quantitative

Assessment of Test Results

(15)

Work Products Supporting Certification

Peer Review Findings and Resolution

–

– SpecificationsSpecifications –

– CodeCode –

– Test PlanTest Plan

Test Results and Resolution of Failures

Release Decision Meeting Minutes

(16)

User Guide

Content

–

– How to install and/or configure the systemHow to install and/or configure the system –

– How to initiate and control all functionsHow to initiate and control all functions –

– How to interpret outputHow to interpret output

Format

–

(17)

Configuration/Release Control

Key Practices

Document/Source control tool

(Subversion)

Centrally managed release tagging

Repeatable automated build procedure

Formal release procedure

Change tracking (Trac)

Audit trail

(18)

Configuration Items

Software Behavior Specification Software Behavior Specification

Software Architecture Specification Software Architecture Specification

Source Code Source Code

Software Test Plan Software Test Plan

Test Record Test Record

User Guide User Guide

(19)

Software Configuration

Management

(20)

Change Request Processing

Open Resolved Suspended Close Developer Action Successful Verification Verification Failed TL/CM Decision TL/CM Decision

(21)

Tickets Drive Defect Correction Tickets Pace Feature Additions

Move Tags for Defect Corrections

Roles of Trac and Subversion in the

Increment Development Sequence

Move Tags for In-scope Features and Fixes

Subversion Time Line

Trac Time Line

Verification Triage Begin Cert Test Increment Complete Feature Tickets Resolved/Closed Request Resolved Ticket Verification Milestone Tickets Closed/Deferred Verification Triage Defect Tickets Resolved/Closed Request Resolved Ticket Verification Milestone Tickets Closed/Deferred

(22)

Process Summary

Rigorous specification of code derived

from informal requirements

– – ArchitectureArchitecture – – BehaviorBehavior Certification Certification –

– Independent Work Product ReviewIndependent Work Product Review –

– Quantitative TestingQuantitative Testing

Configuration management

–

– Centralized build/release controlCentralized build/release control –