Continuous???? Copyright 2015 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

(1)

Continuous ????

(2)

Application Delivery is Accelerating

What customers expect

• Functionality now

• Accessed through multiple devices

2010

4 per app

2015

36 per app

2020

120 per app

Surge in # of releases per app per year

30x

Application

Releases

(3)

Slow

time to

market

Long application release cycles

Poor

user

experience

_{Low application quality}

Poor

predictability

Lack of end to end visibility

High

costs

Poor resource utilization,

rework cycles

(4)

Evolution in Testing

Demanding a “shift-left” approach

…make it critical to “shift-left”

Agile and DevOps

challenge testing processes

• Lack of proven agile testing approach and testing expertise within agile teams

• Applying test automation to agile projects

• Lack of right tools to build reusable test sets

Current challenges…

Improve development efficiency - Reduce the test burden - Accelerate application delivery

Open Source and

Continuous Integration Tools

• Agile & Dev/Ops prefer open source • “Good Enough” test automation • Open integration in dev ecosystem is

very important

Mobile Devices change

definition of quality

• Mobile is everywhere • Mobile development is

“Agile development on steroids” • User experience is EVERYTHING

(5)

Traditional Testing

Focus is placed at the end of the software development cycle

5

Unit Testing

API / Service Testing

User Interface Testing

Traditional test automation is

mostly based on the front end…

…but results in a focus

at the end of the

software development

(6)

© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 6₆ _{© Copyright 2015 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.}

Unit Automation

_{Middle Tier Automation}

Service or

User Interface Testing

Agile’s focus on continuous testing and

continuous delivery causes testing to

“shift left”, occurring much earlier in

the development lifecycle

Agile Testing

Agile and DevOps moves focus to the left

…because of it, developers

become more influential

even in quality assurance

(7)

(8)

Development

Test

Build

Source Control

Commit

CI Process

Testing

Report

Domain

(9)

Development

Test

Build

Source Control

Commit

CI Process

Testing

Report

Tools

Landscape

HP Testing

Tools

(10)

Planning _DevelopmentApp App Testing _decisionRelease App release

High velocity innovation, multiple constraints

Rapidly increasing WIP

Lack of effective customer insight and high latency drives “kitchen sink” requirements

Waiting time for build and test environments drives “desk-side” builds

Poor confidence in test data fosters “release aversion” driving more WIP Isolated build and

integration processes Deployed App Manual Testing increases latency or drives limited test coverage

Manual and error prone app deployments Error prone manual hand-offs and processes

High # defects

Locally optimized teams, measures and tools drives escalating WIP, lack of end-to-end visibility and trust

One way flow

Poor

user

experience

“patch in production” leads to snowflake systems InfoSec & compliance engaged late driving vulnerabilities & re-work

Business

demands

(11)

Optional subtitle

19

Removing the constraints…

#TakingFlight – Technology Briefing Series

October 14, 2015 HP Confidential

(12)

Increase

automation

Automate everything you can

• Accelerate routine tasks

• Ensure repeatability

• Reduce errors

(13)

Reduce

latency

Continuous integration and testing

• Continuous builds

• API driven testing

• Testing virtualization

Continuous delivery and deployment

Continuous security from dev thru prod

(14)

Continuous assessment:

• Monitor and measure everything

• Constant feedback loops

• Prioritize business, security and dev planning

• Improve and iterate

Visibility across teams

Increase

visibility

High velocity innovation

(15)

High # defects

Locally optimized teams, measures and tools drives escalating WIP, lack of end-to-end visibility and trust

One way flow

Poor

user

experience

High velocity innovation,

Business

demands

Continuous assessment

continuous delivery

Continuous assessment

Continuous

Operations

Continuous

Delivery &

Deployment

Continuous

Integration

& Testing

(16)

High # defects

Locally optimized teams, measures and tools drives escalating WIP, lack of end-to-end visibility and trust

One way flow

Poor

user

experience

Continuous integration and testing

Business

demands

Continuous assessment

Continuous

operations

Continuous

delivery &

deployment

Continuous

integration

& testing

Automate test

infrastructure

deployment

Automated

vulnerability and

static analysis

Automate test

execution from

CI systems

Network and

service

virtualization

Intelligent testing

Business

demands

Continuous assessment

Continuous

Operations

Continuous

Delivery &

Deployment

Continuous

Integration

& Testing

(17)

Continuous integration

and testing

Developers

IDE + HP ALI Dev

Implement requirements defined in HP ALM

HP DevInspect’s Security Assistant scans as code is written within IDE

SCM System

Source code is stored in SCM Continuous Integration

Build System

Req/Tasks/Defects Bu ilds Check-in Check-out ALI Reports

HP ALM

and

HP AGM

HP ALM and AGM:

Automate test

_{infrastructure}

Automate test

execution

deployment

Intelligent testing

Network and

service virtualization

(18)

Continuous integration

and testing

Manage Test resources (ALM)

Integration with Jenkins: Automate

test scheduling, execute tests and

view results as part of the build plan

Summarize test results in ALM/AGM

Trigger lab deployment from CI

including mobile devices

Virtualize services and infrastructure

Automated testing:

Automate test

_{infrastructure}

Automate test

execution

deployment

Intelligent testing

Network and

service virtualization

(19)

Continuous integration

and testing

Scripted functional testing

Purpose built for test-driven

development (TDD), Agile Testing,

natural CI integration and Dev/Ops

UFT Object Recognition

Visual Studio / Eclipse (C# or Java)

Nunit/Junit

Small footprint

HP LeanFT:

Automate test

_{infrastructure}

Automate test

execution

deployment

Intelligent testing

Network and

service virtualization

(20)

Automate test

execution

Automate test

infrastructure

deployment

Intelligent testing

Network and

service virtualization

Continuous integration

and testing

Add LoadRunner APIs to unit tests in

Visual Studio or Eclipse

StormRunner Load: Cloud based load

testing from 1 to over 1m Vus

Verify output within the IDE

Execute unit tests using the

LoadRunner Engine

Create LoadRunner scenario from the

IDE

(21)

Automate test

execution

Automate test

infrastructure

deployment

Intelligent testing

Network and

service virtualization

Continuous integration

and testing

Integrated to DevOps Tool Chain

Open APIs, Jenkins, Release and Lab Automation

Simulate services, and system

components as needed

Quickly ‘learn’ and configure virtual

services

.

Eliminate delays or availability of

services or components (Dev, Test)

(22)

Automate test

execution

Automate test

infrastructure

deployment

Intelligent testing

Network and

service virtualization

Continuous integration

and testing

Integrated to DevOps Tool Chain

Open APIs, Vendor Neutral, Automated Results.

Recreate network conditions in Dev,

Test and Ops environments

Discover and capture network

conditions from real-world

Analysis with code level automated

optimization recommendations

(23)

Automate test

execution

Automate test

infrastructure

deployment

Intelligent testing

Continuous integration and testing

HP Fortify

Application Security Testing

Hackers & Actual Attacks

Static Analysis – Fortify SCA

Source Code

Mgt. System Static Analysis Via Build Integration

Dynamic Analysis – WebInspect

Dynamic Testing in QA or Production

Application Protection –

HP Application Defender

Real-time Protection of Running Application

Vulnerability Management

Normalization (Scoring, Guidance) Correlation

(Static, Dynamic, Runtime)

Threat Intelligence Rules Management

Vulnerabilit y Database

Remediation

IDE Plug-ins (Eclipse, Visual Studio, etc.)

Developers (onshore or offshore) Correlate Target Vulnerabilities with Common Guidance and Scoring Defects, Metrics and KPIs Used to Measure Risk

Application

Lifecycle

Development, Project and Management Stakeholders

Software Security Center Fortify on Demand

(24)

High # defects

Locally optimized teams, measures and tools drives escalating WIP, lack of end-to-end visibility and trust

One way flow

Poor

user

experience

Continuous delivery and deployment

Infrastructure

as code

Declarative based

topology models

management

Pipeline

Business

demands

Continuous assessment

Continuous

operations

Continuous

delivery &

deployment

Continuous

integration

& testing

Business

demands

Continuous assessment

Continuous

Operations

Continuous

Delivery &

Deployment

Continuous

Integration

& Testing

Fail forward

vulnerability

protection

(25)

Pipeline

management

Declarative

based topology

models

Infrastructure as

code

Continuous delivery

and deployment

HP Codar:

Integrated to DevOps Tool Chain

Open APIs & Jenkins integ.

Increased visibility

Pipeline Management

Model Based Deployments

Infrastructure as Code

Quick time to automate

Topology App Designs

Fail forward

With protection

(26)

Pipeline

management

Declarative

based topology

models

Infrastructure as

code

Continuous delivery

and deployment

Service Virtualization

Integrated to DevOps Tool Chain

Open APIs, Jenkins, Release and Lab Automation

Simulate services, and system

components as needed

Quickly ‘learn’ and configure virtual

services

.

Eliminate delays or availability of

services or components (Dev, Test)

Fail forward

With protection

(27)

Pipeline

management

Declarative

based topology

models

Infrastructure as

code

Continuous delivery

and deployment

Service Virtualization

Protect vulnerabilities in production

Stop attacks from inside the

application

Identify threats in real-time

Pinpoint vulnerabilities for efficient

remediation

Fail forward

With protection

(28)

High # defects

Locally optimized teams, measures and tools drives escalating WIP, lack of end-to-end visibility and trust