Artificial Intelligence and Testing. Kishore Durg AccentureTechnology June 2016

(1)

1

Artificial Intelligence and Testing

Kishore Durg

AccentureTechnology

June 2016

(2)

Intelligent

automation:

The essential

co-worker for

the digital age

2

(3)

What is intelligent automation?

Foundation

Robotic

Integrated

analytics

Artificial

Intelligence

Project level, ad-hoc

automation for areas like

scripts

Standardization of

scripts and reduced

manual efforts

Use of analytical tools

to predict and

recommend

Self-learning systems

and self-evolving tools

A constellation of automation technologies that, when used together, spell success.

Building blocks for intelligent automation success

Judgement-driven

Transactional

Increased “intelligence”

More proven

(4)

State of the market: Automation is awesome!

GROWING EXECUTIVE AWARENESS

85% of execs use automation to improve their business

process.

1 POWERFUL RESULTS

43% of execs achieved cost savings of greater than 15%

from intelligent process automation in the past 2 years.

2 INCREASED INVESTMENT AND

INNOVATION

AI startups in the US alone have increased 20-fold in the

past four years.

3 1, 2

_{Accenture Technology Vision 2016 Survey.}

3 _{”Artificial Intelligence Startups See 302% Funding}

(5)

Doing things differently

Doing different things

Intelligent Automation: Exponential business

potential

(6)

When is the right time to invest?

Where do I start?

Can I trust it to run critical business processes?

Can I trust it to give me reliable insight?

But…business leaders have questions about

intelligent automation:

(7)

3 lessons learned to guide

your intelligent automation

path

1. Think beyond costs

2. Get the basics right

3. Embrace open innovation

(8)

A Few examples of A.I. across all of Accenture businesses

ATA for

PharmacoVigilance

ATA for Credit

_Services

ATA for Sentiment

Analysis

Unstructured Cyber

Threat Detection

–

This drug doesn’t harm the body at its best

(9)

A.I. Platform for Automation

(10)

A.I. in Application Management

Log Analysis,

and Ticket

Prevention

Automated

resolution of

Incidents

Knowledge

Engineering

support for

Problem

resolution

• Big Data based operational log analysis

• Use NLP and Machine Learning to discover similar,

co-related incidents

• Create knowledge graph on application domain

and data.

NELL

Prevention and closure

of common / repetitive

errors at source.

Faster Resolution

through cognitive

Ticket Solving Agent

Guided problem

resolution / support

through

Knowledge

Search Agent

Automation Themes

Cognitive Mechanism

Benefit

(11)

A.I. in Testing Services

Accenture

Technology Labs

(ATL)

A.I. and Digital

Testing

US Research

Institute

Open Innovation

Niche Cognitive

Vendor

(12)

Sense

Comprehend

Act

and

Recommend

Learn

• Computer Vision

• ASR

• NLU (Text)

• Question Answering

• Deep Learning

• Case Based

Reasoning

• Knowledge Graph

• Dialog Strategy

• BOTS

• Process Recommendation

• _NLG

• _{Active learning}

/ reinforcement

learning

Degree

of

Complexity

Degree

of

Complexity

Degree

of

Complexity

Cognitive Capability

Stack

Software Engineering

_{Application Areas}

Robotic Humanoid Expert Assistant –

(Say hello to R.H.E.A.!!!)

Enablers

(ACE+ OS)

ACE

Testin

g

Savant

Application

Management

Social

Impact

(13)

A.I. in Testing Services:

Intelligent Test Automation

(14)

Accelerating Test Automation Platform (ATAP)

Challenges :

• Author the automation test scripts using tools.

• The amount of manual effort it takes to maintain an automation script.

WebObjects in ATAP

TestScripts in ATAP

ATAP allows a tester to author the automation in simple English like sentences.

These sentences are then automatically converted to JAVA code using Selenium.

(15)

A.I. in Testing Services :

Addressing vendor transitions and Knowledge transfer - Accenture Test

Coverage Analyzer

Benefits:

The tool showed over 70%

accuracy and reduced the

manual effort needed to 20% of

original.

Test Scripts from Client

Generate a

Application Functionality

Diagram (AFD) from all the

test scripts for an application

(16)

A.I. in Testing Services:

Pretotype to Test Case Generator Asset

Benefits to Accenture



Automatically generates test cases

and executable test scripts from

visual requirements.



Improve productivity.

Benefits to Clients

• Superior quality software

• Faster time to market

• Customer satisfaction

(17)

A.I. in Testing Services:

Accelerated Accenture Test Repository Reuse (Accelerated ATR Reuse)

Benefits:



The tool would significantly

reduce the amount of

manual effort required to

identify and reuse testing

artifacts from ATR for

generic testing

engagements.



Accelerated ATR Reuse

PoC has been piloted with

Banking CoE. The tool

showed around 80%

accuracy and could reduce

up to 70% of the manual

effort during reuse process.

New Requirement or Change Request as

Search Query

Search ATR to identify all testing artefacts meeting

search criterion and rank them in order of relevance

Accenture Test

Repository

(18)

Cognitive Computing Solution

Cost

Saving

Faster time

to market

End user

N umerou s D efect s ide nti fied Grouping of the defects De-duplication of defects Assign appropriate tester / developer to the

defect

Test Planning Phase

Test cases &

test scripts

Risk based testing for regression

High priority test cases

Application modules with more defects

System Testing Phase

System builds co-relation between production, test cases & defects

R eg ressi on test plann ing R eg ressi on testi n g

Risk based Testing -Execution limited to co-related test cases

Input

Solution

Output

Benefits

(19)

A.I. in Testing Services :

(20)

Applications

Offline channels

Online channels

WEB MOBILE SMART TV

SOCIAL EMAIL KIOSK POS TELE MARKET DELIVERY STORE BROCHURES

Omni channel testing

20 A.I. in Testing Services :

Digital Testing and Intelligent Execution analytics

Customer

Employee

Partner

Automation Libraries

Framework APIs

Accenture Digital standard test

_{practices [reusable].}

Compatibility

Testing

Interoperability Testing

Functional

Testing Security Testing

Performance Testing Conformance Testing

Web front

end

Consolidated & shared

master data

Cognitive

Analytics

Integration

Data

Management

OCTAVE

Test across all

online and offline

channels

Full control on

retail

infrastructure

through internal

hosting

Test on all key

back end

services Analytics,

integration and

data management

Dedicated test

infrastructure

easily adaptable

to dynamic

business case

changes

End to end

automation

capability

Retail platform

Database

Services

Platforms

X-Industry Enablers

Mobile App Test Lab

Digital Crowd Test Platform

Security Test Platform

Omni channel Test & Validation Engine

Products

Device Test Analytics Digital Optimization Web Browser Test Lab

Connected World Testing

(21)

21

The Touchless test Automation platform

Log Mining based System Usage Modeling & Failure

Modeling Engine

CAST

Program Profiler

Accenture Test

Repository

ConformiQ

Model Generator

Analytics Guided Test Cockpit

API & Services

testing

Java Platform Testing TDM SVT Performance Testing

Usage Model & Failure Model

inputs

Machine Learning based Defect Discovery & Localization Engine

DEFECT

LOCALIZATION

ENGINE

Multi-layer Automation Defect patterns and bug profiler,

Dig-deeper-or-search-elsewhere decision rules Test Scripts

Multi-layer Test

Automation

SEARCH

Defect Prediction Heat Maps Defect Management System

DISCOVER &

LOCALIZE

Production logs and execution traces (masked)

Production tickets Development logs and execution traces defects

Code and code change metrics Authorship metrics Module structure graphs Program Analysis Data