Genetic Improvement for Adaptive Software Engineering

(1)

Genetic Improvement for

Adaptive Software Engineering

Mark Harman

University College London

SEAMS2014

joint work with Yue Jia, Bill Langdon, Iman Moghadam,

Justyna Petke, Shin Yoo & Fan Wu

(2)

Genetic Improvement for Adaptive Software Engineering _{Mark Harman, UCL CREST}

SEAMS2014

20 mins walk

National

Gallery

Nelson’s

Column

Eros

RoyalCourts

of Justice

St. Paul’s

Tate Modern

Theatre

Globe

Covent

Garden

Market

Westminster

Abbey

House of Parliament

London Eye

British

Museum

Madame Tussaud’s

Sherlock Holmes

Museum

Marble Arch

National

History

Museum

(3)

SEAMS2014

COWs

CREST Open Workshop

Roughly one per month

!

Discussion based

(4)

SEAMS2014

COWs

CREST Open Workshop

!

Discussion based

(5)

SEAMS2014

COWs

CREST Open Workshop

!

Discussion based

(6)

SEAMS2014

(7)

SEAMS2014

(8)

SEAMS2014

What is SBSE

In SBSE we apply search techniques to search

large search spaces, guided by a fitness function

that captures properties of the acceptable

software artefacts we seek.

like google search?

like code search?

like breadth first search?

potentially

exhaustive

pick one at

random

sweet spot

(9)

SEAMS2014

What is SBSE

Search Based

Optimization

Software

Engineering

S

B

S

E

(10)

SEAMS2014

What is SBSE

In SBSE we apply

search techniques

to search

that captures properties of the acceptable

software artefacts we seek.

Genetic Algorithms

Hill Climbing

Simulated Annealing

Random

Tabu Search

Estimation of Distribution Algorithms

Particle Swarm Optimization

Ant Colonies

LP

Genetic Programming

(11)

SEAMS2014

What is SBSE

In SBSE we apply

search techniques

to search

that captures properties of the acceptable

software artefacts we seek.

Genetic Algorithms

Hill Climbing

Simulated Annealing

Random

Tabu Search

Estimation of Distribution Algorithms

Particle Swarm Optimization

Ant Colonies

LP

Genetic Programming

(12)

SEAMS2014

What is SBSE

let’s listen to software engineers ...

!

(13)

SEAMS2014

Software Engineers Say

Requirements:

We need to satisfy business and technical concerns

Management:

We need to reduce risk while maintaining completion time

Design:

We need increased cohesion and decreased coupling

Testing:

We need fewer tests that find more nasty bugs

(14)

SEAMS2014

Requirements:

Management:

Design:

Testing:

(15)

SEAMS2014

Requirements:

Management:

Design:

Testing:

(16)

SEAMS2014

Requirements:

Management:

Design:

Testing:

(17)

SEAMS2014

Requirements:

Management:

Design:

Testing:

(18)

SEAMS2014

Requirements:

Management:

Design:

Testing:

(19)

SEAMS2014

Requirements:

Management:

Design:

Testing:

Refactoring:

We need to optimise for all metrics M1,..., Mn

Requirements: We need to satisfy business and technical concerns

Management: We need to reduce risk while maintaining completion time

Design: We need increased cohesion and decreased coupling

Testing: We need fewer tests that find more nasty bugs

(20)

SEAMS2014

Requirements: We need to satisfy business and technical concerns

Management: We need to reduce risk while maintaining completion time

Design: We need increased cohesion and decreased coupling

Testing: We need fewer tests that find more nasty bugs

Refactoring: We need to optimise for all metrics M1,..., Mn

(21)

SEAMS2014

(22)

SEAMS2014

(23)

SEAMS2014

(24)

SEAMS2014

Author statistics

more than 1250 authors

more than 1150 papers

more than 390 institutions

more than 50 countries

!

(25)

SEAMS2014

(26)

SEAMS2014

(27)

SEAMS2014

27

Just some of the

many

SBSE

applications

Agent Oriented

Aspect Oriented

Assertion Generation

Bug Fixing

Component Oriented

Design

Effort Estimation

Heap Optimisation

Model Checking

Predictive Modelling

Probe distribution

Program Analysis

Program Comprehension

Program Transformation

Project Management

Protocol Optimisation

QoS

Refactoring

Regression Testing

Requirements

Reverse Engineering

SOA

Software Maintenance and Evolution

Test Generation

(28)

SEAMS2014

EPSRC

Grant

(29)

SEAMS2014

EPSRC

Grant

Stirling

Birmingham

York

UCL

(30)

SEAMS2014

EPSRC

Grant

Stirling

York

UCL

Birmingham

(31)

SEAMS2014

EPSRC

Grant

(32)

SEAMS2014

EPSRC

Grant

(33)

SEAMS2014

EPSRC

Grant

Stirling

Birmingham

York

UCL

2012-2018

PhD and post docs offered

(34)

SEAMS2014

(35)

SEAMS2014

Mark Harman, Phil McMinn, Jerffeson Teixeira de Souza and Shin Yoo.

Search Based Software Engineering: Techniques, Taxonomy, Tutorial.

Springer, 2012.

!

google: SBSE tutorial

!

Mark Harman, Afshin Mansouri and Yuanyuan Zhang.

Search Based Software Engineering: Trends, Techniques and Applications

ACM Computing Surveys.

45(1): Article 11, 2012.

!

google: SBSE survey

(36)

SEAMS2014

Search Based Software Engineering: Techniques, Taxonomy, Tutorial.

Springer, 2012.

!

Mark Harman, Afshin Mansouri and Yuanyuan Zhang.

45(1): Article 11, 2012.

!

google: SBSE survey

(37)

SEAMS2014

Springer, 2012.

!

ACM Computing Surveys.

45(1): Article 11, 2012.

!

google: SBSE survey

(38)

SEAMS2014

Springer, 2012.

!

45(1): Article 11, 2012.

!

google: SBSE survey

(39)

SEAMS2014

Springer, 2012.

!

ACM Computing Surveys.

45(1): Article 11, 2012.

!

google: SBSE survey

(40)

SEAMS2014

SB

SE

Testing

Management

Design

Requirements

Verification

Maintenance

Debugging

...

GA

GP

ES

SA

HC

ACO

...

(41)

SEAMS2014

SB

SE

Testing

Management

Design

Requirements

Verification

Maintenance

Debugging

...

Modularisation

GA

GP

ES

SA

HC

ACO

...

(42)

SEAMS2014

Search based modularisation

Mancoridis and Mitchell: IWPC 1998 and TSE 2006

Praditwong, Harman and Yao: TSE 2011

(43)

SEAMS2014

m

Praditwong, Harman and Yao: TSE 2011

(44)

SEAMS2014

Praditwong, Harman and Yao: TSE 2011

Barros: GECCO 2012

(45)

SEAMS2014

SB

SE

Testing

Management

Design

Requirements

Verification

Maintenance

Debugging

...

GA

GP

ES

SA

HC

ACO

...

(46)

SEAMS2014

SB

SE

Testing

Management

Design

Requirements

Verification

Maintenance

Debugging

...

Next Release

Problem

GA

GP

ES

SA

HC

ACO

...

(47)

SEAMS2014

Search based requirements

Bagnall, Rayward-Smith and Whittley: IST 2001

Zhang, Harman and Mansouri: GECCO 2007

Saliu and Ruhe: FSE 2007

(48)

SEAMS2014

Search based requirements

Bagnall, Rayward-Smith and Whittley: IST 2001

Zhang, Harman and Mansouri: GECCO 2007

Saliu and Ruhe: FSE 2007

(49)

SEAMS2014

SB

SE

Testing

Management

Design

Requirements

Verification

Maintenance

Debugging

...

GA

GP

ES

SA

HC

ACO

...

(50)

SEAMS2014

SB

SE

Testing

Management

Design

Requirements

Verification

Maintenance

Debugging

...

Search Based

Management

GA

GP

ES

SA

HC

ACO

...

(51)

SEAMS2014

Dolado: IST 2001

Chicano and Alba: MIC 2005

Ferrucci, Harman, Ren and Sarro: ICSE 2013

(52)

SEAMS2014

Dolado: IST 2001

Chicano and Alba: MIC 2005

Ferrucci, Harman, Ren and Sarro: ICSE 2013

Search based management

(53)

SEAMS2014

SB

SE

Testing

Management

Design

Requirements

Verification

Maintenance

Debugging

...

GA

GP

ES

SA

HC

ACO

...

(54)

SEAMS2014

SB

SE

Testing

Management

Design

Requirements

Verification

Maintenance

Debugging

...

Program Repair

GA

GP

ES

SA

HC

ACO

...

(55)

SEAMS2014

Search based repair

Arcuri and Yao: CEC 2008

Weimer, Nguyen, Le Goues and Forrest: ICSE 2009

Kim, Nam, Song and Kim: ICSE 2013

(56)

SEAMS2014

Search based repair

GP

Arcuri and Yao: CEC 2008

Weimer, Nguyen, Le Goues and Forrest: ICSE 2009

Kim, Nam, Song and Kim: ICSE 2013

(57)

SEAMS2014

SB

SE

Testing

Management

Design

Requirements

Verification

Maintenance

Debugging

...

GA

GP

ES

SA

HC

ACO

...

(58)

SEAMS2014

SB

SE

Testing

Management

Design

Requirements

Verification

Maintenance

Debugging

...

Test case

generation

GA

GP

ES

SA

HC

ACO

...

(59)

SEAMS2014

SB

SE

Testing

Management

Design

Requirements

Verification

Maintenance

Debugging

...

GA

GP

ES

SA

HC

ACO

...

(60)

SEAMS2014

SB

SE

Testing

Management

Design

Requirements

Verification

Maintenance

Debugging

...

Test suite

optimisation

GA

GP

ES

SA

HC

ACO

...

(61)

SEAMS2014

Test suite optimisation

Yoo and Harman: ISSTA 2007

Li, Harman and Hierons: TSE 2007

(62)

SEAMS2014

Test suite selection

(63)

SEAMS2014

Test suite selection

Yoo, Nilsson and Harman, FSE 2011

Speedup Speedup (x times) 0 1 2 3 4 5 6 NSGA2/JOMP1 NSGA2/JOMP2 NSGA2/JOMP4 NSGA2/GPU SPEA2/JOMP1 SPEA2/JOMP2 SPEA2/JOMP4 SPEA2/GPU TAEA/JOMP1 TAEA/JOMP2 TAEA/JOMP4 TAEA/GPU pr inttok ens − 1 pr inttok ens − 2 pr inttok ens − 3 pr inttok ens − 4 schedule − 1 schedule − 2 schedule − 3 schedule − 4 pr inttok ens schedule pr inttok ens2 schedule2 Speedup (x times) 0 5 10 15 20 25 30 NSGA2/JOMP1 NSGA2/JOMP2 NSGA2/JOMP4 NSGA2/GPU SPEA2/JOMP1 SPEA2/JOMP2 SPEA2/JOMP4 SPEA2/GPU TAEA/JOMP1 TAEA/JOMP2 TAEA/JOMP4 TAEA/GPU tcas totinf o fle x gzip sed space − 1 space − 2 space − 3 space − 4 replace bash haif a

Yoo, Harman and Ur, EMSE 2013

Find faults faster

Optimise faster using GPGPU

Mirarab, Akhlaghi and Tahvildari: TSE 2012

(64)

SEAMS2014

SB

SE

Testing

Management

Design

Requirements

Verification

Maintenance

Debugging

...

GA

GP

ES

SA

HC

ACO

...

(65)

SEAMS2014

SB

SE

Testing

Management

Design

Requirements

Verification

Maintenance

Debugging

...

optimising

model checking

GA

GP

ES

SA

HC

ACO

...

(66)

SEAMS2014

SB

SE

Testing

Management

Design

Requirements

Verification

Maintenance

Debugging

...

GA

GP

ES

SA

HC

ACO

...

(67)

SEAMS2014

SB

SE

Testing

Management

Design

Requirements

Verification

Maintenance

Debugging

...

Search based

Refactoring

GA

GP

ES

SA

HC

ACO

...

(68)

SEAMS2014

O'Keeffe and Ó Cinnéide: CSMR 2003

Harman and Tratt: GECCO 2007

Jensen and Cheng: GECCO 2010

(69)

SEAMS2014

O'Keeffe and Ó Cinnéide: CSMR 2003

Harman and Tratt: GECCO 2007

Jensen and Cheng: GECCO 2010

Search based refectoring

(70)

SEAMS2014

SB

SE

Testing

Management

Design

Requirements

Verification

Maintenance

Debugging

...

Program

Repair

GA

GP

ES

SA

HC

ACO

...

(71)

SEAMS2014

SB

SE

Testing

Management

Design

Requirements

Verification

Maintenance

Debugging

...

Program

Improvement

GA

GP

ES

SA

HC

ACO

...

(72)

SEAMS2014

GISMOE

t

Pareto program surface is automatically

constructed to support dialog with the

software designer concerning trade offs in

the solution space of programs

(73)

SEAMS2014

There is a paper to

(74)

(75)

SEAMS2014

(76)

SEAMS2014

Requirements

Functional

Requirements

Non-Functional

Requirements

(77)

SEAMS2014

Functional

Requirements

Non-Functional

Requirements

Memory

Execution Time

Battery

Size

Bandwidth

functionality of

the Program

(78)

SEAMS2014

(79)

SEAMS2014

(80)

SEAMS2014

(81)

SEAMS2014

(82)

SEAMS2014

(83)

SEAMS2014

(84)

SEAMS2014

Multiplicity

Multiple

Devices

Conflicting

Objectives

Multiple

Platforms

(85)

SEAMS2014

Why is the

(86)

SEAMS2014

Functional

Requirements

Non-Functional

Requirements

Which requirements must be human coded ?

humans have to

define these

a machine can

optimise these

(87)

SEAMS2014

Functional

Requirements

Non-Functional

Requirements

Which requirements are essential to human ?

humans have to

define these

a machine can

optimise these

(88)

SEAMS2014

(89)

SEAMS2014

Pareto Front

each circle is a

program found

by a machine

(90)

SEAMS2014

Pareto Front

different non

functional

properties have

different pareto

program fronts

(91)

SEAMS2014

(92)

SEAMS2014

Why can’t functional properties

be optimisation objectives ?

(93)

(94)

SEAMS2014

(95)

SEAMS2014

Optimisation

2.5 times faster but

failed 1 test case?

(96)

(97)

SEAMS2014

Optimisation

double the battery life

but failed 2 test cases?

(98)

SEAMS2014

Genetic Programming for

Software Transplantation

(99)

SEAMS2014

Genetic Programming for

Software Transplantation

covered in more detail in the

WCRE 2013

(100)

SEAMS2014

(101)

SEAMS2014

GP for Transplants

(102)

SEAMS2014

GP for Transplants

Localise

A

bstract

Target

I

nterface

Insert

Validate

R

epeat

---Doner

---Host

Featur

e

(103)

SEAMS2014

GP for Transplants

L

ocalise

A

bstract

Target

I

nterface

Insert

Validate

R

epeat

---Doner

---Host

if(ptr!=NULL)

foo(*ptr);

(104)

SEAMS2014

GP for Transplants

L

ocalise

A

bstract

Target

I

nterface

Insert

Validate

R

epeat

---Doner

---Host

if(ptr!=NULL)

(105)

SEAMS2014

GP for Transplants

L

ocalise

A

bstract

T

arget

I

nterface

Insert

Validate

R

epeat

---Doner

---Host

SBSE

(106)

SEAMS2014

GP for Transplants

L

ocalise

A

bstract

T

arget

I

nterface

Insert

Validate

R

epeat

---Doner

---Host

Target

Interface

(107)

SEAMS2014

GP for Transplants

L

ocalise

A

bstract

T

arget

I

nterface

I

nsert

Validate

R

epeat

---Doner

---Host

Target

Interface

if( !=NULL)

( );

2

1

(108)

SEAMS2014

GP for Transplants

L

ocalise

A

bstract

T

arget

I

nterface

I

nsert

Validate

R

epeat

---Doner

---Host

Target

Interface

if(

host_ptr

!=NULL)

(109)

SEAMS2014

GP for Transplants

L

ocalise

A

bstract

T

arget

I

nterface

I

nsert

V

alidate

R

epeat

---Doner

---Host

++++

1 new feature tests

2 regression tests

3 quality tests

(110)

SEAMS2014

GP for Transplants

L

ocalise

A

bstract

T

arget

I

nterface

I

nsert

V

alidate

R

epeat

---Doner

---Host

++++

(111)

SEAMS2014

(112)

SEAMS2014

Offline

Genetic Improvement

(113)

SEAMS2014

Bowtie2

GP

Programs

Bowtie2

Improved

Non-functional

property Test

harness

Fitness

Test

data

Sensitivity

Analysis

Genetic Improvement of Programs

W. B. Langdon and M. Harman

Optimising Existing Software with Genetic Programming. TEC 2014 (TR available)

70 times faster

30+ interventions

HC clean up: 7

(114)

SEAMS2014

GP

Programs

MiniSat

Improved

Non-functional

property Test

harness

Fitness

Test

data

Sensitivity

Analysis

Genetic Improvement of Programs

Justyna Petke, Mark Harman, William B. Langdon and Westley Weimer

Using Genetic Improvement & Code Transplants to Specialise a C++ program

to a Problem Class (EuroGP’14)

Multi-doner transplant

Specialized for CIT

17% faster

MiniSat

v1

v2

vn

(115)

SEAMS2014

SB

SE

GA

GP

EA

SA

HC

ACO

...

Testing

Management

Design

Requirements

Verification

Maintenance

Debugging

...

Program

Improvement

(116)

SEAMS2014

SB

SE

GA

GP

EA

SA

HC

ACO

...

Testing

Management

Design

Requirements

Verification

Maintenance

Debugging

...

Program

Improvement

(117)

SEAMS2014

Genetic Improvement

Babel Pidgin: Grow new functionality

Mark Harman, Yue Jia and William B Langdon

Babel Pidgin: SBSE can grow and graft entirely new functionality into a real

world system (SSBSE’14 Challenge)

Growing

Grafting

GP

Bi-language

translation

Fitness

Test

data

hello

…

안녕하세요

Olá

(118)

SEAMS2014

(119)

SEAMS2014

Online

Genetic Improvement

(120)

SEAMS2014

Online phase

Offline phase

User specify

operation

(121)

SEAMS2014

Online phase

Offline phase

User specify

operation

characteristics

Environmental and

usage profile

Learning

Data Collection

Phase 1

(122)

SEAMS2014

Online phase

Offline phase

User specify

operation

characteristics

Environmental and

usage profile

Learning

Data Collection

Phase 1

Program

Tuneable Parameters

(123)

SEAMS2014

Online phase

Offline phase

User specify

operation

characteristics

Environmental and

usage profile

Learning

Data Collection

Phase 1

Program

Tuneable Parameters

Exposing Implicit Parameters

(124)

SEAMS2014

Online phase

Offline phase

User specify

operation

characteristics

Environmental and

usage profile

Learning

Data Collection

Phase 1

Program

Optimisation Tuning

SBSE

(125)

SEAMS2014

Online phase

Offline phase

User specify

operation

characteristics

Environmental and

usage profile

Learning

Data Collection

Phase 1

Program

Optimisation Tuning

SBSE

Patch generation

(126)

SEAMS2014

Online phase

Offline phase

User specify

operation

characteristics

Environmental and

usage profile

Learning

Data Collection

Phase 1

Program

Optimisation Tuning

SBSE

Patch generation

Phase 1- n

Data Collection

Deploy patches

(127)

SEAMS2014

Online phase

Offline phase

User specify

operation

characteristics

Environmental and

usage profile

Learning

Data Collection

Phase 1

Program

Optimisation Tuning

SBSE

Patch generation

Phase 1- n

Data Collection

Deploy patches

(128)

SEAMS2014

(129)

SEAMS2014

Example:

Wikipedia Mobile

Overnight

Charging

Mode

Daily

Operating

Mode

(130)

SEAMS2014

Example:

Wikipedia Mobile

Daily

Operating

(131)

SEAMS2014

Example:

Wikipedia Mobile

Daily

Operating

(132)

SEAMS2014

Example:

Wikipedia Mobile

Daily

Operating

(133)

SEAMS2014

Example:

Wikipedia Mobile

Overnight

Charging

(134)

SEAMS2014

Example:

Wikipedia Mobile

Overnight

Charging

Mode

(135)

SEAMS2014

Online Genetic Improvement

… could provide an

SBSE for Self Adaptive Systems

!

Genetic Improvement

+

Grow and Graft Patches

+

Expose parameters and Autotune

+

Learning and deployment: Catch, Dream, Optimise