Visualization in Multiobjective Optimization

Visualization in Multiobjective Optimization

Bogdan Filipič and Tea Tušar

Tutorial at GECCO ’20

Department of Intelligent Systems

Jožef Stefan Institute

Ljubljana, Slovenia

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author. GECCO ’20 Companion, July 8–12, 2020, Cancún, Mexico

©2020 Copyright is held by the owner/author(s). ACM ISBN 978-1-4503-7127-8/20/07. https://doi.org/10.1145/3377929.3389867

Instructors

Bogdan Filipič is a senior researcher and head of

Computational Intelligence Group at the Department of

Intelligent Systems of the Jožef Stefan Institute, Ljubljana,

Slovenia, and associate professor of Computer Science at the

Jovzef Stefan International Postgraduate School. His research

interests are in artificial intelligence, stochastic optimization,

and evolutionary computation.

Tea Tušar is a research fellow at the Department of Intelligent

Systems of the Jožef Stefan Institute in Ljubljana, Slovenia.

Her research interests include evolutionary algorithms for

singleobjective and multiobjective optimization with

emphasis on visualizing and benchmarking their results and

applying them to real-world problems.

2

Final version

These slides as well as all the approximation sets used in this tutorial

are available at

http://dis.ijs.si/tea/research.htm

3

Contents

Introduction

A taxonomy of visualization methods

Visualizing approximation sets

Visualizing single approximation sets

Visualizing repeated approximation sets

Visualizing multiobjective landscapes

Summary

References

Introduction

Multiobjective optimization problem

Minimize

f : X

_{→ F}

f : (x

1

, . . . , x

n

)

7→ (f1

(x

1

, . . . , x

n

), . . . , f

m

(x

1

, . . . , x

n

))

• X is an n-dimensional

decision space

(or

search space

)

• F

⊆ R

m

_{is an m-dimensional}

_{objective space}

_(m

_{≥ 2)}

Conflicting objectives

→ a set of optimal solutions

•

Pareto set

in the decision space

•

Pareto front

in the objective space

5

Introduction

Visualization in multiobjective optimization

•

Solution sets

in the decision or objective space (or both)

•

Multiobjective landscapes

—objective values in the decision

space

Visualization of solution sets useful for:

• Analysis of solutions and solution sets

• Decision support in interactive optimization

• Analysis of algorithm performance

Visualization of multiobjective landscapes useful for:

• Revealing problem properties and difficulties

• Identifying basins of attraction of local optima

Introduction

Visualizing solution sets in the decision space

• Problem-specific

• If X

⊆ R

m

_{, any method for visualizing multidimensional}

solutions can be used

• Not the focus of this tutorial

Visualizing solution sets in the objective space

• Interested in sets of mutually nondominated solutions called

approximation sets

• Different from ordinary multidimensional solution sets

Visualization of multiobjective landscapes

Introduction

Challenges of visualizing solution sets in the objective space

• High dimension and large number of solutions

• Limitations of computing and displaying technologies

• Cognitive limitations

8

Introduction

Visualization can be hard even in 2-D

Stochastic optimization algorithms

• Single run

→ single approximation set

• Multiple runs

→ multiple approximation sets

0

0.2

0.4

0.6

0.8

1

1.2

0 0.2 0.4 0.6 0.8 1 1.2

Single run

0

0.2

0.4

0.6

0.8

1

1.2

0 0.2 0.4 0.6 0.8 1 1.2

Three runs

0

0.2

0.4

0.6

0.8

1

1.2

0 0.2 0.4 0.6 0.8 1 1.2

Ten runs

Run 1

Run 2

Run 3

Run 4

Run 5

Run 6

Run 7

Run 8

Run 10

Run 9

The

Empirical Attainment Function (EAF)

[23] or the

Average Runtime

Attainment Function (aRTA)

[10] can be used in such cases

9

Introduction

This tutorial does not cover

• Visualization of a few solutions for decision making purposes

(see [41])

• Visualization of solution sets in the decision space

• General multidimensional visualization methods not previously

used on approximation sets

This tutorial covers

• Visualization of solution sets in the objective space

• Single approximation sets [2]

• Repeated approximation sets [3, 10]

• Visualization of multiobjective landscapes

10

A taxonomy of visualization

methods

Visualization in Multiobjective Optimization Pareto front approximations Single approximation sets Individual solutions (visualizing solutions independently from each other) Showing original objective values Showing transformed objective values Set properties (visualizing solutions dependently from each other) Showing individual solution properties Not optimization based Optimization based Showing aggregated properties Repeated approximation sets Showing performance at a time Showing performance over time Multiobjective landscapes Showing original objective values Showing transformed objective values Showing optima networks

11

Visualizing approximation sets

Visualizing approximation sets

Visualizing single approximation sets

Methodology

Evaluating and comparing visualization methods

• No established methodology for evaluating or comparing

visualization methods

• Propose

benchmark approximation sets

(analog to benchmark

problems in multiobjective optimization)

• Visualize the sets using different methods

• Observe which set properties are distinguishable after

visualization

• Only applicable to methods showing individual solutions or

individual solution properties

Benchmark approximation sets

Three different sets that can be instantiated in any dimension

•

Spherical

with a

clustered distribution

of solutions (more at the

corners and less at the center)

•

Linear

with a

uniform distribution

of solutions

•

Knee-shaped

with an

even distribution

of solutions

Size of each set

• 2-D: 50 solutions

• 3-D: 500 solutions

• 4-D: 500 solutions

13

Benchmark approximation sets

Spherical

Linear

Knee-shaped

0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 f2 f1 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 f2 f1 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 f2 f1 0.2 0.4 0.6 0.8 1 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 f1 f₂ f3 0.2 0.4 0.6 0.8 1 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 f1 f₂ f3 0.2 0.4 0.6 0.8 1 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 f1 f₂ f3

14

Benchmark approximation sets

An additional set with

redundant

objectives

• Adapted from [21]

• 12 objectives

• Can be instantiated for any number of 10n solutions (here 100)

0

20

40

60

80

100

f

₁

f

₂

f

₃

f

₄

f

₅

f

₆

f

₇

f

₈

f

₉

f

₁₀

f

₁₁

f

₁₂

15

Desired properties of visualization methods

Demonstration on the 4-D spherical, linear and knee-shaped sets

• Preservation of the

• Dominance relation between solutions

• Front shape

• Objective range

• Distribution of solutions

• Robustness

• Handling of large sets

• Simultaneous visualization of multiple sets

• Scalability in number of objectives

• Simplicity

Demonstration on the 12-D approximation set

• Showing relations between objectives

Visualizing single approximation sets

approximations

Pareto front

Single

approximation sets

Individual solutions

(visualizing solutions independently from each other)

Showing original

objective values

– Scatter Plot Matrix – Bubble Chart – Parallel Coordinates [28] – Radar Chart – Chord Diagram [32] – Heat Map [45] – Interactive Decision Map [37] – Nasseh Method [37]

Showing transformed

objective values

– Radial Coordinate Visualization [26] – 3-D Radial Coordinate Visualization [27] – Tetrahedron Coordinates Model [7] – Polar Plots [25] – Hyper-Radial Visualization [12] – Level Diagrams [8] – Prosection Plots [2] – PaletteViz [47]

Set properties

(visualizing solutions dependently from each other)

Showing individual

solution properties

Not optimization

based

– Distance and Distribution Charts [5] – Pareto Shells [52] – Hyper-Space Diagonal

Counting [4] – Tree Map [54] – Trade-off Region Map [44]

Optimization

based

– Principal Component Analysis [57] – Sammon Mapping [50] – Neuroscale [15] – Multidimensional Scaling [53] – Isomap [34] – Seriated Heatmap [53] – Two-Stage Mapping [33] – Distance-Based and Dominance-Based Mappings [16]

Showing aggregated

properties

– Self-Organizing Map [43] – Aggregation Tree [21] – MoGram [49] – Trend Mining [6]

Repeated

approximation sets

Showing

performance

at a time

– Line Plot [20] – Heat Map [36] – Visualization of Facets [24] – Grid-Based Sampling [30] – Slicing [3] – Maximum Intensity Projection [3] – Direct Volume Rendering [3] – Circular Indicator Plot [55]

Showing

performance

over time

– Grid-Based Sampling [10]

Multiobjective

landscapes

Showing original

objective values

– Level Sets – Line Walks [11, 51]

Showing transformed

objective values

– Dominance Rank [19] – Local Dominance [17] – Cumulative Gradient [29]

Showing optima

networks

– Pareto local optimal solutions network [35] – Pareto local optima

network [18] – Dominance-neutral

optima network [18]

17

Scatter plot matrix

Most often

• Scatter plot in a 2-D space

• Matrix of all possible combinations of objectives

• m objectives

→

m(m

−1)

2

different combinations

Alternatively

• Scatter plot in a 3-D space

• m objectives

→

m(m

−1)(m−2)

6

different combinations

18

Scatter plot matrix

Spherical

Linear

Knee-shaped

0 0.5 1 0 0.5 1 f2 f1 0 0.5 1 0 0.5 1 f3 f1 0 0.5 1 0 0.5 1 f4 f1 0 0.5 1 0 0.5 1 f3 f2 0 0.5 1 0 0.5 1 f4 f2 0 0.5 1 0 0.5 1 f4 f3 0 0.5 1 0 0.5 1 f2 f1 0 0.5 1 0 0.5 1 f3 f1 0 0.5 1 0 0.5 1 f4 f1 0 0.5 1 0 0.5 1 f3 f2 0 0.5 1 0 0.5 1 f4 f2 0 0.5 1 0 0.5 1 f4 f3 0 0.5 1 0 0.5 1 f2 f1 0 0.5 1 0 0.5 1 f3 f1 0 0.5 1 0 0.5 1 f4 f1 0 0.5 1 0 0.5 1 f3 f2 0 0.5 1 0 0.5 1 f4 f2 0 0.5 1 0 0.5 1 f4 f3

Scatter plot matrix

Spherical

Linear

Knee-shaped

0.5 1 0 0.5 1 0 0.5 1 f2 f3 f 4 0.5 1 0 0.5 1 0 0.5 1 f1 f 3 f 4 0.5 1 0 0.5 1 0 0.5 1 f1 f2 f4 0.5 1 0 0.5 1 0 0.5 1 f1 f 2 f3 0.5 1 0 0.5 1 0 0.5 1 f2 f3 f 4 0.5 1 0 0.5 1 0 0.5 1 f1 f3 f 4 0.5 1 0 0.5 1 0 0.5 1 f1 f2 f4 0.5 1 0 0.5 1 0 0.5 1 f1 f2 f3 0.5 1 0 0.5 1 0 0.5 1 f2 f3 f 4 0.5 1 0 0.5 1 0 0.5 1 f 1 f3 f 4 0.5 1 0 0.5 1 0 0.5 1 f1 f2 f 4 0.5 1 0 0.5 1 0 0.5 1 f 1 f2 f3 Preservation of the

Robustness Handling of_{large sets} Simultaneous_{visualization} Scalability Simplicity dominance

relation

front shape objective range

distribution of solutions

Parallel coordinates

• m objectives

→ m parallel axes

• Solution represented as a polyline with vertices on the axes

• Position of each vertex corresponds to that objective value

• No loss of information

0.0

0.2

0.4

0.6

0.8

1.0

f

1

f

2

f

3

f

4

21

Parallel coordinates

Spherical

Linear

Knee-shaped

0 0.2 0.4 0.6 0.8 1 f₁ f₂ f₃ f₄ 0 0.2 0.4 0.6 0.8 1 f₁ f₂ f₃ f₄ 0 0.2 0.4 0.6 0.8 1 f₁ f₂ f₃ f₄ Preservation of the

Robustness Handling of_{large sets} Simultaneous_{visualization} Scalability Simplicity dominance

relation

front shape objective range distribution of solutions ≈ × X ≈ X × × X X

22

Parallel coordinates

Original

0 20 40 60 80 100 f1 f2 f3 f4 f5 f6 f7 f8 f9 f10 f11 f12

Ordered

0 20 40 60 80 100 f11 f3 f10 f2 f4 f6 f8 f12 f1 f9 f5 f7

23

Interactive decision maps

The

Edgeworth-Pareto hull (EPH)

of an approximation set A contains

all points in the objective space that are weakly dominated by any

solution in A.

Interactive decision maps

• Visualize the surface of the EPH, not the actual approximation

set

• Plot a number of axis-aligned sampling surfaces of the EPH

• Color used to denote third objective

• Fixed value of the forth objective

Interactive decision maps

f

4

= 0.2

Spherical

Linear

Knee-shaped

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

f

2

f

1

Spherical

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

f

2

f

1

Linear

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

f

2

f

1

Knee-shaped

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

0.2

0.4

0.6

0.8

1

f

3

0.2

0.4

0.6

0.8

1

0.2

0.4

0.6

0.8

1

f

3

0.2

0.4

0.6

0.8

1

0.2

0.4

0.6

0.8

1

f

3

0.2

0.4

0.6

0.8

1

Preservation of the

Robustness Handling of_{large sets} Simultaneous_{visualization} Scalability Simplicity dominance

relation

front shape objective range

distribution of solutions

× ≈ X ≈ X X × × ≈

25

Visualizing single approximation sets

approximations

Pareto front

Single

approximation sets

Individual solutions

(visualizing solutions independently from each other)

Showing original

objective values

– Scatter Plot Matrix – Bubble Chart – Parallel Coordinates [28] – Radar Chart – Chord Diagram [32] – Heat Map [45] – Interactive Decision Map [37] – Nasseh Method [37]

Showing transformed

objective values

– Radial Coordinate Visualization [26] – 3-D Radial Coordinate Visualization [27] – Tetrahedron Coordinates Model [7] – Polar Plots [25] – Hyper-Radial Visualization [12] – Level Diagrams [8] – Prosection Plots [2] – PaletteViz [47]

Set properties

(visualizing solutions dependently from each other)

Showing individual

solution properties

Not optimization

based

– Distance and Distribution Charts [5] – Pareto Shells [52] – Hyper-Space Diagonal

Counting [4] – Tree Map [54] – Trade-off Region Map [44]

Optimization

based

– Principal Component Analysis [57] – Sammon Mapping [50] – Neuroscale [15] – Multidimensional Scaling [53] – Isomap [34] – Seriated Heatmap [53] – Two-Stage Mapping [33] – Distance-Based and Dominance-Based Mappings [16]

Showing aggregated

properties

– Self-Organizing Map [43] – Aggregation Tree [21] – MoGram [49] – Trend Mining [6]

Repeated

approximation sets

Showing

performance

at a time

– Line Plot [20] – Heat Map [36] – Visualization of Facets [24] – Grid-Based Sampling [30] – Slicing [3] – Maximum Intensity Projection [3] – Direct Volume Rendering [3] – Circular Indicator Plot [55]

Showing

performance

over time

– Grid-Based Sampling [10]

Multiobjective

landscapes

Showing original

objective values

– Level Sets – Line Walks [11, 51]

Showing transformed

objective values

– Dominance Rank [19] – Local Dominance [17] – Cumulative Gradient [29]

Showing optima

networks

– Pareto local optimal solutions network [35] – Pareto local optima

network [18] – Dominance-neutral

optima network [18]

26

Radial coordinate visualization

Also called

RadViz

• Inspired from physics

• Objectives treated as anchors,

equally spaced around the

circumference of a unit circle

• Solutions attached to anchors with

‘springs’

• Spring stiffness proportional to the

objective value

• Solution placed where the spring

forces are in equilibrium

f

1

f

2

f

3

f

4

Radial coordinate visualization

Spherical

Linear

Knee-shaped

f1 f2 f3 f4 f1 f2 f3 f4 f1 f2 f3 f4 Preservation of the

Robustness Handling of_{large sets} Simultaneous_{visualization} Scalability Simplicity dominance

relation

front shape objective range

distribution of solutions

Level diagrams

• m objectives

→ m diagrams

• Plot solutions with objective f

i

on the x axis and distance to the

ideal point on the y axis

29

Level diagrams

Spherical

Linear

Knee-shaped

0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1

Distance to the ideal point

f1 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1

Distance to the ideal point

f2 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1

Distance to the ideal point

f3 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1

Distance to the ideal point

f4 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1

Distance to the ideal point

f1 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1

Distance to the ideal point

f2 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1

Distance to the ideal point

f3 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1

Distance to the ideal point

f4 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1

Distance to the ideal point

f1 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1

Distance to the ideal point

f 2 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1

Distance to the ideal point

f3 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1

Distance to the ideal point

f

4

Preservation of the

Robustness Handling of_{large sets} Simultaneous_{visualization} Scalability Simplicity dominance

relation

front shape objective range distribution of solutions × ≈ X × X ≈ X X X

30

Prosections

• Visualize only part of the objective space

• Dimensionality reduction by projection of solutions in a section

• Need to choose prosection plane, angle and section width

0.2

0.4

0.6

0.8

1

0.2

0.4

0.6

0.8

1

0

0.2

0.4

0.6

0.8

1

f

1

f

₂

f

₃

0

0.2

0.4

0.6

0.8

1

0

0.2

0.4

0.6

0.8

1

f

3

f

1

f

2

Before prosection

After prosection

31

Prosections

Spherical

Linear

Knee-shaped

0.2

0.4

0.6

0.8

1

0.2

0.4

0.6

0.8

1

0

0.2

0.4

0.6

0.8

1

f

₁

f

₂

f

₃

f

4

0.2

0.4

0.6

0.8

1

0.2

0.4

0.6

0.8

1

0

0.2

0.4

0.6

0.8

1

f

₁

f

₂

f

₃

f

4

0.2

0.4

0.6

0.8

1

0.2

0.4

0.6

0.8

1

0

0.2

0.4

0.6

0.8

1

f

₁

f

₂

f

₃

f

4 Preservation of the

Robustness Handling of_{large sets} Simultaneous_{visualization} Scalability Simplicity dominance

relation

front shape objective range

distribution of solutions

X X ≈ X X X X × ≈

Prosections

Spherical and Linear

0.2

0.4

0.6

0.8

1

0.2

0.4

0.6

0.8

1

0

0.2

0.4

0.6

0.8

1

f

₁

f

₂

f

₃

f

₄

Preservation of the

Robustness Handling of_{large sets} Simultaneous_{visualization} Scalability Simplicity dominance

relation

front shape objective range

distribution of solutions

X X ≈ X X X X × ≈

33

Visualizing single approximation sets

approximationsPareto front

Single approximation sets Individual solutions (visualizing solutions independently from each other) Showing original objective values

– Scatter Plot Matrix – Bubble Chart – Parallel Coordinates [28] – Radar Chart – Chord Diagram [32] – Heat Map [45] – Interactive Decision Map [37] – Nasseh Method [37] Showing transformed objective values – Radial Coordinate Visualization [26] – 3-D Radial Coordinate Visualization [27] – Tetrahedron Coordinates Model [7] – Polar Plots [25] – Hyper-Radial Visualization [12] – Level Diagrams [8] – Prosection Plots [2] – PaletteViz [47] Set properties (visualizing solutions dependently from each other) Showing individual solution properties Not optimization based – Distance and Distribution

Charts [5] – Pareto Shells [52] – Hyper-Space Diagonal

Counting [4] – Tree Map [54] – Trade-off Region Map [44]

Optimization based – Principal Component Analysis [57] – Sammon Mapping [50] – Neuroscale [15] – Multidimensional Scaling [53] – Isomap [34] – Seriated Heatmap [53] – Two-Stage Mapping [33] – Distance-Based and Dominance-Based Mappings [16] Showing aggregated properties – Self-Organizing Map [43] – Aggregation Tree [21] – MoGram [49] – Trend Mining [6] Repeated approximation sets Showing performance at a time – Line Plot [20] – Heat Map [36] – Visualization of Facets [24] – Grid-Based Sampling [30] – Slicing [3] – Maximum Intensity Projection [3] – Direct Volume Rendering [3] – Circular Indicator Plot [55]

Showing performance over time – Grid-Based Sampling [10] Multiobjective landscapes Showing original objective values – Level Sets – Line Walks [11, 51] Showing transformed objective values – Dominance Rank [19] – Local Dominance [17] – Cumulative Gradient [29] Showing optima networks

– Pareto local optimal solutions network [35] – Pareto local optima

network [18] – Dominance-neutral

optima network [18]

34

Hyper-space diagonal counting

• Inspired by Cantor’s proof that shows

|N| = |N

2

_{| = |N}

3

_{| . . .}

(1, 1)

(2, 1)

(1, 2)

(3, 1)

(2, 2)

(1, 3)

(4, 1)

(3, 2)

(2, 3)

. . .

• Discretize each objective (choose a number of bins)

• In the 4-D case

• Enumerate the bins for objectives f

1

and f

2

• Enumerate the bins for objectives f

3

and f

4

• Plot the number of solutions in each pair of bins

Hyper-space diagonal counting

Spherical

Linear

Knee-shaped

0 20 40 60 80 0 20 40 60 80 0 5 10 15 20 f 1f2 bins f3f4 bins Number of solutions 0 20 40 60 80 0 20 40 60 80 0 5 10 15 20 f 1f2 bins f3f4 bins Number of solutions 0 20 40 60 80 0 20 40 60 80 0 5 10 15 20 f 1f2 bins f3f4 bins Number of solutions Preservation of the

Robustness Handling of_{large sets} Simultaneous_{visualization} Scalability Simplicity dominance

relation

front shape objective range

distribution of solutions

Visualizing single approximation sets

Visualization in Multiobjective Optimization

Pareto front approximations Single approximation sets Individual solutions (visualizing solutions independently from each other) Showing original objective values

– Scatter Plot Matrix – Bubble Chart – Parallel Coordinates [28] – Radar Chart – Chord Diagram [32] – Heat Map [45] – Interactive Decision Map [37] – Nasseh Method [37] Showing transformed objective values – Radial Coordinate Visualization [26] – 3-D Radial Coordinate Visualization [27] – Tetrahedron Coordinates Model [7] – Polar Plots [25] – Hyper-Radial Visualization [12] – Level Diagrams [8] – Prosection Plots [2] – PaletteViz [47] Set properties (visualizing solutions dependently from each other) Showing individual solution properties Not optimization based

– Distance and Distribution Charts [5] – Pareto Shells [52] – Hyper-Space Diagonal

Counting [4] – Tree Map [54] – Trade-off Region Map [44]

Optimization based – Principal Component Analysis [57] – Sammon Mapping [50] – Neuroscale [15] – Multidimensional Scaling [53] – Isomap [34] – Seriated Heatmap [53] – Two-Stage Mapping [33] – Distance-Based and Dominance-Based Mappings [16] Showing aggregated properties – Self-Organizing Map [43] – Aggregation Tree [21] – MoGram [49] – Trend Mining [6] Repeated approximation sets Showing performance at a time – Line Plot [20] – Heat Map [36] – Visualization of Facets [24] – Grid-Based Sampling [30] – Slicing [3] – Maximum Intensity Projection [3] – Direct Volume Rendering [3] – Circular Indicator Plot [55]

Showing performance over time – Grid-Based Sampling [10] Multiobjective landscapes Showing original objective values – Level Sets – Line Walks [11, 51] Showing transformed objective values – Dominance Rank [19] – Local Dominance [17] – Cumulative Gradient [29] Showing optima networks

– Pareto local optimal solutions network [35] – Pareto local optima

network [18] – Dominance-neutral optima network [18]

37

Sammon mapping

• A non-linear mapping

• Aims to preserve distances between solutions

• d

∗

ij

distance between solutions x

i

and x

j

in the objective space

• d

ij

distance between solutions x

i

and x

j

in the visualized space

• Stress function to be minimized

S =

∑

i

∑

j>i

(d

∗

_ij

− d

_ij

)

2

• Minimization by gradient descent or other (iterative) methods

38

Sammon mapping

Spherical

Linear

Knee-shaped

-0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 -1.2 -1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 -1.2 -1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 -1.2 -1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 Preservation of the

Robustness Handling of_{large sets} Simultaneous_{visualization} Scalability Simplicity dominance

relation

front shape objective range

distribution of solutions

× × × X ≈ ≈ X X ×

39

Distance- and dominance-based mappings

Both mappings

• Use nondominated sorting to split solutions to fronts

• Project solutions onto the circumference of circles (with circle

radius proportional to front number)

Distance-based mapping

• Tries to preserve closeness

of solutions

• Two solutions are very close

if their relations to other

solutions are mostly equal

Dominance-based mapping

• Aims at preserving dominance

relations among solutions

• All x

≺ y can be shown

correctly

• Tries to minimize cases where

x

_{⊀ y is not shown correctly}

40

Distance-based mapping

Dominance-based mapping

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2

0

0.5

1

1.5

2

Linear

Spherical

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

Linear

Spherical

Preservation of the

Robustness Handling of_{large sets} Simultaneous_{visualization} Scalability Simplicity dominance

relation

front shape objective range distribution of solutions × / X × × × / ≈ ≈ × X X ×

41

Pareto front

approximations

Single

approximation sets

Individual solutions

(visualizing solutions independently from each other)

Showing original

objective values

– Scatter Plot Matrix – Bubble Chart – Parallel Coordinates [28] – Radar Chart – Chord Diagram [32] – Heat Map [45] – Interactive Decision Map [37] – Nasseh Method [37]

Showing transformed

objective values

– Radial Coordinate Visualization [26] – 3-D Radial Coordinate Visualization [27] – Tetrahedron Coordinates Model [7] – Polar Plots [25] – Hyper-Radial Visualization [12] – Level Diagrams [8] – Prosection Plots [2] – PaletteViz [47]

Set properties

(visualizing solutions dependently from each other)

Showing individual

solution properties

Not optimization

based

– Distance and Distribution Charts [5] – Pareto Shells [52] – Hyper-Space Diagonal

Counting [4] – Tree Map [54] – Trade-off Region Map [44]

Optimization

based

– Principal Component Analysis [57] – Sammon Mapping [50] – Neuroscale [15] – Multidimensional Scaling [53] – Isomap [34] – Seriated Heatmap [53] – Two-Stage Mapping [33] – Distance-Based and Dominance-Based Mappings [16]

Showing aggregated

properties

– Self-Organizing Map [43] – Aggregation Tree [21] – MoGram [49] – Trend Mining [6]

Repeated

approximation sets

Showing

performance

at a time

– Line Plot [20] – Heat Map [36] – Visualization of Facets [24] – Grid-Based Sampling [30] – Slicing [3] – Maximum Intensity Projection [3] – Direct Volume Rendering [3] – Circular Indicator Plot [55]

Showing

performance

over time

– Grid-Based Sampling [10]

Multiobjective

landscapes

Showing original

objective values

– Level Sets – Line Walks [11, 51]

Showing transformed

objective values

– Dominance Rank [19] – Local Dominance [17] – Cumulative Gradient [29]

Showing optima

networks

– Pareto local optimal solutions network [35] – Pareto local optima

network [18] – Dominance-neutral

optima network [18]

42

Self-organizing maps

• Self-organizing maps (SOMs) are neural networks

• Nearby solutions are mapped to nearby neurons in the SOM

• A SOM can be visualized using the unified distance matrix

• Distance between adjacent neurons is denoted with color

• Similar neurons

→ light color

• Different neurons (cluster boundaries)

→ dark color

Self-organizing maps

Aggregation trees

• Binary trees that show relationships between objectives

• Iterative clustering of objectives based on their harmony

• Computation of different types of conflict

• Percentages quantify the conflict between objectives

• Colors used to show type of conflict

• global conflict (black)

• local conflict on ’good’ values (red)

• local conflict on ’bad’ values (blue)

• Can be used to sort objectives in other representations (parallel

coordinates, radial charts, heat maps)

45

Aggregation trees

Spherical

Linear

Knee-shaped

f3+ f1+ f4+ f2− 91.51% f3+ f1− 74.52% f3 f1 f4+ f2− 73.99% f4 f2 f4+ f2+ f3+ f1− 97.92% f4+ f2− 76.86% f4 f2 f3+ f1− 75.91% f3 f1 f2+ f1+ f3+ f4− 95.76% f2+ f1+ f3− 79.79% f2+ f1− 67.52% f2 f1 f3 f4

46

f

8

+ f

6

+ f

4

+ f

2

+ f

12

+ f

10

+ f

9

+ f

1

+ f

5

+ f

7

+ f

11

+ f

3

− 100%

f

8

+ f

6

+ f

4

+ f

2

+ f

12

+ f

10

+ f

9

+ f

1

+ f

5

+ f

7

− 26%

f

8

+ f

6

+ f

4

+ f

2

+ f

12

+ f

10

+ f

9

+ f

1

+ f

5

− 24%

f

8

+ f

6

+ f

4

+ f

2

+ f

12

+ f

10

+ f

9

+ f

1

− 3.2%

f

8

+ f

6

+ f

4

+ f

2

+ f

12

+ f

10

+ f

9

− 2%

f

8

+ f

6

+ f

4

+ f

2

+ f

12

+ f

10

− 0%

f

8

+ f

6

+ f

4

+ f

2

− 0%

f

8

+ f

6

− 0%

f

8

f

6

f

4

+ f

2

− 0%

f

4

f

2

f

12

+ f

10

− 0%

f

12

f

10

f

9

f

1

f

5

f

7

f

11

+ f

3

− 24%

f

11

f

3

Aggregation trees

0 20 40 60 80 100 f8 f6 f4 f2 f12 f10 f9 f1 f5 f7 f11 f3

47

Visualizing approximation sets

Visualizing repeated approximation sets

Pareto front approximations Single approximation sets Individual solutions (visualizing solutions independently from each other) Showing original objective values

– Scatter Plot Matrix – Bubble Chart – Parallel Coordinates [28] – Radar Chart – Chord Diagram [32] – Heat Map [45] – Interactive Decision Map [37] – Nasseh Method [37] Showing transformed objective values – Radial Coordinate Visualization [26] – 3-D Radial Coordinate Visualization [27] – Tetrahedron Coordinates Model [7] – Polar Plots [25] – Hyper-Radial Visualization [12] – Level Diagrams [8] – Prosection Plots [2] – PaletteViz [47] Set properties (visualizing solutions dependently from each other) Showing individual solution properties Not optimization based

– Distance and Distribution Charts [5] – Pareto Shells [52] – Hyper-Space Diagonal

Counting [4] – Tree Map [54] – Trade-off Region Map [44]

Optimization based – Principal Component Analysis [57] – Sammon Mapping [50] – Neuroscale [15] – Multidimensional Scaling [53] – Isomap [34] – Seriated Heatmap [53] – Two-Stage Mapping [33] – Distance-Based and Dominance-Based Mappings [16] Showing aggregated properties – Self-Organizing Map [43] – Aggregation Tree [21] – MoGram [49] – Trend Mining [6] Repeated approximation sets Showing performance at a time – Line Plot [20] – Heat Map [36] – Visualization of Facets [24] – Grid-Based Sampling [30] – Slicing [3] – Maximum Intensity Projection [3] – Direct Volume Rendering [3] – Circular Indicator Plot [55]

Showing performance over time – Grid-Based Sampling [10] Multiobjective landscapes Showing original objective values – Level Sets – Line Walks [11, 51] Showing transformed objective values – Dominance Rank [19] – Local Dominance [17] – Cumulative Gradient [29] Showing optima networks

– Pareto local optimal solutions network [35] – Pareto local optima

network [18] – Dominance-neutral

optima network [18]

Showing performance at a time with the

Empirical Attainment

Function (EAF)

[23]

48

Empirical attainment function

Goal-attainment

• Approximation set A

• A point in the objective space z is

attained

by A when z is

weakly dominated by at least one solution from A

0

0.2

0.4

0.6

0.8

1

1.2

0

0.2

0.4

0.6

0.8

1

1.2

49

Empirical attainment function

EAF values [23]

• Algorithm

A, approximation sets A1

, A

2

, . . . , A

r

• EAF of z is the frequency of attaining z by A

1

, A

2

, . . . , A

r

• Summary (or k%-) attainment surfaces

0

0.2

0.4

0.6

0.8

1

1.2

0

0.2

0.4

0.6

0.8

1

1.2

1st run

2nd run

3rd run

0

0.2

0.4

0.6

0.8

1

1.2

0

0.2

0.4

0.6

0.8

1

1.2

best

median

worst

0 1/3 2/3 1

• Visualization with line plots and heat maps

Empirical attainment function

Differences in EAF values [36]

• Algorithm

A, approximation sets A1

, A

2

, . . . , A

r

• Algorithm

B, approximation sets B1

, B

2

, . . . , B

r

• Visualize differences between EAF values

0

0.2

0.4

0.6

0.8

1

1.2

0

0.2

0.4

0.6

0.8

1

1.2

1st run of A

2nd run of A

1st run of B

2nd run of B

0

0.2

0.4

0.6

0.8

1

1.2

0

0.2

0.4

0.6

0.8

1

1.2

25%-att. surf.

50%-att. surf.

75%-att. surf.

100%-att. surf.

-1 -1/2 0 1/2 1

Visualization of 3-D EAF

Need to compute and visualize a large number (over 10 000) of

cuboids

Exact case

• EAF values:

Slicing [3]

, Visualization of facets [13, 24]

• EAF differences:

Slicing

, Maximum intensity projection [56, 3]

Approximated case

• EAF values:

Grid-based sampling [30]

, Slicing, Direct volume

rendering [14, 3]

• EAF differences: Slicing, Maximum intensity projection, Direct

volume rendering

52

Benchmark approximation sets

Two groups of spherical approximation sets

• 5

spherical

approximation sets with a

clustered distribution

of

solutions (different radii, 100 solutions in each)

• 5

spherical

approximation sets with a

uniform distribution

of

solutions (different radii, 100 solutions in each)

Clustered spherical

Uniform spherical

0 0.2 0.4 0.6 0.8 1 1.2 0 0.2 0.4 0.6 0.8 1 1.2 0.2 0.4 0.6 0.8 1 1.2 f₁ f2 f₃ 0 0.2 0.4 0.6 0.8 1 1.2 0 0.2 0.4 0.6 0.8 1 1.2 0.2 0.4 0.6 0.8 1 1.2 f₁ f2 f₃

53

Exact 3-D EAF values and differences

Slicing

• Visualize cuboids intersecting the slicing plane

• Need to choose coordinate and angle

r

1

z

z

′

o

o

′

f

1

f

2

f

3

ϕ

54

Exact 3-D EAF values and differences

Slicing

Clustered

Uniform

Difference

φ = 5

◦

0 0.3 0.6 0.9 1.2 0 0.3 0.6 0.9 1.2 f3 f1, f2 0 0.3 0.6 0.9 1.2 0 0.3 0.6 0.9 1.2 f3 f1, f2 0 0.3 0.6 0.9 1.2 0 0.3 0.6 0.9 1.2 f3 f1, f2

Clustered

Uniform

Difference

φ = 45

◦

0 0.3 0.6 0.9 1.2 0 0.3 0.6 0.9 1.2 f3 f1, f2 0 0.3 0.6 0.9 1.2 0 0.3 0.6 0.9 1.2 f3 f1, f2 0 0.3 0.6 0.9 1.2 0 0.3 0.6 0.9 1.2 f3 f1, f2

55

Approximated attainment surfaces

Grid-based sampling

Repeat for all f

i

f

j

, i < j

(i.e. f

1

f

2

, f

1

f

3

and f

2

f

3

):

• Construct a k

× k grid on the plane f

i

f

j

• Compute intersections between the attainment surface and the

axis-aligned lines on the grid

Clustered

Uniform

0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 f1 f2 f3 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 f1 f2 f3

Median attainment surfaces

56

Visualizing repeated approximation sets

Pareto front approximations Single approximation sets Individual solutions (visualizing solutions independently from each other) Showing original objective values

– Scatter Plot Matrix – Bubble Chart – Parallel Coordinates [28] – Radar Chart – Chord Diagram [32] – Heat Map [45] – Interactive Decision Map [37] – Nasseh Method [37] Showing transformed objective values – Radial Coordinate Visualization [26] – 3-D Radial Coordinate Visualization [27] – Tetrahedron Coordinates Model [7] – Polar Plots [25] – Hyper-Radial Visualization [12] – Level Diagrams [8] – Prosection Plots [2] – PaletteViz [47] Set properties (visualizing solutions dependently from each other) Showing individual solution properties Not optimization based

– Distance and Distribution Charts [5] – Pareto Shells [52] – Hyper-Space Diagonal

Counting [4] – Tree Map [54] – Trade-off Region Map [44]

Optimization based – Principal Component Analysis [57] – Sammon Mapping [50] – Neuroscale [15] – Multidimensional Scaling [53] – Isomap [34] – Seriated Heatmap [53] – Two-Stage Mapping [33] – Distance-Based and Dominance-Based Mappings [16] Showing aggregated properties – Self-Organizing Map [43] – Aggregation Tree [21] – MoGram [49] – Trend Mining [6] Repeated approximation sets Showing performance at a time – Line Plot [20] – Heat Map [36] – Visualization of Facets [24] – Grid-Based Sampling [30] – Slicing [3] – Maximum Intensity Projection [3] – Direct Volume Rendering [3] – Circular Indicator Plot [55]

Showing performance over time – Grid-Based Sampling [10] Multiobjective landscapes Showing original objective values – Level Sets – Line Walks [11, 51] Showing transformed objective values – Dominance Rank [19] – Local Dominance [17] – Cumulative Gradient [29] Showing optima networks

– Pareto local optimal solutions network [35] – Pareto local optima

network [18] – Dominance-neutral

optima network [18]

Showing performance over time with the

Average Runtime

Attainment Function (aRTA)

[10]

57

Average Runtime Attainment Function

aRTA value

• Algorithm

A run r times

• All solutions that are nondominated at creation are recorded

• aRTA(z) is the average number of evaluations needed to attain z

aRTA ratio

• Algorithms

A and B

• Visualize ratio between aRTA(z) values for

A and B

•

Benchmark approximation sets

Two groups of sets mimicking convergence to a spherical front

• 5 sets mimicking

logarithmic convergence

to a

spherical

front

with a

clustered distribution

(100 solutions each)

• 5 sets mimicking

linear convergence

to a

spherical

front with a

uniform distribution

(100 solutions each)

Clustered with

Uniform with

logarithmic convergence

linear convergence

0 0.5 1 1.5 2 0 0.5 1 1.5 2 f2 0 0.5 1 1.5 2 0 0.5 1 1.5 2 f2

Average Runtime Attainment Function

Grid-based sampling

Clustered with

Uniform with

Ratio

logarithmic convergence

linear convergence

0.0 0.5 1.0 1.5 2.0 f1 0.0 0.5 1.0 1.5 2.0 f2 100 101 102 0.0 0.5 1.0 1.5 2.0 f1 0.0 0.5 1.0 1.5 2.0 f2 100 101 102 0.0 0.5 1.0 1.5 2.0 f1 0.0 0.5 1.0 1.5 2.0 f2 −6 −4 −2 0 2 4 6

60

Visualizing multiobjective

landscapes

Visualizing problem landscapes

• Single objective: visualize objective values in the decision space

• Multiple objectives: ?

61

Benchmark problems

The bbob-biobj test suite [11]

• Each bi-objective function constructed as the combination of

two single-objective bbob functions

• Problems scalable in the number of decision variables

• Known single-objective optima, but not the Pareto set (or front)

• Included in the COCO platform

(https://github.com/numbbo/coco)

• Double sphere problem (F

1

= (f

1

, f

1

)

in 2-D and 5-D, instance 1)

• Sphere-Gallagher problem (F

10

= (f

1

, f

21

)

in 2-D and 5-D,

instance 1)

• Double Gallagher problem (F

55

= (f

21

, f

21

)

in 2-D and 5-D,

instance 1)

*Gallagher = Gallagher’s Gaussian 101-me Peaks Function

63

Visualization in Multiobjective Optimization

Pareto front

approximations

Single

approximation sets

Individual solutions

(visualizing solutions independently from each other)

Showing original

objective values

– Scatter Plot Matrix – Bubble Chart – Parallel Coordinates [28] – Radar Chart – Chord Diagram [32] – Heat Map [45] – Interactive Decision Map [37] – Nasseh Method [37]

Showing transformed

objective values

– Radial Coordinate Visualization [26] – 3-D Radial Coordinate Visualization [27] – Tetrahedron Coordinates Model [7] – Polar Plots [25] – Hyper-Radial Visualization [12] – Level Diagrams [8] – Prosection Plots [2] – PaletteViz [47]

Set properties

(visualizing solutions dependently from each other)

Showing individual

solution properties

Not optimization

based

– Distance and Distribution Charts [5] – Pareto Shells [52] – Hyper-Space Diagonal

Counting [4] – Tree Map [54] – Trade-off Region Map [44]

Optimization

based

– Principal Component Analysis [57] – Sammon Mapping [50] – Neuroscale [15] – Multidimensional Scaling [53] – Isomap [34] – Seriated Heatmap [53] – Two-Stage Mapping [33] – Distance-Based and Dominance-Based Mappings [16]

Showing aggregated

properties

– Self-Organizing Map [43] – Aggregation Tree [21] – MoGram [49] – Trend Mining [6]

Repeated

approximation sets

Showing

performance

at a time

– Line Plot [20] – Heat Map [36] – Visualization of Facets [24] – Grid-Based Sampling [30] – Slicing [3] – Maximum Intensity Projection [3] – Direct Volume Rendering [3] – Circular Indicator Plot [55]

Showing

performance

over time

– Grid-Based Sampling [10]

Multiobjective

landscapes

Showing original

objective values

– Level Sets – Line Walks [11, 51]

Showing transformed

objective values

– Dominance Rank [19] – Local Dominance [17] – Cumulative Gradient [29]

Showing optima

networks

– Pareto local optimal solutions network [35] – Pareto local optima

network [18] – Dominance-neutral

optima network [18]

64

Level sets

• Curves connecting points with the same value

•

Orange = first objective

, blue = second objective

• Demonstration on the 2-D benchmark problems

Double sphere problem

Sphere-Gallagher problem

Double Gallagher problem

Line walks

• Equidistant sampling of the decision space along a line

• The line does not have to be parallel to an axis

• Not constrained by the decision space dimension

• Two display options

• Show resulting values for each objective separately

•

Show resulting values in the objective space

Line walks

Double sphere problem in 5-D

Decision space

Objective space

67

Line walks

Sphere-Gallagher problem in 5-D

Decision space

Objective space

68

Line walks

Double Gallagher problem in 5-D

Decision space

Objective space

69

Visualizing multiobjective landscapes

Visualization in Multiobjective Optimization

Pareto front

approximations

Single

approximation sets

Individual solutions

(visualizing solutions independently from each other)

Showing original

objective values

– Scatter Plot Matrix – Bubble Chart – Parallel Coordinates [28] – Radar Chart – Chord Diagram [32] – Heat Map [45] – Interactive Decision Map [37] – Nasseh Method [37]

Showing transformed

objective values

– Radial Coordinate Visualization [26] – 3-D Radial Coordinate Visualization [27] – Tetrahedron Coordinates Model [7] – Polar Plots [25] – Hyper-Radial Visualization [12] – Level Diagrams [8] – Prosection Plots [2] – PaletteViz [47]

Set properties

(visualizing solutions dependently from each other)

Showing individual

solution properties

Not optimization

based

– Distance and Distribution Charts [5] – Pareto Shells [52] – Hyper-Space Diagonal

Counting [4] – Tree Map [54] – Trade-off Region Map [44]

Optimization

based

– Principal Component Analysis [57] – Sammon Mapping [50] – Neuroscale [15] – Multidimensional Scaling [53] – Isomap [34] – Seriated Heatmap [53] – Two-Stage Mapping [33] – Distance-Based and Dominance-Based Mappings [16]

Showing aggregated

properties

– Self-Organizing Map [43] – Aggregation Tree [21] – MoGram [49] – Trend Mining [6]

Repeated

approximation sets

Showing

performance

at a time

– Line Plot [20] – Heat Map [36] – Visualization of Facets [24] – Grid-Based Sampling [30] – Slicing [3] – Maximum Intensity Projection [3] – Direct Volume Rendering [3] – Circular Indicator Plot [55]

Showing

performance

over time

– Grid-Based Sampling [10]

Multiobjective

landscapes

Showing original

objective values

– Level Sets – Line Walks [11, 51]

Showing transformed

objective values

– Dominance Rank [19] – Local Dominance [17] – Cumulative Gradient [29]

Showing optima

networks

– Pareto local optimal solutions network [35] – Pareto local optima

network [18] – Dominance-neutral

optima network [18]

Showing transformed objective values

• Decision space approximated with a grid of points

• Show value using color (contours or the third dimension)

• Suitable only for 2-D decision spaces

• Discretized decision space (1000

× 1000 grid)

• Rank = number of grid points that dominate the current point

• All nondominted points have a rank of zero

• Visualize normalized ranks in logarithmic scale

71

Double sphere problem

Sphere-Gallagher problem

Double Gallagher problem

72

Visualizing local dominance

• Discretized decision space (1000

× 1000 grid)

• Moore neighborhood = eight surrounding points

• Compute three different kinds of regions

Green Locally dominance-neutral regions

• Points that are mutually nondominated with all

their neighbors

• Not equal to local Pareto sets

Pink Basins of attraction

White Boundary regions

• Can take a long time to compute

Visualizing local dominance

Visualizing cumulative gradients

• Discretized decision space (1000

× 1000 grid)

• Compute the bi-objective gradient for all grid points

v =

v

1

||v1||

+

v

2

||v2||

• From a grid point, follow the path in the direction of the

bi-objective gradient

• Sum all bi-objective gradient values along the path

• Visualize cumulative gradients in logarithmic scale

75

Visualizing cumulative gradients

Double sphere problem

Sphere-Gallagher problem

Double Gallagher problem

76

Global vs. local information

Sphere-Gallagher problem

Dominance rank

Local dominance

Cumulative gradient

77

Visualizing multiobjective landscapes

How to handle such visualization when n > 2?

Level sets, dominance ranks, local dominance and cumulative

gradients

• Require cuts through the decision space (cf. slicing)

• Challenging to compute and interpret these methods in n-D

Line walks

• A useful alternative for high-dimensional decision spaces

• The presented information is very limited

Visualization in Multiobjective Optimization

Pareto front

approximations

Single

approximation sets

Individual solutions

(visualizing solutions independently from each other)

Showing original

objective values

– Scatter Plot Matrix – Bubble Chart – Parallel Coordinates [28] – Radar Chart – Chord Diagram [32] – Heat Map [45] – Interactive Decision Map [37] – Nasseh Method [37]

Showing transformed

objective values

– Radial Coordinate Visualization [26] – 3-D Radial Coordinate Visualization [27] – Tetrahedron Coordinates Model [7] – Polar Plots [25] – Hyper-Radial Visualization [12] – Level Diagrams [8] – Prosection Plots [2] – PaletteViz [47]

Set properties

(visualizing solutions dependently from each other)

Showing individual

solution properties

Not optimization

based

– Distance and Distribution Charts [5] – Pareto Shells [52] – Hyper-Space Diagonal

Counting [4] – Tree Map [54] – Trade-off Region Map [44]

Optimization

based

– Principal Component Analysis [57] – Sammon Mapping [50] – Neuroscale [15] – Multidimensional Scaling [53] – Isomap [34] – Seriated Heatmap [53] – Two-Stage Mapping [33] – Distance-Based and Dominance-Based Mappings [16]

Showing aggregated

properties

– Self-Organizing Map [43] – Aggregation Tree [21] – MoGram [49] – Trend Mining [6]

Repeated

approximation sets

Showing

performance

at a time

– Line Plot [20] – Heat Map [36] – Visualization of Facets [24] – Grid-Based Sampling [30] – Slicing [3] – Maximum Intensity Projection [3] – Direct Volume Rendering [3] – Circular Indicator Plot [55]

Showing

performance

over time

– Grid-Based Sampling [10]

Multiobjective

landscapes

Showing original

objective values

– Level Sets – Line Walks [11, 51]

Showing transformed

objective values

– Dominance Rank [19] – Local Dominance [17] – Cumulative Gradient [29]

Showing optima

networks

– Pareto local optimal solutions network [35] – Pareto local optima

network [18] – Dominance-neutral optima network [18]

79

Summary

Summary

Visualization in Multiobjective Optimization

Pareto front approximations Single approximation sets Individual solutions (visualizing solutions independently from each other) Showing original objective values

– Scatter Plot Matrix – Bubble Chart – Parallel Coordinates [28] – Radar Chart – Chord Diagram [32] – Heat Map [45] – Interactive Decision Map [37] – Nasseh Method [37] Showing transformed objective values – Radial Coordinate Visualization [26] – 3-D Radial Coordinate Visualization [27] – Tetrahedron Coordinates Model [7] – Polar Plots [25] – Hyper-Radial Visualization [12] – Level Diagrams [8] – Prosection Plots [2] – PaletteViz [47] Set properties (visualizing solutions dependently from each other) Showing individual solution properties Not optimization based

– Distance and Distribution Charts [5] – Pareto Shells [52] – Hyper-Space Diagonal

Counting [4] – Tree Map [54] – Trade-off Region Map [44]

Optimization based – Principal Component Analysis [57] – Sammon Mapping [50] – Neuroscale [15] – Multidimensional Scaling [53] – Isomap [34] – Seriated Heatmap [53] – Two-Stage Mapping [33] – Distance-Based and Dominance-Based Showing aggregated properties – Self-Organizing Map [43] – Aggregation Tree [21] – MoGram [49] – Trend Mining [6] Repeated approximation sets Showing performance at a time – Line Plot [20] – Heat Map [36] – Visualization of Facets [24] – Grid-Based Sampling [30] – Slicing [3] – Maximum Intensity Projection [3] – Direct Volume Rendering [3] – Circular Indicator Plot [55]

Showing performance over time – Grid-Based Sampling [10] Multiobjective landscapes Showing original objective values – Level Sets – Line Walks [11, 51] Showing transformed objective values – Dominance Rank [19] – Local Dominance [17] – Cumulative Gradient [29] Showing optima networks

– Pareto local optimal solutions network [35] – Pareto local optima

network [18] – Dominance-neutral

optima network [18]

Summary

• Visualization in multiobjective optimization useful for various

purposes

• Customized methods are needed to address the peculiarities of

approximation set visualization as well as multiobjective

landscape visualization

•

New visualization methods should first be analyzed using

approximation sets and problems with known properties

•

Visualization methods should also be evaluated with user

studies

(never done in multiobjective optimization and seldom

Acknowledgement

The authors acknowledge the financial support from the

Slovenian Research Agency (Research core funding No.

P2-0209).

82

References

References i

[1] B. Filipič and T. Tušar.

A Taxonomy of Methods for Visualizing Pareto Front Approximations.

GECCO 2018, pages 649–656, 2018.

[2] T. Tušar and B. Filipič.

Visualization of Pareto front approximations in evolutionary

multiobjective optimization: A critical review and the prosection

method.

IEEE Transactions on Evolutionary Computation, 19(2):225-245, 2015.

[3] T. Tušar and B. Filipič.

Visualizing exact and approximated 3D empirical attainment functions.

Mathematical Problems in Engineering, Article ID 569346, 18 pages, 2014.

[4] G. Agrawal, C. L. Bloebaum, and K. Lewis.

Intuitive design selection using visualized n-dimensional Pareto

frontier.

American Institute of Aeronautics and Astronautics, 2005.

83

References ii

[5] K. H. Ang, G. Chong, and Y. Li.

Visualization technique for analyzing nondominated set comparison.

SEAL ’02, pages 36–40, 2002.

[6] S. Bandaru and A. H. C. Ng.

Trend mining: A visualization technique to discover variable trends in

the objective space.

EMO 2019, pages 605–617, 2019.

[7] X. Bi and B. Li.

The visualization decision-making model of four objectives based on

the balance of space vector.

IHMSC 2012, pages 365–368, 2014.

[8] X. Blasco, J. M. Herrero, J. Sanchis, and M. Martínez.

A new graphical visualization of n-dimensional Pareto front for

decision-making in multiobjective optimization.

Information Sciences, 178(20):3908–3924, 2008.

[9] X. Blasco, G. Reynoso-Mezab, E. A. Sanchez Perez, and J. V. Sanchez Perez.

Asymmetric distances to improve n-dimensional Pareto fronts

graphical analysis.

Information Sciences, 340-341:228–249, 2016.

[10] D. Brockhoff, A. Auger, N. Hansen, and T. Tušar.

Quantitative performance assessment of multiobjective optimizers:

The average runtime attainment function.

EMO 2017, pages 103–119, 2017.

[11] D. Brockhoff, T. Tušar, A. Auger, and N. Hansen.

Using well-understood single-objective functions in multiobjective

black-box optimization test suites.

ArXiv e-prints, 1604.00359v3, 2019.

85

[12] P.-W. Chiu and C. Bloebaum.

Hyper-radial visualization (HRV) method with range-based preferences

for multi-objective decision making.

Structural and Multidisciplinary Optimization, 40(1–6):97–115, 2010.

[13] M. T. M. Emmerich and C. M. Fonseca.

Computing Hypervolume Contributions in Low Dimensions:

Asymptotically Optimal Algorithm and Complexity Results.

EMO 2011, pages 121–135, 2011.

[14] K. Engel, M. Hadwiger, J. M. Kniss, C. Rezk-Salama, and D. Weiskopf.

Real-time Volume Graphics.

A. K. Peters, Natick, MA, USA, 2006.

[15] R. M. Everson and J. E. Fieldsend.

Multi-class ROC analysis from a multi-objective optimisation

perspective.

Pattern Recognition Letters, 27(8):918–927, 2006.

86

References v

[16] J. E. Fieldsend and R. M. Everson.

Visualising high-dimensional Pareto relationships in two-dimensional

scatterplots.

EMO 2013, pages 558–572, 2013.

[17] J. E. Fieldsend, T. Chugh, R. Allmendinger, and K. Miettinen.

A feature rich distance-based many-objective visualisable test

problem generator.

GECCO 2019, pages 541–549, 2019.

[18] J. E. Fieldsend and K. AlYahya.

Visualising the landscape of multi-objective problems using local

optima networks.

GECCO 2019 Companion, pages 1421–1429, 2019.

References vi

[19] C. M. Fonseca.

Multiobjective Genetic Algorithms with Application to Control

Engineering Problems.

Ph.D. thesis, University of Sheffield, 1995.

[20] C. M. Fonseca and P. J. Fleming.

On the performance assessment and comparison of stochastic

multiobjective optimizers.

PPSN IV, pages 584–593, 1996.

[21] A. R. R. de Freitas, P. J. Fleming, and F. G. Guimaraes.

Aggregation trees for visualization and dimension reduction in

many-objective optimization.