Plot your data

(1)

Statistics for Materials Engineers

MATLS 3J03

(2)

Plot your data

(3)

Usage examples

I

Co-worker

: Here are the yields from a batch system for the

last 3 years (1256 data points), can you help me:

I understand more about the time-trends in the past 3 year?

I efficientlysummarizetheyieldfromallbatchesrunin201

2

?

I

Manager

: effectively summarize the (a) number and (b) types

of defects on 17 aluminum grades for the past 12 months

I

Yourself

: 24 different variables being measured

vs

time (5

readings per minute, over 300 minutes) for each batch we

produce; how can we visualize these 36,000 data points?

(4)

Batch systems: large quantities of valuable data

[From Cecilia Rodrigues’ M.A.Sc thesis, 2006, McMaster University, used with permission]

[Flickr:#2516220152]

(5)

Batch systems: large quantities of valuable data

(6)

References

1. Edward Tufte,

Envisioning Information

, Graphics Press, 1990.

(10th printing in 2005)

2. Edward Tufte,

The Visual Display of Quantitative Information

,

Graphics Press, 2001.

3. Edward Tufte,

Visual Explanations: Images and Quantities,

Evidence and Narrative

, 2nd edition, Graphics Press, 1997.

4. William Cleveland,

Visualizing Data

, and

The Elements of

Graphing Data

, Hobart Press; 2nd edition, 1994.

5. Stephen Few,

Show Me the Numbers

, and “Now You See It”,

Analytics Press.

6. Su, It’s easy to produce chartjunk using Microsoft Excel 2007

but hard to make good graphs,

Computational Statistics and

Data Analysis

,

52 (10), 4594-4601, 2008,

http://dx.doi.org/10.1016/j.csda.2008.03.007

(7)

Background

This class might seem too easy, too obvious. It is!

I

The human eye and brain are excellent at pattern recognition,

sorting

through

signal

and

noise.

I

We

can

cope

with

bad

plots;

but

good

plots

save

time

and show a clearer, more honest picture.

I

Cliches: “Let the data speak for themselves”, “Plot the data”

(8)

Source: Presentation Graphics, Leland Wilkinson, SPSS Inc & Northwestern University

Position along a common scale:

bar chart, time series, scatter plot

Best information transfer

Worst information transfer

Position along nonaligned axis:

side by side graphs

Length:

stacked bar charts, waterfall charts

Angle / Slope:

pie charts

Area:

pie charts, matrix charts, radar charts

Volume:

3d charts

Colour

Position along a common scale:

bar chart, time series, scatter plot

Best information transfer

Worst information transfer

Position along nonaligned axis:

side by side graphs

Length:

stacked bar charts, waterfall charts

Angle / Slope:

pie charts

Area:

pie charts, matrix charts, radar charts

Volume:

3d charts

Colour

(9)

Time-series plots

I

It is a 2-dimensional plot:

I (usually) horizontal x-axis: time or sequence order

I other axis: the data values

I

Univariate plot

I

Our eyes can deal with high data density:

I sinusoids

I spikes

I outliers

(10)

Time-series plots

Good, automated labelling is important.

Here’s an example of bad labelling

(and bad axis scaling and colour choices)

(11)

Time-series plots

I

Multiple lines (trajectories): should not cross and jumble

(12)

Time-series plots

Use separate, parallel axes rather; and minimal ink

These non-default settings can take a long time to set (10 minutes

for this example)

(13)

Time-series plots

Sparklines

I

this website

(link also in the

notes)

I

Used for financial trends (see Google Finance, for example)

I

Built into Excel 2010

I

Good for iPods, cell phones, tablet computers:

(14)

Time-series plots

Example of sparklines in everyday use:

[Wikipedia: File:12leadECG.jpg]

(15)

Time-series plots

Further tips

I

Keep the x-axis spacing constant: helps interpretation

I Keep constant spacing on a time-axis (months)

(16)

Time-series plots

I

Adjust for inflation when plotting money values against time

I sales of polymer to DuPont over the past 10 years

I example of car sales:

http://www.duke.edu/∼rnau/411infla.htm

(17)

Time-series plots

(18)

Bar plots

I

A univariate plot on a two dimensional axis.

I

Has a

category axis

and

value axis

Use a bar plot when:

I

many categories

I

interpretation does not change if category axis is reordered

(19)

Bar plots

Rather use a time-series plot if the data have a sequence:

(20)

Bar plots

Bar plots can be wasteful as each data point is repeated several

times:

1. left edge (line) of

each bar

2. right edge (line) of

each bar

3. the height of the

colour in the bar

4. the number’s

position (up and

down along the

y-axis)

5. the top edge of

each bar, just below

the number

6. the number itself

(21)

Bar plots

Maximize data ink ratio within reason

Maximize data ink ratio

=

total ink for data

total ink for graphics

= 1

−

proportion of ink that can be erased

without loss of data information

(22)

Bar plots

I

Don’t use cross-hatching, textures, or unusual shading in the

plots: it creates visual vibrations

(23)

A

B

C

D

E

F

G

H

I

J

(24)

A 0 5 10 15 20 25 30 35 40 45 50 Jan-09 Feb-09 Mar-09 Apr-09 May-09 Jun-09 Jul-09 Aug-09 Sep-09 Oct-09 Nov-09 Dec-09 Jan-10 B 0 5 10 15 20 25 30 35 40 45 50 Jan-09 Feb-09 Mar-09 Apr-09 May-09 Jun-09 Jul-09 Aug-09 Sep-09 Oct-09 Nov-09 Dec-09 Jan-10 C 0 5 10 15 20 25 30 35 40 45 50 Jan-09 Feb-09 Mar-09 Apr-09 May-09 Jun-09 Jul-09 Aug-09 Sep-09 Oct-09 Nov-09 Dec-09 Jan-10 D 0 5 10 15 20 25 30 35 40 45 50 Jan-09 Feb-09 Mar-09 Apr-09 May-09 Jun-09 Jul-09 Aug-09 Sep-09 Oct-09 Nov-09 Dec-09 Jan-10 E 0 5 10 15 20 25 30 35 40 45 50 Jan-09 Feb-09 Mar-09 Apr-09 May-09 Jun-09 Jul-09 Aug-09 Sep-09 Oct-09 Nov-09 Dec-09 Jan-10 F 0 5 10 15 20 25 30 35 40 45 50 Jan-09 Feb-09 Mar-09 Apr-09 May-09 Jun-09 Jul-09 Aug-09 Sep-09 Oct-09 Nov-09 Dec-09 Jan-10 G 0 5 10 15 20 25 30 35 40 45 50 Jan-09 Feb-09 Mar-09 Apr-09 May-09 Jun-09 Jul-09 Aug-09 Sep-09 Oct-09 Nov-09 Dec-09 Jan-10 H 0 5 10 15 20 25 30 35 40 45 50 Jan-09 Feb-09 Mar-09 Apr-09 May-09 Jun-09 Jul-09 Aug-09 Sep-09 Oct-09 Nov-09 Dec-09 Jan-10 I 0 5 10 15 20 25 30 35 40 45 50 Jan-09 Feb-09 Mar-09 Apr-09 May-09 Jun-09 Jul-09 Aug-09 Sep-09 Oct-09 Nov-09 Dec-09 Jan-10 J 0 5 10 15 20 25 30 35 40 45 50 Jan-09 Feb-09 Mar-09 Apr-09 May-09 Jun-09 Jul-09 Aug-09 Sep-09 Oct-09 Nov-09 Dec-09 Jan-10

It’s now easier to see patterns related to time.

Having a common axis also allows for a comparison of

average and variability between “letters”.

(25)

Bar plots

I

Use horizontal bars if:

I there is a some ordering to the categories

I the labels do not fit side-by-side

I

You can place the labels inside the bars

(26)

D

iagnos

is

Top 30 presenting diagnoses of patients arriving at ED from LTC facility - All Sites

01APR2008-31AUG2009

Data source: NACRS 5/10/2010 Total Counts 202 ( 6. 17% ) 187 ( 5. 71% ) 121 ( 3. 69% ) 66 ( 2. 02% ) 64 ( 1. 95% ) 60 ( 1. 83% ) 58 ( 1. 77% ) 55 ( 1. 68% ) 53 ( 1. 62% ) 52 ( 1. 59% ) 51 ( 1. 56% ) 50 ( 1. 53% ) 45 ( 1. 37% ) 40 ( 1. 22% ) 38 ( 1. 16% ) 37 ( 1. 13% ) 34 ( 1. 04% ) 34 ( 1. 04% ) 34 ( 1. 04% ) 34 ( 1. 04% ) 33 ( 1. 01% ) 32 ( 0. 98% ) 31 ( 0. 95% ) 31 ( 0. 95% ) 29 ( 0. 89% ) 29 ( 0. 89% ) 26 ( 0. 79% ) 26 ( 0. 79% ) 25 ( 0. 76% ) 24 ( 0. 73% ) U ri nar y t rac t i nf ec tion s ite not s pec P neum oni a uns pec ifi ed C onges tiv e hear t f ai lur e Int er tr oc hant er ic f rac tur e, c los ed S ept ic aem ia uns pec ifi ed C hes t pai n uns pec ifi ed S y nc

ope and c

ol laps e C O P D w ith ac ut e ex ac er bat ion uns pec ifi ed O ther f rac tur e of f em or al nec k , c los ed G as tr oi nt es tinal haem or rhage N O S O pen w ound of s c al p unc om pl ic at ed A bn c v f unc t s tudy s ugges ts N S T E M I( F Y 07) O pen w ounds ot h par ts head, unc om pl ic at e E p is ta x is S tr ok e not s pec haem or rhage or inf ar c t M al ai s

e and f

at igue C e llu lit is o f lo w e r lim b A c ut e r enal f ai lur e uns pec ifi ed D is or ient at ion uns pec ifi ed O th & uns pec int es tinal obs tr uc tion D y s pnoea T rans ient c er ebr al is c haem ic at tac k N O S C O P D w ith ac ut e l ow er r es p i nf ec tion C ons tipat ion D ehy dr at ion P neum oni tis due t o f

ood and v

om it A tr ia l f ib rilla tio n U ns pec ifi ed i nj ur y of head B eni gn hy per tens ion U ns pec f rac tur e of nec k of f em ur , c los ed Count _Percent 0% 1% 2% 3% 4% 5% 6% 0 50 100 150 200 D iagnos is

Top 30 presenting diagnoses of patients arriving at ED from LTC facility - All Sites

01APR2008-31AUG2009

ope and c

e and f

ood and v

Top 30 presenting diagnoses of patients arriving at ED from LTC facility - All Sites

01APR2008-31AUG2009

ope and c

e and f

ood and v

Top 30 presenting diagnoses of patients arriving at ED from LTC facility - All Sites

01APR2008-31AUG2009

ope and c

e and f

ood and v

Top 30 presenting diagnoses of patients arriving at ED from LTC facility - All Sites

01APR2008-31AUG2009

ope and c

e and f

ood and v

Top 30 presenting diagnoses of patients arriving at ED from LTC facility - All Sites

01APR2008-31AUG2009

ope and c

e and f

ood and v

Top 30 presenting diagnoses of patients arriving at ED from LTC facility - All Sites

01APR2008-31AUG2009

ope and c

e and f

ood and v

Top 30 presenting diagnoses of patients arriving at ED from LTC facility - All Sites

01APR2008-31AUG2009

ope and c

e and f

ood and v

Top 30 presenting diagnoses of patients arriving at ED from LTC facility - All Sites

01APR2008-31AUG2009

ope and c

e and f

ood and v

Top 30 presenting diagnoses of patients arriving at ED from LTC facility - All Sites

01APR2008-31AUG2009

ope and c

e and f

ood and v

Top 30 presenting diagnoses of patients arriving at ED from LTC facility - All Sites

01APR2008-31AUG2009

ope and c

e and f

ood and v

(27)

D

iagnos

is

Top 30 presenting diagnoses of patients arriving at ED from LTC facility - All Sites

01APR2008-31AUG2009 5/10/2010 Total Counts Count Percent 0.73% 0.76% 0.79% 0.79% 0.89% 0.89% 0.95% 0.95% 0.98% 1.01% 1.04% 1.04% 1.04% 1.04% 1.13% 1.16% 1.22% 1.37% 1.53% 1.56% 1.59% 1.62% 1.68% 1.77% 1.83% 1.95% 2.02% 3.69% 5.71% 6.17%

0% 1% 2% 3% 4% 5% 6%

0 50 100 150 200

Urinary tract infection site not spec Pneumonia unspecified

Congestive heart failure

Intertrochanteric fracture, closed Septicaemia unspecified

Chest pain unspecified Syncope and collapse

COPD w ith acute exacerbation unspecified Other fracture of femoral neck, closed Gastrointestinal haemorrhage NOS Open w ound of scalp uncomplicated Abn cv funct study suggests NSTEMI(FY07) Open w ounds oth parts head, uncomplicate Epistaxis

Stroke not spec haemorrhage or infarct Malaise and fatigue

Cellulitis of low er limb

Acute renal failure unspecified Disorientation unspecified

Oth & unspec intestinal obstruction Dyspnoea

Transient cerebral ischaemic attack NOS COPD w ith acute low er resp infection Constipation

Dehydration

Pneumonitis due to food and vomit Atrial fibrillation

Unspecified injury of head Benign hypertension

Unspec fracture of neck of femur, closed

D

iagnos

is

Top 30 presenting diagnoses of patients arriving at ED from LTC facility - All Sites

0% 1% 2% 3% 4% 5% 6%

0 50 100 150 200

Dehydration

D

iagnos

is

Top 30 presenting diagnoses of patients arriving at ED from LTC facility - All Sites

0% 1% 2% 3% 4% 5% 6%

0 50 100 150 200

Dehydration

D

iagnos

is

Top 30 presenting diagnoses of patients arriving at ED from LTC facility - All Sites

0% 1% 2% 3% 4% 5% 6%

0 50 100 150 200

Dehydration

D

iagnos

is

Top 30 presenting diagnoses of patients arriving at ED from LTC facility - All Sites

0% 1% 2% 3% 4% 5% 6%

0 50 100 150 200

Dehydration

D

iagnos

is

Top 30 presenting diagnoses of patients arriving at ED from LTC facility - All Sites

0% 1% 2% 3% 4% 5% 6%

0 50 100 150 200

Dehydration

D

iagnos

is

Top 30 presenting diagnoses of patients arriving at ED from LTC facility - All Sites

0% 1% 2% 3% 4% 5% 6%

0 50 100 150 200

Dehydration

D

iagnos

is

Top 30 presenting diagnoses of patients arriving at ED from LTC facility - All Sites

0% 1% 2% 3% 4% 5% 6%

0 50 100 150 200

Dehydration

D

iagnos

is

Top 30 presenting diagnoses of patients arriving at ED from LTC facility - All Sites

0% 1% 2% 3% 4% 5% 6%

0 50 100 150 200

Dehydration

D

iagnos

is

Top 30 presenting diagnoses of patients arriving at ED from LTC facility - All Sites

0% 1% 2% 3% 4% 5% 6%

0 50 100 150 200

Dehydration

D

iagnos

is

Top 30 presenting diagnoses of patients arriving at ED from LTC facility - All Sites

0% 1% 2% 3% 4% 5% 6%

0 50 100 150 200

Dehydration

(28)

Box plots

A graphical display of the “5-number summary” for 1 variable

I

whisker = minimum sample value [

or

: median

−

1 .

5

· IQR]

I

25th percentile (1st quartile)

I

50th percentile (median)

I

75th percentile (3rd quartile)

I

whisker = maximum sample value [

or

: median +1

.

5

· IQR]

Notes

:

1. 25th percentile is the value below which 25 percent of the

observations in the sample are found

2. distance from 3rd to 1st quartile = interquartile range (IQR)

Box plots are effective for comparing similar variables (same units

of measurement)

(29)

(30)

Box plots

Video of data

source: sawmill in

Qu´

ebec

I

4 degrees of

rotation of log

as it moves

through the

saws

(31)

Box plots

Thickness

measured

at

6 locations

(

target

=

1680

mils

)

(32)

Box plots

(33)

Box plots

Some variations:

I

use the mean instead of the median

I

outliers shown as dots, where an outlier is most commonly

defined as any point 1.5 IQR distance units above and below

the median.

I

use the 2nd percentile (instead of median

−

1 .

5

· IQR)

I

use the 98th percentile (instead of median + 1

.

5

· IQR)

I

add the density histogram onto the box plot:

violin plot

(34)

Box plot variation: violin plot

(35)

Scatter plots

I

Used to help understand the relationship between two

variables: a bivariate plot

I

Collection of points in the 2 axes

I

Each point is the intersection of the values on each axis

Intention of a scatter plot

(36)

Scatter plots

(37)

Scatter plots

(38)

Scatter plots

Strive for graphical excellence by:

I

making each axis as tight as possible

I

avoid heavy grid lines

I

use the least amount of ink

I

do not distort the axes

(39)

Scatter plots

There is an unfounded fear that others won’t understand your 2D

scatter plot.

I

Tufte study (VDQI): no scatter plots in a sample (1974 to

1980) of Western dailies

I

12 year olds can interpret such plots.

I

Japanese newspapers frequently use scatterplots

I

Plant control room: seldom see scatter plots.

Key point

The producers of charts must assume their audience is capable of

interpreting them. Rather, assume that if you can understand the

(40)

Here’s an example (January 2013 publication)

I

Why did the author

use a time-series plot

to show correlation?

I

Would the plot be

more informative as

2D-scatter plot?

I

What if you were to

repeat this analysis

for multiple

regions/-countries/cities. How

would you show

(visualize) the

correlations

effectively?

Pb(CH

2

CH

3

)

4

[Read the full story for more interesting details and geographic visualizations:

http://www.motherjones.com/environment/2013/01/lead-crime-link-gasoline]

(41)

Scatter plots

I

Add a 3rd variable: different marker sizes

I

Add a 4th variable: use colour or grayscale shading

(42)

Scatter plots

I

Web-based demo from

http://gapminder.org

I

Demo by Hans Rosling

(requires internet access)

(43)

Tables

Tables are for

comparative

data analysis on

categorical objects

.

I

categorical objects

: the cars

I

Note the rows are in

default

alphabetical order.

I

We can make the table “tell a story” if we reorder the rows by

some other variable.

(44)

Tables

I

Compare defect types (columns) for different product grades

(rows)

I

Categorical variables appear in the

rows

and

columns

here

I

Which defects cost us the most money?

(45)

Tables

I

Defect frequency

I If 1850 lots of grade A4636 (first row): defect A rate = 1/50

I If 250 lots of grade A2610 (last row): defect A rate = 1/50

I Redraw table on production rate basis

I

If comparing defects over different grades: go down the table

(show fraction within the column)

I

If comparing defects within grade: go across table (show

fraction with the row)

(46)

Tables

Three common pitfalls:

1. using pie charts when tables or bar charts will do

I cannot explain the pitfalls of pie charts as well as Stephen Few

does:

Save the pies for dessert

(please read)

(47)

(48)

Percentage by Province: Value of Building Permits Issued: Oct. 2008

NL PEI NS NB PQ ON MB SK AB BC Yukon NWT Nunavut Total Value: 5,436,600,000 CAD

(49)

Percentage by Province: Value of Building Permits Issued: Oct. 2008

1% 0%

25%

33% 3%

3% 15%

16%

0% 0%

0%

2% 2%

(50)

Percentage by Province: Value of Building Permits Issued: Oct. 2008

PQ 25%

ON 33% MB

3% SK 3% AB 15%

BC 16%

NL 1%

PEI 0% Nunavut

0% NWT

0% Yukon

0%

NS 2%

NB 2% Total Value: 5,436,600,000 CAD

(51)

Percentage Value of Building Permits Issued

By Province

October 2008

Total Value: 5,436,600,000 CAD

Province

Percentage

ON

33%

PQ

25%

BC

16%

AB

15%

MB

3%

SK

3%

NS

2%

NB

2%

NL

1%

PEI

0.3%

NWT

0.1%

Yukon

0.1%

Nunavut

0.1%

(52)

Percentage Value of Building Permits Issued By Province

October 2008

0% 5% 10% 15% 20% 25% 30% 35%

ON

PQ

BC

AB

MB

SK

NS

NB

NL

PEI

NWT

Yukon

Nunavut Province

Total Value: 5,436,600,000 Source: Statistics Canada td/9JAN2009

(53)

Tables

(54)

Tables

3. using excessive grid lines

(55)

Province

Percentage

ON

33%

PQ

25%

BC

16%

AB

15%

MB

3%

SK

3%

NS

2%

NB

2%

NL

1.4%

PEI

0.32%

NWT

0.11%

Yukon

0.10%

Nunavut

0.09%

Total

100%

Province

Percentage

ON

33%

PQ

25%

BC

16%

AB

15%

MB

3%

SK

3%

NS

2%

NB

2%

NL

1.4%

PEI

0.32%

NWT

0.11%

Yukon

0.10%

Nunavut

0.09%

Total

100%

(56)

Tables

Interesting example: comparing two treatments

I

Coating A or B are applied to different products

I

K-series, P-series, S-series

I

How does the coating affect corrosion and surface roughness?

(57)

(58)

Data frames

Frames are the basic containers that surround the data and give

context to our numbers. Here are some tips:

1. Use round numbers

2. Tighten the axes as much as possible, except ...

3. when showing comparison plots:

all axes must have the same

minima and maxima

(59)

Aesthetics and style

(60)

Colour

I

Colour is effective, but:

I readers could be colour-blind,

I document read from a gray-scale print out

I

There is

no standard colour progression

(blues, greens,

yellows, orange, red).

I

Safest colour progression is gray-scale axis: from black to

white

I satisfies colour-blind readers

I looks good in printed form

(61)

(62)

Life Expectancy at particular age

0 10 20 30 40 50 60 70 80 90 100

Birth Age 65 Age 75

L

if

e E

xp

ect

an

cy (

year

s)

Male Female Male Female Male Female

(63)

General summary

No general advice that applies in every instance

. Useful tips

nevertheless:

I

To understand causality, you must show causality: use

bivariate scatter plots (sometimes line plots also work well)

I

Plots and text go together: a plot = paragraph of text

I add labels to plots for outliers and interesting points

I add equations

I add small summary tables

(64)

General summary

I

Avoid unnecessary “extras” to enliven the plot

I

“

If the statistics are boring, then you’ve got the wrong

numbers

”.

(65)

General summary

I

Adjust for inflation if plot involves money and time

I

Maximize the data-ink ratio = (ink for data) / (total ink for

graphics).

1. eliminate non-data ink

2. erase redundant data-ink.