Design of Experiments Advanced Factorial Designs

(1)

Design of Experiments – Advanced Factorial Designs

Fractional Factorial Experiments 3k Factorial Designs 2k Factorial Designs with Center Points

Topics

I. 2

⁴

Full factorial

II. 2

^k-p

Fractional Factorial Designs III. 3

^k

Example

IV. 2

^k

Experiments with Center Points

(2)

3

I. Engine Casting DOE* – 2

⁴

 Case Study – Aircraft Engine Component

 Output Variable: Material Cracking (length in mm)

 Objective: Smaller-the-better (Minimize Cracking Length)

 Measurements based on a stress test to see the effect of several factors on potential cracking

 Study: Four Factors (each factor at 2 levels)

*Example adapted from Montgomery, Design and Analysis of Experiments, Wiley and Sons

A – Pouring temperature (Low: 1100 ^oC; High: 1150 ^oC)

B – Amount of titanium used in alloy material (Low: 13%; High: 15%) C – Heat treatment process (Method 1 and 2)

D – Amount grain refiner type (Low: 90; High: 100)

4

Engine Casting Data Set

 4 factors @ 2 levels each (2 replications)  32 samples

 Terms = 2⁴– 1 =15(4 Main, 6 2-way; 4 3-way; 1 4-way interaction)

Replicate A B C D Length (mm) Replicate A B C D Length (mm)

1 -1 -1 -1 -1 1.75 2 -1 -1 -1 -1 1.72

1 -1 -1 -1 1 1.97 2 -1 -1 -1 1 2.03

1 -1 -1 1 -1 1.71 2 -1 -1 1 -1 1.72

1 -1 -1 1 1 2.06 2 -1 -1 1 1 1.97

1 -1 1 -1 -1 1.74 2 -1 1 -1 -1 1.74

1 -1 1 -1 1 2.03 2 -1 1 -1 1 2.01

1 -1 1 1 -1 1.72 2 -1 1 1 -1 1.71

1 -1 1 1 1 1.98 2 -1 1 1 1 2.06

1 1 -1 -1 -1 1.42 2 1 -1 -1 -1 1.41

1 1 -1 -1 1 1.86 2 1 -1 -1 1 1.85

1 1 -1 1 -1 1.41 2 1 -1 1 -1 1.43

1 1 -1 1 1 1.85 2 1 -1 1 1 1.88

1 1 1 -1 -1 1.42 2 1 1 -1 -1 1.44

1 1 1 -1 1 1.83 2 1 1 -1 1 1.85

1 1 1 1 -1 1.45 2 1 1 1 -1 1.42

1 1 1 1 1 1.86 2 1 1 1 1 1.89

(3)

5

Factorial Analysis



Here, 3 Factors explain ~98% of Variance in Y (Cracking length)

Factorial Fit: Length (mm) versus A, B, C, D Term Effect Coef SE Coef T P Constant 1.7559 0.004811 364.99 0.000 A -0.2281 -0.1141 0.004811 -23.71 0.000

B 0.0069 0.0034 0.004811 0.71 0.485 C 0.0031 0.0016 0.004811 0.32 0.750 D 0.3606 0.1803 0.004811 37.48 0.000

A*B -0.0006 -0.0003 0.004811 -0.06 0.949 A*C 0.0106 0.0053 0.004811 1.10 0.286 A*D 0.0731 0.0366 0.004811 7.60 0.000 B*C 0.0006 0.0003 0.004811 0.06 0.949 B*D -0.0019 -0.0009 0.004811 -0.19 0.848 C*D 0.0119 0.0059 0.004811 1.23 0.235 A*B*C 0.0056 0.0028 0.004811 0.58 0.567 A*B*D -0.0069 -0.0034 0.004811 -0.71 0.485 A*C*D -0.0031 -0.0016 0.004811 -0.32 0.750 B*C*D 0.0019 0.0009 0.004811 0.19 0.848 A*B*C*D 0.0044 0.0022 0.004811 0.45 0.655 S = 0.0272144 R-Sq = 99.22% R-Sq(adj) = 98.48%

Full Factorial Analysis - Pareto Results

 Pareto Results: Full Factorial Analysis

 Select Pareto under “graphs” under analyze factorial design

So, we created an experiment to study 15 terms, but only found 3 significant Surprising?

(4)

7 Analysis of Variance for Length (mm) (coded units)

Source DF Seq SS Adj SS Adj MS F P Main Effects 4 1.45719 1.45719 0.364297 600.46 0.000 2-Way Interactions 6 0.04484 0.04484 0.007474 12.32 0.000 Residual Error 21 0.01274 0.01274 0.000607

Lack of Fit 5 0.00089 0.00089 0.000178 0.24 0.939 Pure Error 16 0.01185 0.01185 0.000741

Total 31 1.51477

Pure Error –

estimate based on replications

Lack of Fit Error – Estimate based on df from ignored terms S=0.0426

Higher Order Interaction Effects



Suppose you re-fit these data and only consider main effects and two-way interactions (i.e., ignore 3-way and 4-way)

 Did the 3-way and 4-way interactions matter?

8

II. Fractional Factorial Designs

 As one increases the number of factors, the likelihood of them all being significant is low

 Suppose you study 9 factors using a full factorial, one would not expect 511 (2⁹-1) significant main effects and interactions!

 Sparsity of Effects Principle* – Given numerous variables in a factorial experiment, only a few factors will likely be significant

 Hierarchical ordering principal(Wu and Hamada) – Main effects and low-order interactions are more likely to be important than higher order

 One Strategy to reduce experimental costs:

 Run Fractional Factorial Design (“Screening experiment”)

 Seek to maximize information at lowest cost by reducing the number of combinations studied by assuming higher order interactions are negligible

*Wu, C.F.J. and Hamada, M.S. (2000). Experiments: Planning, Analysis, and Parameter Design Optimization, Wiley

(5)

9

Fractional Factorial – Confounding

 Consider Simple Case: Factors’ A and B

Suppose you wish to add a third factor C without any more runs

Confounding Strategy: Run Factor C using the ‘same settings’ as those naturally occurring for the A*B Interaction (AB_Lowvs. AB_High)

Fractional factorial designs involve ‘confounding terms’ – i.e., making terms indistinguishable from another

Make C equivalent to A*B Study 3 factors in 4 Runs Study 2 factors in 4 Runs

A*B_(-1): L/H & H/L A*B₍₊₁₎: L/L & H/H

Confounding:

Benefits/Drawbacks

 Consider the 2

⁴

Casting Experiment – Suppose we confound Variable D with ABC interaction (same settings)

 This strategy has both benefits and potential drawbacks

 Benefit: Allows one to use fewer experimental runs (less cost)

 Drawback: If a confounded term is significant, one will not know for certain which is the causal variable (D and/or A*B*C)

 Though we may conjecture that D is more likely the causal effect

(6)

11

Types of Fractional Factorial Designs

 2

^k

Fractional Factorial Designs may be expressed as 2

^k-p

For p=1 experiments, the number of combinations is cut in half (this is also known as a Half Fractional Design)

For instance, we may run a 2⁴factorial experiment using only 8 combinations (2^4-1) by confounding terms

 Fractional Designs and values for “p”

p = 2, reduces # of combinations by 1/4, e.g., 2^5-2(from 32  8)

p = 3, reduces # of combinations by 1/8, e.g., 2^6-3(from 64  8)

 By identifying a value for p in 2

^k-p

, Minitab will automatically create experiments with confounded terms

Manually creating fractional designs is outside this class scope

12

Design Resolution – Fractional Designs

Design Resolution is used to categorize fractional designs by the terms confounded (e.g., main effects, 2-way, 3-way)

Resolution III (Most Risky) – Some main effects are confounded with 2-way interactions, however, no main effects are confounded with other main effects

Resolution IV – Some two-factor interactions are confounded with each other, however, no main effect is confounded with any other main effect or with any two-factor interaction

Resolution V (Least Risky) – Some 2-way are confounded with 3-way, however, no main effect or two-way interaction is confounded with any other main effect or two-way interaction

The higher the Design Resolution number, the less ‘risk’

(7)

13

Creating Fractional Factorial Experiments Using Minitab

 Use Minitab to create fractional designs (e.g., 2

^4-1

)

 May click on

‘Display Available Design’ for Options based on Desired Resolution

Minitab Available Designs

E.g., May examine 4 factors at 2 levels using Resolution IV - 8 run combinations (‘Half Fractional Design’)

Note: Minitab Highlights Resolution III In Red

(8)

15

Minitab Factorial Design Options

 Given the number of factors, Minitab will offer choices of experiments to create For this option,

Minitab will set D = ABC (default generator) Why might one confound terms, but still use 2 replications?

(vs. a Full Factorial with 1 replication)

16

Minitab – “Alias Structure”

 Minitab identifies confounded variables using ‘alias structure’

Confounded Main Effects: D=ABC; A=BCD, B=ACD, etc.

Confounded 2-Way: A*B = C*D; A*C = B*D, etc.

So, if A*B is significant, there is no way to know for certain if the significant effect is A*B and/or C*D

Factors: 4 Base Design: 4, 8 Resolution: IV Runs: 8 Replicates: 1 Fraction: 1/2

Design Generators: D = ABC Alias Structure

I + ABCD, A + BCD, B + ACD C + ABD, D + ABC, AB + CD AC + BD, AD + BC

D is confounded

with ABC

(9)

17

Casting Example – 2

^4-1

(Based on 2 Replications)

 Casting Example: D = ABC

Row 1: A*B*C = -1*-1*-1 = -1 which is same as Factor D setting

Row 2: A*B*C = -1*-1*1 = 1 …

Equate D=ABC

Run ‘Analyze Factorial Design’

 Minitab will automatically remove confounded terms

 Alternatively, may run an initial model with only main effects and 2-way interactions (assume higher order are negligible)

Alias Information for Terms in the Model. Totally confounded terms were removed from the analysis.

I + A*B*C*D A + B*C*D B + A*C*D C + A*B*D D + A*B*C A*B + C*D A*C + B*D

(10)

19

Fractional Factorial Results

Are these findings the same as the Full Factorial?

Factorial Fit: Length (mm) versus A, B, C, D Estimated Effects and Coefficients for Length (mm) (coded units)

Term Effect Coef SE Coef T P Constant 1.7581 0.006644 264.62 0.000 A -0.2262 -0.1131 0.006644 -17.03 0.000

B 0.0037 0.0019 0.006644 0.28 0.785 C -0.0037 -0.0019 0.006644 -0.28 0.785 D 0.3663 0.1831 0.006644 27.56 0.000

A*B 0.0113 0.0056 0.006644 0.85 0.422 A*C 0.0087 0.0044 0.006644 0.66 0.529 A*D 0.0738 0.0369 0.006644 5.55 0.001 S = 0.0265754 R-Sq = 99.27% R-Sq(adj) = 98.62%

20

Evaluating Alias Structure

 Recall: Factor D is confounded with the ABC interaction

 Which do you think is most likely significant: D and/or ABC?

 Still, you cannot know for sure unless you verify

 This example worked well because the model did not have a lot of significant interactions and only 2 of 4 main effects were significant

 If an experiment has many significant terms, it may be difficult to identify ‘likely’ significant terms among those that are confounded

 There are usually some drawbacks to reducing sample size!

(11)

21

Identify Settings:

Factorial Plots

 Next, we may create factorial plots to help identify settings

 Significant Interaction

 A*D

 Recommendations?

 A B

 C D

III. 3

^k

Factorial Experiments

 3

^k

Experiments – k factors at 3 levels such as:

 3

²

– 9 combinations (2 factors at 3 levels)

 3

³

– 27 combinations (3 factors at 3 levels)

 Why might we choose a 3

^k

over a 2

^k

?

 What are some limitations of 3

^k

?

Note: Another option is a mixed experiment (usually done with most factors at 2 levels, but one or two factors at 3 levels)

(12)

23

Casting Example – Identifying Settings for a 3

^k

Experiment

 Output – Length (mm)

 Smaller-the-Better

 Suppose you decide to add a 3^rd Level to our casting experiment

 Given that we observed small cracking at A High and D low, we may investigate further with:

 Run A at a Higher Level = 1200

 Run D at a Lower Level = 80

 Suppose you also wish to examine any effects related to side of oven

 May apply Blocking Variable Original Study Factors:

A – pouring temperature (Low: 1100 ^oC, High: 1150 ^oC) D – amount grain refiner type (Low: 90 High: 100)

Interaction:

Pour Temp (A) and Amount Refiner (D)

24

3

²

Data Set

 Here, we are studying 2 factors (Pouring Temperature and Amount of Grain Refiner) @ 3 Levels each (2 Replications)

 3²* 2 = 18 How many samples at each level of A (Low, Med, High)?

Pouring temperature (1100, 1150, 1200) Amount grain refiner (80, 90, 100)

Blocking Variable: Oven Side - use 1 replicate from RH and 1 replicate from LH Side

Replicate: Side

…

(13)

25

Define/Analyze Custom Factorial Design

Caution: will not have option to use Response Optimizer Tool

Must Identify Blocking Variable in separate column

Analyze Factorial Design

Or, Create New ‘3 Level Design’

Note: Would have only used if started from scratch – no available data

(14)

27

Factorial Analysis – Results

 At 3 Levels, we have sufficient degrees of freedom to study all terms

PourTemp fixed 3 -1, 0, 1 Refiner fixed 3 -1, 0, 1

Analysis of Variance for Length (mm), using Adjusted SS for Tests Source DF Seq SS Adj SS Adj MS F P Side (1-LH) 1 0.00001 0.00001 0.00001 0.01 0.930 PourTemp 2 0.46643 0.46643 0.23322 342.69 0.000 Refiner 2 0.40623 0.40623 0.20312 298.46 0.000 PourTemp*Refiner 4 0.08093 0.08093 0.02023 29.73 0.000 Error 8 0.00544 0.00544 0.00068

Total 17 0.95905

S = 0.0260875 R-Sq = 99.43% R-Sq(adj) = 98.79%

Did Blocking Matter here?

28

Results – Without Blocking Variable

 Both main and interactions are significant

 Note: Estimate of error term changed slightly

Factor Type Levels Values PourTemp fixed 3 -1, 0, 1 Refiner fixed 3 -1, 0, 1

Analysis of Variance for Length (mm), using Adjusted SS for Tests Source DF Seq SS Adj SS Adj MS F P PourTemp 2 0.46643 0.46643 0.23322 385.13 0.000 Refiner 2 0.40623 0.40623 0.20312 335.42 0.000 PourTemp*Refiner 4 0.08093 0.08093 0.02023 33.41 0.000 Error 9 0.00545 0.00545 0.00061

Total 17 0.95905

S = 0.0246080 R-Sq = 99.43% R-Sq(adj) = 98.93%

Uncheck to remove blocking

(15)

29

Factorial Plots

 Post Analysis:

Was it useful to run this experiment at 3 levels?

Why?

 Recommend settings per Interaction Plot (‘minimize’ response)

Pour Temp:

Refiner:

 How might one verify this recommendation?

3D Surface Plot

 For two continuous X factors, another visualization tool is a 3D surface plot

Effective for 2 X’s and one Y, particularly if some non-linearity exists in main effect/interaction plots

 Note: Another option is a ‘contour plot’

(16)

31

Example: 3D Surface Plot

 Would we expect to keep reducing cracking length with higher pour temperature and less grain refiner?

32

IV. DOE (2

^k

) with Center Points

 If all factors are continuous, rather than replicating each corner point, another strategy is to replicate using center points only

 Suppose you run a new experiment with 3 factors at 2 levels

 Add Treatment Time (Low = 25; High = 35) to factors Pour Temp/Refiner

 For the 3 Factors at 2 Levels shown below, identify center point values x 2

x 1 x 3

(-1,-1,-1) (+1,-1,-1) (-1,+1,+1)





 









Pour Temp (1100, 1200) Amt Refiner

(80,100) Treatment Time

(25, 35)

(17)

33

Experiment with Center Points

We typically add ~5 center points (at least 3) to provide replications for estimating error term and testing for non-linear effects (curvature)

 Note: Average of the 8 corner points is 1.5975 (remember this number)

2

³

Center Points

Define Custom Factorial Design and Analyze Factorial Design

Identify Center Point Column

Response: Length (mm)

(18)

35

Factorial Analysis – Results Estimated Effects Table

 Did the Center Points Matter?

Factorial Fit: Length (mm) versus PourTemp, AmtRefiner, TreatTime Estimated Effects and Coefficients for Length (mm) (coded units)

Term Effect Coef SE Coef T P Constant 1.5975 0.007665 208.42 0.000 PourTemp -0.3800 -0.1900 0.007665 -24.79 0.000 AmtRefiner 0.2550 0.1275 0.007665 16.63 0.000 TreatTime -0.1400 -0.0700 0.007665 -9.13 0.001 PourTemp*AmtRefiner -0.1300 -0.0650 0.007665 -8.48 0.001 PourTemp*TreatTime -0.0050 -0.0025 0.007665 -0.33 0.761 AmtRefiner*TreatTime -0.0200 -0.0100 0.007665 -1.30 0.262 PourTemp*AmtRefiner*TreatTime 0.0050 0.0025 0.007665 0.33 0.761 Ct Pt -0.2755 0.012359 -22.29 0.000

S = 0.0216795 R-Sq = 99.74% R-Sq(adj) = 99.23%

‘Center of Cube’

- all coefficients are relative to this value including ‘Ct Pt’

36

Factorial Analysis – ANOVA Table Results

 With center points, we may use ANOVA table to assess if we have ‘curvature’. Is curvature significant?

Analysis of Variance for Length (mm) (coded units)

Source DF Seq SS Adj SS Adj MS F P Main Effects 3 0.458050 0.458050 0.152683 324.86 0.000 2-Way Interactions 3 0.034650 0.034650 0.011550 24.57 0.005 3-Way Interactions 1 0.000050 0.000050 0.000050 0.11 0.761 Curvature 1 0.233539 0.233539 0.233539 496.89 0.000 Residual Error 4 0.001880 0.001880 0.000470

Pure Error 4 0.001880 0.001880 0.000470 Total 12 0.728169

(19)

37

Factorial Plots

 Next Step –Create 3D surface plot for pour temp and refiner (significant interaction)

Response Optimizer



One advantage of using 2^K with center points is that we may use the response optimizer

 Suppose Final Model include: Factors’ A, B, C, and A*B interaction

 Suppose Target is 1 (with upper limit of 1.5) – Goal is Minimize Response

(20)

39

Optimizer Results

 Best Result Available: A High; B Low; C High

HighCur 0.45000OptimalD Low

d = 0.45000 Minimum

Length ( y = 1.2750

0.45000 Desirability Composite

-1.0 1.0 -1.0

1.0 -1.0

1.0 AmtRefin TreatTim

PourTemp

1.0 -1.0 1.0

Best result given observed data

40

Summary

 As one increases the number of factors, the likelihood of them all being significant is low 

Sparsity of Effects Principle – Given numerous variables in a factorial experiment, only a few will likely be significant (typically main effects and 2-way interactions – hierarchical ordering principle)

 Common Strategy to reduce experimental costs is to run a fractional factorial design at Resolution IV or V (preferred) and confound higher-order interactions with main effects and 2-way to reduce experimentation cost

 To test for potential non-linear effects, may run a 3^kFactorial Experiment

Or, we may use 2^kwith center points (if X’s are continuous) and test for statistically significant curvature and

 Center Points provide replications for error without replicating corner points

Non-linear effects may be visualized using 3D Surface Plots

Design of Experiments Advanced Factorial Designs