Factor Analysis and Structural equation modelling

(1)

Factor Analysis and Structural equation

modelling

Herman Adèr

Previously:

Department Clinical Epidemiology and Biostatistics,

VU University medical center, Amsterdam

(2)

Overview

1

E

←→

C

2

EFA

3

CFA

4

SEM

Exploratory

←→

Confirmatory methods

Exploratory Factor Analysis

Confirmatory Factor Analysis

Structural Equation Modelling

(3)

Overview

1

E

←→

C

2

EFA

3

CFA

4

SEM

Exploratory

←→

(4)

Overview

1

E

←→

C

2

EFA

3

CFA

4

SEM

Exploratory

←→

(5)

Overview

1

E

←→

C

2

EFA

3

CFA

4

SEM

Exploratory

←→

(6)

Overview

1

E

←→

C

2

EFA

3

CFA

4

SEM

Exploratory

←→

(7)

Exploratory and confirmatory methods Exploratory factor analysis Confirmatory factor analysis Structural equation modelling References

Part VI

(8)

(9)

Characterization of Tukey (1977)

Exploratory data analysis is

detective

in character.

Confirmatory data analysis is

judicial

or quasi-judicial in

character. . . Unless the detective finds the clues, judge or jury

has nothing to consider. Unless exploratory data analysis

uncovers indications, usually quantitative ones, there is likely to

be nothing for confirmatory data analysis to consider.

On the other hand:

Exploratory data analysis can never be the whole story, but

nothing else can serve as the foundation stone – the first step.

(10)

Characterization of Tukey (1977)

Exploratory data analysis is

detective

in character.

Confirmatory data analysis is

judicial

or quasi-judicial in

character. . . Unless the detective finds the clues, judge or jury

has nothing to consider. Unless exploratory data analysis

uncovers indications, usually quantitative ones, there is likely to

be nothing for confirmatory data analysis to consider.

On the other hand:

Exploratory data analysis can never be the whole story, but

nothing else can serve as the foundation stone – the first step.

(11)

Questions and answers

Three steps

Determining the meaningful factors Rotation

Interpretation of the factor structure

F

=

M ∪ U ∪ E

,

of which

M

indicate the meaningful factors,

U

so-called

‘unique’ factors (factors on which only one item loads) and

E

error factors.

Questions we try to settle using EFA

1

How many meaningful dimensions are present?

2

What is the structure of those dimensions?

(12)

Three steps

F

=

M ∪ U ∪ E

,

of which

M

indicate the meaningful factors,

U

so-called

‘unique’ factors (factors on which only one item loads) and

E

error factors.

Questions we try to settle using EFA

1

How many meaningful dimensions are present?

(13)

Three steps

determine meaningful factors rotate

determine factor meaning

unrotated

Exploratory Factor Analysis Items

k meaningful factors

final factor solution factor structure factor structure Subject Data set: D ₌n o ? ? ?

Three steps

1

Determine the meaningful factors

2

Rotate

(14)

Questions and answers Three steps

Determining the meaningful factors

Rotation

determine meaningful factors

:

scree plot

total variance explained more than 50%

select factors with

intelligible

loading patterns

and

name

them

communalities

≈

reliabilities

eigenvalues larger than 1

doubtful

GoF change

significant

between solutions

(15)

Rotation

CES-D: A multidimensional scale to assess Depression

FACTOR

/VARIABLES cesd1 cesd2 cesd3 cesd4 cesd5

cesd6 cesd7 cesd8 cesd9 cesd10

/MISSING LISTWISE

/ANALYSIS cesd1 cesd2 cesd3 cesd4 cesd5

/PRINT

all

/FORMAT BLANK(.30)

/PLOT EIGEN

/CRITERIA MINEIGEN(0) ITERATE(100) factors(3)

/EXTRACTION pc

(16)

Rotation

CES-D: A multidimensional scale to assess Depression

FACTOR

/VARIABLES cesd1 cesd2 cesd3 cesd4 cesd5

/MISSING LISTWISE

/ANALYSIS cesd1 cesd2 cesd3 cesd4 cesd5

/PRINT

all

/FORMAT BLANK(.30)

/PLOT EIGEN

/CRITERIA MINEIGEN(0) ITERATE(100) factors(3)

/EXTRACTION pc

(17)

Rotation

Scree Plot

Component Number

20

19

18

17

16

15

14

13

12

11

10

9

8

7

6

5

4

3

2

1

E

ig

e

n

v

a

lu

e

10

8

6

4

2

0

(18)

Rotation

8.516 42.578 42.578 1.775 8.876 51.454 1.229 6.147 57.600 1.040 5.201 62.802 .944 4.719 67.520 .767 3.836 71.356 .738 3.689 75.045 .684 3.419 78.465 .606 3.030 81.495 .577 2.886 84.381 .508 2.542 86.923 .442 2.211 89.134 .426 2.129 91.263 .339 1.697 92.960 .318 1.592 94.552 .286 1.429 95.981 .274 1.371 97.352 .202 1.010 98.362 .178 .891 99.253 Component 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Total % of Variance Cumulative % Initial Eigenvalues

(19)

Rotation

Interpretation of the unrotated factor structure

.525 .372 -.373 .757 -.318 .802 .389 -.573 .538 .687 .336 .689 .726 .349 -.635 .531 .721 -.353 .672 .398 .623 .415 -.690 .489 .599 -.513 .710 .432 .490 -.627 -.758 .383 .657 .500 .763 .349 .482 -.447 -.376 .784 .359 CESD1 CESD2 CESD3 CESD4 CESD5 CESD6 CESD7 CESD8 CESD9 CESD10 CESD11 CESD12 CESD13 CESD14 CESD15 CESD16 CESD17 CESD18 CESD19 CESD20 1 2 3 4 5 6 Component

(20)

Rotation

Communalities: four factors and all factors

Communalities 1.000 .418 1.000 .751 1.000 .798 1.000 .683 1.000 .492 1.000 .526 1.000 .533 1.000 .763 1.000 .695 1.000 .543 1.000 .597 1.000 .734 1.000 .655 1.000 .546 1.000 .442 1.000 .721 1.000 .699 1.000 .709 1.000 .631 CESD1 CESD2 CESD3 CESD4 CESD5 CESD6 CESD7 CESD8 CESD9 CESD10 CESD11 CESD12 CESD13 CESD14 CESD15 CESD16 CESD17 CESD18 CESD19 Initial Extraction Communalities 1.000 .966 1.000 .970 1.000 .868 1.000 .817 1.000 .919 1.000 .912 1.000 .830 1.000 .834 1.000 .812 1.000 .883 1.000 .896 1.000 .841 1.000 .861 1.000 .829 1.000 .961 1.000 .829 1.000 .828 1.000 .793 1.000 .963 CESD1 CESD2 CESD3 CESD4 CESD5 CESD6 CESD7 CESD8 CESD9 CESD10 CESD11 CESD12 CESD13 CESD14 CESD15 CESD16 CESD17 CESD18 CESD19 Initial Extraction

(21)

Rotation

Communalities: two and three factors

Communalities 1.000 .414 1.000 7.631E-02 1.000 .795 1.000 .617 1.000 .481 1.000 .489 1.000 .531 1.000 .686 1.000 .544 1.000 .539 1.000 .399 1.000 .716 1.000 .389 1.000 .540 1.000 .427 1.000 .721 1.000 .434 1.000 .587 1.000 .291 1.000 .615 CESD1 CESD2 CESD3 CESD4 CESD5 CESD6 CESD7 CESD8 CESD9 CESD10 CESD11 CESD12 CESD13 CESD14 CESD15 CESD16 CESD17 CESD18 CESD19 CESD20 Initial Extraction

Extraction Method: Principal Component Analysis.

Communalities 1.000 .418 1.000 .650 1.000 .796 1.000 .639 1.000 .491 1.000 .517 1.000 .532 1.000 .720 1.000 .669 1.000 .541 1.000 .571 1.000 .733 1.000 .392 1.000 .546 1.000 .437 1.000 .721 1.000 .449 1.000 .587 1.000 .490 1.000 .621 CESD1 CESD2 CESD3 CESD4 CESD5 CESD6 CESD7 CESD8 CESD9 CESD10 CESD11 CESD12 CESD13 CESD14 CESD15 CESD16 CESD17 CESD18 CESD19 CESD20 Initial Extraction

(22)

Rotation

Goodness-of-fit criterium

# Facts

GoF

χ

2

Df

χ

2

change

Df

χ

2_α=.05

1

444.399

170

2

278.177

151

166.222

19

30.144

3

218.422

133

59.755

18

28.869

4

170.845

116

47.577

17

27.587

5

126.361

100

44.484

16

26.296

(23)

Rotation

Unrotated Rotated

Factor Factor

Item I II III I II III

CESD01 + + + CESD02 + + CESD03 + + + CESD04 − + − CESD05 + + + CESD06 + + + CESD07 + + + CESD08 − + − CESD09 + ₋ + + CESD10 + + CESD11 + + + + CESD12 − + − CESD13 + + CESD14 + + CESD15 + + + CESD16 − + − CESD17 + + + CESD18 + + + CESD19 + + CESD20 + + +

(24)

Interpretation of table I

Unrotated Rotated

Factor Factor Item I II III I II III CESD01 + + + CESD02 + + CESD03 + + + CESD04 ₋ + ₋ CESD05 + + + CESD06 + + + CESD07 + + + CESD08 ₋ + ₋ CESD09 + ₋ + + CESD10 + + CESD11 + + + + CESD12 ₋ + ₋ CESD13 + + CESD14 + + CESD15 + + + CESD16 ₋ + ₋ CESD17 + + + CESD18 + + +

All items with high positive loadings on

the unrotated Factor I still load on this

factor after rotation, but some of them

(5, 6, 7, 9, 17, 18 and 20) also load on

factor II. This makes that Factor I is

now separated into two sublists, of

which the above could be called the

pure depression

scale while (1, 3, 10,

11, 13, 14, 15, 19) includes a sublist

indicating

less severe

symptoms of

depression.

(25)

Interpretation of table II

Unrotated Rotated

Factor Factor Item I II III I II III CESD01 + + + CESD02 + + CESD03 + + + CESD04 ₋ + ₋ CESD05 + + + CESD06 + + + CESD07 + + + CESD08 ₋ + ₋ CESD09 + ₋ + + CESD10 + + CESD11 + + + + CESD12 ₋ + ₋ CESD13 + + CESD14 + + CESD15 + + + CESD16 ₋ + ₋ CESD17 + + + CESD18 + + + CESD19 + + CESD20 + + +

Factor II now also contains all the

items that loaded negatively on Factor

I before (4, 8, 12 and 16). We called

this a factor that measures:

(26)

Interpretation of table III

Unrotated Rotated

Factor Factor Item I II III I II III CESD01 + + + CESD02 + + CESD03 + + + CESD04 ₋ + ₋ CESD05 + + + CESD06 + + + CESD07 + + + CESD08 ₋ + ₋ CESD09 + ₋ + + CESD10 + + CESD11 + + + + CESD12 ₋ + ₋ CESD13 + + CESD14 + + CESD15 + + + CESD16 ₋ + ₋ CESD17 + + + CESD18 + + +

After rotation, Factor III now contains

only item 2 (‘

I did not feel like eating:

my appetite was poor

’) and item 11

(‘

My sleep was restless

’), suggesting

that it is a factor that has to do with

physical aspects of depression.

(27)

Example of EQS input

Conclusions of EFA and CFA combined

/TITLE Cesd; /SPECIFICATIONS VARIABLES=6; CASES= 200; DATAFILE=’cesd.ess’; MATRIX=RAW; ME = ML; /EQUATIONS V1 = 2 * F1 + E1; V2 = 2 * F1 + E2; V3 = 2 * F2 + E3; V4 = 2 * F2 + E4; V5 = 2 * F3 + E5; V6 = 2 * F3 + E6; F1 = 2 * F2 + 2*F3+ D1; /VARIANCES E1 to E10 = 0.2*; D1, D2, D3 = 0.2*; /COVARIANCES F2,F3 = .5*; /END

(28)

Example of EQS input Example of EQS input

CES-D: Confirmatory factor analysis

Conclusions of the CFA

The varimax solution is not fully confirmed: item 5, 6, 7, 18

and 20 have negligible coefficients on the second factor.

The assumption of a orthogonal factor structure is

unfounded: The correlations between the factors are

(29)

CES-D: Confirmatory factor analysis

Conclusions of the CFA

The varimax solution is not fully confirmed: item 5, 6, 7, 18

and 20 have negligible coefficients on the second factor.

The assumption of a orthogonal factor structure is

unfounded: The correlations between the factors are

(30)

Example of EQS input Example of EQS input

Conclusions of EFA and CFA combined

1

The first factor is a

general depression factor

(item 1, 3, 5,

6, 7, 10, 13, 14, 15, 18, 19 and 20)

2

The second factor contains the positively formulated items

(4, 8, 9, 12, 16, 17). It represents a

general attitude

towards life

.

3

The third factor contains items that have to do with

(31)

Example: Post-traumatic stress disorder

Example of a postulated structure

(32)

Example of a postulated structure

(33)

Example of a postulated structure

(34)

Postulated structure

(35)

Recommended literature

Factor analysis:

Friendly (1995)

(

http://www.psych.yorku.ca/lab/psy6140/fa/factorbi.htm

)

Principal component analysis:

Jackson (1991)

(36)

(37)

Summary

In factor analysis the contrast between exploratory and

confirmatory approaches is quite clear: the available

techniques are different although related.

Exploratory factor analysis can well be performed with

principal component analysis, possibly combined with

Maximum likelihood factor analysis. PCA is first used to

determine the meaningful factors, MLFA for the final factor

structure.

Structural Equation Modelling makes it possible to analyze

a research problem that has been represented as a

(38)

Bollen, K. A. (1989).

Structural equations with latent variables.

New

York: John Wiley and Sons.

Friendly, M. (1995).

Annotated Factor Analysis Bibliography.

Jackson, J. E. (1991).

A user’s guide to principal components.

New

York: Wiley.

Kaplan, D. (2000).

Structural Equation Modeling. Foundations and

Extensions.

Thousand Oaks London New Delhi: Sage

Publications.

Tukey, J. W. (1977).

Exploratory data analysis.

Reading, MA:

Addison Wesley.

(http://www.psych.yorku.ca/lab/psy6140/fa/factorbi.htm