Controlling for Common Method Variance in PLS Analysis: The Measured Latent Marker Variable Approach

Controlling for Common Method Variance in PLS Analysis:

The Measured Latent Marker Variable Approach

Wynne W. Chin Jason Bennett Thatcher

Ryan T. Wright

Douglas J. Steel

Outline

• Common Method Bias

• Study 1: Evaluating Unmeasured Latent Marker Variable Approach

• Study 2: A Measured Latent Marker Variable Approach

• Conclusions

Common Method Bias

• Several authors have attempted to create

approaches for addressing common method bias in SEM which have been applied to PLS

• Williams et al (1989)

• Podsakoff et al (2003)

Study One: Evaluating UMLC Approach in PLS

• Liang et al (2007) created a PLS Unmeasured Latent Marker Construct (UMLC) approach to control for common method variance.

• Constructing a Model

– Take all the indicators for each construct and reusing them to create single indicator constructs.

– Link the original constructs to their respective single indicator constructs.

– The method construct consisting of all indicators used in

the study is linked to all the single indicator constructs

Liang ULMC Approach in PLS

• Evaluating Common Method Variance

– Estimate a model using bootstrapping

– Compare the statistical significance of loadings on the method factor

– Examine variance explained in loadings and constructs

• Squared variance of the method loadings was

interpreted as variance explained by common method

– Lack of significant loadings & smaller method

variances viewed as indicators of absence of CMB

Problem: ULMC Not vetted

• Issues with Liang et al

– No proofs

– No simulations

– No evidence that it worked

• Issues with UMLC

– Richardson et al (2009) demonstrated through a

series of simulations that it rarely worked in ML

SEM.

Evaluating the ULMC Method

Evaluating the ULMC Method

• Monte Carlo simulations

– 500 datasets of 5,000 in prelis

• Method bias of different forms and at different levels

– Congeneric

– Non-Congeneric

• Estimated models using PLSGraph 3.0

Table 3a. Summary Results of the Scenarios’ PLS ULMC Analysis *

Scenario S1 S2 S3 S4 S5

Path Mean Std. Dev. Mean Std. Dev. Mean Std. Dev. Mean Std. Dev. Mean Std. Dev.

X →Y (true score) 0.596 0.009 0.596 0.009 0.599 0.009 0.601 0.009 0.595 0.009

XX→YY 0.495 0.011 0.636 0.008 0.748 0.006 0.64 0.008 0.752 0.006

XX→A1 0.758 0.019 0.796 0.017 0.927 0.013 0.869 0.014 0.965 0.007

XX→A2 0.741 0.019 0.884 0.017 0.909 0.013 0.874 0.013 0.965 0.007

XX→A3 0.773 0.019 0.834 0.018 0.952 0.013 0.844 0.017 0.949 0.014

XX→A4 0.732 0.017 0.834 0.018 0.951 0.013 0.828 0.018 0.921 0.013

XX→A5 0.769 0.019 0.853 0.016 0.941 0.01 0.813 0.02 0.927 0.018

XX→A6 0.779 0.019 0.85 0.017 0.933 0.013 0.822 0.021 0.884 0.019

YY→B1 0.734 0.019 0.842 0.017 0.908 0.013 0.876 0.014 0.956 0.006

YY→B2 0.756 0.018 0.842 0.016 0.935 0.014 0.889 0.013 0.956 0.006

YY→B3 0.758 0.017 0.849 0.018 0.938 0.013 0.823 0.017 0.919 0.014

YY→B4 0.746 0.019 0.867 0.018 0.928 0.014 0.846 0.018 0.943 0.013

YY→B5 0.752 0.018 0.802 0.016 0.951 0.013 0.806 0.022 0.897 0.018

YY→B6 0.782 0.018 0.85 0.017 0.946 0.014 0.799 0.02 0.952 0.018

M (Method)→A1 0.015 0.021 -0.04 0.019 0.025 0.014 0.03 0.014 0.021 0.007

M→A2 -0.006 0.021 0.053 0.018 0.01 0.014 0.034 0.015 0.021 0.007

M→A3 -0.006 0.02 0 0.02 -0.016 0.012 0 0.018 -0.015 0.015

M→A4 0.034 0.018 0.01 0.019 -0.015 0.014 0.014 0.019 0.018 0.014

M→A5 -0.013 0.021 -0.011 0.018 -0.008 0.01 -0.039 0.021 -0.047 0.019

M→A6 -0.023 0.021 -0.012 0.019 0.004 0.014 -0.051 0.023 -0.003 0.019

M→B1 0.011 0.021 0 0.017 0.03 0.014 0.026 0.015 0.032 0.007

M→B2 -0.003 0.019 0.001 0.018 -0.002 0.014 0.011 0.014 0.032 0.007

M→B3 0.002 0.017 -0.007 0.019 -0.004 0.014 0.018 0.019 0.018 0.014

M→B4 0.019 0.021 -0.028 0.019 0.007 0.015 -0.004 0.019 -0.003 0.014

M→B5 0.002 0.02 0.043 0.018 -0.018 0.014 -0.038 0.023 -0.014 0.018

M→B6 -0.03 0.02 -0.009 0.018 -0.013 0.015 -0.021 0.021 -0.072 0.019

* Scenario 1 (S1) = Latent Item Loadings (LIL) are noncongeneric (NC), Method Loadings (ML) are 0, S2 = LIL are NC, ML are NC at .4,

S3 = LIL are NC and ML are NC at .6,

S4 = LIL are congeneric (C) and ML are NC at .4, S5 = LIL are C and ML are NC at .6.

Table 3b. Summary Results of the Scenarios’ PLS ULMC Analysis *

Scenario S6 S7 S8 S9 S10

Path Mean Std. Dev. Mean Std. Dev. Mean Std. Dev. Mean Std. Dev. Mean Std. Dev.

X →Y (true score) 0.587 0.009 0.596 0.009 0.596 0.009 0.596 0.009 0.599 0.009

XX→YY 0.635 0.008 0.646 0.008 0.495 0.011 0.636 0.008 0.748 0.006

XX→A1 0.905 0.02 0.953 0.017 0.752 0.007 0.847 0.006 0.936 0.005

XX→A2 0.926 0.021 0.924 0.018 0.753 0.006 0.85 0.005 0.93 0.005

XX→A3 0.896 0.017 0.856 0.017 0.768 0.006 0.832 0.006 0.94 0.004

XX→A4 0.88 0.015 0.866 0.017 0.762 0.006 0.843 0.006 0.945 0.005

XX→A5 0.751 0.013 0.723 0.018 0.757 0.006 0.841 0.006 0.93 0.005

XX→A6 0.734 0.014 0.71 0.017 0.759 0.007 0.838 0.006 0.933 0.004

YY→B1 0.878 0.019 0.935 0.018 0.743 0.007 0.843 0.006 0.933 0.004

YY→B2 0.936 0.021 0.933 0.018 0.754 0.006 0.841 0.006 0.929 0.005

YY→B3 0.866 0.017 0.863 0.017 0.76 0.006 0.843 0.006 0.935 0.005

YY→B4 0.869 0.017 0.833 0.017 0.762 0.006 0.847 0.006 0.931 0.005

YY→B5 0.746 0.014 0.729 0.018 0.754 0.006 0.833 0.006 0.944 0.005

YY→B6 0.762 0.013 0.736 0.017 0.756 0.007 0.844 0.006 0.934 0.005

Method (M)→A1 -0.184 0.022 -0.117 0.019 0.001 0.01 -0.005 0.009 0.005 0.007

M→A2 -0.155 0.022 -0.144 0.019 0.001 0.009 -0.007 0.009 0.001 0.007

M→A3 -0.058 0.019 -0.01 0.018 0.008 0.009 0.006 0.009 -0.004 0.006

M→A4 -0.032 0.017 -0.024 0.018 0 0.01 0 0.009 -0.014 0.007

M→A5 0.173 0.014 0.135 0.019 -0.012 0.009 0.004 0.009 0.005 0.007

M→A6 0.191 0.015 0.149 0.018 0.003 0.009 0.003 0.009 0.006 0.006

M→B1 -0.132 0.021 -0.118 0.019 0.013 0.01 -0.002 0.009 0.005 0.006

M→B2 -0.203 0.022 -0.129 0.019 0.007 0.009 0.003 0.008 0.007 0.007

M→B3 -0.027 0.018 -0.02 0.019 -0.008 0.009 -0.002 0.009 -0.001 0.007

M→B4 -0.033 0.018 0.01 0.018 0 0.009 -0.012 0.009 0.005 0.007

M→B5 0.173 0.015 0.125 0.018 -0.01 0.01 0.019 0.009 -0.015 0.007

M→B6 0.158 0.014 0.123 0.018 -0.001 0.01 -0.006 0.009 -0.001 0.007

* Scenario 6 (S1) = Latent Item Loadings (LIL) are noncongeneric (NC), Method Loadings (ML) are congeneric (C) at an ave. of .4, S7 = LIL are C, ML are C at an average of .4,

S8 = LIL are NC and ML are represented by the method (M) score at 0, S9 = LIL are NC and ML are represented by the method (M) score at .4, S10 = LIL are NC and ML are represented by the method (M) score at .6.

Conclusions of Part One

• Where Richardson et al (2009) found that a UMLC approach had limited utility for

detecting CMB using ML SEM, the same

technique applied to PLS had no ability to

detect and control for CMB.

Part Two: Measured Latent Marker Variable Approach

• A measured latent marker variable approach uses a set of unrelated items to each other or to the constructs of interest.

• By doing so, we capture the just the variance attributable to method, not just to covariance

among theoretically connected latent constructs.

• Can perform construct or item level corrections.

• Illustrate using simulation with the same

parameters as Study 1.

Creating Indicators for MLMV

• Each indicator must not be in the same domain as constructs found in the research model.

• Each indicator must be drawn from different unit of analysis than that investigated in the research model.

• Rather than reliability, ensure all unique and

error variances are independent among the set of measures chosen

• The MLMV must include a minimum of 4 items.

• A well-designed survey should include these

indicators at the end of the instrument.

Construct Level Corrections

• Create as many CMV control constructs as there are constructs in the model.

• Each CMV control uses the same entire set of MLMV items.

• CMV construct is modeled as impacting each model construct.

• The residuals obtained now represent the model constructs with the CMV effects

removed.

PLS Estimates Using CMV of .36

Construct Level Correlations

• Enter the 12 MLMV as controls by creating a CLC construct for the two model constructs

• Results in reducing the inflated path of 0.741 to more closely match the population parameter with an estimate of 0.606

• This represents the impact of construct XX on construct YY holding CLC constant.

• CLC scores are then used partial out the CMV from both constructs to obtain the partial

correlation between XX and YY.

First Step in CLC

PLS Estimates using items with 0.36

Assessment of Construct Level Correlations

Table 1. Number of Latent Marker Measures and Percentage Reduction in CMV Using CLC

# of MLMV

Items in CLC 12 11 10 9 8 7 6 5 4 3 2 1

Structural

Estimate 0.6 0.604 0.608 0.613 0.616 0.619 0.624 0.63 0.639 0.652 0.67 0.696 Percent

Reduction 100% 97% 94% 91% 89% 87% 83% 79% 72% 63% 50% 32%

While a 12-item CLC effectively captured the simulated CMV, our simulation

illustrates that one can use a four item LMV to remove 72 percent variance due to CMV.

Given that researchers tend to have limited space on survey instruments to

include additional items, these results illustrate that our MLMV approach is flexible

enough to be included in surveys of varying lengths.

Item Level Corrections

• Use the MLMV items to partial out the CMV effects at the measurement item level.

• Each item measure is regressed on the entire set of MLMV items.

• The residuals for each item now represent the construct items with the CMV effects

removed.

Item Level Corrections

• The CMV should be replaced with an equivalent amount of random error to be equivalent to the variance of the original measures sans bias.

• Must obtain an assessment of reliability of the original items by assessing the reliability of

original items

– R-square obtained from each item to MLMV regression is used.

– Specifically, the square root of the R-square multiplied with a number drawn from a normal distribution of mean 0 and standard deviation of 1 is added to each item residual.

– This represents the final ILC items used in a PLS

analyses.

Item Level Correlations

• Results in item loadings that are more consistent with a PLS analyses without CMV effects.

• The estimated loadings varying from 0.76 to 0.789 are consistent with the tendency of PLS to overestimate the loadings by approximately 10 percent.

• The estimated structural path of 0.552 is consistent with an approximate 10 percent underestimation of the population parameter of 0.60.

• To correct this, you need to add noise back into the

model.

Results of ILC Approach

CLC and ILC Results

• Overall, both approaches seem to converge to the same results.

• With CLC, we obtain an accurate estimate of the path estimate at the expense of the loadings.

• With ILC, we obtain more accurate item loading estimates at the expense of the structural path.

– But, the path estimates can be obtained if we

compensate for the CMV partialed out.

Results of ILC with Error Compensation

Varying Level and Form of Error

• Introduced different levels and forms of error into the simulations.

• Two items for each construct had true score and method loadings of 0.8 and 0.2.

• Two items were set at 0.7 and 0.4

• Two items were set at 0.6 and 0.6 (i.e., equal

amounts of true and method effects).

Results with Varying Level

of Trait and Method

Results of CLC with Different

Trait and Method Impact

Results of ILC with Different Trait and

Method (No Error Compensation)

Results of ILC Different Trait and

Method with Error Compensation

Comparing Approaches

Unmeasured Latent Marker Measured Latent Marker

Post-Hoc A Priori

Unplanned Carefully planned by researcher

Item Level Item and Construct Level

Based on same set of items Items drawn at the same time;

yet span completely different sets of knowledge

Low effort Modest effort

Discredited, unvetted approach

(Chin et al, Forthcoming) Rigorously vetted via

simulation; collecting field data

Conclusion

• We have presented initial evidence of two methods for correcting for common method bias in PLS Path Modelling.

• CLC is more easily implemented.

• ILC yields more insight into CMB influence in the measurement model.

• Approach must be tailored to each study.