Sample Size Estimation

In structural equation modeling, sample size is usually estimated by power analysis. The main framework of power analysis in model fit evaluation is based on the test of close fit or not close fit (MacCallum et al., 1996, 2006). Sample size should be large enough so that severely misspecified model can be rejected with sufficient power for the test of close fit or the trivially misspecified model can be retained with sufficient power for the test of not close fit. Note that power analysis can be done based on the significance testing of specific parameters using a Monte Carlo simulation

(Muthén & Muthén, 2002). Another approach to estimate sample size is based on the width of the confidence interval for parameter estimates (Lai & Kelley, 2011) or RMSEA (Kelley & Lai, 2011). The confidence interval should be sufficiently narrow to achieve accurate estimates of parameters and avoid the inconclusive result (i.e., confidence interval brackets the maximal trivial misspecification value) in test of close fit / not close fit. I will show that both approaches for sample size estimation are applicable for the unified approach.

Based on the simulation studies provided above, larger sample size decreases the proportion of inconclusive results in the local fit evaluation. The global fit evaluation is less sensitive to sample size and takes long computational time so I recommend to ignore the sample size estimation for the global fit evaluation. For power analysis, the unified approach provides two types of statistical power: the power in retaining trivially misspecified models and the power in rejecting severely misspecified models. Thus, researchers should provide two models: a trivially misspecified model

that researchers really wish to retain with a high rate (referred to as MT) and a severely misspecified

model that researchers really wish to reject with a high rate (referred to as MS). For example,

researchers may specify the level of maximal trivial misspecification as the measurement error

correlations of .2. They may set MT and MSas a model with the measurement error correlations of

.1 and .5, respectively. Researchers may use the repeated sampling method to find MT or MS(see

Steps 2 and 5 of the unified approach in Chapter 3, respectively).

Then, multiple data sets with a given sample size can be created from MT and MS. Then, the

local fit evaluation can be used to check the proportion of inconclusive results from MT and MS.

Researchers can adjust the sample size until they achieve their desired proportions of inconclusive

results, such as less than .2 for both MT and MS. Researchers may consider different types of

misspecifications depending on their research purposes. For example, as suggested in Wu et al. (2009), misspecification in the mean structure and covariance structure should be both investigated in power analysis of growth curve model.

The local fit evaluation uses confidence intervals of EPCs to evaluate models. The width of confidence intervals should be narrow enough for two purposes: (a) to get accurate EPCs (or other

parameter estimates in a model) or (b) to lower the chance of getting the inconclusive or under- powered results. I will focus on the latter objective. The width of confidence intervals of EPCs should be narrow enough to avoid inconclusive results. Initially, researchers need to find the de- sired width of the confidence intervals of EPC. This can be determined by the level of maximal

trivial misspecification and the specification of MT and MS. Then, for each fixed parameter, the

differences between the maximal trivial misspecification and the values from MT and MSare com-

puted. These differences represent one side of the confidence intervals of EPCs. The smallest difference is picked and the value is multiplied by 2. The result is the desired width. For example,

for a measurement error correlation, the maximal trivial misspecification is .2 and the values on MT

and MSare .1 and .5 so the differences are .1 and .3, respectively. The desired width is .1 × 2 = .2.

If MT and MSare not specified, researchers may specify the width that avoids underpowered results

such that the width is less than the range of trivial misspecification. For example, if the maximal trivial misspecification is .2, the desired width is .4 or less.

A Monte Carlo simulation can be used to estimate sample size to get the expected width of the confidence intervals of EPC. The procedure is similar to power analysis described above. First,

researchers generate multiple data sets from MT and MS. Then, these simulated data sets are are

fitted by the hypothesized model. The width of the confidence intervals of EPCs are calculated. Researchers can find the average of the widths and adjust their sample size values until the desired widths of all confidence intervals of EPCs are achieved. Note that the observed width of the confidence interval is random across repeated sampling. Researchers may want to make sure that the observed width is narrower the desired width by C%, which is referred to as degree of assurance (Kelley & Lai, 2011; Kelley & Rausch, 2006). To find the sample size for a certain degree of assurance, researchers find the C-th percentile value of the widths across simulated samples. Next, sample size is adjusted until the C-th percentile value of the width is equal to the desired width. Then, if researchers obtain a sample with the estimated sample size, there is C% probability that they get confidence intervals narrower than the desired width.