Chapter 4- Data Analysis Techniques and Procedures
4.12 Factor Scores: Further Analysis
The factors derived from EFA, can firstly be used in additional studies following confirmatory factor analysis, as part of a scale for a questionnaire within the field of mega-events, legacy and policy implementation. Adaptations can be made to the context of the funding stream, the type of organisation or mega-event involved to allow further use. This is a common use of factor analytical techniques, as the researcher starts with a large sample of individual questions and reduces these items into smaller clusters of sub scales (Pallant, 2013). Additionally, as factors are reduced into more manageable groupings of related variables this analysis is used prior to further analysis such as Multivariate Analysis Of Variance (MANOVA) or multiple regression for example (Pallant, 2013). This transition from individual Likert items into Likert scales through EFA ensured enhanced reliability, through the justification of the CLT allowing further parametric analysis of averages to be undertaken.
4.13 MANOVA
MANOVA is an extension of analysis of variance and is used by researchers when the analysis involves more than one dependent variable (Field, 2013). It is important to consider the theoretical underpinnings of the dependent variables and there must be a clear conceptual reason for considering them together (Field, 2013; Pallant, 2013). Within the current research there is a clear theoretical link between the variables, due to the literature surrounding SMEs, legacy and policy implementation that was used to aid the creation of the survey questions.
With statistical methods such as analysis of variance and T- test there are always possibilities of reaching the wrong conclusion (Pallant, 2013). There are two main error types, Type 1 and Type 2. Field (2013) states a type 1 error is when it is believed that there is a genuine effect within the data sample, when in fact there is not.
On the other hand, a type 2 error occurs when it is believed there is no effect in the population, whereas there is an effect (Field, 2013). The relationship between these two error types is problematic, since when a researcher attempts to mitigate a type 1 error, the risk of committing a type 2 error is enlarged. The aim is to ensure that the statistical tests used by the researcher correctly identify a difference between groups if there is one (Pallant, 2013). This is known as the power of a test. Stevens (1996) highlighted that the power of a test is highly related to the sample size and if the sample is over 100, the power of a test, is not a concern. Thus, due to the sample of the current study being 105 subjects, the concern of power, is eradicated. In order to reduce the likelihood of type 1 error, MANOVAs were conducted to test the multiple hypotheses for this study, which are:
H1: There will be a significant difference between Olympic and non-Olympic sports
and the way NGB managers perceive legacy production.
H0: There will be no significant difference between Olympic and non-Olympic sports
and the way NGB managers perceive legacy production.
H2: There will be a significant difference between employment levels (CEO, Senior
or regional manager) in Olympic and non-Olympic sports and the way NGB managers perceive legacy production.
H0: There will be no significant difference between employment levels (CEO, Senior
or regional manager) in Olympic and non-Olympic sports and the way NGB managers perceive legacy production.
H3: There will be a significant difference in the way NGB managers from different-
size sport’s (participation: small, medium, large) perceive legacy production.
H0: There will be no significant difference in the way NGB managers from different-
size sport’s (participation: small, medium, large) perceive legacy production.
H4: There will be a significant difference in the way NGB managers perceive legacy
production depending on their sports funding level (increased or decreased).
H0: There will be no significant difference in the way NGB managers perceive legacy
Pallant (2013) states that you could run a series of individual Analysis of Variance (ANOVAs) for each dependent variable, however this inflates the risk of type 1 error. This is because by conducting a series of individual analysis it is more likely that a significant result will be found, even if there are no differences within those groups. MANOVAs control this increased risk of a type 1 error (Pallant, 2013). By using MANOVAs an increased number of assumptions need to be considered, and these are outlined below.
The first assumption is that the two or more dependent variables being used for the analysis need to be measured at the ratio or interval level (Pallant, 2013). The four factors (Objectives, Standards and Resources, Opportunities for Capitalisation, Monitoring and Evaluation and Club Engagement and Implementation) meet this assumption. Laerd Statistics (2016) states that the second assumption is that your independent variables should consist of two or more categorical, independent groups. The four groups being used for the MANOVA analysis highlighted below meet this requirement:
- Sport Type (Olympic and non-Olympic sports)
- Funding level (increased, decreased5)
- Sport size (participation: small [under 49,999], medium [50,000-299,999 or large [300,000+])
- Employment type (CEO, senior or regional manager)
The third assumption states that there needs to be independence of observations, meaning that the datasets are independent from each other (Field, 2013). With the current data set this assumption is met, as the same participant cannot be in more than one group and as the observations were not collected in a group setting (Laerd Statistics, 2016; Pallant, 2013). The researcher did note that in some cases multiple responses were collected from the same NGB, however this was often at multiple levels within the organisation. This was also due to the survey being online and delivered on an individual opinion basis, meaning risk to independence within the samples is limited. Therefore, it is worth noting some organisations refused
5 The third category in which funding ‘stayed the same’ was removed due to a small sample n=4,
participation as they were not happy for their staff to complete the survey, due to upcoming funding decisions, as well as, time and resource restrictions. This reinforces the independence of samples and cases, where individuals could have participated and being scripted in their responses were removed.
The fourth assumption that must be considered when understanding MANOVAs relates to having an adequate sample size. Pallant (2013) states that as a minimum, you are required to have more cases in each group than the number of dependent variables being analysed. Hence, in this research a minimum value of four was needed, yet the sample (n=105) exceeds this. Furthermore, by having a larger
sample this can also allow the researcher to “‘get away with’ violations of some other assumptions (e.g. normality)” (Pallant, 2013, p.295). The fifth assumption that must
be accounted for by the researcher is to ensure there are no univariate or multivariate outliers. Univariate outliers were checked using SPSS software through the ‘Explore’ function (Pallant, 2013). Reflecting on the histograms and boxplots the distribution of scores were reasonably normal, with most scores following the shape of a normal curve. The histogram looked somewhat normally distributed and the boxplot showed a few outliers, nonetheless when the data set was checked all respondents were all from the expected sample group and the responses where within the expected response range (between 1 and 7). The Kolmogorov-Smirov for two of the dependent variables
(“Opportunities for Capitalisation” and “Club Engagement and Implementation”)
were 0.00, thus violating normality.
Efron and Tibshirani (1993) suggest that to overcome this deviation from normality, bootstrapping can be used. For this research bootstrapping was undertaken through the SPSS 21 programme, as this technique does not rely on the assumptions of normality being achieved. Bootstrapping estimates the sample distribution from the
sample of data used within the research. In effect, the sample data (n=105) are “treated as the population from which smaller samples, known as bootstrap samples are taken”
(Field, 2013, p.163). Through this method, a mean is calculated from each sample,
then by using many samples (SPSS’s recommended default setting of 1,000 iterations
was used) the sampling distribution can be estimated (IBM, 2012). This process is
known as a ‘robust method’ and ensures the data is accurate when some assumptions
are not met or are in doubt (Field, 2013; IBM, 2012). Due to the sample size of the current investigation, bootstrapping was used to ensure that robust estimates were
derived, as well as to provide reassurance in the case of any hesitation or doubt concerning the assumptions for the statistical procedures.
Multivariate outliers, were checked using a measure called Mahalanobis distance, which is calculated through SPSS (Pallant, 2013). The Mahalanobis distance allows the researcher to identify if any cases have a strange pattern of scores across the dependent variables. Tabachnick and Fiddell (2013) state that the Mahalanobis distance highlights the distance of an individual case, from the centroid of all other cases and the centroid is the point that is created by the means of all the variables within the analysis. When identifying if there are any multivariate outliers, the
researcher compared the “Mahalanobis distance value against a critical value, which
is determined using a chi-square table” (Pallant, 2013, p.298). If the Mahalanobis distance value exceeds the critical value, it was considered to be an outlier. The critical values table (Pallant, 2013, p.298) was used to test for multivariate outliers in this investigation and as this study had four dependent variables, a critical value of 18.47 was used. Hence, as the maximum value obtained from the output was 15.6, this was below the critical value, meaning no multivariate outliers were present. The sixth assumption to consider is whether there is a linear relationship between each pair of dependent variables for each group of independent variables. Linearity between two variables is assessed by reviewing bivariate scatterplots. When the variables are linearly related and are normally distributed, the scatterplots are oval shaped (Tabachnick and Fiddell, 2013). After reviewing the scatterplots, there is no obvious evidence of non-linearity meaning the assumption of linearity was satisfied.
The seventh assumption relates to the homogeneity of variance-covariance matrices and this is produced as part of the MANOVA output. The test used to assess homogeneity of variance is Box’s M test of Equality of Covariance Matrices (Pallant, 2013). If the significance value is larger than .001, the assumption of homogeneity of variance-covariance matrices, has not been violated. The Box’s M Sig. value was
above .001 for three of the MANOVA’s conducted (funding level .023, sport size .611
and employment type .921), yet for sport type the Box’s M Sig. value was .000, thus violating the assumption. Pallant (2013) states that the next step is to look at Levene's Test of Equality of Error Variance, and any values that are less than .05, would indicate
a violation. This was the case for one of the dependent variables (‘capitalise’ .000)
Tabachnick and Fiddell (2013) suggest that when this assumption is violated a more conservative alpha value is needed to determine the significance within that variable. As such, an alpha value of .01 was set to account for this assumption violation, when
analysing the data for the dependent variable ‘capitalise’ within the sport type
MANOVA.
The final assumption relates to multicollinearity. MANOVAs are most effective when the dependent variables are moderately correlated (Pallant, 2013). If the correlations are low, running separate univariate analysis of variance could be used. Similarly, correlations that are too high around .8 or .9 are cause for concern (Pallant, 2013). To test for this a correlation was run through SPSS. The correlations ranged between 0.29-0.44, which is deemed acceptable and satisfies an acceptable level of multicollinearity. Similar to the study by Doherty (2009) on volunteer legacy
from a major sports event, Pillai’s Trace F-values were inspected, as they are robust
against violations of homogeneity of variance which may occur when there are unequal N values, or if there is a small sample or any assumptions are violated (Pallant, 2013; Tabachnick and Fidell, 2001).
4.14 Conclusion
This chapter outlined and illustrated the analysis techniques that were used for this investigation, both descriptive and statistical. Firstly, this chapter reviewed the processes involved with descriptive data, in terms of variable measurement and Likert items and Likert scales. Then the chapter defined the processes involved and the justification for the selection of EFA and MANOVA. Chapters 6 and 7, provide more content on the analysis and the specific findings themselves that were a result of the data analysis methods applied to this research.