Table 14 t-test Mean Difference t df p (2-tailed) 95% Confidence Interval of the Difference Lower Upper Weighted-Indicator Scores Protocol 0.000 0.000 34 1.000 -0.344 0.344 Multilevel Achievement Estimation Protocol 0.011 0.064 33 .949 -0.336 0.358 Correlations
The examination of relationships between estimation models, in terms of Pearson correlations, revealed that statistically significant relationships existed between all the models, Primary School Examinations, the Base-Model, Weighted-Indicator Scores Protocol (WISP), and Multilevel Achievement Estimation Protocol (MAEP). The
relationship between Primary School Examinations and the Base-Model was (r = .829, p < .001), and between Primary School Examinations and WISP, the correlation was (r = .903, p < .001) (Table 15). The relationship between Primary School Examinations and MAEP (r = .354, p < .05) was also statistically significant, but at the .05 level. Similarly, the relationships between the Base-Model and the WISP (r = .696, p < .001) and with the MAEP (r = .792, p < .001) were also statistically significant at the .01 level (Table 15).
Table 15
Correlations between Standardized PSE and Model Estimates
PSE BASE WISP MAEP
PSE 1
BASE .829** 1
WISP .903** .987** 1
Primary School Examinations (PSE), the Base-Model (BASE), Weighted-Indicator Scores Protocol (WISP), and Multilevel Achievement Estimation Protocol (MAEP)
** Correlation is significant at the .01 level (1-tailed). * Correlation is significant at the .05 level (1-tailed). Listwise n = 33, M = 0, SD = 1
Multiple Regression Analysis
The final measure to determine if there was a difference between the two models, Weighted-Indicator Scores Protocol (WISP) and Multilevel Achievement Estimation Protocol (MAEP), was to identify which was the better predictor of Primary School Examinations. A multiple regression was run to predict Primary School Examinations with WISP and MAEP as the predictor variables.
The regression model met the assumptions for: continuous variables, linear distribution, no apparent outliers, independence of observation, homoscedasticity, and normally distributed residual errors. As previously established, the two predictor variables were significantly correlated. Therefore, there were low levels (within acceptable ranges) of autocorrelation (Durbin-Watson = 1.97) for the model. The negative value in the Beta statistics for MAEP (β = -.532, p < .05) was within reason since the multicollinearity statistic (VIF=1.939) was not at a level that warranted concern (Table 16).
It was determined that both WISP and MAEP predicted Primary School
Examinations at a statistically significant level, (F(2, 30) = 366.678, p < .001 , R2 = .961). Together, both WISP and MAEP added significantly to the model (p < .001) and the adjusted R2 indicating 95.8% of the variance accounted. Nevertheless, no significant
differences were teased out to determine which of WISP and MAEP was the better predictor.
Table 16
Regression with WISP and MAEP as the Predictor Variables Model Summary
R R2
Adjusted R2 SE
Change Statistics Durbin- Watson R2 F df1 df2 p .980 .961 .958 .202 .961 366.678 2 30 .000 1.957 Coefficients Unstandardized Standardized t p VIF B SE Beta (constant) -.046 .035 -1.300 .204 1.939 WISP 1.266 .050 1.273* 25.257 .000 MAEP -.560 .053 -.532* -10.560 .000
Dependent variable: Primary School Examinations (PSE)
Predictor variables: Weighted-Indicator Scores Protocol (WISP), Multilevel Achievement Estimation Protocol (MAEP)
* Statistically significant at the .05 level
Indications from the various measures of differences between the standardized estimates for WISP and MAEP did not yield either apparent or statistically significant results. Differences were evaluated using descriptive statistics (including score ranges and rank-order comparisons). Differences were also measured in terms of standard deviations, relationships described by Pearson correlations, a t-test, and a linear
regression (predicting the 2018 Primary School Examinations). It was determined that the School-Level Achievement estimates derived from WISP and MAEP were not
differences in the overall estimates for school-level educational achievement using a latent approach with a MAEP versus a classic summative approach with the WISP. The estimates produced by both methods were very similar and equally viable for estimating School-Level Achievement with the kind of sample assessed (a school district in Belize). Question 2
Were there differences in the School-Level Achievement estimates by selected
school characteristics such as School Type (Catholic, Non-parochial, or Other Denominations) or Location (Urban, Rural, or Remote)?
The analyses to answer the question were in two parts, first to determine if School Type impacted educational achievement with statistical significance (Table 17) and, second, to understand whether educational achievement was impacted by physical location (Table 18).
Using one-way analyses of variance tests (ANOVA) of school-level achievement, it was determined that the differences were not statistically significant for Catholic, Non- parochial, or Other Denominations, regardless of which mode of estimation was used. The School Type, or schools managed by Catholic, Non-parochial, or Other
Denominations, did not appear to have statistically significant bearing on the School- Level Achievement estimates derived using Weighted-Indicator Scores Protocol (WISP) (F(2, 30) = 1.198, p > .05) Similarly, the differences in School-Level Achievement estimates derived from Multilevel Achievement Estimation Protocol (MAEP) were not statistically significant by School Type, or schools managed by Catholic, Non-parochial, or Other Denominations (F(2, 30) = 2.48, p > .05). Therefore, membership or
school-level educational achievement. Statistically, classification by School Type accounted for 7.4% (η2 = .074) of total variance in the School-Level Achievement
estimates derived using Weighted-Indicator Scores Protocol, and 14.0% (η2 = .140) of variance in the School-Level Achievement estimates derived via the Multilevel Achievement Estimation Protocol (Table 17).
Table 17
Differences between Model Estimates by School Type
School Types Sum of
Squares
df Mean Square
F p η2
WISP
Between School Types 2.337 2 1.169 1.198 .316 .074 Within School Types 29.266 30 0.976
Total 31.603 32 Sum of Squares df Mean Square F p η2 MAEP
Between School Types 3.957 2 1.978 2.448 .104 .140 Within School Types 24.249 30 0.808
Total 28.206 32
School-Level Achievement estimates derived from: Weighted-Indicator Scores Protocol (WISP), Multilevel Achievement Estimation Protocol (MAEP)
Eta squared (η2) provides indication of total variance in the School-Level Achievement estimates associated with the School Type classification: Catholic, Non-parochial, or Other Denominations
The location of a school was revealed as having a statistically significant effect on the School-Level Achievement estimates, where a school was located mattered. There were statistically significant differences in the School-Level Achievement estimates derived using WISP (F(2, 30) = 3.365, p < .05). A school’s location in Urban, Rural, or Remote Settings accounted for 18.3% (η2 = .183) of the total variance in the estimate as determined in a one-way ANOVA (Table 18). For the School-Level Achievement
estimates derived using the MAEP, the one-way ANOVA also revealed a statistically significant difference based on a school’s location (F(2, 30) = 4.494, p < .05). Using the MAEP, it was determined that 23.1% (η2 = .231) of the total variance was attributable to location: Urban, Rural, or Remote (Table 18).
A school’s location mattered with statistical significance (p < .05) in terms of School-Level Achievement regardless of the protocol used. Subsequently, a Tukey honestly significant difference (Tukey HSD) post hoc test on the location was conducted to establish statistical significance by location category. The Tukey HSD revealed that the differences in School-Level Achievement were statistically significant between the Urban (higher) and Remote (lower) schools using both WISP and MAEP (p < .05) achievement estimates (Table 18).
For the achievement estimates from the Multilevel Achievement Estimation Protocol, there was also a statistically significant difference between the Rural and Remote locations (p < .05), but not for WISP (p >.05). Comparatively, for the schools located in urban and rural areas, there were no statistically significant differences with WISP estimates (p > .05) or with the MAEP estimates (p > .05) (Table 18).
Table 18
Differences between Model Estimates by School Location Sum of Squares df Mean Square F p η2 WISP Between Location 5.790 2 2.895 3.365* .048 .183 Within Location 25.813 30 .860 Total 31.603 32 MAEP Between Location 6.503 2 3.251 4.494* .020 .231 Within Location 21.703 30 0.723 Total 28.206 32
Tukey HSD Post Hoc Test
(I) Location (J) Location
(I-J) Mean Difference SE p WISP Urban Rural 0.356 .379 .620 Urban Remote 1.125* .440 .041 Rural Remote 0.769 .406 .158 MAEP Urban Rural 0.145 .347 .909 Urban Remote 1.113* .403 .026 Rural Remote 0.968* .372 .037
School-Level Achievement estimates derived from: Weighted-Indicator Scores Protocol (WISP), Multilevel Achievement Estimation Protocol (MAEP)
Eta squared (η2) provides indication of total variance in the School-Level Achievement estimates
associated with the location classifications: Urban, Rural, Remote * Statistically significant
Therefore, the response to the second question follows that there were no
differences in the School-Level Achievement estimates revealed between the three school types. However, statistically significant differences in the School-Level Achievement estimates existed in both the WISP and the MAEP estimates between the location groupings.
Question 3
How much variance in school-level academic outcomes was accounted for by selected Educator-Efficacy, Educators’ Perceptions of Their School Environment, and the Opportunity to Learn variables?
The response to this question was broken down into three sets of analyses, one for each of the academic outcome variables addressed in this study: Primary School
Examinations (Table 19), School-Based Averages (Table 20), and School-Rigor (Table 21). In each analysis (Stepwise), the independent variables (predictors) were Educator Efficacy, Opportunity to Learn, and Educators’ Perceptions of Their School
Environment. The “probability-of-F-to-enter ≤ .050, probability-of-F-to-remove ≥ .100” were the Stepwise criteria used. The Durbin-Watson test was used to detect
autocorrelation, or similarities across series that can lead to underestimation of the standard error and false positive significance for predictors. The generally acceptable range for the Durbin-Watson test is between 1.5 and 2.5, with 2.0 indicating no autocorrelation, less than 2.0 as positive, and greater than 2.0 as negative (Glen et al., 2016). Multicollinearity, or the redundancy caused by predictor variables that are highly correlated or could be used to predict the other, was reported in terms of variance inflation factor (VIF). Uncorrelated predictors have VIF = 1.0; the generally acceptable range is between 1.0 and 5.0, while VIF > 5.0 indicates unacceptable highly correlated predictors (Glen, 2015).
Predicting Primary School Examinations
In the model with 2018 Primary School Examinations as the academic outcome variable, both Educator Efficacy and Educators’ Perceptions of Their School
Environment did not meet the criteria (p > .05) and therefore were excluded. In the final model that met criteria for predicting Primary School Examinations, Opportunity to Learn was the only variable that predicted Primary School Examinations with statistical significance (R2 = .199, F (1, 33) 8.175, p < .05) (Table 19). The model showed a slightly negative autocorrelation (DW= 2.3), which was within the acceptable range. No
multicollinearity was detected (VIF = 1.00). Therefore, given the sample of schools in this study, Opportunity to Learn accounted for 19.9% of the total variance in School- Rigor with statistical significance. That Opportunity to Learn matters for school-level achievement was reflected in school-level averages for Primary School Examinations.
Table 19
PSE as the Academic Outcome Variable and EFF, OTL, and PRC as Predictors Model Summary R R2 Adj. R2 SE F df1 df2 p DW .446 .199 .174 .909 8.175 1 33 .007 2.313 Coefficients Model Unstandardized Standardized t p VIF B SE Beta EFF^ .136 0.862 .395 .985 OTL .446 .162 .446 2.859 .007 .100 PRC^ .018 0.095 .925 .683
Dependent variable: Primary School Examinations (PSE) Predictor: Opportunity to Learn (OTL);
^Excluded (p>.10): Educator Efficacy (EFF), Educators’ Perceptions of Their School Environment (PRC)
Predicting School-Based Averages
With School-Based Averages as the academic outcome variable, in the Stepwise regression analysis both Educator Efficacy and Opportunity to Learn variables did not meet the criterion (p > .05), thus excluded. In the model that was retained, autocorrelation and multicollinearity were detected, but within the acceptable ranges (DW = 1.56, VIF = 1.0). In this model to predict School-Based Averages, Educators’ Perceptions of Their School Environment was the only statistically significant predictor variable (R2 = .200,
F(1, 34) 8.513, p < .05) (Table 20). Educators’ Perceptions of Their School Environment accounted for 20.0% of the variance in School-Based Averages. Therefore, given the schools sampled in this study, Educators’ Perceptions of Their School Environment matters for school-level educational achievement, as reflected in School-Based Averages.
Table 20
SBA as the Academic Outcome Variable and EFF, OTL, and PRC as Predictors Model Summary R R2 Adj. R2 SE F df1 df2 p Durbin- Watson .447 .200 .177 .907 8.513 1 33 .006 1.558 Coefficients Model Unstandardized Standardized t p VIF B SE Beta EFF^ -.305 -2.021 .051 .946 OTL^ .147 0.774 .445 .661 PRC .447 .153 .447 2.918 .006 1.00
Dependent variable: School-Based Averages (SBA)
Predictor: Educators’ Perceptions of Their School Environment (PRC) ^Excluded: Educator Efficacy (EFF), Opportunity to Learn (OTL) Method: Stepwise, n = 35
Predicting School-Rigor
In the third Stepwise regression analysis, in which School-Rigor was the academic outcome variable, neither Educator Efficacy nor Educators’ Perceptions of Their School Environment met the input criteria. In the model that met the Stepwise criteria, autocorrelation was detected, but within the acceptable range (DW = 2.38). No multicollinearity was detected (VIF = 1.00). In this model, the Opportunity to Learn was the only variable of the three that predicted School-Rigor with statistical significance (R2
= .173, F(1, 33) 6.893, p < .05) (Table 21). In other words, given the schools sampled in this study, Opportunity to Learn accounted for 17.3% of the total variance in School- Rigor. Opportunity to Learn mattered for school-level educational achievement as reflected in School-Rigor.
Table 21
School-Rigor (RIG) as the Academic Outcome variable and EFF, OTL, and PRC as Predictors Model Summary R R2 Adj. R2 SE F df1 df2 p Durbin- Watson .416 .173 .148 .944 6.893 1 33 .013 2.368 Coefficients Model Unstandardized Standardized t p VIF B SE Beta EFF^ .177 1.116 .273 1.015 OTL .441 .168 .416 2.625 .013 .100 PRC^ -.036 -.183 .856 .683
Dependent variable: School-Rigor (RIG) Predictor: Opportunity to Learn (OTL)
^Excluded: Educator Efficacy (EFF), Educators’ Perceptions of Their School Environment (PRC) Method: Stepwise, n = 35
Predicting Academic Outcomes
The variances in the three measures of academic outcome, Primary School Examinations, School-Based Averages, and School-Rigor, were not equally accounted for (explained) by the educator-related variables (Educator Efficacy, Opportunity to Learn, and Educators’ Perceptions of Their School Environment). Opportunity to Learn was the best predictor of both Primary School Examinations and School-Rigor with statistical significance. The Primary School Examinations was an external measure reflecting the mean performance of eighth-grade students as a reflection of the school’s achievement. School-Rigor was computed with consideration of both the Primary School Examinations and School-Based Averages (all grades). Therefore, for the schools
sampled in this study, Opportunity to Learn explained a statistically significant portion of the variances in measures that included external assessments. However, the Educators’ Perceptions of Their School Environment variable better explained variances in the School-Based Averages among the schools. The educators’ self-reported ratings encompassing their internal capacity and likelihood to succeed in their job Educator Efficacy did not appear to significantly account for variances in academic outcome variables. School-level academic achievements, both externally and internally sourced, did not appear to be significantly impacted by Educator Efficacy for the schools
represented in this study. Question 4
Was there a difference between class averages for upper, middle, and lower divisions and school-level performance on the Primary School Examinations?
When determining if there were differences between class averages grouped by division (upper, middle, and lower), due to sample sizes, analyses were only possible using the overall data (district level). The analyses to answer this research question could include neither responses from preschool-only teachers nor those from educators who were not teachers. Sample composition also impacted the group sizes, which were unevenly distributed across school divisions: lower division (n = 59), middle division (n = 103), and upper division (n = 72). The harmonic mean was used in the Tukey HSD post hoc analyses; therefore, Type 1 errors (false positives) were possible.
As a whole, it was determined that there was a statistically significant difference for School-Based Averages estimates by class-level divisions (F(2, 231) = 4.741, p < .05, η2 = .039) (Table 22). Class-level divisions accounted for 3.9% of the total variance in
School-Based Averages. In the multigroup comparisons (post hoc test), the only statistically significant difference for class averages by divisions was revealed between lower and the upper divisions (Mdiff = 0.565, SE = 0.186, p < .05) (Table 22). For the
Primary School Examinations, it was determined that there were no statistically
significant differences in estimates by class-level divisions (F(2, 229) = .020, p > .05, η2
< .001) (Table 22). Since division was not a significant predictor of Primary School Examinations, a school-level variable, no further post hoc tests were conducted.
Table 22
Differences between School-Based Averages by Class Level/Divisions ANOVA: Model Estimates by Division
Sum of Squares df Mean Square F p η2 SBA Between Division 10.662 2 5.331 4.741 .010 .039 Within Division 259.738 231 1.124 Total 270.400 233 PSE Between Division 0.040 2 0.020 0.020 .981 0 Within Division 234.711 229 1.025 Total 234.751 231
Tukey HSD Post Hoc Test
(I) Division (J) Division
(I-J) Mean Difference SE P SBA Lower Middle 0.386 .173 .069 Lower Upper 0.565* .186 .008 Middle Upper 0.179 .163 .517
Dependent variable: School-Based Averages (SBA) Independent variable: Division (Lower, Middle, Upper) * Statistically significant at the .05 level.
Therefore, the fourth question about differences in School-Based Averages and Primary School Examinations by division revealed that overall there was a significant difference in School-Based Averages by division, but the difference was only statistically significant between upper and lower divisions. Overall, the class division was not a significant predictor of Primary School Examinations.
Question 5
What was the relationship between selected school characteristic variables (School Size, Location, and Infrastructure) and variables for Educator-Efficacy and Educators’ Perceptions of Their School Environment?
The relationships between school characteristics and educator-related variables were assessed and described in terms of Pearson correlation coefficients and p-values (Table 23). The analyses included two measures for the School Size: student population and the number of educators in schools. In addition to School Size, there were two other school characteristic variables, Location and Infrastructure. The educator-related
variables of particular interest were Educator Efficacy and Educators’ Perceptions of Their School Environment of the school environment.
At the school level, the correlational analyses (Table 23) indicated that there were negative associations between school-size variables (student population, number of educators) and educator-related variables (Educator Efficacy and Educators’ Perceptions of Their School Environment). In school-level comparisons, the relationships were not statistically significant (p > .05) between School Size and Educator Efficacy or between School Size and Educators’ Perceptions of Their School Environment (Table 23).
In other school-level correlational analyses, the relationship between a school’s location and educator-related variables presented negative coefficients. The negative coefficient for the relationship between the school’s location and Educator Efficacy was not statistically significant (p> .05). However, the negative relationship between Location and Educators’ Perceptions of Their School Environment was statistically significant (r (35) = -.492, p < .001) (Table 23). The Location grouping codes were, Urban = 1, Rural =
2, and Remote = 3; therefore, Educators’ Perceptions of Their School Environment were less favorable for schools located farther away from the urban areas.
At the school level, Pearson correlation analyses indicated that there were statistically significant positive associations between the physical attributes of the school’s infrastructure and the educator-related variables. The relationship was statistically significant between physical attributes of the school’s infrastructure and Educator Efficacy (r(35) = .486, p < .05). Similarly, the Pearson correlation analyses showed that the relationship between the physical attributes of the school’s infrastructure and the Educators’ Perceptions of Their School Environment (r(35) = .398, p < .05) was statistically significant.
Table 23
Correlations between Select School-Characteristic Variables and Educator-Related Variables Data
Level
Educator Variables School Characteristic Variables M SD EFF PRC Size-E Size-S LOCN PHYS
School EFF 386.97 26.35 1 .519** -.189 -.243 -.149 .486** PRC 377.70 40.57 .519** 1 -.218 -.205 -.492** .398** Size-E 12.44 7.45 -.189 -.218 1 .951** .183 -.189 Size-S 259.81 190.16 -.243 -.205 .951** 1 .229 -.200 LOCN 112.50 30.18 -.149 -.492** .183 .229 1 -.295* PHYS 61.70 12.74 .486** .398** -.189 -.200 -.295* 1 Overall (District) EFF 384.21 54.68 1 .510** -.105* -.115* -.082 .225** PRC 375.72 65.36 .510** 1 -.091 -.093 -.379** .303** Size-E 16.49 9.32 -.105* -.091 1 .969** .171** -.099* Size-S 370.21 245.25 -.115* -.093 .969** 1 .207** -.101* LOCN 108.25 30.04 -.082 -.379** .171** .207** 1 -.282** PHYS 60.12 22.09 .225** .303** -.099* -.101* -.282** 1
** Correlation is significant at the 0.01 level (1-tailed). * Correlation is significant at the 0.05 level (1-tailed). School-Level: Listwise n = 36
Overall, at the district level, Pearson correlation coefficients did not reveal any statistically significant relationships between School Size and Educators’ Perceptions of Their School Environment. However, there were statistically significant negative
associations at the .05 significance level between Student Population (Size-S) and Educator Efficacy (r(294) = -.115, p < .05). Similarly, the relationship between number of educators (Size-E) and Educator Efficacy (r(294) = -.105, p < .05) was statistically significant (Table 23). Correlational analysis also revealed positive statistically significant associations between physical attributes of the school’s infrastructure and Educator Efficacy (r(294) = .225, p < .001) and between physical attributes of the school’s infrastructure and Educators’ Perceptions of Their School Environment (r(294) = .303, p < .001). Furthermore, the relationship between Location and Educators’ Perceptions of Their School Environment (r(294) = -.379, p < .001) was statistically significant, but the relationship between the school’s Location and Educator Efficacy was not significant.
In response to the fifth question, there were statistically significant relationships between select school characteristic variables and educator-related variables at the school level and the overall district level. There were positive statistically significant
associations for the physical attributes of schools and both educator-related variables in every comparison. Where a school was located (Location variable) also revealed statistically significant associations, at both levels, with the Educators’ Perceptions of Their School Environment. Correlations between the school-size variables and educator- related variables revealed only one significant association with Educator Efficacy at the district level and none at all with Educators’ Perceptions of Their School Environment.
Overall at the district level, for the schools included in this particular study sample, School Size appeared to have a proportional impact on the how educator’s perceived their schools, but that relationship changed when considered at the school level. What seemed to have had a greater impact was the condition or physical attributes of the schools.
Chapter 5 Discussion Overview
This chapter discusses the five primary research questions associated with this study, the data that was used, and some considerations for the Belize education system. The first question in this study asked if there were differences between the School-Level Achievement estimates, as derived from the Base-Model using the Weighted-Indicator