(Areas of high importance/low satisfaction)
Principals'
and
Teachers'
Perceptions
(Areas of high importance/low
agreement)
Students'
Perceptions
(Areas of high importance/low
study were the Grade 8 stakeholders’ five-point Likert-scale questionnaire about their perceptions related to the intention and implementation of French immersion programming.
The Noel-Levitz’s website (2004) suggested, 100% of a given population should be asked to participate in order to assure a response rate of 20-30% for their original electronic version of the SSI. The Grade 8 stakeholders from all the elementary French immersion schools in the participating school divisions were solicited via email based on the method approved by those divisions. In most cases, these invitation requests were followed up by several phone calls to elicit a response regarding their participation in this study. Furthermore, my doctoral advisory committee provided additional support by writing to the principals and introducing my research study to them.
Despite many attempts, only one school in the first division participated in my research study. Although participants from this division took the online surveys, only a few responses were returned from the Grade 8 students and parents on the survey. These responses totaled 7 from Grade 8 students (11% response rate), 7 from the principals and Grade 8 teachers (47% response rate), and only 1 from Grade 8 parents (2% response rate).
Considering this low response rate, on February 9, 2016 (six weeks after the initial contact was made with the schools in the division), my research supervisor and I discussed the possibility of increasing sample size through a request for participation to a second school division. This discussion was broadened to include the other members of the Research Advisory Committee. Members of the Committee suggested that adding incentives for participation might result in an increase of prospective participants. As a result, ten $25 iTunes Gift Cards were purchased and offered to participants through the use of a lottery system. Subsequently, and in compliance with the policy of the University, I asked for an amendment from the University
Behavioural Research Ethics Board regarding this planned increase in the sample size and the use of incentives; the amended Behavioural Research Ethics Board Approval Letter was submitted to the second division, along with a request to conduct this research study therein.
As of April 2016, the number of the participants from the Grade 8 stakeholders in elementary French immersion schools involved in this study were as follows: 38 Grade 8 students (63% response rate), 14 principals and Grade 8 teachers (93% response rate), and 4 Grade 8 parents (8% response rate).
Instrumentation and Measures
As explained in Chapter three, the measurement instrument used in this study was adapted from the Noel-Levitz® Student Satisfaction Inventory (SSI), the Institutional Priorities Survey (IPS), and the Parent Satisfaction Inventory (PSI), as developed by Schreiner and
Juillerat (1994) to better align with an elementary school context. Each of the adapted SSI, IPS, and PSI are discussed separately in the following paragraphed subsections.
Student Satisfaction Inventory (SSI).
The SSI instrument is comprised of 53 statements, with each statement correlating to one of 9 scales: School Climate, Concern about Individuals, School Staff and Teacher Effectiveness, Safety and Security, Academic Services, Instructional Effectiveness, School Resources and Facilities, Student Centeredness, and School Support Services. The adapted version used for this study varies slightly in scales from Schreiner and Juillerat’s (1994) original version, with slight changes in the related items (see Table 4. 1).
The SSI scores the importance and satisfaction of each item. The importance
score tells us how important it is for your school to meet this expectation and the satisfaction score shows how satisfied you are that your school has met this expectation (Schreiner, 2009).
Table 4. 1- Comparison between Noel-Levitz® (2010) and the adapted questionnaire (2016)
Scale Original 12 Scales (2010) Adapted 9 Scales (2016) 1 Campus Climate School Climate (SC)
2 Responsiveness to Diverse
Populations b N/A
3 Instructional Effectiveness Instructional Effectiveness (IE)
4 Recruitment and Financial Aid School Resources and Facilities(SR&F) 5 Campus Support Services School Support Services (SSS)
6 Campus Life N/A 7 Registration Effectiveness N/A
8 Safety and Security Safety and Security (S&S) 9 Concern for the Individuals Concern for the Individuals (CI) 10 Service Excellence (SE) N/A
11 Student Centeredness Student Centeredness (SC) 12 Academic Advising School Staff and Teacher
Effectiveness Effectiveness(ST&E)
Each response is indicated using a five-point Likert scale. The range and values of the Likert scales are listed in Table 4. 2.
Table 4. 2-Likert Values and Indications for Importance and Satisfaction Scores
Importance Satisfaction
1= not important at all 1= not satisfied at all
2 = not very important 2 = not very satisfied
4 = Important 4 = Satisfied
5 = Very important 5 = Very satisfied
Analysis of Satisfaction Instruments for Students
Many researchers such as Bresciani, Zelna, and Anderson (2004), and Schuh and Upcraft, (2001), demonstrated that students’ satisfaction instruments are useful tools to evaluate how satisfied students are with respect to services and programming. The goal of using these
instruments is to seek if students’ experiences align with the educational organizations’ mission and vision. According to Heiserman (2013), “to best serve students, it is imperative to know how satisfied they are with their educational experience, within the context of what they value most” (p. 35).
The Noel-Levitz SSI is an instrument that measures student satisfaction and priorities, indicating how satisfied students are both inside and outside of the classroom as well as what students expect most from their educational experience (Noel-Levitz, 2012). The assessment of both importance and satisfaction helps the administrators to be more focused on the areas that matter most to students, rather than directing resources to improve areas of dissatisfaction that students also consider a low priority. In figure 4.3, I provide the concept of how importance and satisfaction are related for the identification of schools strengths and weaknesses.
According to Low (2000) in order to measure student satisfaction, it is essential to analyze students’ experiences, as well as their perceptions. In order to best serve their students, schools must understand how satisfied their students are within the context of what matters most to them. The Noel-Levitz surveys (2012) are unique in that they capture three different scores per item, including an importance score, a satisfaction score, and a performance gap score.
Figure 4.3- A matrix for Prioritizing School Action.
Very important
School Challenges School Strengths
Not satisfied at all Very satisfied
Not important at all
The use of performance gaps in understanding student satisfaction.
The performance gap is determined by subtracting the satisfaction score from the importance score. The performance gap can also be viewed as an indicator of how the
educational organization is doing at meeting student expectations (Noel-Levitz, 2009). In other words, Gap analyses are also influential catalysts for effective decision-making, as they confirm how well schools are meeting a student’s expectations (Low, 2000). A larger performance gap means that schools are far from meeting students’ expectations, similarly a smaller performance gap indicates that schools are close to meeting the expectations (Bryant, 2006).
Strengths.
Items with high importance and high satisfaction are school strengths that should be identified and emphasized. The strength indicates the areas at the school that we should celebrate. Sharing the strengths within the school affects the school in a positive way, as
these important areas. Therefore, identifying and emphasizing the strengths as powerful components of the assessment process should not be overlooked (Noel-Levitz, 2015).
Celebration and communication of the areas that are highly valued by students, where the school is also performing well provides positive feedback to French immersion stakeholders on what works well in the school. Furthermore, the positive activities related to one area of strength could be emulated as a model in another area that has been identified as a challenge.
Consequently, strengths should be highlighted in school reports to stakeholders and the school community build a more positive reputation among French immersion stakeholders.
Strengths are identified as items at or above the mid-point in importance (top half) and in the upper quartile (25%) of your satisfaction scores (Noel-Levitz, 2015). Following this scheme, in the amended SSI survey (with 53 total items) used in the current research project, the mid- point would be rank position 26 and the upper quartile would be rank position 13. Items indicated as the strengths are shown with a star
( ) in the below tables, while items indicated as areas for consideration are shown with a flag ( ). Means for importance and satisfaction for each item are calculated by adding the students’ ratings and dividing by the number of students. Students respond to each item on a 1 to 5 Likert scale, with 5 being highest. In table 4.3, I indicate the average importance score for the students for each item provided through the descriptive statistics by the SPSS. The importance score means for students are listed in descending order.
Table 4.3- Student Importance Means Descriptive Statistics
N Mean Rank
[26.0] I am able to learn at this school. 38 4.76 1
[6.0] I feel safe and secure in this school. 38 4.74 2
[24.0] The school has clear procedures for emergencies. 38 4.66 3 [39.0] The quality of instruction I receive in my subjects is good. 37 4.59 4
[34.0] French immersion improves my future job opportunities. 36 4.58 5 [22.0] Male and female students have equal opportunities to participate
in school activities. 38 4.58 6
[31.0] I feel welcome in this school. 37 4.57 7
[5.0] The principal is helpful and approachable to students. 38 4.55 8
[3.0] Teachers are knowledgeable. 38 4.55 9
[10.0] My teachers are concerned about my successes as an individual.
38 4.53 10 [35.0] The school is a comfortable place for me. 37 4.51 11
[4.0] Teachers are caring and helpful. 38 4.50 12
[19.0] Transportation to school is available if I need it. 38 4.50 13 [8.0] The vice-principal is helpful and approachable to students. 38 4.50 14 [20.0] It is an enjoyable experience to be a student in this school. 38 4.47 15
[41.0] Student discipline is fair. 36 4.44 16
[46.0] There is enough English language instruction for Grade 8. 36 4.42 17 [38.0] When I ask questions of the teachers or principal, they provide
helpful answers. 37 4.41 18
[32.0] I can easily get involved in school activities. 37 4.41 19 [42.0] New students are made to feel welcome in this school. 37 4.41 20 [36.0] Teachers take into consideration student differences as they teach
a subject. 36 4.39 21
[11.0] The quality of instruction I receive in French is good. 38 4.34 22 [28.0] There is a commitment to my success in this school. 37 4.32 23 [50.0] I am encouraged to remain in French immersion throughout high
school. 37 4.32 24
[47.0] My teachers use a variety of teaching techniques. 37 4.32 25
[27.0] School rules are reasonable and fair. 37 4.32 25
[23.0] The exams, quizzes, and tests help me to learn. 38 4.32 25 [16.0] Teachers are fair in their treatment of me. 38 4.29
[45.0] I am able to express complaints or concerns. 36 4.28 [1.0] I feel that I belong (am accepted and liked) at this school. 38 4.26
[17.0] Computers are available and work well. 38 4.26
[33.0] Teachers provide timely feedback about my progress in a subject
(e.g., report cards). 36 4.25
[40.0] This school shows concerns for me as an individual. 36 4.25 [15.0] The classrooms have good, lighting, heat, air conditioning, and
comfortable desks. 38 4.21
[7.0] A variety of extra-curricular activities are offered. 38 4.18 [21.0] The vice-principal is concerned about me as an individual. 38 4.16
[25.0] I am proud of my school. 38 4.13 [13.0] My teacher helps me set my own learning goals. 38 4.13 [44.0] There are a good number of subject choices. 36 4.11 [37.0] My homework assignments are reasonable (hours spent on
homework). 36 4.08
[12.0] Library resources are good for my needs. 37 4.08 [18.0] The main office staff are helpful and approachable to students. 38 4.08 [29.0] I have lots of opportunities to communicate with members of the
French community to help me to become bilingual (e.g., exchanges, visits to Québec).
38 4.05 [49.0] There are people who talk with us about the advantages and
disadvantages of staying in the program. 36 4.03
[30.0] The French immersion program is designed to meet my individual
needs. 38 4.03
[2.0] The Principal cares about me as an individual. 38 4.03 [43.0] My teachers are available after school hours for help. 36 4.00 [48.0] My teachers use French actively in all subjects (except English). 37 3.95 [9.0] Library staff are helpful and approachable to students. 38 3.89 [14.0] The school is located in my neighborhood. 38 3.16
Valid N (listwise) 34
In table 4.4, calculated and presented through the descriptive statistics provided by SPSS, I list satisfaction score means in descending order of satisfaction. The standard deviation (SD) in the satisfaction score columns represents variability in the satisfaction scores; the greater the standard deviation, the greater variability in the responses. That is to say, greater variability means that some students are very satisfied, while some students are not satisfied at all with respect to a particular item.
Table 4.4- Student Satisfaction Means Descriptive Statistics
N Mean Std.
Deviation Rank [19.0] Transportation to school is available if I need it. 38 4.39 .679 1 [22.0] Male and female students have equal opportunities to
[26.0] I am able to learn at this school. 38 4.32 .873 3 [24.0] The school has clear procedures for emergencies. 38 4.32 .662 4 [50.0] I am encouraged to remain in French immersion
throughout high school. 37 4.30 .740 5
[34.0] French immersion improves my future job opportunities.
37 4.30 .845 6
[32.0] I can easily get involved in school activities. 37 4.24 .760 7 [10.0] My teachers are concerned about my successes as an
individual. 37 4.19 .701 8
[31.0] I feel welcome in this school. 37 4.16 1.014 9
[8.0] The vice-principal is helpful and approachable to
students. 38 4.16 1.001 10
[42.0] New students are made to feel welcome in this school.
37 4.14 .787 11
[3.0] Teachers are knowledgeable. 38 4.13 .529 12
[2.0] The Principal cares about me as an individual. 38 4.11 .764 13 [21.0] The vice-principal is concerned about me as an
individual. 38 4.08 .912
[6.0] I feel safe and secure in this school. 38 4.08 .941 [5.0] The principal is helpful and approachable to students. 38 4.08 .850 [35.0] The school is a comfortable place for me. 37 4.05 .880 [4.0] Teachers are caring and helpful. 38 4.03 .885 [20.0] It is an enjoyable experience to be a student in this
school. 38 4.03 .972
[28.0] There is a commitment to my success in this school. 37 4.00 .745
[25.0] I am proud of my school. 38 3.97 .885
[1.0] I feel that I belong (am accepted and liked) at this school. 38 3.95 .837 [47.0] My teachers use a variety of teaching techniques. 36 3.94 .826 [11.0] The quality of instruction I receive in French is good. 38 3.92 .818 [7.0] A variety of extra-curricular activities are offered. 38 3.92 .882 [23.0] The exams, quizzes, and tests help me to learn. 38 3.92 .712 [39.0] The quality of instruction I receive in my subjects is
good. 37 3.89 .875
[40.0] This school shows concerns for me as an individual. 35 3.86 .974 [38.0] When I ask questions of the teachers or principal, they
provide helpful answers. 37 3.81 .877
[46.0] There is enough English language instruction for Grade
[30.0] The French immersion program is designed to meet my
individual needs. 38 3.76 .883
[16.0] Teachers are fair in their treatment of me. 38 3.74 .950 [44.0] There are a good number of subject choices. 36 3.72 .974 [14.0] The school is located in my neighborhood. 38 3.68 .904 [37.0] My homework assignments are reasonable (hours spent
on homework). 36 3.64 .833
[27.0] School rules are reasonable and fair. 38 3.63 1.051 [13.0] My teacher helps me set my own learning goals. 38 3.61 .946 [33.0] Teachers provide timely feedback about my progress in
a subject (e.g., report cards). 36 3.58 1.156 13
[49.0] There are people who talk with us about the advantages
and disadvantages of staying in the program. 36 3.56 .969 12
[41.0] Student discipline is fair. 36 3.56 .969 11
[9.0] Library staff are helpful and approachable to students. 38 3.55 .891 10 [48.0] My teachers use French actively in all subjects (except
English). 37 3.54 .989 9
[12.0] Library resources are good for my needs. 38 3.53 .922 8 [18.0] The main office staff are helpful and approachable to
students. 38 3.47 1.084 7
[29.0] I have lots of opportunities to communicate with members of the French community to help me to become bilingual (e.g., exchanges, visits to Québec).
38 3.45 1.224 6 [36.0] Teachers take into consideration student differences as
they teach a subject. 35 3.43 1.092 5
[17.0] Computers are available and work well. 38 3.37 .998 4 [43.0] My teachers are available after school hours for help. 36 3.33 1.146 3 [45.0] I am able to express complaints or concerns. 36 3.28 1.059 2 [15.0] The classrooms have good, lighting, heat, air
conditioning, and comfortable desks. 38 3.13 1.018 1
Valid N (listwise) 31
As noted earlier (p. 121), items indicated as the school strengths are shown with a star ( ) in the above two tables 4.3 and 4.4, while items indicated as areas for consideration are shown with a flag ( ).
Challenges or areas for consideration.
Challenges are items with higher importance and lower satisfaction or a larger
performance gap. Challenges are the areas where students’ most important expectations are not met by the school. Schools usually conduct student satisfaction assessments in order to identify areas for improvement. In my research study, these improvement priorities are highlighted as a list of areas for consideration rather than a list of challenges. This was purposeful and intended to avoid any misconception-generating among Grade 8 stakeholders and further any prejudice or bias in stakeholder responses to the individual and focus group interview questions as these related to the areas shown in the diagrams. The areas for consideration should be “discussed, explored, prioritized, and responded to” (Noel-Levitz, 2015, p. 9). According to Noel-Levitz (2015), ignoring these areas can further risk increased student dissatisfaction, and finally impact the retention of the students.
Discussions about the areas for consideration are important, as we cannot assume that we know what students mean on each individual item from the data alone. Focus group interviews with students, followed by the SSI survey provide powerful insights, discussing the areas that the students value most, addressing specific issues, and ultimately providing suggestions to improve these situations towards positive change.
Similar to the studies conducted by the Noel-Levitz (2009) using these scales, areas for consideration are specifically identified as items at or above the mid-point in importance (top half) and in the lower quartile (25 percent) of your satisfaction scores or items at or above the mid-point in importance (top half) and in the top quartile (25 percent) of your performance gap scores. Items indicated as areas for consideration with higher importance and lower satisfaction are shown with a flag ( ) in the two above tables 4.3 and 4.4. Moreover, presented in the table
4.5 are items (appendix A) with higher importance and larger performance gap. The areas for consideration are listed in descending order of performance gap score and shown with a flag ( ) in the table below.
Table 4.5- Student Performance Gap Means Descriptive Statistics
N Minimum Maximum Rank
Gap_performance15 1 1.08 1.08 1 Gap_performance45 1 1.00 1.00 2 Gap_performance36 1 .96 .96 3 Gap_performance17 1 .89 .89 4 Gap_performance41 1 .88 .88 5 Gap_performance39 1 .70 .70 6 Gap_performance27 1 .69 .69 7 Gap_performance43 1 .67 .67 8 Gap_performance33 1 .67 .67 9 Gap_performance6 1 .66 .66 10 Gap_performance46 1 .64 .64 11 Gap_performance18 1 .61 .61 12 Gap_performance38 1 .60 .60 13 Gap_performance29 1 .60 .60 13 Gap_performance12 1 .55 .55 Gap_performance16 1 .55 .55 Gap_performance13 1 .52 .52 Gap_performance49 1 .47 .47 Gap_performance5 1 .47 .47 Gap_performance4 1 .47 .47 Gap_performance35 1 .46 .46 Gap_performance37 1 .44 .44 Gap_performance26 1 .44 .44 Gap_performance11 1 .42 .42 Gap_performance3 1 .42 .42 Gap_performance48 1 .41 .41 Gap_performance31 1 .41 .41 Gap_performance23 1 .40 .40 Gap_performance44 1 .39 .39
Gap_performance40 1 .39 .39 Gap_performance47 1 .38 .38 Gap_performance9 1 .34 .34 Gap_performance24 1 .34 .34 Gap_performance10 1 .34 .34 Gap_performance8 1 .34 .34 Gap_performance28 1 .32 .32 Gap_performance1 1 .31 .31 Gap_performance34 1 .28 .28 Gap_performance42 1 .27 .27 Gap_performance30 1 .27 .27 Gap_performance7 1 .26 .26 Gap_performance22 1 .21 .21 Gap_performance32 1 .17 .17 Gap_performance20 1 .17 .17 Gap_performance25 1 .16 .16 Gap_performance19 1 .11 .11 Gap_performance21 1 .08 .08 Gap_performance50 1 .02 .02 Gap_performance2 1 -.08 -.08 Gap_performance14 1 -.52 -.52 Valid N (listwise) 1
In figure 4.4, I depict all the areas for consideration, comparing the importance and satisfaction means for each item on a five-point Likert scale. All these areas are listed in descending order of importance score. The data collected from the focus group interviews with students in the next chapter will provide more detailed information about these areas related to students’ perceptions of French immersion programming in the three selected schools, help to better understand the areas for consideration, and finally provide the direction that the schools needs in order to improve the situation.
Figure 4.4- Areas for Consideration by Students
According to Erwin (1991), an instrument employed in research (such as that found in the present study) should have both good reliability and evidence of validity so that the assessment information are helpful in decision making. The reliability of the SSI has proven to be
exceptionally high. Cronbach’s alpha, as calculated by SPSS, was .93 for the set of importance scores and .94 for the set of satisfaction scores.
3.63 3.43 3.81 3.78 3.56 3.89 4.08 4.32 4.39 4.41 4.42 4.44 4.59 4.74 1 2 3 4 5
SCHOOL RULES ARE REASONABLE AND FAIR.
TEACHERS TAKE INTO CONSIDERATION STUDENT DIFFERENCES AS THEY TEACH
A SUBJECT.
WHEN I ASK QUESTIONS OF THE