4. Results
4.2. The purpose of using data in primary schools
4.2.2. Data use for school development in high and low performing schools
Regarding data use for school development, the respondents were asked to explain how they use data for school development planning process, setting priorities, goals and targets, shaping professional
26
development, supporting conversation with parents, teachers and administrators as well as reflecting on their own teaching practices. A separate within-case analysis of data use for school development purposes in high and low performing schools is presented below followed by a cross-case examination.4.2.2.1. Within-case analysis: data use for school development in high performing schools
The respondents were first asked to explain how the school improvement and teachers’ professional development plans were developed. The respondents unanimously expressed that school development planning process was participatory, where the school improvement committee mainly organizes the participation of teachers, students, and parents. For example, a principal from School 1 explains that “all teachers participate to express their views, about 60% of the students participate in filling the questionnaires, and some selected parents (about 40-50%) also participate” in the planning process. Although the planning process was described as participatory, the nature and extent of participation seemed to vary between stakeholders. For example, the participation of parents was described mostly in relation to fundraising, and monitoring student behavior and discipline. The principal continues to say the following:
Parents participate in raising fund to fulfill school resources. They also help the school in shaping student behavior and discipline, where tripartite target setting is a good example. The school, the parent and the student set time bound goals to improve achievement; and parents agree to follow up the child’s behavior and discipline, make regular school visit to consult teachers.
The nature of stakeholders’ participation in school development planning, however, remains different within high performing schools. While some schools often used analysis of student achievement scores, classroom observation and supervision reports, and student satisfaction survey results to prepare teacher professional development plans (School 1), others base their decisions on discussions made during teachers’ general meetings. Explaining how the contents for professional development
were determined, a PD facilitator from School 2 stated that teachers made discussions on the challenges of teaching and learning. Following the discussion, “teachers forward the list of challenges [as professional development topics], of which they select the top three that are most commonly faced by them”. Evidences from the schools’ documents also appear to support what was obtained through interview responses, where schools follow almost similar PD format and structure, but differ on how they determine the content. Analysis of student achievement scores was the basis for determining the needs of professional development in School 1, but that was not the case in School 2. Rather, the school “identified twenty two” potential ‘problems’ during discussions with the teachers, from which ten of them were selected as topics for professional development.
The responses also suggest that there are distinctions in identifying areas of need for policy development and planning, and setting priorities, goals and targets for improvement. For example, School 1 appears to have used multiple sources of data, including assessment and classroom observation data disaggregated by different student characteristics, such as socio-economic status, to identify strengths and weaknesses that make comparison of baseline, actual and target achievement easier. Whereas in School 2 often used data were continuous assessment and final examination results disaggregated only in terms of the genders. When asked to explain what and how data are used for school development planning, a principal from School 1 reported that:
…often a five year trend analysis of student results is conducted. For example, the analysis mostly indicated achievement gaps between students from the community and students from orphanage supported by our school where the former achieve better. So, one of the targets of this year’s plan is reducing the achievement gaps between community and orphanage students by 20%. Data are used to monitor and evaluate the school improvement plan, compare baseline achievement with the target achievement and compare current actual achievement with a similar period in the previous year on the basis of student characteristics.
27
Using data for shaping teacher professional development was also reflected in the responses of teachers, principals and PD facilitators. However, these perspectives were not widely shared by respondents from all of the high performing schools, rather confined to some schools which seemed to have better data use practices. Only the responses of principals and PD facilitators from School 1 indicated that analysis of teachers’ instructional planning, classroom observation, student satisfaction survey data, and student assessment results is often followed by identification of knowledge and skills gap of the teachers for intervention planning. A principal from School 1 stated that:The analysis of achievement scores may show some groups of students achieve below the target, as a result, we may need to design special support programs like tutorials. Based on student satisfaction survey, classroom observation and supervision reports as well as teacher satisfaction survey, we may organize training programs for teachers on specific skill development programs often in the form of professional development activities.
Setting priorities, goals and targets for professional development and shaping its nature and characteristics ongoing is partly related with how its content was determined initially. The PD and school improvement documents of School 1 indicated that the school set a goal to increase the “percentage of teachers, in the lessons observed, who adjusted their teaching strategies according to the interests of individual students and groups to 92.3%” based on teachers’ interview and survey data. On the contrary, School 2 bases its decision on teachers’ discussions. The responses also indicate that schools used data for supporting conversations with parents, students, and teachers and with the school administration. For example, teachers use assessment results to reflect on their own teaching practice often in department level evaluative meetings and professional development sessions, including how they developed differential instruction for students, student achievement, attendance and participation in classroom activities and the functioning of peer-led team learning approach. A PD facilitator in School 1 states that:
For example, when student achievement score shows problems, we discuss with the student, parents, and teachers; then we identify whether the problems are related to the exam, the student, or the teacher. We collect parents’ views, teachers’ records and views about the student, and then propose interventions to change the achievement problem.
This point of view was also echoed by another PD facilitator from School 2 who says that “teachers reflect on their own teaching practices in PD sessions, and often followed by questions and propositions from other teachers”. Evidence from the schools’ documents also indicate that high performing schools exhibit noticeable difference within themselves in using data for meeting accountability demands by complying with regulations. While School 1 sets performance target to conduct “four meetings per year with parents to present and discuss student assessment methods, learning achievements and annual learning targets” and intends to use school records and self- assessment data for verification, School 2 often focuses on phony targets that appears to be difficult for measurement. For example, School 2 sets targets to “establish partnership with external stakeholders and improve teaching and learning”. However, there was no mention of what data can be used to measure performance level and how.
The evidences suggest that schools created different mechanisms to participate stakeholders in school development planning though the nature and extent of participation differ. While some schools participate the staff to some extent in data collection and analysis (School 1), others define participation in terms of discussion to set out areas of professional development needs (School 2). They also differ in the amount of data collected and used to identify areas of need and set priorities, goals and targets. School 1 collected multiple types of data that were fairly disaggregated by different student characteristics than School 2 and used to set baseline, actual and target achievement goals. Moreover, differences were observed when using data to shape professional development. Several kinds of data (e.g. instructional plans, achievement and classroom observation data) were analyzed to base decisions for intervention planning to bridge teachers’ knowledge and skills gap. High performing schools however displayed similarities in using data for supporting conversation. Also supported with data from school development documents, the differences within high performing schools override the similarities in using data for school development purposes.
28
4.2.2.2. Within-case analysis: data use for school development in low performing schools
Developing the school improvement plan in low performing schools follows almost the same process as in the high performing schools. The process starts with the formation of a school improvement committee charged with collecting data through questionnaires, interviews and documents (e.g. student achievement records and school minutes). Explaining the process of formulating the school development plan and what kinds of data were used to identify the areas of need for improvement and set priorities, goals and targets, a principal from School 5 stated that:
The school improvement committee distributes questionnaires to teachers, conduct focus group discussion with students and parents, collects information from different documents, such as student achievement rosters, and school minutes. The committee then rates the extent of student achievement in different subjects as very high, high, low and very low, and decides target areas.
Although schools collected different kinds of data to determine areas of need for school development planning, they lack the necessary autonomy for setting priorities, goals and targets. The schools’
effort to contextualize their school improvement planning seemed to be challenged as they were supposed to directly adopt targets set at national level. As such, student achievement data were mainly used to determine school level and classroom level goals. A principal from School 5 reported that:
The school set targets in line with the government’s national goals in the education sector which says that all students should score above 50%, 50% of students score 50-74%, 35% of students score 75-84%, and 15% of students score above 85%.
A teacher and PD facilitator from School 8 also explains how schools and individual teachers determine improvement goals. He stated that:
The Woreda (district) Education Office expects schools to improve student achievement levels by 10% annually. The school then set its targets based on the direction coming from the education office. Similarly, teachers are supposed to improve student achievement by 10% from the previous year.
The evidence from the schools’ development plans also indicate that low performing schools determine areas of need for teacher professional development through discussions with teachers on the overall problems influencing student learning and achievement. For example, in School 5 “twenty nine” potential areas or ‘problems’ were suggested, of which the school selected nine of them with a simple criteria whether the issue is related with professional development or not. Concerning the use of data for shaping teacher professional development, none of the respondents explicitly mentioned how data were used to identify the knowledge and skills gap of teachers, and plan for intervention. Hence, the possibility of schools to address the special professional learning needs of teachers on the basis of data and tackle emerging and trending issues in teaching and learning process would be greatly reduced.
Regarding data use for supporting conversation with students, teachers, parents and school leaders, continuous assessment and final examination results were commonly often used to initiate discussion during professional development, department and school level meetings. However, the discussions with parents were often dominated by issues of raising funds, student attendance and discipline (School 5 development plan and financial report). Explaining the benefits of data for initiating discussions among teachers, a principal from School 5 states that:
In department meeting teachers discuss the practice of continuous assessment, peer-led learning, usage of instructional materials, etc. They compare their own performance with others, and share experiences to scale up best experiences.
Also, a teacher from School 8 mentioned how student assessment data were used for reflecting on teachers’ own teaching practice in department level meetings. A teacher reported that:
I administer tests to measure students’ performance and report the result every month to department coordinator. At department we discuss on analyzes of average student
29
performance and on the strengths and weaknesses, and possible solutions for improvement. The analysis also compares performance of students’ in terms of gender.The responses indicated that schools collect different data (e.g. student achievement score) to prepare their school development. The participation of stakeholders (mainly teachers and students) was described in terms of providing information and filling out questionnaires to generate data for school development planning – setting priorities, goals and targets. Even though a lot of data were collected and made available, schools’ autonomy to identify development needs and set goals and targets on the basis of the data was reduced as they were required to adopt nationally set student achievement targets. Also, the possibility of schools to contextualize school development planning in a way that explains their internal capacity will be affected. Low performing schools were also experience more similarity in using data for supporting conversation with students, teachers, parents, and the school administration. While the discussion with teachers and school administration often centers on achievement results, their conversation with parents were mainly focus on student attendance and disciplinary issues.
4.2.2.3. Cross-case analysis: data use for school development in high and low performing schools
The respondents in high performing schools almost unanimously indicated that school development planning was participatory in nature. However, the way participation was described differs across schools. While in some schools teachers participate in data collection and analysis (School 1), others explain participation in terms of discussions to determine professional development needs (School 2). The nature and characteristics of data collected and used to identify priorities, goals and targets also vary, where School 1 used more data to set baseline, actual and target achievement goals than School 2. Moreover, high performing schools displayed difference when using data for shaping teacher professional development. Data on student achievement, classroom observation and instructional planning were used to identify professional development needs and for intervention planning. A slight resemblance however was shown on data use for supporting conversation often in the form of department level evaluative meetings and professional development sessions. High performing schools were exhibit more differences than similarities in using data for school development purposes, where School 1 appeared to use data more constructively than School 2.
Low performing schools also participate stakeholders in development planning process. Stakeholders provide information during discussions and survey questionnaires for the schools to determine priority areas and set goals and targets. However, the data were not sufficiently used as schools were required to adopt targets set by the Woreda (district) Education Office which determines the goals based on nationally set standards. Lastly, low performing schools used data (e.g. student achievement, attendance, and discipline) for supporting conversation with stakeholders during department based weekly evaluative meetings and professional development sessions, where teachers often reflect on their own teaching practice and share experiences.
Several instances of data use for school development purposes have been raised both in high and low performing schools. Respondents reported data use for identifying professional development needs, setting goals and targets, shaping professional development and initiating conversation with stakeholders. However, these aspects of data use were less distributed among schools; high performing schools appeared to be more diverse than low performing schools which displayed more similarity between themselves. When comparing high and low performing schools, the high performing schools (particularly School 1) were generally better in using data for school development purposes.