Data and data collection

In the attempt to answer my research questions, I regarded it appropriate to collect different kinds of data. Although classroom observation and in- terviews play a considerable role in my study, I also draw upon data from

students’ written work, from meetings with the teacher about the teaching, and learning, the class and the school, and from some meetings within the TBM/LBM project when teachers’ views on topics covered in the ob- served class came to the fore. In this section I will go into what data I have collected and what methods I have used. An overview of the collect- ed data is presented in the figure below.

Figur 7-1 Types of collected data

In the case study method different kinds of data might be collected, both quantitative and qualitative data (Yin, (2009). This thesis will rely on qualitative data, and is located within the interpretivist paradigm. 7.3.1 Classroom observations

My main data corpus is data from the classroom. The collection started two weeks after the start of school, after one week with mathematics les- sons. I divide the observations into two parts according to the mathemati- cal topic taught and to the period of time. The first period was devoted to number operations and elementary algebra, the second was devoted to lin- ear functions and systems of equations. The teaching followed the text- book, and these periods correspond to chapter 1 and chapter 4 in the text- book. The data from the second period is drawn upon as a comparison to the teaching of the topics in the first chapter. In addition, students’ work with tasks related to the learning goals for the first chapter is drawn upon.

To collect data, I videotaped each observed lesson from the start to the end. In addition, the teacher carried with her an audio recorder. The rea- son for this was to audio tape the conversations between the teacher and the students during seatwork. During the plenary sessions I either admin- istered the video camera, or I sat at the back or in the front of the class- room taking notes. This depended on whether a colleague assisted me or not during the data collection. Mostly we were two.

During the seatwork, I went around with an extra camera, videotaping students’ PC-screens and thus students’ work. On occasions, I interacted with them during the seatwork asking them questions about what they were doing. It happened that students asked me or my colleague for help. We had discussed this and agreed that ethically we could not refuse to help, when there were others waiting for the teacher. However, mostly they waited for the teacher, or asked their peers.

7.3.2 Interviews

During the collaboration with the teacher, I had many meetings with her, and sometimes with her and some of her colleagues. All the meetings were audio taped. Most of these meetings I will characterise as unstruc- tured interviews, conversational in style. Some of those interviews took place right after a lesson, when we were discussing the lesson, some were planned in the sense that we had made an appointment to meet and to hold an interview. Other interviews were semi-structured in the sense that I had a list of questions to ask, however, the teacher was free to lead the inter- view in the direction she wanted. For me it was important to listen to her opinions and thoughts. Although this was a strategy, it was as well for me a genuine interest in her practice.

All the students had a short interview twice as to their opinion about using computers instead of paper and pen. Another time they were asked about their experiences of working in groups to find the solutions without the teachers’ explanation beforehand. These interviews were structured in the sense that they all were asked the same few specific questions; more in the style of a survey.

Five students were interviewed based on tasks picked out from both the algebra test, and the first ordinary test. Four students were interviewed twice; once in the autumn and once again in the springtime right before finishing the school year. One interviewed student, a girl, changed class during the school year. Another girl, from whom I had a considerable amount of data from the classroom observation, was chosen for interview in the spring.

The reason for choosing task-based interviews was to illuminate the reasoning behind students’ solutions. Test results alone give limited in- formation about students’ reasoning. Goldin (2000) claims:

In comparison with conventional paper-and-pencil test-based methods, task based interviews make it possible to focus research attention more directly on the sub- jects’ processes of addressing mathematical tasks, rather than just on patterns of correct and incorrect answers in the result they produce (ibid, p. 520).

The questions were focused on the solution of the tasks, descriptions of the strategies, and assessment of the solution.

The five students were chosen based on their results on tests. I wanted students representing both top, bottom and middle level. Two girls and three boys were chosen. In the class the proportion between girls and boys was 10:17.

In the autumn the interviews were video and audio taped. Some stu- dents felt the filming uncomfortable, and the camera was moved to vide- otape from a relatively long distance. In the spring the interviews were only audio taped.

From my master study I had positive experiences with focus group in- terviews. When the students talked together during task solutions, they were more open than when being interviewed alone. Therefore, my inten- tion was to do the same in these interviews, however, this did not function as intended. One of the girls remained rather quiet during the whole inter- view session. She had to be prompted. This resulted in the boys being in- terviewed one by one and in the spring all were interviewed individually.

All students had paper and pencil available and their notes were col- lected. The interviews were of different time duration. They lasted from 6 to 40 minutes. The short interview was with Tord. He had made few er- rors and was extremely quick in his answers. In addition, he wanted to reach the next lesson, he answered all the questions though.

7.3.3 Tests

Based on my findings from my master study, I conducted an algebra test, this was a paper and pencil test. This test was administered by the teacher after the first week of the school year. At that time algebra had not been the focus for teaching. The tasks were taken from studies such as the CSMS study (see section 4.1) and the KIM-study (see section 1). Most of the tasks were also present in the tests in the LCM-study14_{. The reason for}

not making my own tasks, was to be sure that they had been revealed as purposeful in earlier studies. Another reason was that I could compare the results in this study to the results from the LCM, KIM, and CSMS studies. This test was given twice for students to solve; in August 2007, and 8 months later in May 2008.

In September, 25 out of 27 students were present. In April, 26 students solved the test tasks. At that point in time two students had moved to an- other class, and two new students had moved in. One student was missing.

14 _{The LCM study was part of the KUL Programme (Kunnskap, Utdanning og Læring –}

Knowledge, Education and Learning) of the Norwegian Research Council (Norges For-

skningsraad, NFR). My master study was part of this study. The study relied mainly on tests in grade 4, 7, 9, and 11. The tests were administered by the teachers in the participating schools and were given twice a year in three consecutive years.

That means that only 22 students participated in both test situations. Since only 24 out of the students had followed the same teaching in the autumn, the other students are left out when I refer to the test in the spring. To conclude: 25 students in the autumn and 24 in the spring are in the sam- ple. 22 of them can be compared from autumn to spring. Both tests were administered by the teacher. In May I was there observing. It seemed as if the students worked seriously trying to do their best.

All the ordinary tests were written on the computers and delivered to the teacher as computer files, and I was allowed access to these files. In the first lessons after both periods of observation, students were tested in the topics taught during these observations. Some tasks from the algebra test and some from the ordinary test in September were the basis for the interviews with the students. In addition, one test in December, concern- ing all the topics taught during the autumn is part of my material. From the latter, only tasks related to the learning goals expressed in the first chapter of the textbook are taken into account. (All tests are presented in appendix 6.)

7.3.4 Questionnaire

At the outset of my study, I planned to give the students a questionnaire with some questions about their views and experiences of mathematics. This questionnaire was mainly a translation of a questionnaire applied by Persson in his study (Persson, 2005). In order not to disturb the teaching, it was decided to send this to the students electronically and ask for an- swers by e-email. This was not a success, as less than half the students replied (12 students). In the parts of the thesis describing individual stu- dents, the answers in the questionnaire will be drawn upon for the students who answered. (The full questionnaire in Norwegian is to be found in Ap- pendix 7).

7.3.5 The ‘learning book’ and students’ solutions

The teacher and her colleagues had created a web site for all the students attending the mathematics course in Vg1. They had worked out examples, mainly from the textbook, and short versions of rules and didactical ad- vice stored in data files; one file to each sub-chapter in the textbook. The collections of these files were called the ‘learning book’, and were created to help students to be focused on the white board and to participate in the plenary session. For the fast writers the teachers had experienced that stu- dents were often distracted by the computers; with a keystroke they could mentally leave the classroom. Thus all worked examples, rules and advice were to be found later on the web-site. In addition, the teachers encour-

aged students later to write their own notes in the same files, and save them on their private computers.

I was interested to see if students had saved their own notes and thus made changes in their ‘learning books’. All students answered a short sur- vey, in addition some students down loaded their files in order to show me their work. This was done at the end of the second period in November. Later when looking into their files I realised that some had sent me their complete work done during the autumn term.

At the outset I had rejected the idea of looking into their work with as- signed tasks. One reason for this was that students had answers to all the tasks in the back of the textbook, another was the overwhelming amount of data. However, when I looked into the files I had been given access to, I realised that many students had not bothered to check their answers, or they had adjusted them to the right solution in different ways. At that time, I also saw the advantage of comparing the files with the observa- tions made in class. I went back to school, and the students willingly shared the solutions to their assigned work during all the time of observa- tion.

From the date on the files I could see who had changed the files or who had down loaded ready-made solutions from the publisher. The latter files were rejected. In addition, some students had suffered a computer crash which had deleted all their files. Data from 18 out of 27 students constitutes the data from students’ saved computer files.

For the students it was helpful to have their computer well organised, because they were allowed to use any resource apart from communication and internet access on tests.

7.3.6 Other data

Other data are official documents from the school, the textbook and its resources, official documents related to the syllabus, and data from meet- ings and workshops within, or related to the TBM/LBM projects.

In the table below most of the data is presented, with date for the col- lection, the type of data and who was present.

Table 7-1: Data collected (R-researcher, T teacher)*

Date Event/Purpose Kind of data Present

070508 Meeting with teachers: Information about the project

Audio R1, R2, R3

teachers 070619 Meeting in order to start cooperation Audio + notes R1, T1, T2 070827 Planning the cooperation Audio + notes R1, T1, T2 070904 Classroom observation (2 lessons) Audio/video/notes R1, R2

070904 Algebra test Test on paper

070910 Classroom observation (3 lessons) Audio/video/notes R1, R2 070911 Classroom observation (2 lessons) Audio/video/notes R1, R2 070917 Ordinary test algebra + numbers Students’ test on

computers August-

Sept. students) Students work on textbook tasks (18 Computer files 070918 Short meeting with T1 Audio

070925 Interview with T1, Watch video Audio R1 + T1 070925 Interview with five students Audio/video

+ students’ notes R1 Oct.-Nov. Meetings with teachers in the school Audio R1, T1, T2,

T3, T4 071112 Classroom observation (2 lessons) Audio/video +

students’ notes

R1 + T1 071113 Classroom observation (3 lessons) Audio, 2 videos

+ students’ notes R1 + T1 071119 Classroom observation (2 lessons) Audio, 2 videos R1 071126 Classroom observation (2 lessons) Audio, 2 videos R1 071127 Classroom observation (3 lessons) Audio, 2 videos R1, R5 071204 Ordinary test - Linear function Students’ tests

071218 Ordinary test - All topics taught Students’ tests

080122 Interview teacher + class meeting Audio R1, T1 080310 Short questionnaire - digital files

080519 Interview 4 students Audio + students’

notes R1

080602 Interview one student Audio + student’s

notes R1

Nov. 2008 Collecting students’ computer files Computer files From 18 students Meeting

Dec. 2008 Meeting in TBM/LBM Notes Sept. 2009 Conference presentation sept 2009 Notes shortly

afterwards

R1 + confer- ence partici- pants 2010 Comments on workshop presentation Video TBM/LBM

members 2010 Presentation – County’s School leader

meeting Notes afterwards School lead-ers

* T1 is the class teacher, R1 is me.

All data occurring in the TBM project is stored systematically with per- mission from the NSD - Norwegian Centre for Research Data. In accord- ance with regulations from the Inspectorate, the data can be stored for an undefined period, however, from 2015 the data is made anonymous and all name lists are deleted. Access is restricted to the participants in the TBM project at the University.

All names of data files have the same format. One example could be: SM-070580-TBM-C-au-ABF-MB-HE.MSV. The first two initials indicate which type of event is recorded. The next is the date, then the initials for the project, and initials for the location. The abbreviation au indicates that this recording is an audio recording. The next initials are for the partici- pants in the event.

This particular file is an audio recording from a school meeting on the 8th _{of May 2007 within the TBM project. The meeting took place in}

school C and the participating researchers were ABF, MB, and HE. Msv is the type of file, a voice file. In this way the file names give precise in- formation about what, when, where, who, and which type of recording.

7.4 Work with the data

Since the data corpus is of different types, different methods of analysis were applied in the work with the data. In this section these methods will be described.

7.4.1 Video and audio recordings

Many hours of video recordings and audio files were watched and listened to before a data reduction was made.

Data from school meetings in the particular school were partly tran- scribed after being searched for statements about the particular course and class. Recordings from interviews and meetings with the teacher were ful- ly transcribed, and regular statements were categorised. All references from the teacher are shown to her. She was sent the manuscript for the whole thesis, and in a meeting all text that directly refers to her was shared with her in a process of informant checking.

All the interviews with students were fully transcribed. Afterwards the students’ reasoning behind the solutions, and their strategies were catego- rised according to earlier studies reviewed in chapter 4. This work served as a background for the interpretation of other solutions of the same task and solutions of similar tasks solved by other students on tests and in reg- ular classroom sessions.

The short interviews, more in the style of surveys taken in the class- room about the students’ opinions on the paperless classroom and about the use of the ‘learning book’, were also fully transcribed. Categories were created based on students answers. These were then compared with what was observed in the classroom or said in other interviews.

The data from classroom observation were watched and listened to and a data reduction was made. Then parts of the data material were tran-

and listened to time after time, more and more of the data were fully tran- scribed. As the the transcripts were read over and over again, some pat- terns of interactions appeared to be regularly repeated. These were catego- rised.

Since the recordings from the classroom were stored in different files, some of the work with the data was to combine the files. This was done in that the transcripts from the different sources were combined in one text file where the time schedule was the common reference.

This file was later combined with the data from students’ written work. The computer files with students’ written work helped to illuminate what tasks students worked on, and thereby gave a better understanding of the students’ responses when solving the tasks in the classroom. The same was the case with the data from the interviews with students.

7.4.2 Students’ written task solutions

The computer program excel was used as the main tool when structuring the data. One file was made for each test and one was made for the as- signed tasks. First, all students were listed in rows, then the tasks in the columns. All task solutions were investigated for the way they were solved and for errors.

All results were documented with the code 1 for correct solutions, code 0 for not solved, and 11 for errors. Afterwards the worksheets were copied to other worksheets where the errors were categorised according to findings in earlier studies reviewed in chapter 4.

From these excel files it was easy to get an overview of each student and to see if an error occurred only once or more frequently. It also

showed how many tasks each students had solved both of the assigned tasks and of test tasks. Changing rows and columns also gave a good