Data Collection Methods

influences the kinds of data that the researcher subsequently gathers”, and that qualitative students generally heavily rely on observation and/or interviews (2005, p. 144) as data sources. This is certainly true for my action research case study.

I used a dual method to collecting data based on both classroom-based data collection as well as literature-based research. These two information sources worked in a cycle: First, I conducted, in several phases, a preliminary investigation into the basic motivational forces at work in my classroom. Then I turned to literature-based research to better understand the topics that emerged from the class-based research. I then returned to classroom-based research to gain an even deeper perspective of what motivates students and why. My final delve into existing literature helped me to understand these deeper perspectives. Repeatedly returning to the literature served as a form of further data interpretation that had a formative fashion on my research as a whole.

The literature-based research I undertook in chapter two proved that motivation is a very complex construction. This means that the information learned from students only represented a small part of their entire motivational picture. My initial classroom based research was focused on what was learned from Dörnyei during my literature- based research in chapter two: Motivational Influences in the Actional Phase.

According to Dörnyei and Otto (1998), student appraisal is ongoing during the

Actional Phase. During appraisal, the learner attaches values to environmental stimuli such as the teacher, the class, the subject, the material, and learning. This is where attitudes are formed. I wanted to find out what classroom factors were shaping student motivation. The main factor turned out to be a class activity – the state German

competition. Another factor was a vehicle of instruction –Total Physical Response Storytelling (TPRS).

Since I am primarily interested in motivational factors that I can adjust to tailor my instruction to my students, information on Motivational Influences in the Preactional Phase, which are focused on influences that have occurred prior to the student entering my classroom, was not collected. Examples of Preactional Motivational Influences would include individual students’ internalized value systems, as well as parental influences on students’ thinking. (Dörnyei and Otto 1998) If I were to

conduct the study again from the beginning, I would have included an initial

questionnaire concerning the factors influencing the students’ joining the class in the first place. Knowing these factors would have added more depth to the study as information about students’ Preactional Motivational Influences would have been discovered. This, in turn, would have enabled me to find out if a relationship existed between Preactional Motivational Influences and Motivational Influences in the Actional Phase at the Learning Situation Level.

I used various methods of collecting my initial data, as I wanted to ensure gaining a full picture. This use of several forms of data collection enriched the depth of results as well as made my findings more credible as (ideally) similar conclusions would appear in all types of data collection. Such use of multiple sources of data is called triangulation. (Mills 2000, p. 49) Data collected was transcribed and archived for confirmability.

Open ended questionnaires

My initial methods were written, open ended questionnaires. These were very easy to administer, allowed me to see a very broad (too broad in the beginning) range of information. The open ended format did not force students to make choices, but left it wide open for them to write what they truly felt. Since the questionnaires were administered in class, and were often tied to a short explanatory lesson (for example the motivation lesson given before the initial motivation questionnaire), the

questionnaire was not given to students absent on that day, as they had missed the short lesson that clarified the object of the questionnaire. Therefore, there are often varying numbers of responses dependant upon attendance on the day of delivery of the questionnaire.

Written questionnaires acted as a springboard for group and individual interviews. It was decided to use this method first as a way to give students quiet time to begin thinking about what motivates them. This written questionnaire can be seen as a warm up exercise to the interviews. Taped interviews without this “quiet time” to think might have made students feel “on the spot” and more likely to provide an “I don’t know” answer during the interviews. I wanted to give students a chance to take an

I left initial questionnaires in conversational format. I write my tests in the same way. I find that the “Tell me about…” question format has an informal feel to it and invites students to give an answer instead of demanding one from them. This very open ended format also worked well as no predetermined categories were imposed on the data collected. Categories are allowed to emerge from the data itself.

Dörnyei (2003, p. 63) recommends piloting the questionnaire before actually giving it to the actual participant group. On October 31, 2009, I piloted the questionnaire on 16 3rd and 4th grade Social Studies students and asked them questions concerning the clarity of instructions and clarity of the questions themselves.

It was found in my pilot study that getting students to provide a detailed response meant that it was necessary to give them a little food for thought. As I passed out the questionnaire to the pilot study group of 16 3rd and 4th grade Social Studies students, the first question came within 60 seconds of handing out the questionnaire: “What is motivation?” An opening speech with the group was needed to explain what

motivation was. Students were told that I would get back with them by the end of the day and we would continue with the pilot study.

I used my break time to research how I could teach “motivation” as a mini lesson, and found a model on which to base my lesson on line (Mills and McKnight 1988). I modified the lesson to better suit my elementary and middle school audience, and then presented it that afternoon as follows.

Pilot Study for Questionnaire I:

Getting Students ready to write about Motivation

First, I explained that motivation is something that can come from inside or outside of a person, and that it is the force that makes you want to continue doing something or to start doing something. I also explained that they already knew a lot about

motivation, and that I was going to prove it to them.

I then orally reviewed adjectives and synonyms before going on to the second part of this exercise – asking students to tell me some other words for “motivation” or “being

motivated”. Students came up with quite a few words including “excited”, “wanting to do something”, “enthusiasm”, and “pumped up”.

Students were then asked to tell about a time they were motivated. As different students volunteered to tell, I wrote down key vocabulary they used into two categories – a negative and a positive category. Following the instructions given by Mills and McKnight (1988), those categories were not yet labeled, but rather students were allowed to discover the labels themselves after quite a few samples had been collected.

Students quickly did discover that the categories were positive and negative. I then explained that we can be motivated in both positive and negative ways. Positive motivation resulting in “seeking behaviors” (when we are motivated to do something because we are looking for something like pleasure or a good feeling) or negative motivation resulting in “avoidance behaviors” (when we are motivated to do something to avoid something else, like a punishment). I then returned to the individual students who volunteered instances of motivation and helped them to define if the instances they described were resulting in motivation as a seeking or an avoidance behavior.

Before going on to the next part of this small lesson, I reviewed what motivation was once again, stressing that is can come from within a person or from an outside source. I helped students to recognize where the motivation came from in each instance volunteered in class – from inside of the students, or from an outside source.

We then discussed the following question: If motivation is being “excited”, “wanting to do something”, or “pumped up”, what does it mean to be UNmotivated? I wrote down students answers which included “bored”, “tuned out”, and “wishing you were somewhere else”.

I asked students which state they most preferred to be in: it was unanimous. All preferred to be in a positively motivated state. After this mini-lesson, students and I all felt that they were ready to write about what motivated them in class. Another student asked a question dealing with length of response. It was decided to tell

interested in knowing their thoughts, and to write as much as they needed to in order to make me understand their thinking.

In addition to the questions, a few “yes/no” questions were added to help me learn what students thought of the quality of questions. These simple “yes/no” questions were added because students had already been through the mini lesson and the actual answering. I did not want to prolong the pilot study with a lengthy Q and A session concerning question quality. It was decided that if several students indicated that the quality of questions was not good, specifics would then be asked for. However, this was not the case. (See the results in Appendix 1)

The pilot study taught me that I can’t assume that young students will understand what motivation is, and that the best way for them to understand it was to tap into their own experiences. Based on the experiences made through the pilot study, it was decided to give the mini-lesson on motivation at the beginning of the actual

questionnaire for the German class and keep the questions as they were.

Semi-Structured Interviews

Interviewing, according to Leedy and Ormrod (2005, p. 144) is a data collection technique that is common in qualitative studies, and that interviews are either open- ended or semi-structured. Semi-structured meaning that the interview revolves around a few central questions. (Leedy and Ormrod 2005, p. 146)

My interviews are semi-structured in nature as I formulated the main question, based on the results of the initial questionnaire students had done in class, in advance, but left it open ended and also allowed for myself to modify or add questions as needed. These semi-structured interviews allowed for flexibility as the dynamics of the conversation changed. I feel this method worked well, as I already have a relatively high comfort level with my students – this hopefully lent itself to them speaking freely. They have also had experiences in front of the camera in group discussion formats, making them feel more comfortable speaking with the camera rolling.

I used the results of the questionnaire to help make these interviews as far reaching as possible. I clarified those written results with students as a starting-off point for the

interview, and then saw where each student took it from there. Since I worked with a relatively small sample, I conducted personal interviews with individual students or in small groups. My sample was a convenience sample, and consisted of the students from my class who were available for interviews that day.

Students basically focused on the topics derived from the questionnaires; however, they had the freedom to go off on other tangents. Using the written questionnaires as a base allows for retrospection on the part of the students, I asked questions aimed at both clarifying previous information and providing new information. I tried to get students to add depth to their responses by searching for opinions, asking for

explanations, using a mirroring technique, and using hypothetical questions. These are all techniques I use on a regular basis when talking to students both in formal

classroom situations as well as in informal school situations, and therefore were not unfamiliar to students.

Participant Observation Recordings

Stringer (2004, p. 80) says, “The principal purpose of observation is to familiarize researchers with the context in which issues and events are played out, or to provide participants with opportunities to stand back from their everyday involvement and watch purposefully as events unfold.” My situation is the latter: I am an active participant observer in my own research and needed the opportunity to be able to watch my classroom with a different view than the one I have while teaching the class. Observation was a powerful data collection method in my research.

Besides questionnaires and interviews, in-class observations served as a further triangulation of data. Making observations is particularly challenging as I am a participant observer – teaching the class and observing at the same time. Using a video camera was a great help, allowing me to teach the class, then observe student behavior afterwards by reviewing the tape I made of the class. I hung a small video camera from the top of the chalkboard at the front of the room so that students’ faces and actions would be able to be seen. Recording started as class began, and students seemed to forget that the video camera was there at all. The camera was not referred to during class, as I wanted students to act naturally. Leedy and Ormrod (2005) tell

hand as I videotaped my own classroom. More than once, the subjects I was trying to get on video moved about the classroom, or sat at different seats.

According to Leedy and Ormrod (2005, p. 145), observations in a qualitative study are generally unstructured, allowing for unforeseen data sources to naturally emerge. For triangulation purposes, I wanted to watch for specific instances of on and off-task behavior in this case and decided to use simple descriptive statistics to be able to better portray what I was observing. Leedy and Ormrod (2005, p. 180) explain that an observational study focuses on “…a particular aspect of behavior. Furthermore, the behavior is quantified in some way. In some situations, each occurrence of behavior is counted to determine its overall frequency.”

It was decided to carry through with such observations based on what was learned in my literature-based research. Keller (1987) gave me the idea of examining attention as a measure of motivation. Getting and sustaining attention means that students must be able to focus their attention on the appropriate stimuli – the stimuli the teacher is providing that leads to learning. If the stimuli are not attention provoking, then students will seek out other, inappropriate stimuli instead - missing the learning opportunity in class.

According to Keller, motivation is based on four conditions that must be fulfilled - attention is one of them, and attention is one area that I am able to monitor in class using an observational rubric. Turning my observations from the rubric into simple statistics (example - James was on task 89% of the time) is just a way of making data easier to work with. Even though this method of data collection seems more

quantitative in nature, my research remains inherently a qualitative case study as using statistics to generalize my findings is not my goal – in my case – using simple statistics is just a means to a much bigger end. Mills (2000, p. 106) reminds

qualitative action researchers, “Do not feel compelled to include elaborate statistical measures simply to add a perceived sense of rigor or credibility to your inquiry.” Denzin and Lincoln (2000) also assert that various methods are appropriate for qualitative research, “Nor does qualitative research have a distinct set of methods or practices that are entirely its own. Qualitative researchers use semiotics, narrative,

content, discourse, archival, and phonemic analysis—even statistics, tables, graphs, and numbers.” (p. 6)

My goal was to observe and record on-task and off-task behavior during class. This off-task behavior signifies periods of less motivation- periods where the action control offensive linemen working during the Actional Phase are not fending off distractions. I noted the topic at hand and type of presentation when the off-task behavior was taking place. I then compared the amount of off task behavior during one topic/type of presentation to another. These results were then compared to the information students had supplied in the written questionnaires.

A second set of in-class recordings was made after having completed TPRS training as well as the literature based research into second language learning and made changes to the curriculum. This second set of observations was analyzed the same way (looking at amount of on—task and off- task behaviors, as well as considering the topic at hand), but it was also used to compare the amount of off-task behaviors viewed in it to the amount of off-task behavior in the instruction being given prior to my new curriculum. In other words, did the new TPRS curriculum make students produce less off-task behavior? Did students exhibit more on-task behavior when the topic at hand was the state German competition? Results are featured in chapter four.

This second round of classroom observations was coupled with the second round of questionnaires, and served as a validation of my attempt to improve my instruction through the implementation of TPRS. Did my changes deliver outcomes that positively affected student motivation? Again, the results are in chapter four.

The same exact students were not on camera each day I taped instruction – absences, disappearances to the guidance counselor, and simply not being in view of the camera made it impossible for me to video tape each individual student in the natural

classroom setting each time the class met.

However, three students were on camera most of the time for both the 2009/2010 school year recordings as well as the 2010/2011 school year recordings –Alan, James,

behaviors of these three key people. As Mills reminds action researchers (2000, p. 50), when making observations, “A good rule of thumb here is to try to do less – better.”

This observation based, non-interview data is used as a further round of triangulation of the data collected from the initial round of questionnaire and interview data. Along with this non-interview based data, further literature based research was also

conducted in order to better understand the outcomes of the second round of questionnaires (those done AFTER making the TPRS method the main delivery