This chapter will address the research questions put forth in this paper. In order to answer the research questions, the sample data from the classes will be discussed. This will be followed by an analysis of the data obtained from this data set. Then the sample data from the students will be discussed. The student data will be followed by an analysis of that data. The final analysis will be to compare the class data and the student data.
Class Data
The class data were obtained through a combination of sources. The first source was the course syllabi from the various classes used in this analysis. The second source of data was taken from interviews with selected instructors of the various classes used for this analysis. The third source of data was the textbooks used for each of the courses. These three sources of data were used to triangulate the data used to identify the degree to which each class incorporated a reform approach to teaching the methods course.
The course syllabi were sent to the researcher either electronically or as a hardcopy included with the student surveys. The initial evaluation of these syllabi revealed many themes which were relevant in identifying how the instructors incorporated the NCTM’s PSSM and aspects of CFTK. These themes were compared and common themes were combined, which reduced the number of themes from 28 to 10. The identified themes are shown in Table 6.
In order to create a rubric to be used in identifying the level of reform teaching in each class (based upon the syllabi); the themes were further broken down into levels of
66 Table 6: Themes Found in Syllabi
Theme Number of Schools
NCTM Membership 16
Readings 12
Use of the NCTM’s PSSM 11
Unit Plans/Lesson Plans 11
Presentation of Lesson to Peers 11
Assessment Practices 11
Observation of Teaching 10
Technology 9
Manipulatives 6
Attention to Diversity 5
description of CFTK, as proposed by the principal designers of CFTK (Ronau & Taylor, 2008), and the ways that each of the components of CFTK were found to occur in the course syllabi. The instances where various courses addressed one of the aspects of CFTK were compared to identify how the implementation of that aspect matched the intent of the aspect as proposed by Ronau and Taylor. In order to ensure that the levels of implementation were comparable to the intent of CFTK, the rubric was sent to a designer of CFTK for his expert opinion about the rubric’s effectiveness at examining the degree of implementation of the aspects of CFTK. He examined the rubric and made recommendations to ensure that this rubric followed the intent of CFTK. These recommendations were implemented in the rubric prior to the use of the rubric to
67
evaluate the degree of implementation of CFTK in each course. This was done in the same manner as the identification of the themes where each syllabus was read and read again to ensure that the key points in each syllabus would fit into one of the levels of implementation of each theme.
The way that these themes were used in creating the rubric required examining the way that the themes were used in each course. The manner that the theme was implemented indicated various levels of implementation of that theme. One example was the way in which NCTM membership was implemented in each course was used to identify the levels of involvement for the Environment aspect. The manner in which attention to diversity was implemented in each course was used to help identify the levels of involvement included in the Individual aspect. Similarly, the implementation of the assessment practices was used in determining the Cognition aspect. The various ways that the instructors used presentations by the teachers and preservice teachers in the courses were used to help create the pedagogy and content aspects of the rubric. The manner that instructors used manipulatives and technology provided some help in creating the content and pedagogy aspects of the rubric. The orientation section of the rubric required a synthesis of several themes. These themes included the readings, unit plans, lesson plans, presentations, and observations. The reason a synthesis of these themes was needed was that each of these themes included attempts at examining the beliefs of the preservice teachers. Finally, the ways that the readings and the PSSM were incorporated into the courses helped with several of the aspects, but were difficult due to the lack of information about the exact articles and the lack of detail about how the articles were used.
Once this was completed, then the rubric was ready to use for examining the syllabi. The same rubric was used to examine the interviews in order to draw a comparison between the
68
interviews and the syllabi. In order to use the rubric on the interviews, the transcripts of the interviews had to be read and analyzed in the same manner as the syllabi. The analysis of the interview transcripts included an initial reading to search for additional themes that were not present in the syllabi. This review did not provide insight into any new themes that did not appear in the syllabi of the various courses.
The third piece of information used to triangulate the degree of reform teaching in each class was the textbook used in the course. The name and author of the textbook was obtained from the course syllabi, where available. In other cases the course instructor was contacted in order to obtain this information. Copies of each of the textbooks were obtained for use in the analysis. A rubric was created in order to evaluate the degree to which the textbooks
incorporated the ideas of the Comprehensive Framework for Teacher Knowledge (CFTK). This rubric produces a range of scores from 0 to 30 for each book where 0 represents no integration of CFTK in the textbook and 30 represents complete alignment between CFTK and the textbook. In determining how to derive a total score for textbooks, several options were considered. The mean of the scores was determined to not suffice as a course that used multiple books would not be differentiated from a course that used only one book. A rubric could have been developed to incorporate the number of books used in a course, but that rubric would need to be in addition to the CFTK rubric used to evaluate each book. Another approach was to use the sum of the rubrics for all books used in a course. This would allow courses using multiple books to have a higher score than a course using only a single book.
In comparing the data from the 3 sources of information (the syllabi, the interviews and the textbooks used), it was necessary to find a method to compare the data since the measures used to analyze the syllabi and interviews provided different ranges of possible scores than the
69
textbook analyses. The method used to compare these sources of data was to convert each score into a rank so that the ranks could be compared to identify if the course receiving the highest rank for the textbooks was the same as the rank for the course syllabi and the rank of the
interviews. When comparing the ranks of the entire group of courses, the rank of the syllabi was found by taking the results of the syllabi, found by scoring the syllabi with the CFTK rubric designed to determine the degree of CFTK mentioned in the syllabi, and labeling the highest score with a 1, the second highest score with a 2 and likewise until the lowest score on the rubric resulted in a rank of 16. In the event of a tie, each score was treated as a separate ranked score and then the ranks of all of the positions occupied by the tied scores were used to find a median rank for the tied scores. For example, if there were three scores tied for a rank position of 2, then they were treated as ranks 2, 3 and 4 and the median of these three scores was 3. The rank for each of the tied scores was then replaced with this median. This was repeated for the results of the textbook and interview analyses. A similar process was then conducted on the MBI scores for both section A and section B to provide a ranking system for them. It was discovered that there was little similarity between the three sources of data from the courses. An example of this is that the school using the most books and the most reform-oriented books only ranked in a tie for 9th and 10th place in the rankings of the syllabi, out of a total of 16. Similarly, the course who used no reform textbooks and ranked last in that category was ranked 6th for the course syllabus. The ranks of each course’s mean MBI score for Sections A and B were then used to conduct a correlation analysis to see if each score’s relative standing provided any insight into the amount of correlation between the various scores on the syllabi, interviews and textbooks. Due to the nature of the data in these analyses, a Spearman’s rho was calculated to identify the degree of linear correlation. The results of the correlation analysis using the entire group of courses and
70
section A of the MBI indicated a significant negative correlation between the syllabi rank and the results on section A of the MBI, ρ(14)= -.608, p =.012. The scatter plot of the data can be seen in Figure 2 below. This indicated that as the rank of the syllabi increased, the rank of the MBI score decreased. Another way to say this is that the syllabi that indicated the lowest degree of
implementation of CFTK correspond to the highest MBI scores. The results of the textbook ranks did not show a significant correlation to the MBI scores on section A, ρ(14)=-.156, p=.563.
Figure 2: Syllabus rank and MBI Section A rank
The results of the correlation analysis using the entire group of courses and section B of the MBI indicated no significant correlation between the syllabi rank and the results on section B of the MBI, ρ(14)=-.249, p=.351. The results of the textbook ranks did not show any significant correlation to the MBI scores on section B, ρ(14)=.086, p=.753. As before, due to only 7
71
instructors participating in the interview process, the interview data is not included here, but instead discussed later.
When taking into consideration only the instructors who were interviewed, there is a significant correlation between the ranks of the results of the interviews and section B on the MBI. When ranking the interviews based on the results of the rubric used to evaluate the interviews and the ranks of the results of section B of the MBI there is a significant correlation, ρ(7)=.826, p=.022. These results indicated that there is a significant correlation between the rank of the results of the interview analysis and the results on section B of the MBI. For the same seven instructors, the correlation analysis comparing the ranks of the Syllabi, based on their rubric results, and the ranks of the textbooks, according to their rubrics, all showed no significant correlations. The correlation of the syllabi ranks and the MBI section B scores indicate a
negative correlation that is not significant, ρ(7)=-.444, p=.323. The correlation of the textbook ranks and the MBI section B scores indicated that there is not a significant correlation between the ranks of the results of the textbook analysis and the rank of the results of section b of the MBI, ρ(7)=.345, p=.448. This indicates that only the interviews show a significant correlation with the ranked results of the MBI section B.
The results for section A of the MBI are less significant than for section B when comparing only the courses that included instructors who participated in the interviews. The correlation of the interview ranks and the MBI section A ranks also indicated that there was no significant correlation between the rank of the results of the interview analyses and the ranks of the results on section A of the MBI, ρ(5)=.382, p=.398. The correlation analysis of the course syllabi ranks compared to the ranks of the MBI section A scores indicated there was not a significant
72
section A scores indicated that there is not a significant correlation between the two variables, ρ(5)=.108, p=.818. These findings indicated that there is no correlation between the ranks of any of the collected data for the instructors who were interviewed and the ranks of the results on the MBI section A.
The analysis of the ranks did not find consistent results between the syllabi, the interviews and the textbooks. Therefore, the qualitative data is used to find commonalities among the three data sources. The syllabi analysis and interview analysis did reveal many common themes that can be seen in the qualitative analysis for Question 1.
Developing Reform Oriented Beliefs
Question 1: What characteristics of the methods courses challenge traditional beliefs and encourage the development of beliefs aligned with reform teaching practice?
In order to answer this question, the course syllabi were examined to identify methods that would challenge the beliefs of the preservice teachers. Of the 16 syllabi used in this study, none of the syllabi indicated that any methods were used for the express purpose of challenging the preservice teachers’ beliefs. The syllabi did list many types of activities that were employed by the various instructors in the methods courses. The most frequently used assignment in the syllabi was having preservice teachers create lesson plans. Lessons plans were included in 12 of the 16 syllabi. The second most commonly used assignment was to have preservice teachers read journal articles. The use of these articles varied greatly in the class from just reading to writing a reflection paper to whole class discussions. There were two assignments that were included in seven of the syllabi. These two included membership in the NCTM and observing classroom teachers. The NCTM membership was necessary in some classes in order for the preservice teachers to access literature and on-line materials, while other instructors did not indicate the
73
reason for requiring membership. There were two instructors who encouraged attendance at NCTM events. The purpose of the classroom observations was not consistent from class to class. Some of the instructors required a written summary of the classroom observations while other instructors did not specify any writing on the observations.
Due to the challenge of determining which of these activities were designed to challenge the beliefs of the preservice teachers, the interviews were used to glean more information from the instructors on what activities were used in the individual courses for the purpose of
challenging the beliefs of the preservice teachers. The interviews did list some of the same topics as the syllabi, but there were several topics that were used by the instructors that did not appear in the course syllabi. Due to the lack of evidence in the syllabi regarding the nature and purpose of the activities in regard to challenging the preservice teachers’ beliefs, for the purpose of this section of this study, the interview information will be used to identify the methods used by the instructors for the express purpose of challenging the beliefs of the preservice teachers.
In order to examine the interview transcripts to identify what methods were used to challenge the preservice teachers’ beliefs, the transcripts were read. Then each of the transcripts was reviewed to identify methods used by that instructor. Once the methods were identified in each of the transcripts, the lists of methods were compared. Common methods were identified and a list of methods was compiled. This list was then used to review the transcripts again in order to ensure that each method in every class fit one of the identified methods. This process indicated the original list did not sufficiently cover all of the methods used by the instructors. Therefore, the process was repeated. Once it was determined that all methods fit into a category, then the list of methods was examined for the number of times that each method on that list was used by the various instructors. This not only provided a list of methods used by these instructors
74
to challenge the preservice teachers’ beliefs, but it also indicated how much overlap exists between these independently taught courses.
Dilemmas that Lead to Change
Question 1.a: Which of the identified methods support the purpose of generating dilemmas for the purpose of challenging preservice teachers’ existing belief systems?
In order to address this question, it is necessary to identify the methods used to challenge preservice teachers’ beliefs. The most common method identified in the interviews, which was used by 4 of the instructors, was to have the preservice teachers take on the role of a student as the teacher taught a class that would be similar to a class that the preservice teachers will teach once they are in the schools. In this manner, the preservice teachers were to assume the role of student in order to get a feel for the way that the class looks from that perspective, or as one instructor puts it “they can interact with the activity as in almost the same way as their students would”. According to another instructor, this allows the preservice teachers “the opportunity to kind of engage in either the doing, the learning…” Typically these instructors used a reform- based curriculum to teach the lesson. One of the goals of this type of lesson for the instructors was to create dilemmas by challenging the preservice teachers because, although the activities incorporated mathematics that they already knew, “the activities are somewhat designed to bring forth ideas that they really don’t understand or that they haven’t thought about.” One of the instructors summed this method up as:
We just play school through that. And then we go back and talk about how that was